From 6e0fc144b061f6a039eea2f3bee15d867ab712fd Mon Sep 17 00:00:00 2001 From: NiveditJain Date: Wed, 12 Aug 2026 23:53:40 +0530 Subject: [PATCH 1/2] docs: rebuild the documentation as one product The site had two top-level tabs, "Enforcement" and "Observability", which asked every reader to work out which half of a single product they were in before reading anything. There is now one navigation, ordered as a journey: install and guard a machine, see what happened on it, then govern the fleet from the cloud. The observability half is repositioned as FailproofAI Cloud and moves from the agenteye section to the cloud section, with a redirect for every old URL. Documents the integration surface that had no docs at all. `failproofai config --connect` existed only in --help: new pages cover connecting a machine (both capabilities, what leaves the machine, fleet provisioning, troubleshooting), managed policies (deployments, digest verification, observe-mode rollout), the fleet view, and session capture across all 12 CLIs -- replacing three per-CLI capture pages written for a separate collector that the daemon has since absorbed. Adds the pages a reader kept needing and could not find: How it works (tool call to decision to dashboard, with the failure-mode table), the failproofaid service (fail-closed, supervision, how the binary arrives, upgrades), Supported agents (per-CLI matrix of what a deny actually blocks, derived from the same source the product asserts against), Policies as a hub, one merged Concepts glossary, a files-and-paths reference, and CLI pages for config, harness, backfill, flush and uninstall. Cloud pages are value- and contract-first: internal architecture claims are out, what a user needs to operate the product stays. Per-CLI hook-schema internals move off the Configuration page and into the support matrix. Two tests hardcoded the old docs directory as a fixture path and are updated to the new location. The 14 language trees are moved and relinked; pages with no counterpart are seeded from English so every nav entry resolves, and the scheduled translation job replaces them on its next run (its cache is keyed on English source hashes, all of which changed). Adds one screenshot of the local session viewer, captured through headless Chrome against synthetic fixture data rather than a real machine's sessions. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_014ZHcJRck9qdLpGEZeL51js --- CHANGELOG.md | 9 + README.md | 8 +- .../translate-docs/mdx-translator.test.ts | 4 +- __tests__/scripts/validate-mdx.test.ts | 10 +- docs/agent-support.mdx | 204 ++ docs/agenteye/cli-and-agents.mdx | 80 - docs/agenteye/cli-skill.mdx | 159 -- docs/agenteye/codex-capture.mdx | 55 - docs/agenteye/concepts.mdx | 87 - docs/agenteye/evaluator-skill.mdx | 167 -- docs/agenteye/hermes-capture.mdx | 53 - docs/agenteye/observability.mdx | 23 - docs/agenteye/openclaw-capture.mdx | 49 - docs/agenteye/overview.mdx | 108 - docs/agenteye/python-sdk-skill.mdx | 131 - docs/agenteye/security.mdx | 68 - docs/ar/agent-support.mdx | 204 ++ docs/ar/agenteye/cli-and-agents.mdx | 81 - docs/ar/agenteye/cli-skill.mdx | 160 -- docs/ar/agenteye/codex-capture.mdx | 56 - docs/ar/agenteye/concepts.mdx | 88 - docs/ar/agenteye/evaluator-skill.mdx | 168 -- docs/ar/agenteye/hermes-capture.mdx | 54 - docs/ar/agenteye/observability.mdx | 24 - docs/ar/agenteye/openclaw-capture.mdx | 50 - docs/ar/agenteye/python-sdk-skill.mdx | 129 - docs/ar/agenteye/security.mdx | 68 - docs/ar/{cli => }/audit.mdx | 0 docs/ar/cli/backfill.mdx | 75 + docs/ar/cli/config.mdx | 145 ++ docs/ar/cli/flush.mdx | 64 + docs/ar/cli/harness.mdx | 126 + docs/ar/cli/migrate.mdx | 117 + docs/ar/cli/uninstall.mdx | 95 + docs/ar/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 14 +- docs/ar/cloud/agent-skills.mdx | 219 ++ docs/ar/{agenteye => cloud}/alerts.mdx | 20 +- docs/ar/{agenteye => cloud}/assistant.mdx | 16 +- docs/ar/{agenteye => cloud}/audits.mdx | 14 +- docs/ar/cloud/capture.mdx | 177 ++ docs/ar/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/ar/{agenteye => cloud}/cli.mdx | 18 +- docs/ar/cloud/connect.mdx | 289 +++ docs/ar/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/ar/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/ar/{agenteye => cloud}/event-stream.mdx | 14 +- docs/ar/cloud/fleet.mdx | 120 + docs/ar/{agenteye => cloud}/incidents.mdx | 10 +- docs/ar/cloud/managed-policies.mdx | 182 ++ docs/ar/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/ar/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/ar/cloud/security.mdx | 68 + docs/ar/{agenteye => cloud}/sessions.mdx | 14 +- docs/ar/concepts.mdx | 196 ++ docs/ar/daemon.mdx | 267 +++ docs/ar/dashboard.mdx | 2 +- .../ar/{architecture.mdx => how-it-works.mdx} | 0 docs/ar/introduction.mdx | 2 +- docs/ar/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/ar/reference/files.mdx | 117 + docs/architecture.mdx | 332 --- docs/{cli => }/audit.mdx | 2 +- docs/built-in-policies.mdx | 16 +- docs/cli/backfill.mdx | 75 + docs/cli/config.mdx | 145 ++ docs/cli/dashboard.mdx | 2 +- docs/cli/flush.mdx | 64 + docs/cli/harness.mdx | 126 + docs/cli/hook.mdx | 2 +- docs/cli/install-policies.mdx | 2 +- docs/cli/list-policies.mdx | 2 +- docs/cli/migrate.mdx | 2 +- docs/cli/remove-policies.mdx | 2 +- docs/cli/uninstall.mdx | 95 + docs/cli/update.mdx | 2 +- docs/cli/version.mdx | 2 +- .../api-keys.mdx => cloud/access.mdx} | 50 +- docs/cloud/agent-skills.mdx | 219 ++ docs/{agenteye => cloud}/alerts.mdx | 20 +- docs/{agenteye => cloud}/assistant.mdx | 18 +- docs/{agenteye => cloud}/audits.mdx | 16 +- docs/cloud/capture.mdx | 177 ++ docs/{ru/agenteye => cloud}/cli-recipes.mdx | 14 +- docs/{agenteye => cloud}/cli.mdx | 26 +- docs/cloud/connect.mdx | 289 +++ docs/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 18 +- docs/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 71 +- docs/{agenteye => cloud}/event-stream.mdx | 16 +- docs/cloud/fleet.mdx | 120 + docs/{agenteye => cloud}/images/alert-new.png | Bin docs/{agenteye => cloud}/images/alerts.png | Bin docs/{agenteye => cloud}/images/api-keys.png | Bin docs/{agenteye => cloud}/images/assistant.png | Bin docs/{agenteye => cloud}/images/audits.png | Bin .../images/dashboard-fleet.png | Bin .../images/dashboard-quality.png | Bin docs/{agenteye => cloud}/images/errors.png | Bin .../images/events-stream.png | Bin docs/{agenteye => cloud}/images/hooks.png | Bin .../images/incident-detail.png | Bin docs/{agenteye => cloud}/images/incidents.png | Bin docs/{agenteye => cloud}/images/login.png | Bin docs/{agenteye => cloud}/images/models.png | Bin docs/{agenteye => cloud}/images/queries.png | Bin docs/{agenteye => cloud}/images/query-lab.png | Bin .../images/session-detail.png | Bin .../images/sessions-list.png | Bin docs/{agenteye => cloud}/images/settings.png | Bin docs/{agenteye => cloud}/images/tools.png | Bin docs/{agenteye => cloud}/images/users.png | Bin .../images/video-audit.jpg | Bin .../images/video-tracing.jpg | Bin docs/{agenteye => cloud}/incidents.mdx | 12 +- docs/cloud/managed-policies.mdx | 182 ++ docs/cloud/overview.mdx | 183 ++ .../telemetry.mdx => cloud/performance.mdx} | 18 +- docs/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 31 +- docs/cloud/security.mdx | 187 ++ docs/{agenteye => cloud}/sessions.mdx | 18 +- docs/concepts.mdx | 196 ++ docs/configuration.mdx | 260 +- docs/custom-policies.mdx | 17 +- docs/daemon.mdx | 267 +++ docs/dashboard.mdx | 203 +- docs/de/agent-support.mdx | 204 ++ docs/de/agenteye/cli-and-agents.mdx | 80 - docs/de/agenteye/cli-skill.mdx | 159 -- docs/de/agenteye/codex-capture.mdx | 55 - docs/de/agenteye/concepts.mdx | 87 - docs/de/agenteye/evaluator-skill.mdx | 167 -- docs/de/agenteye/hermes-capture.mdx | 53 - docs/de/agenteye/observability.mdx | 23 - docs/de/agenteye/openclaw-capture.mdx | 49 - docs/de/agenteye/python-sdk-skill.mdx | 131 - docs/de/agenteye/security.mdx | 68 - docs/de/{cli => }/audit.mdx | 0 docs/de/cli/backfill.mdx | 75 + docs/de/cli/config.mdx | 145 ++ docs/de/cli/flush.mdx | 64 + docs/de/cli/harness.mdx | 126 + docs/de/cli/migrate.mdx | 117 + docs/de/cli/uninstall.mdx | 95 + docs/de/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 14 +- docs/de/cloud/agent-skills.mdx | 219 ++ docs/de/{agenteye => cloud}/alerts.mdx | 20 +- docs/de/{agenteye => cloud}/assistant.mdx | 16 +- docs/de/{agenteye => cloud}/audits.mdx | 14 +- docs/de/cloud/capture.mdx | 177 ++ docs/de/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/de/{agenteye => cloud}/cli.mdx | 18 +- docs/de/cloud/connect.mdx | 289 +++ docs/de/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/de/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/de/{agenteye => cloud}/event-stream.mdx | 16 +- docs/de/cloud/fleet.mdx | 120 + docs/de/{agenteye => cloud}/incidents.mdx | 12 +- docs/de/cloud/managed-policies.mdx | 182 ++ docs/de/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/de/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/de/cloud/security.mdx | 68 + docs/de/{agenteye => cloud}/sessions.mdx | 16 +- docs/de/concepts.mdx | 196 ++ docs/de/daemon.mdx | 267 +++ docs/de/dashboard.mdx | 2 +- .../de/{architecture.mdx => how-it-works.mdx} | 0 docs/de/introduction.mdx | 2 +- docs/de/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/de/reference/files.mdx | 117 + docs/docs.json | 2107 +++++++++-------- docs/es/agent-support.mdx | 204 ++ docs/es/agenteye/cli-and-agents.mdx | 80 - docs/es/agenteye/cli-skill.mdx | 159 -- docs/es/agenteye/codex-capture.mdx | 55 - docs/es/agenteye/concepts.mdx | 87 - docs/es/agenteye/evaluator-skill.mdx | 167 -- docs/es/agenteye/hermes-capture.mdx | 53 - docs/es/agenteye/observability.mdx | 23 - docs/es/agenteye/openclaw-capture.mdx | 49 - docs/es/agenteye/python-sdk-skill.mdx | 135 -- docs/es/agenteye/security.mdx | 68 - docs/es/{cli => }/audit.mdx | 0 docs/es/cli/backfill.mdx | 75 + docs/es/cli/config.mdx | 145 ++ docs/es/cli/flush.mdx | 64 + docs/es/cli/harness.mdx | 126 + docs/es/cli/migrate.mdx | 117 + docs/es/cli/uninstall.mdx | 95 + docs/es/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 10 +- docs/es/cloud/agent-skills.mdx | 219 ++ docs/es/{agenteye => cloud}/alerts.mdx | 20 +- docs/es/{agenteye => cloud}/assistant.mdx | 16 +- docs/es/{agenteye => cloud}/audits.mdx | 14 +- docs/es/cloud/capture.mdx | 177 ++ docs/es/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/es/{agenteye => cloud}/cli.mdx | 10 +- docs/es/cloud/connect.mdx | 289 +++ docs/es/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/es/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/es/{agenteye => cloud}/event-stream.mdx | 12 +- docs/es/cloud/fleet.mdx | 120 + docs/es/{agenteye => cloud}/incidents.mdx | 12 +- docs/es/cloud/managed-policies.mdx | 182 ++ docs/es/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/es/{agenteye => cloud}/queries.mdx | 14 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/es/cloud/security.mdx | 68 + docs/es/{agenteye => cloud}/sessions.mdx | 14 +- docs/es/concepts.mdx | 196 ++ docs/es/daemon.mdx | 267 +++ docs/es/dashboard.mdx | 2 +- .../es/{architecture.mdx => how-it-works.mdx} | 0 docs/es/introduction.mdx | 2 +- docs/es/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/es/reference/files.mdx | 117 + docs/for-agents.mdx | 15 +- docs/fr/agent-support.mdx | 204 ++ docs/fr/agenteye/cli-and-agents.mdx | 79 - docs/fr/agenteye/cli-skill.mdx | 159 -- docs/fr/agenteye/codex-capture.mdx | 55 - docs/fr/agenteye/concepts.mdx | 87 - docs/fr/agenteye/evaluator-skill.mdx | 167 -- docs/fr/agenteye/hermes-capture.mdx | 53 - docs/fr/agenteye/observability.mdx | 23 - docs/fr/agenteye/openclaw-capture.mdx | 49 - docs/fr/agenteye/python-sdk-skill.mdx | 135 -- docs/fr/agenteye/security.mdx | 68 - docs/fr/{cli => }/audit.mdx | 0 docs/fr/cli/backfill.mdx | 75 + docs/fr/cli/config.mdx | 145 ++ docs/fr/cli/flush.mdx | 64 + docs/fr/cli/harness.mdx | 126 + docs/fr/cli/migrate.mdx | 117 + docs/fr/cli/uninstall.mdx | 95 + docs/fr/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 10 +- docs/fr/cloud/agent-skills.mdx | 219 ++ docs/fr/{agenteye => cloud}/alerts.mdx | 16 +- docs/fr/{agenteye => cloud}/assistant.mdx | 16 +- docs/fr/{agenteye => cloud}/audits.mdx | 14 +- docs/fr/cloud/capture.mdx | 177 ++ docs/fr/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/fr/{agenteye => cloud}/cli.mdx | 10 +- docs/fr/cloud/connect.mdx | 289 +++ docs/fr/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/fr/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/fr/{agenteye => cloud}/event-stream.mdx | 12 +- docs/fr/cloud/fleet.mdx | 120 + docs/fr/{agenteye => cloud}/incidents.mdx | 12 +- docs/fr/cloud/managed-policies.mdx | 182 ++ docs/fr/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/fr/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/fr/cloud/security.mdx | 68 + docs/fr/{agenteye => cloud}/sessions.mdx | 14 +- docs/fr/concepts.mdx | 196 ++ docs/fr/daemon.mdx | 267 +++ docs/fr/dashboard.mdx | 2 +- .../fr/{architecture.mdx => how-it-works.mdx} | 0 docs/fr/introduction.mdx | 2 +- docs/fr/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/fr/reference/files.mdx | 117 + docs/getting-started.mdx | 201 -- docs/he/agent-support.mdx | 204 ++ docs/he/agenteye/cli-and-agents.mdx | 80 - docs/he/agenteye/cli-skill.mdx | 160 -- docs/he/agenteye/codex-capture.mdx | 56 - docs/he/agenteye/concepts.mdx | 87 - docs/he/agenteye/evaluator-skill.mdx | 168 -- docs/he/agenteye/hermes-capture.mdx | 54 - docs/he/agenteye/observability.mdx | 23 - docs/he/agenteye/openclaw-capture.mdx | 50 - docs/he/agenteye/python-sdk-skill.mdx | 132 -- docs/he/agenteye/security.mdx | 67 - docs/he/{cli => }/audit.mdx | 0 docs/he/cli/backfill.mdx | 75 + docs/he/cli/config.mdx | 145 ++ docs/he/cli/flush.mdx | 64 + docs/he/cli/harness.mdx | 126 + docs/he/cli/migrate.mdx | 117 + docs/he/cli/uninstall.mdx | 95 + docs/he/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 14 +- docs/he/cloud/agent-skills.mdx | 219 ++ docs/he/{agenteye => cloud}/alerts.mdx | 20 +- docs/he/{agenteye => cloud}/assistant.mdx | 16 +- docs/he/{agenteye => cloud}/audits.mdx | 14 +- docs/he/cloud/capture.mdx | 177 ++ docs/he/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/he/{agenteye => cloud}/cli.mdx | 18 +- docs/he/cloud/connect.mdx | 289 +++ docs/he/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/he/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/he/{agenteye => cloud}/event-stream.mdx | 14 +- docs/he/cloud/fleet.mdx | 120 + docs/he/{agenteye => cloud}/incidents.mdx | 12 +- docs/he/cloud/managed-policies.mdx | 182 ++ docs/he/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/he/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/he/cloud/security.mdx | 67 + docs/he/{agenteye => cloud}/sessions.mdx | 16 +- docs/he/concepts.mdx | 196 ++ docs/he/daemon.mdx | 267 +++ docs/he/dashboard.mdx | 2 +- .../he/{architecture.mdx => how-it-works.mdx} | 0 docs/he/introduction.mdx | 2 +- docs/he/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/he/reference/files.mdx | 117 + docs/hi/agent-support.mdx | 204 ++ docs/hi/agenteye/cli-and-agents.mdx | 80 - docs/hi/agenteye/cli-skill.mdx | 159 -- docs/hi/agenteye/codex-capture.mdx | 56 - docs/hi/agenteye/concepts.mdx | 87 - docs/hi/agenteye/evaluator-skill.mdx | 171 -- docs/hi/agenteye/hermes-capture.mdx | 53 - docs/hi/agenteye/observability.mdx | 23 - docs/hi/agenteye/openclaw-capture.mdx | 50 - docs/hi/agenteye/python-sdk-skill.mdx | 133 -- docs/hi/agenteye/security.mdx | 68 - docs/hi/{cli => }/audit.mdx | 0 docs/hi/cli/backfill.mdx | 75 + docs/hi/cli/config.mdx | 145 ++ docs/hi/cli/flush.mdx | 64 + docs/hi/cli/harness.mdx | 126 + docs/hi/cli/migrate.mdx | 117 + docs/hi/cli/uninstall.mdx | 95 + docs/hi/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 14 +- docs/hi/cloud/agent-skills.mdx | 219 ++ docs/hi/{agenteye => cloud}/alerts.mdx | 20 +- docs/hi/{agenteye => cloud}/assistant.mdx | 16 +- docs/hi/{agenteye => cloud}/audits.mdx | 14 +- docs/hi/cloud/capture.mdx | 177 ++ docs/hi/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/hi/{agenteye => cloud}/cli.mdx | 18 +- docs/hi/cloud/connect.mdx | 289 +++ docs/hi/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/hi/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/hi/{agenteye => cloud}/event-stream.mdx | 14 +- docs/hi/cloud/fleet.mdx | 120 + docs/hi/{agenteye => cloud}/incidents.mdx | 12 +- docs/hi/cloud/managed-policies.mdx | 182 ++ docs/hi/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/hi/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/hi/cloud/security.mdx | 68 + docs/hi/{agenteye => cloud}/sessions.mdx | 16 +- docs/hi/concepts.mdx | 196 ++ docs/hi/daemon.mdx | 267 +++ docs/hi/dashboard.mdx | 2 +- .../hi/{architecture.mdx => how-it-works.mdx} | 0 docs/hi/introduction.mdx | 2 +- docs/hi/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/hi/reference/files.mdx | 117 + docs/how-it-works.mdx | 313 +++ docs/images/local-session-viewer.png | Bin 0 -> 198779 bytes docs/introduction.mdx | 163 +- docs/it/agent-support.mdx | 204 ++ docs/it/agenteye/cli-and-agents.mdx | 80 - docs/it/agenteye/cli-skill.mdx | 159 -- docs/it/agenteye/codex-capture.mdx | 56 - docs/it/agenteye/concepts.mdx | 87 - docs/it/agenteye/evaluator-skill.mdx | 170 -- docs/it/agenteye/hermes-capture.mdx | 53 - docs/it/agenteye/observability.mdx | 23 - docs/it/agenteye/openclaw-capture.mdx | 49 - docs/it/agenteye/python-sdk-skill.mdx | 132 -- docs/it/{cli => }/audit.mdx | 0 docs/it/cli/backfill.mdx | 75 + docs/it/cli/config.mdx | 145 ++ docs/it/cli/flush.mdx | 64 + docs/it/cli/harness.mdx | 126 + docs/it/cli/migrate.mdx | 117 + docs/it/cli/uninstall.mdx | 95 + docs/it/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 14 +- docs/it/cloud/agent-skills.mdx | 219 ++ docs/it/{agenteye => cloud}/alerts.mdx | 20 +- docs/it/{agenteye => cloud}/assistant.mdx | 16 +- docs/it/{agenteye => cloud}/audits.mdx | 14 +- docs/it/cloud/capture.mdx | 177 ++ docs/it/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/it/{agenteye => cloud}/cli.mdx | 14 +- docs/it/cloud/connect.mdx | 289 +++ docs/it/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/it/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/it/{agenteye => cloud}/event-stream.mdx | 14 +- docs/it/cloud/fleet.mdx | 120 + docs/it/{agenteye => cloud}/incidents.mdx | 12 +- docs/it/cloud/managed-policies.mdx | 182 ++ docs/it/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/it/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/it/{agenteye => cloud}/security.mdx | 24 +- docs/it/{agenteye => cloud}/sessions.mdx | 16 +- docs/it/concepts.mdx | 196 ++ docs/it/daemon.mdx | 267 +++ docs/it/dashboard.mdx | 2 +- .../it/{architecture.mdx => how-it-works.mdx} | 0 docs/it/introduction.mdx | 2 +- docs/it/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/it/reference/files.mdx | 117 + docs/ja/agent-support.mdx | 204 ++ docs/ja/agenteye/cli-and-agents.mdx | 80 - docs/ja/agenteye/cli-skill.mdx | 159 -- docs/ja/agenteye/codex-capture.mdx | 55 - docs/ja/agenteye/concepts.mdx | 87 - docs/ja/agenteye/evaluator-skill.mdx | 167 -- docs/ja/agenteye/hermes-capture.mdx | 53 - docs/ja/agenteye/observability.mdx | 23 - docs/ja/agenteye/openclaw-capture.mdx | 49 - docs/ja/agenteye/python-sdk-skill.mdx | 131 - docs/ja/agenteye/security.mdx | 68 - docs/ja/{cli => }/audit.mdx | 0 docs/ja/cli/backfill.mdx | 75 + docs/ja/cli/config.mdx | 145 ++ docs/ja/cli/flush.mdx | 64 + docs/ja/cli/harness.mdx | 126 + docs/ja/cli/migrate.mdx | 117 + docs/ja/cli/uninstall.mdx | 95 + docs/ja/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 14 +- docs/ja/cloud/agent-skills.mdx | 219 ++ docs/ja/{agenteye => cloud}/alerts.mdx | 20 +- docs/ja/{agenteye => cloud}/assistant.mdx | 14 +- docs/ja/{agenteye => cloud}/audits.mdx | 14 +- docs/ja/cloud/capture.mdx | 177 ++ docs/ja/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/ja/{agenteye => cloud}/cli.mdx | 18 +- docs/ja/cloud/connect.mdx | 289 +++ docs/ja/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/ja/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/ja/{agenteye => cloud}/event-stream.mdx | 14 +- docs/ja/cloud/fleet.mdx | 120 + docs/ja/{agenteye => cloud}/incidents.mdx | 12 +- docs/ja/cloud/managed-policies.mdx | 182 ++ docs/ja/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/ja/{agenteye => cloud}/queries.mdx | 14 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/ja/cloud/security.mdx | 68 + docs/ja/{agenteye => cloud}/sessions.mdx | 16 +- docs/ja/concepts.mdx | 196 ++ docs/ja/daemon.mdx | 267 +++ docs/ja/dashboard.mdx | 2 +- .../ja/{architecture.mdx => how-it-works.mdx} | 0 docs/ja/introduction.mdx | 2 +- docs/ja/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/ja/reference/files.mdx | 117 + docs/ko/agent-support.mdx | 204 ++ docs/ko/agenteye/cli-and-agents.mdx | 80 - docs/ko/agenteye/cli-skill.mdx | 159 -- docs/ko/agenteye/codex-capture.mdx | 55 - docs/ko/agenteye/concepts.mdx | 87 - docs/ko/agenteye/evaluator-skill.mdx | 167 -- docs/ko/agenteye/hermes-capture.mdx | 53 - docs/ko/agenteye/observability.mdx | 23 - docs/ko/agenteye/openclaw-capture.mdx | 49 - docs/ko/agenteye/python-sdk-skill.mdx | 130 - docs/ko/agenteye/security.mdx | 68 - docs/ko/{cli => }/audit.mdx | 0 docs/ko/cli/backfill.mdx | 75 + docs/ko/cli/config.mdx | 145 ++ docs/ko/cli/flush.mdx | 64 + docs/ko/cli/harness.mdx | 126 + docs/ko/cli/migrate.mdx | 117 + docs/ko/cli/uninstall.mdx | 95 + docs/ko/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 14 +- docs/ko/cloud/agent-skills.mdx | 219 ++ docs/ko/{agenteye => cloud}/alerts.mdx | 20 +- docs/ko/{agenteye => cloud}/assistant.mdx | 16 +- docs/ko/{agenteye => cloud}/audits.mdx | 14 +- docs/ko/cloud/capture.mdx | 177 ++ docs/ko/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/ko/{agenteye => cloud}/cli.mdx | 18 +- docs/ko/cloud/connect.mdx | 289 +++ docs/ko/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 14 +- docs/ko/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 30 +- docs/ko/{agenteye => cloud}/event-stream.mdx | 14 +- docs/ko/cloud/fleet.mdx | 120 + docs/ko/{agenteye => cloud}/incidents.mdx | 12 +- docs/ko/cloud/managed-policies.mdx | 182 ++ docs/ko/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/ko/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/ko/cloud/security.mdx | 68 + docs/ko/{agenteye => cloud}/sessions.mdx | 16 +- docs/ko/concepts.mdx | 196 ++ docs/ko/daemon.mdx | 267 +++ docs/ko/dashboard.mdx | 2 +- .../ko/{architecture.mdx => how-it-works.mdx} | 0 docs/ko/introduction.mdx | 2 +- docs/ko/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/ko/reference/files.mdx | 117 + docs/package-aliases.mdx | 2 +- docs/policies.mdx | 267 +++ docs/pt-br/agent-support.mdx | 204 ++ docs/pt-br/agenteye/cli-and-agents.mdx | 80 - docs/pt-br/agenteye/cli-skill.mdx | 159 -- docs/pt-br/agenteye/codex-capture.mdx | 55 - docs/pt-br/agenteye/concepts.mdx | 87 - docs/pt-br/agenteye/evaluator-skill.mdx | 167 -- docs/pt-br/agenteye/hermes-capture.mdx | 53 - docs/pt-br/agenteye/observability.mdx | 23 - docs/pt-br/agenteye/openclaw-capture.mdx | 49 - docs/pt-br/agenteye/python-sdk-skill.mdx | 136 -- docs/pt-br/agenteye/security.mdx | 68 - docs/pt-br/{cli => }/audit.mdx | 0 docs/pt-br/cli/backfill.mdx | 75 + docs/pt-br/cli/config.mdx | 145 ++ docs/pt-br/cli/flush.mdx | 64 + docs/pt-br/cli/harness.mdx | 126 + docs/pt-br/cli/migrate.mdx | 117 + docs/pt-br/cli/uninstall.mdx | 95 + docs/pt-br/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 10 +- docs/pt-br/cloud/agent-skills.mdx | 219 ++ docs/pt-br/{agenteye => cloud}/alerts.mdx | 16 +- docs/pt-br/{agenteye => cloud}/assistant.mdx | 16 +- docs/pt-br/{agenteye => cloud}/audits.mdx | 14 +- docs/pt-br/cloud/capture.mdx | 177 ++ .../pt-br/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/pt-br/{agenteye => cloud}/cli.mdx | 12 +- docs/pt-br/cloud/connect.mdx | 289 +++ docs/pt-br/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- .../pt-br/{agenteye => cloud}/evaluations.mdx | 18 +- .../evaluators.mdx} | 44 +- .../{agenteye => cloud}/event-stream.mdx | 12 +- docs/pt-br/cloud/fleet.mdx | 120 + docs/pt-br/{agenteye => cloud}/incidents.mdx | 12 +- docs/pt-br/cloud/managed-policies.mdx | 182 ++ docs/pt-br/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/pt-br/{agenteye => cloud}/queries.mdx | 14 +- .../python-sdk.mdx => cloud/sdk.mdx} | 10 +- docs/pt-br/cloud/security.mdx | 68 + docs/pt-br/{agenteye => cloud}/sessions.mdx | 14 +- docs/pt-br/concepts.mdx | 196 ++ docs/pt-br/daemon.mdx | 267 +++ docs/pt-br/dashboard.mdx | 2 +- .../{architecture.mdx => how-it-works.mdx} | 0 docs/pt-br/introduction.mdx | 2 +- docs/pt-br/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/pt-br/reference/files.mdx | 117 + docs/quickstart.mdx | 261 ++ docs/reference/files.mdx | 117 + docs/ru/agent-support.mdx | 204 ++ docs/ru/agenteye/cli-and-agents.mdx | 80 - docs/ru/agenteye/cli-skill.mdx | 159 -- docs/ru/agenteye/codex-capture.mdx | 55 - docs/ru/agenteye/concepts.mdx | 87 - docs/ru/agenteye/evaluator-skill.mdx | 171 -- docs/ru/agenteye/hermes-capture.mdx | 53 - docs/ru/agenteye/observability.mdx | 23 - docs/ru/agenteye/openclaw-capture.mdx | 49 - docs/ru/agenteye/python-sdk-skill.mdx | 131 - docs/ru/{cli => }/audit.mdx | 0 docs/ru/cli/backfill.mdx | 75 + docs/ru/cli/config.mdx | 145 ++ docs/ru/cli/flush.mdx | 64 + docs/ru/cli/harness.mdx | 126 + docs/ru/cli/migrate.mdx | 117 + docs/ru/cli/uninstall.mdx | 95 + docs/ru/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 14 +- docs/ru/cloud/agent-skills.mdx | 219 ++ docs/ru/{agenteye => cloud}/alerts.mdx | 20 +- docs/ru/{agenteye => cloud}/assistant.mdx | 16 +- docs/ru/{agenteye => cloud}/audits.mdx | 14 +- docs/ru/cloud/capture.mdx | 177 ++ docs/{agenteye => ru/cloud}/cli-recipes.mdx | 12 +- docs/ru/{agenteye => cloud}/cli.mdx | 18 +- docs/ru/cloud/connect.mdx | 289 +++ docs/ru/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/ru/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/ru/{agenteye => cloud}/event-stream.mdx | 14 +- docs/ru/cloud/fleet.mdx | 120 + docs/ru/{agenteye => cloud}/incidents.mdx | 12 +- docs/ru/cloud/managed-policies.mdx | 182 ++ docs/ru/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/ru/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/ru/{agenteye => cloud}/security.mdx | 24 +- docs/ru/{agenteye => cloud}/sessions.mdx | 16 +- docs/ru/concepts.mdx | 196 ++ docs/ru/daemon.mdx | 267 +++ docs/ru/dashboard.mdx | 2 +- .../ru/{architecture.mdx => how-it-works.mdx} | 0 docs/ru/introduction.mdx | 2 +- docs/ru/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/ru/reference/files.mdx | 117 + docs/testing.mdx | 2 +- docs/tr/agent-support.mdx | 204 ++ docs/tr/agenteye/cli-and-agents.mdx | 80 - docs/tr/agenteye/cli-skill.mdx | 161 -- docs/tr/agenteye/codex-capture.mdx | 56 - docs/tr/agenteye/concepts.mdx | 87 - docs/tr/agenteye/evaluator-skill.mdx | 167 -- docs/tr/agenteye/hermes-capture.mdx | 53 - docs/tr/agenteye/observability.mdx | 23 - docs/tr/agenteye/openclaw-capture.mdx | 50 - docs/tr/agenteye/overview.mdx | 107 - docs/tr/agenteye/python-sdk-skill.mdx | 130 - docs/tr/agenteye/security.mdx | 69 - docs/tr/{cli => }/audit.mdx | 0 docs/tr/cli/backfill.mdx | 75 + docs/tr/cli/config.mdx | 145 ++ docs/tr/cli/flush.mdx | 64 + docs/tr/cli/harness.mdx | 126 + docs/tr/cli/migrate.mdx | 117 + docs/tr/cli/uninstall.mdx | 95 + docs/tr/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 10 +- docs/tr/cloud/agent-skills.mdx | 219 ++ docs/tr/{agenteye => cloud}/alerts.mdx | 20 +- docs/tr/{agenteye => cloud}/assistant.mdx | 16 +- docs/tr/{agenteye => cloud}/audits.mdx | 14 +- docs/tr/cloud/capture.mdx | 177 ++ docs/tr/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/tr/{agenteye => cloud}/cli.mdx | 18 +- docs/tr/cloud/connect.mdx | 289 +++ docs/tr/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/tr/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/tr/{agenteye => cloud}/event-stream.mdx | 14 +- docs/tr/cloud/fleet.mdx | 120 + docs/tr/{agenteye => cloud}/incidents.mdx | 12 +- docs/tr/cloud/managed-policies.mdx | 182 ++ docs/tr/cloud/overview.mdx | 107 + .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/tr/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/tr/cloud/security.mdx | 69 + docs/tr/{agenteye => cloud}/sessions.mdx | 16 +- docs/tr/concepts.mdx | 196 ++ docs/tr/daemon.mdx | 267 +++ docs/tr/dashboard.mdx | 2 +- .../tr/{architecture.mdx => how-it-works.mdx} | 0 docs/tr/introduction.mdx | 2 +- docs/tr/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/tr/reference/files.mdx | 117 + docs/vi/agent-support.mdx | 204 ++ docs/vi/agenteye/cli-and-agents.mdx | 80 - docs/vi/agenteye/cli-skill.mdx | 159 -- docs/vi/agenteye/codex-capture.mdx | 55 - docs/vi/agenteye/concepts.mdx | 88 - docs/vi/agenteye/evaluator-skill.mdx | 171 -- docs/vi/agenteye/hermes-capture.mdx | 53 - docs/vi/agenteye/observability.mdx | 23 - docs/vi/agenteye/openclaw-capture.mdx | 49 - docs/vi/agenteye/python-sdk-skill.mdx | 131 - docs/vi/agenteye/security.mdx | 68 - docs/vi/{cli => }/audit.mdx | 0 docs/vi/cli/backfill.mdx | 75 + docs/vi/cli/config.mdx | 145 ++ docs/vi/cli/flush.mdx | 64 + docs/vi/cli/harness.mdx | 126 + docs/vi/cli/migrate.mdx | 117 + docs/vi/cli/uninstall.mdx | 95 + docs/vi/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 14 +- docs/vi/cloud/agent-skills.mdx | 219 ++ docs/vi/{agenteye => cloud}/alerts.mdx | 20 +- docs/vi/{agenteye => cloud}/assistant.mdx | 16 +- docs/vi/{agenteye => cloud}/audits.mdx | 14 +- docs/vi/cloud/capture.mdx | 177 ++ docs/vi/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/vi/{agenteye => cloud}/cli.mdx | 18 +- docs/vi/cloud/connect.mdx | 289 +++ docs/vi/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/vi/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/vi/{agenteye => cloud}/event-stream.mdx | 14 +- docs/vi/cloud/fleet.mdx | 120 + docs/vi/{agenteye => cloud}/incidents.mdx | 12 +- docs/vi/cloud/managed-policies.mdx | 182 ++ docs/vi/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/vi/{agenteye => cloud}/queries.mdx | 16 +- .../python-sdk.mdx => cloud/sdk.mdx} | 18 +- docs/vi/cloud/security.mdx | 68 + docs/vi/{agenteye => cloud}/sessions.mdx | 16 +- docs/vi/concepts.mdx | 196 ++ docs/vi/daemon.mdx | 267 +++ docs/vi/dashboard.mdx | 2 +- .../vi/{architecture.mdx => how-it-works.mdx} | 0 docs/vi/introduction.mdx | 2 +- docs/vi/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/vi/reference/files.mdx | 117 + docs/zh/agent-support.mdx | 204 ++ docs/zh/agenteye/cli-and-agents.mdx | 79 - docs/zh/agenteye/cli-skill.mdx | 159 -- docs/zh/agenteye/codex-capture.mdx | 55 - docs/zh/agenteye/concepts.mdx | 87 - docs/zh/agenteye/evaluator-skill.mdx | 167 -- docs/zh/agenteye/hermes-capture.mdx | 53 - docs/zh/agenteye/observability.mdx | 23 - docs/zh/agenteye/openclaw-capture.mdx | 49 - docs/zh/agenteye/python-sdk-skill.mdx | 131 - docs/zh/agenteye/security.mdx | 68 - docs/zh/{cli => }/audit.mdx | 0 docs/zh/cli/backfill.mdx | 75 + docs/zh/cli/config.mdx | 145 ++ docs/zh/cli/flush.mdx | 64 + docs/zh/cli/harness.mdx | 126 + docs/zh/cli/migrate.mdx | 117 + docs/zh/cli/uninstall.mdx | 95 + docs/zh/cli/update.mdx | 94 + .../api-keys.mdx => cloud/access.mdx} | 14 +- docs/zh/cloud/agent-skills.mdx | 219 ++ docs/zh/{agenteye => cloud}/alerts.mdx | 20 +- docs/zh/{agenteye => cloud}/assistant.mdx | 16 +- docs/zh/{agenteye => cloud}/audits.mdx | 14 +- docs/zh/cloud/capture.mdx | 177 ++ docs/zh/{agenteye => cloud}/cli-recipes.mdx | 12 +- docs/zh/{agenteye => cloud}/cli.mdx | 18 +- docs/zh/cloud/connect.mdx | 289 +++ docs/zh/{agenteye => cloud}/dashboards.mdx | 14 +- .../error-tracking.mdx => cloud/errors.mdx} | 16 +- docs/zh/{agenteye => cloud}/evaluations.mdx | 24 +- .../evaluators.mdx} | 44 +- docs/zh/{agenteye => cloud}/event-stream.mdx | 12 +- docs/zh/cloud/fleet.mdx | 120 + docs/zh/{agenteye => cloud}/incidents.mdx | 12 +- docs/zh/cloud/managed-policies.mdx | 182 ++ docs/zh/{agenteye => cloud}/overview.mdx | 56 +- .../telemetry.mdx => cloud/performance.mdx} | 16 +- docs/zh/{agenteye => cloud}/queries.mdx | 14 +- .../python-sdk.mdx => cloud/sdk.mdx} | 10 +- docs/zh/cloud/security.mdx | 68 + docs/zh/{agenteye => cloud}/sessions.mdx | 14 +- docs/zh/concepts.mdx | 196 ++ docs/zh/daemon.mdx | 267 +++ docs/zh/dashboard.mdx | 2 +- .../zh/{architecture.mdx => how-it-works.mdx} | 0 docs/zh/introduction.mdx | 2 +- docs/zh/policies.mdx | 267 +++ .../{getting-started.mdx => quickstart.mdx} | 0 docs/zh/reference/files.mdx | 117 + scripts/translate-docs/mintlify-nav.ts | 5 + 794 files changed, 47094 insertions(+), 17524 deletions(-) create mode 100644 docs/agent-support.mdx delete mode 100644 docs/agenteye/cli-and-agents.mdx delete mode 100644 docs/agenteye/cli-skill.mdx delete mode 100644 docs/agenteye/codex-capture.mdx delete mode 100644 docs/agenteye/concepts.mdx delete mode 100644 docs/agenteye/evaluator-skill.mdx delete mode 100644 docs/agenteye/hermes-capture.mdx delete mode 100644 docs/agenteye/observability.mdx delete mode 100644 docs/agenteye/openclaw-capture.mdx delete mode 100644 docs/agenteye/overview.mdx delete mode 100644 docs/agenteye/python-sdk-skill.mdx delete mode 100644 docs/agenteye/security.mdx create mode 100644 docs/ar/agent-support.mdx delete mode 100644 docs/ar/agenteye/cli-and-agents.mdx delete mode 100644 docs/ar/agenteye/cli-skill.mdx delete mode 100644 docs/ar/agenteye/codex-capture.mdx delete mode 100644 docs/ar/agenteye/concepts.mdx delete mode 100644 docs/ar/agenteye/evaluator-skill.mdx delete mode 100644 docs/ar/agenteye/hermes-capture.mdx delete mode 100644 docs/ar/agenteye/observability.mdx delete mode 100644 docs/ar/agenteye/openclaw-capture.mdx delete mode 100644 docs/ar/agenteye/python-sdk-skill.mdx delete mode 100644 docs/ar/agenteye/security.mdx rename docs/ar/{cli => }/audit.mdx (100%) create mode 100644 docs/ar/cli/backfill.mdx create mode 100644 docs/ar/cli/config.mdx create mode 100644 docs/ar/cli/flush.mdx create mode 100644 docs/ar/cli/harness.mdx create mode 100644 docs/ar/cli/migrate.mdx create mode 100644 docs/ar/cli/uninstall.mdx create mode 100644 docs/ar/cli/update.mdx rename docs/ar/{agenteye/api-keys.mdx => cloud/access.mdx} (95%) create mode 100644 docs/ar/cloud/agent-skills.mdx rename docs/ar/{agenteye => cloud}/alerts.mdx (77%) rename docs/ar/{agenteye => cloud}/assistant.mdx (83%) rename docs/ar/{agenteye => cloud}/audits.mdx (81%) create mode 100644 docs/ar/cloud/capture.mdx rename docs/ar/{agenteye => cloud}/cli-recipes.mdx (90%) rename docs/ar/{agenteye => cloud}/cli.mdx (90%) create mode 100644 docs/ar/cloud/connect.mdx rename docs/ar/{agenteye => cloud}/dashboards.mdx (72%) rename docs/ar/{agenteye/error-tracking.mdx => cloud/errors.mdx} (69%) rename docs/ar/{agenteye => cloud}/evaluations.mdx (69%) rename docs/ar/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (84%) rename docs/ar/{agenteye => cloud}/event-stream.mdx (79%) create mode 100644 docs/ar/cloud/fleet.mdx rename docs/ar/{agenteye => cloud}/incidents.mdx (91%) create mode 100644 docs/ar/cloud/managed-policies.mdx rename docs/ar/{agenteye => cloud}/overview.mdx (51%) rename docs/ar/{agenteye/telemetry.mdx => cloud/performance.mdx} (83%) rename docs/ar/{agenteye => cloud}/queries.mdx (78%) rename docs/ar/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (92%) create mode 100644 docs/ar/cloud/security.mdx rename docs/ar/{agenteye => cloud}/sessions.mdx (80%) create mode 100644 docs/ar/concepts.mdx create mode 100644 docs/ar/daemon.mdx rename docs/ar/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/ar/policies.mdx rename docs/ar/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/ar/reference/files.mdx delete mode 100644 docs/architecture.mdx rename docs/{cli => }/audit.mdx (99%) create mode 100644 docs/cli/backfill.mdx create mode 100644 docs/cli/config.mdx create mode 100644 docs/cli/flush.mdx create mode 100644 docs/cli/harness.mdx create mode 100644 docs/cli/uninstall.mdx rename docs/{agenteye/api-keys.mdx => cloud/access.mdx} (89%) create mode 100644 docs/cloud/agent-skills.mdx rename docs/{agenteye => cloud}/alerts.mdx (74%) rename docs/{agenteye => cloud}/assistant.mdx (83%) rename docs/{agenteye => cloud}/audits.mdx (80%) create mode 100644 docs/cloud/capture.mdx rename docs/{ru/agenteye => cloud}/cli-recipes.mdx (90%) rename docs/{agenteye => cloud}/cli.mdx (90%) create mode 100644 docs/cloud/connect.mdx rename docs/{agenteye => cloud}/dashboards.mdx (71%) rename docs/{agenteye/error-tracking.mdx => cloud/errors.mdx} (64%) rename docs/{agenteye => cloud}/evaluations.mdx (67%) rename docs/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (85%) rename docs/{agenteye => cloud}/event-stream.mdx (77%) create mode 100644 docs/cloud/fleet.mdx rename docs/{agenteye => cloud}/images/alert-new.png (100%) rename docs/{agenteye => cloud}/images/alerts.png (100%) rename docs/{agenteye => cloud}/images/api-keys.png (100%) rename docs/{agenteye => cloud}/images/assistant.png (100%) rename docs/{agenteye => cloud}/images/audits.png (100%) rename docs/{agenteye => cloud}/images/dashboard-fleet.png (100%) rename docs/{agenteye => cloud}/images/dashboard-quality.png (100%) rename docs/{agenteye => cloud}/images/errors.png (100%) rename docs/{agenteye => cloud}/images/events-stream.png (100%) rename docs/{agenteye => cloud}/images/hooks.png (100%) rename docs/{agenteye => cloud}/images/incident-detail.png (100%) rename docs/{agenteye => cloud}/images/incidents.png (100%) rename docs/{agenteye => cloud}/images/login.png (100%) rename docs/{agenteye => cloud}/images/models.png (100%) rename docs/{agenteye => cloud}/images/queries.png (100%) rename docs/{agenteye => cloud}/images/query-lab.png (100%) rename docs/{agenteye => cloud}/images/session-detail.png (100%) rename docs/{agenteye => cloud}/images/sessions-list.png (100%) rename docs/{agenteye => cloud}/images/settings.png (100%) rename docs/{agenteye => cloud}/images/tools.png (100%) rename docs/{agenteye => cloud}/images/users.png (100%) rename docs/{agenteye => cloud}/images/video-audit.jpg (100%) rename docs/{agenteye => cloud}/images/video-tracing.jpg (100%) rename docs/{agenteye => cloud}/incidents.mdx (85%) create mode 100644 docs/cloud/managed-policies.mdx create mode 100644 docs/cloud/overview.mdx rename docs/{agenteye/telemetry.mdx => cloud/performance.mdx} (78%) rename docs/{agenteye => cloud}/queries.mdx (78%) rename docs/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (90%) create mode 100644 docs/cloud/security.mdx rename docs/{agenteye => cloud}/sessions.mdx (71%) create mode 100644 docs/concepts.mdx create mode 100644 docs/daemon.mdx create mode 100644 docs/de/agent-support.mdx delete mode 100644 docs/de/agenteye/cli-and-agents.mdx delete mode 100644 docs/de/agenteye/cli-skill.mdx delete mode 100644 docs/de/agenteye/codex-capture.mdx delete mode 100644 docs/de/agenteye/concepts.mdx delete mode 100644 docs/de/agenteye/evaluator-skill.mdx delete mode 100644 docs/de/agenteye/hermes-capture.mdx delete mode 100644 docs/de/agenteye/observability.mdx delete mode 100644 docs/de/agenteye/openclaw-capture.mdx delete mode 100644 docs/de/agenteye/python-sdk-skill.mdx delete mode 100644 docs/de/agenteye/security.mdx rename docs/de/{cli => }/audit.mdx (100%) create mode 100644 docs/de/cli/backfill.mdx create mode 100644 docs/de/cli/config.mdx create mode 100644 docs/de/cli/flush.mdx create mode 100644 docs/de/cli/harness.mdx create mode 100644 docs/de/cli/migrate.mdx create mode 100644 docs/de/cli/uninstall.mdx create mode 100644 docs/de/cli/update.mdx rename docs/de/{agenteye/api-keys.mdx => cloud/access.mdx} (95%) create mode 100644 docs/de/cloud/agent-skills.mdx rename docs/de/{agenteye => cloud}/alerts.mdx (76%) rename docs/de/{agenteye => cloud}/assistant.mdx (85%) rename docs/de/{agenteye => cloud}/audits.mdx (80%) create mode 100644 docs/de/cloud/capture.mdx rename docs/de/{agenteye => cloud}/cli-recipes.mdx (90%) rename docs/de/{agenteye => cloud}/cli.mdx (91%) create mode 100644 docs/de/cloud/connect.mdx rename docs/de/{agenteye => cloud}/dashboards.mdx (70%) rename docs/de/{agenteye/error-tracking.mdx => cloud/errors.mdx} (65%) rename docs/de/{agenteye => cloud}/evaluations.mdx (66%) rename docs/de/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (83%) rename docs/de/{agenteye => cloud}/event-stream.mdx (73%) create mode 100644 docs/de/cloud/fleet.mdx rename docs/de/{agenteye => cloud}/incidents.mdx (85%) create mode 100644 docs/de/cloud/managed-policies.mdx rename docs/de/{agenteye => cloud}/overview.mdx (50%) rename docs/de/{agenteye/telemetry.mdx => cloud/performance.mdx} (82%) rename docs/de/{agenteye => cloud}/queries.mdx (76%) rename docs/de/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (92%) create mode 100644 docs/de/cloud/security.mdx rename docs/de/{agenteye => cloud}/sessions.mdx (75%) create mode 100644 docs/de/concepts.mdx create mode 100644 docs/de/daemon.mdx rename docs/de/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/de/policies.mdx rename docs/de/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/de/reference/files.mdx create mode 100644 docs/es/agent-support.mdx delete mode 100644 docs/es/agenteye/cli-and-agents.mdx delete mode 100644 docs/es/agenteye/cli-skill.mdx delete mode 100644 docs/es/agenteye/codex-capture.mdx delete mode 100644 docs/es/agenteye/concepts.mdx delete mode 100644 docs/es/agenteye/evaluator-skill.mdx delete mode 100644 docs/es/agenteye/hermes-capture.mdx delete mode 100644 docs/es/agenteye/observability.mdx delete mode 100644 docs/es/agenteye/openclaw-capture.mdx delete mode 100644 docs/es/agenteye/python-sdk-skill.mdx delete mode 100644 docs/es/agenteye/security.mdx rename docs/es/{cli => }/audit.mdx (100%) create mode 100644 docs/es/cli/backfill.mdx create mode 100644 docs/es/cli/config.mdx create mode 100644 docs/es/cli/flush.mdx create mode 100644 docs/es/cli/harness.mdx create mode 100644 docs/es/cli/migrate.mdx create mode 100644 docs/es/cli/uninstall.mdx create mode 100644 docs/es/cli/update.mdx rename docs/es/{agenteye/api-keys.mdx => cloud/access.mdx} (98%) create mode 100644 docs/es/cloud/agent-skills.mdx rename docs/es/{agenteye => cloud}/alerts.mdx (76%) rename docs/es/{agenteye => cloud}/assistant.mdx (84%) rename docs/es/{agenteye => cloud}/audits.mdx (82%) create mode 100644 docs/es/cloud/capture.mdx rename docs/es/{agenteye => cloud}/cli-recipes.mdx (91%) rename docs/es/{agenteye => cloud}/cli.mdx (97%) create mode 100644 docs/es/cloud/connect.mdx rename docs/es/{agenteye => cloud}/dashboards.mdx (69%) rename docs/es/{agenteye/error-tracking.mdx => cloud/errors.mdx} (66%) rename docs/es/{agenteye => cloud}/evaluations.mdx (68%) rename docs/es/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (83%) rename docs/es/{agenteye => cloud}/event-stream.mdx (85%) create mode 100644 docs/es/cloud/fleet.mdx rename docs/es/{agenteye => cloud}/incidents.mdx (84%) create mode 100644 docs/es/cloud/managed-policies.mdx rename docs/es/{agenteye => cloud}/overview.mdx (51%) rename docs/es/{agenteye/telemetry.mdx => cloud/performance.mdx} (81%) rename docs/es/{agenteye => cloud}/queries.mdx (84%) rename docs/es/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (93%) create mode 100644 docs/es/cloud/security.mdx rename docs/es/{agenteye => cloud}/sessions.mdx (82%) create mode 100644 docs/es/concepts.mdx create mode 100644 docs/es/daemon.mdx rename docs/es/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/es/policies.mdx rename docs/es/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/es/reference/files.mdx create mode 100644 docs/fr/agent-support.mdx delete mode 100644 docs/fr/agenteye/cli-and-agents.mdx delete mode 100644 docs/fr/agenteye/cli-skill.mdx delete mode 100644 docs/fr/agenteye/codex-capture.mdx delete mode 100644 docs/fr/agenteye/concepts.mdx delete mode 100644 docs/fr/agenteye/evaluator-skill.mdx delete mode 100644 docs/fr/agenteye/hermes-capture.mdx delete mode 100644 docs/fr/agenteye/observability.mdx delete mode 100644 docs/fr/agenteye/openclaw-capture.mdx delete mode 100644 docs/fr/agenteye/python-sdk-skill.mdx delete mode 100644 docs/fr/agenteye/security.mdx rename docs/fr/{cli => }/audit.mdx (100%) create mode 100644 docs/fr/cli/backfill.mdx create mode 100644 docs/fr/cli/config.mdx create mode 100644 docs/fr/cli/flush.mdx create mode 100644 docs/fr/cli/harness.mdx create mode 100644 docs/fr/cli/migrate.mdx create mode 100644 docs/fr/cli/uninstall.mdx create mode 100644 docs/fr/cli/update.mdx rename docs/fr/{agenteye/api-keys.mdx => cloud/access.mdx} (98%) create mode 100644 docs/fr/cloud/agent-skills.mdx rename docs/fr/{agenteye => cloud}/alerts.mdx (84%) rename docs/fr/{agenteye => cloud}/assistant.mdx (84%) rename docs/fr/{agenteye => cloud}/audits.mdx (82%) create mode 100644 docs/fr/cloud/capture.mdx rename docs/fr/{agenteye => cloud}/cli-recipes.mdx (91%) rename docs/fr/{agenteye => cloud}/cli.mdx (97%) create mode 100644 docs/fr/cloud/connect.mdx rename docs/fr/{agenteye => cloud}/dashboards.mdx (71%) rename docs/fr/{agenteye/error-tracking.mdx => cloud/errors.mdx} (66%) rename docs/fr/{agenteye => cloud}/evaluations.mdx (67%) rename docs/fr/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (89%) rename docs/fr/{agenteye => cloud}/event-stream.mdx (85%) create mode 100644 docs/fr/cloud/fleet.mdx rename docs/fr/{agenteye => cloud}/incidents.mdx (84%) create mode 100644 docs/fr/cloud/managed-policies.mdx rename docs/fr/{agenteye => cloud}/overview.mdx (51%) rename docs/fr/{agenteye/telemetry.mdx => cloud/performance.mdx} (81%) rename docs/fr/{agenteye => cloud}/queries.mdx (78%) rename docs/fr/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (94%) create mode 100644 docs/fr/cloud/security.mdx rename docs/fr/{agenteye => cloud}/sessions.mdx (80%) create mode 100644 docs/fr/concepts.mdx create mode 100644 docs/fr/daemon.mdx rename docs/fr/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/fr/policies.mdx rename docs/fr/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/fr/reference/files.mdx delete mode 100644 docs/getting-started.mdx create mode 100644 docs/he/agent-support.mdx delete mode 100644 docs/he/agenteye/cli-and-agents.mdx delete mode 100644 docs/he/agenteye/cli-skill.mdx delete mode 100644 docs/he/agenteye/codex-capture.mdx delete mode 100644 docs/he/agenteye/concepts.mdx delete mode 100644 docs/he/agenteye/evaluator-skill.mdx delete mode 100644 docs/he/agenteye/hermes-capture.mdx delete mode 100644 docs/he/agenteye/observability.mdx delete mode 100644 docs/he/agenteye/openclaw-capture.mdx delete mode 100644 docs/he/agenteye/python-sdk-skill.mdx delete mode 100644 docs/he/agenteye/security.mdx rename docs/he/{cli => }/audit.mdx (100%) create mode 100644 docs/he/cli/backfill.mdx create mode 100644 docs/he/cli/config.mdx create mode 100644 docs/he/cli/flush.mdx create mode 100644 docs/he/cli/harness.mdx create mode 100644 docs/he/cli/migrate.mdx create mode 100644 docs/he/cli/uninstall.mdx create mode 100644 docs/he/cli/update.mdx rename docs/he/{agenteye/api-keys.mdx => cloud/access.mdx} (95%) create mode 100644 docs/he/cloud/agent-skills.mdx rename docs/he/{agenteye => cloud}/alerts.mdx (77%) rename docs/he/{agenteye => cloud}/assistant.mdx (84%) rename docs/he/{agenteye => cloud}/audits.mdx (82%) create mode 100644 docs/he/cloud/capture.mdx rename docs/he/{agenteye => cloud}/cli-recipes.mdx (91%) rename docs/he/{agenteye => cloud}/cli.mdx (91%) create mode 100644 docs/he/cloud/connect.mdx rename docs/he/{agenteye => cloud}/dashboards.mdx (73%) rename docs/he/{agenteye/error-tracking.mdx => cloud/errors.mdx} (67%) rename docs/he/{agenteye => cloud}/evaluations.mdx (66%) rename docs/he/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (83%) rename docs/he/{agenteye => cloud}/event-stream.mdx (79%) create mode 100644 docs/he/cloud/fleet.mdx rename docs/he/{agenteye => cloud}/incidents.mdx (85%) create mode 100644 docs/he/cloud/managed-policies.mdx rename docs/he/{agenteye => cloud}/overview.mdx (51%) rename docs/he/{agenteye/telemetry.mdx => cloud/performance.mdx} (81%) rename docs/he/{agenteye => cloud}/queries.mdx (79%) rename docs/he/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (92%) create mode 100644 docs/he/cloud/security.mdx rename docs/he/{agenteye => cloud}/sessions.mdx (75%) create mode 100644 docs/he/concepts.mdx create mode 100644 docs/he/daemon.mdx rename docs/he/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/he/policies.mdx rename docs/he/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/he/reference/files.mdx create mode 100644 docs/hi/agent-support.mdx delete mode 100644 docs/hi/agenteye/cli-and-agents.mdx delete mode 100644 docs/hi/agenteye/cli-skill.mdx delete mode 100644 docs/hi/agenteye/codex-capture.mdx delete mode 100644 docs/hi/agenteye/concepts.mdx delete mode 100644 docs/hi/agenteye/evaluator-skill.mdx delete mode 100644 docs/hi/agenteye/hermes-capture.mdx delete mode 100644 docs/hi/agenteye/observability.mdx delete mode 100644 docs/hi/agenteye/openclaw-capture.mdx delete mode 100644 docs/hi/agenteye/python-sdk-skill.mdx delete mode 100644 docs/hi/agenteye/security.mdx rename docs/hi/{cli => }/audit.mdx (100%) create mode 100644 docs/hi/cli/backfill.mdx create mode 100644 docs/hi/cli/config.mdx create mode 100644 docs/hi/cli/flush.mdx create mode 100644 docs/hi/cli/harness.mdx create mode 100644 docs/hi/cli/migrate.mdx create mode 100644 docs/hi/cli/uninstall.mdx create mode 100644 docs/hi/cli/update.mdx rename docs/hi/{agenteye/api-keys.mdx => cloud/access.mdx} (95%) create mode 100644 docs/hi/cloud/agent-skills.mdx rename docs/hi/{agenteye => cloud}/alerts.mdx (79%) rename docs/hi/{agenteye => cloud}/assistant.mdx (86%) rename docs/hi/{agenteye => cloud}/audits.mdx (82%) create mode 100644 docs/hi/cloud/capture.mdx rename docs/hi/{agenteye => cloud}/cli-recipes.mdx (89%) rename docs/hi/{agenteye => cloud}/cli.mdx (90%) create mode 100644 docs/hi/cloud/connect.mdx rename docs/hi/{agenteye => cloud}/dashboards.mdx (73%) rename docs/hi/{agenteye/error-tracking.mdx => cloud/errors.mdx} (68%) rename docs/hi/{agenteye => cloud}/evaluations.mdx (70%) rename docs/hi/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (81%) rename docs/hi/{agenteye => cloud}/event-stream.mdx (81%) create mode 100644 docs/hi/cloud/fleet.mdx rename docs/hi/{agenteye => cloud}/incidents.mdx (86%) create mode 100644 docs/hi/cloud/managed-policies.mdx rename docs/hi/{agenteye => cloud}/overview.mdx (50%) rename docs/hi/{agenteye/telemetry.mdx => cloud/performance.mdx} (85%) rename docs/hi/{agenteye => cloud}/queries.mdx (82%) rename docs/hi/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (92%) create mode 100644 docs/hi/cloud/security.mdx rename docs/hi/{agenteye => cloud}/sessions.mdx (76%) create mode 100644 docs/hi/concepts.mdx create mode 100644 docs/hi/daemon.mdx rename docs/hi/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/hi/policies.mdx rename docs/hi/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/hi/reference/files.mdx create mode 100644 docs/how-it-works.mdx create mode 100644 docs/images/local-session-viewer.png create mode 100644 docs/it/agent-support.mdx delete mode 100644 docs/it/agenteye/cli-and-agents.mdx delete mode 100644 docs/it/agenteye/cli-skill.mdx delete mode 100644 docs/it/agenteye/codex-capture.mdx delete mode 100644 docs/it/agenteye/concepts.mdx delete mode 100644 docs/it/agenteye/evaluator-skill.mdx delete mode 100644 docs/it/agenteye/hermes-capture.mdx delete mode 100644 docs/it/agenteye/observability.mdx delete mode 100644 docs/it/agenteye/openclaw-capture.mdx delete mode 100644 docs/it/agenteye/python-sdk-skill.mdx rename docs/it/{cli => }/audit.mdx (100%) create mode 100644 docs/it/cli/backfill.mdx create mode 100644 docs/it/cli/config.mdx create mode 100644 docs/it/cli/flush.mdx create mode 100644 docs/it/cli/harness.mdx create mode 100644 docs/it/cli/migrate.mdx create mode 100644 docs/it/cli/uninstall.mdx create mode 100644 docs/it/cli/update.mdx rename docs/it/{agenteye/api-keys.mdx => cloud/access.mdx} (96%) create mode 100644 docs/it/cloud/agent-skills.mdx rename docs/it/{agenteye => cloud}/alerts.mdx (75%) rename docs/it/{agenteye => cloud}/assistant.mdx (85%) rename docs/it/{agenteye => cloud}/audits.mdx (82%) create mode 100644 docs/it/cloud/capture.mdx rename docs/it/{agenteye => cloud}/cli-recipes.mdx (91%) rename docs/it/{agenteye => cloud}/cli.mdx (95%) create mode 100644 docs/it/cloud/connect.mdx rename docs/it/{agenteye => cloud}/dashboards.mdx (69%) rename docs/it/{agenteye/error-tracking.mdx => cloud/errors.mdx} (67%) rename docs/it/{agenteye => cloud}/evaluations.mdx (67%) rename docs/it/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (83%) rename docs/it/{agenteye => cloud}/event-stream.mdx (78%) create mode 100644 docs/it/cloud/fleet.mdx rename docs/it/{agenteye => cloud}/incidents.mdx (84%) create mode 100644 docs/it/cloud/managed-policies.mdx rename docs/it/{agenteye => cloud}/overview.mdx (51%) rename docs/it/{agenteye/telemetry.mdx => cloud/performance.mdx} (81%) rename docs/it/{agenteye => cloud}/queries.mdx (78%) rename docs/it/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (92%) rename docs/it/{agenteye => cloud}/security.mdx (51%) rename docs/it/{agenteye => cloud}/sessions.mdx (74%) create mode 100644 docs/it/concepts.mdx create mode 100644 docs/it/daemon.mdx rename docs/it/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/it/policies.mdx rename docs/it/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/it/reference/files.mdx create mode 100644 docs/ja/agent-support.mdx delete mode 100644 docs/ja/agenteye/cli-and-agents.mdx delete mode 100644 docs/ja/agenteye/cli-skill.mdx delete mode 100644 docs/ja/agenteye/codex-capture.mdx delete mode 100644 docs/ja/agenteye/concepts.mdx delete mode 100644 docs/ja/agenteye/evaluator-skill.mdx delete mode 100644 docs/ja/agenteye/hermes-capture.mdx delete mode 100644 docs/ja/agenteye/observability.mdx delete mode 100644 docs/ja/agenteye/openclaw-capture.mdx delete mode 100644 docs/ja/agenteye/python-sdk-skill.mdx delete mode 100644 docs/ja/agenteye/security.mdx rename docs/ja/{cli => }/audit.mdx (100%) create mode 100644 docs/ja/cli/backfill.mdx create mode 100644 docs/ja/cli/config.mdx create mode 100644 docs/ja/cli/flush.mdx create mode 100644 docs/ja/cli/harness.mdx create mode 100644 docs/ja/cli/migrate.mdx create mode 100644 docs/ja/cli/uninstall.mdx create mode 100644 docs/ja/cli/update.mdx rename docs/ja/{agenteye/api-keys.mdx => cloud/access.mdx} (95%) create mode 100644 docs/ja/cloud/agent-skills.mdx rename docs/ja/{agenteye => cloud}/alerts.mdx (75%) rename docs/ja/{agenteye => cloud}/assistant.mdx (86%) rename docs/ja/{agenteye => cloud}/audits.mdx (82%) create mode 100644 docs/ja/cloud/capture.mdx rename docs/ja/{agenteye => cloud}/cli-recipes.mdx (90%) rename docs/ja/{agenteye => cloud}/cli.mdx (91%) create mode 100644 docs/ja/cloud/connect.mdx rename docs/ja/{agenteye => cloud}/dashboards.mdx (69%) rename docs/ja/{agenteye/error-tracking.mdx => cloud/errors.mdx} (68%) rename docs/ja/{agenteye => cloud}/evaluations.mdx (68%) rename docs/ja/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (82%) rename docs/ja/{agenteye => cloud}/event-stream.mdx (80%) create mode 100644 docs/ja/cloud/fleet.mdx rename docs/ja/{agenteye => cloud}/incidents.mdx (86%) create mode 100644 docs/ja/cloud/managed-policies.mdx rename docs/ja/{agenteye => cloud}/overview.mdx (51%) rename docs/ja/{agenteye/telemetry.mdx => cloud/performance.mdx} (81%) rename docs/ja/{agenteye => cloud}/queries.mdx (85%) rename docs/ja/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (91%) create mode 100644 docs/ja/cloud/security.mdx rename docs/ja/{agenteye => cloud}/sessions.mdx (74%) create mode 100644 docs/ja/concepts.mdx create mode 100644 docs/ja/daemon.mdx rename docs/ja/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/ja/policies.mdx rename docs/ja/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/ja/reference/files.mdx create mode 100644 docs/ko/agent-support.mdx delete mode 100644 docs/ko/agenteye/cli-and-agents.mdx delete mode 100644 docs/ko/agenteye/cli-skill.mdx delete mode 100644 docs/ko/agenteye/codex-capture.mdx delete mode 100644 docs/ko/agenteye/concepts.mdx delete mode 100644 docs/ko/agenteye/evaluator-skill.mdx delete mode 100644 docs/ko/agenteye/hermes-capture.mdx delete mode 100644 docs/ko/agenteye/observability.mdx delete mode 100644 docs/ko/agenteye/openclaw-capture.mdx delete mode 100644 docs/ko/agenteye/python-sdk-skill.mdx delete mode 100644 docs/ko/agenteye/security.mdx rename docs/ko/{cli => }/audit.mdx (100%) create mode 100644 docs/ko/cli/backfill.mdx create mode 100644 docs/ko/cli/config.mdx create mode 100644 docs/ko/cli/flush.mdx create mode 100644 docs/ko/cli/harness.mdx create mode 100644 docs/ko/cli/migrate.mdx create mode 100644 docs/ko/cli/uninstall.mdx create mode 100644 docs/ko/cli/update.mdx rename docs/ko/{agenteye/api-keys.mdx => cloud/access.mdx} (95%) create mode 100644 docs/ko/cloud/agent-skills.mdx rename docs/ko/{agenteye => cloud}/alerts.mdx (75%) rename docs/ko/{agenteye => cloud}/assistant.mdx (84%) rename docs/ko/{agenteye => cloud}/audits.mdx (81%) create mode 100644 docs/ko/cloud/capture.mdx rename docs/ko/{agenteye => cloud}/cli-recipes.mdx (91%) rename docs/ko/{agenteye => cloud}/cli.mdx (91%) create mode 100644 docs/ko/cloud/connect.mdx rename docs/ko/{agenteye => cloud}/dashboards.mdx (69%) rename docs/ko/{agenteye/error-tracking.mdx => cloud/errors.mdx} (75%) rename docs/ko/{agenteye => cloud}/evaluations.mdx (66%) rename docs/ko/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (88%) rename docs/ko/{agenteye => cloud}/event-stream.mdx (80%) create mode 100644 docs/ko/cloud/fleet.mdx rename docs/ko/{agenteye => cloud}/incidents.mdx (87%) create mode 100644 docs/ko/cloud/managed-policies.mdx rename docs/ko/{agenteye => cloud}/overview.mdx (51%) rename docs/ko/{agenteye/telemetry.mdx => cloud/performance.mdx} (82%) rename docs/ko/{agenteye => cloud}/queries.mdx (77%) rename docs/ko/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (92%) create mode 100644 docs/ko/cloud/security.mdx rename docs/ko/{agenteye => cloud}/sessions.mdx (73%) create mode 100644 docs/ko/concepts.mdx create mode 100644 docs/ko/daemon.mdx rename docs/ko/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/ko/policies.mdx rename docs/ko/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/ko/reference/files.mdx create mode 100644 docs/policies.mdx create mode 100644 docs/pt-br/agent-support.mdx delete mode 100644 docs/pt-br/agenteye/cli-and-agents.mdx delete mode 100644 docs/pt-br/agenteye/cli-skill.mdx delete mode 100644 docs/pt-br/agenteye/codex-capture.mdx delete mode 100644 docs/pt-br/agenteye/concepts.mdx delete mode 100644 docs/pt-br/agenteye/evaluator-skill.mdx delete mode 100644 docs/pt-br/agenteye/hermes-capture.mdx delete mode 100644 docs/pt-br/agenteye/observability.mdx delete mode 100644 docs/pt-br/agenteye/openclaw-capture.mdx delete mode 100644 docs/pt-br/agenteye/python-sdk-skill.mdx delete mode 100644 docs/pt-br/agenteye/security.mdx rename docs/pt-br/{cli => }/audit.mdx (100%) create mode 100644 docs/pt-br/cli/backfill.mdx create mode 100644 docs/pt-br/cli/config.mdx create mode 100644 docs/pt-br/cli/flush.mdx create mode 100644 docs/pt-br/cli/harness.mdx create mode 100644 docs/pt-br/cli/migrate.mdx create mode 100644 docs/pt-br/cli/uninstall.mdx create mode 100644 docs/pt-br/cli/update.mdx rename docs/pt-br/{agenteye/api-keys.mdx => cloud/access.mdx} (98%) create mode 100644 docs/pt-br/cloud/agent-skills.mdx rename docs/pt-br/{agenteye => cloud}/alerts.mdx (82%) rename docs/pt-br/{agenteye => cloud}/assistant.mdx (84%) rename docs/pt-br/{agenteye => cloud}/audits.mdx (81%) create mode 100644 docs/pt-br/cloud/capture.mdx rename docs/pt-br/{agenteye => cloud}/cli-recipes.mdx (91%) rename docs/pt-br/{agenteye => cloud}/cli.mdx (95%) create mode 100644 docs/pt-br/cloud/connect.mdx rename docs/pt-br/{agenteye => cloud}/dashboards.mdx (68%) rename docs/pt-br/{agenteye/error-tracking.mdx => cloud/errors.mdx} (65%) rename docs/pt-br/{agenteye => cloud}/evaluations.mdx (83%) rename docs/pt-br/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (89%) rename docs/pt-br/{agenteye => cloud}/event-stream.mdx (85%) create mode 100644 docs/pt-br/cloud/fleet.mdx rename docs/pt-br/{agenteye => cloud}/incidents.mdx (84%) create mode 100644 docs/pt-br/cloud/managed-policies.mdx rename docs/pt-br/{agenteye => cloud}/overview.mdx (50%) rename docs/pt-br/{agenteye/telemetry.mdx => cloud/performance.mdx} (81%) rename docs/pt-br/{agenteye => cloud}/queries.mdx (84%) rename docs/pt-br/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (96%) create mode 100644 docs/pt-br/cloud/security.mdx rename docs/pt-br/{agenteye => cloud}/sessions.mdx (82%) create mode 100644 docs/pt-br/concepts.mdx create mode 100644 docs/pt-br/daemon.mdx rename docs/pt-br/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/pt-br/policies.mdx rename docs/pt-br/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/pt-br/reference/files.mdx create mode 100644 docs/quickstart.mdx create mode 100644 docs/reference/files.mdx create mode 100644 docs/ru/agent-support.mdx delete mode 100644 docs/ru/agenteye/cli-and-agents.mdx delete mode 100644 docs/ru/agenteye/cli-skill.mdx delete mode 100644 docs/ru/agenteye/codex-capture.mdx delete mode 100644 docs/ru/agenteye/concepts.mdx delete mode 100644 docs/ru/agenteye/evaluator-skill.mdx delete mode 100644 docs/ru/agenteye/hermes-capture.mdx delete mode 100644 docs/ru/agenteye/observability.mdx delete mode 100644 docs/ru/agenteye/openclaw-capture.mdx delete mode 100644 docs/ru/agenteye/python-sdk-skill.mdx rename docs/ru/{cli => }/audit.mdx (100%) create mode 100644 docs/ru/cli/backfill.mdx create mode 100644 docs/ru/cli/config.mdx create mode 100644 docs/ru/cli/flush.mdx create mode 100644 docs/ru/cli/harness.mdx create mode 100644 docs/ru/cli/migrate.mdx create mode 100644 docs/ru/cli/uninstall.mdx create mode 100644 docs/ru/cli/update.mdx rename docs/ru/{agenteye/api-keys.mdx => cloud/access.mdx} (96%) create mode 100644 docs/ru/cloud/agent-skills.mdx rename docs/ru/{agenteye => cloud}/alerts.mdx (79%) rename docs/ru/{agenteye => cloud}/assistant.mdx (86%) rename docs/ru/{agenteye => cloud}/audits.mdx (82%) create mode 100644 docs/ru/cloud/capture.mdx rename docs/{agenteye => ru/cloud}/cli-recipes.mdx (91%) rename docs/ru/{agenteye => cloud}/cli.mdx (91%) create mode 100644 docs/ru/cloud/connect.mdx rename docs/ru/{agenteye => cloud}/dashboards.mdx (73%) rename docs/ru/{agenteye/error-tracking.mdx => cloud/errors.mdx} (67%) rename docs/ru/{agenteye => cloud}/evaluations.mdx (70%) rename docs/ru/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (89%) rename docs/ru/{agenteye => cloud}/event-stream.mdx (81%) create mode 100644 docs/ru/cloud/fleet.mdx rename docs/ru/{agenteye => cloud}/incidents.mdx (86%) create mode 100644 docs/ru/cloud/managed-policies.mdx rename docs/ru/{agenteye => cloud}/overview.mdx (52%) rename docs/ru/{agenteye/telemetry.mdx => cloud/performance.mdx} (84%) rename docs/ru/{agenteye => cloud}/queries.mdx (80%) rename docs/ru/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (92%) rename docs/ru/{agenteye => cloud}/security.mdx (51%) rename docs/ru/{agenteye => cloud}/sessions.mdx (76%) create mode 100644 docs/ru/concepts.mdx create mode 100644 docs/ru/daemon.mdx rename docs/ru/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/ru/policies.mdx rename docs/ru/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/ru/reference/files.mdx create mode 100644 docs/tr/agent-support.mdx delete mode 100644 docs/tr/agenteye/cli-and-agents.mdx delete mode 100644 docs/tr/agenteye/cli-skill.mdx delete mode 100644 docs/tr/agenteye/codex-capture.mdx delete mode 100644 docs/tr/agenteye/concepts.mdx delete mode 100644 docs/tr/agenteye/evaluator-skill.mdx delete mode 100644 docs/tr/agenteye/hermes-capture.mdx delete mode 100644 docs/tr/agenteye/observability.mdx delete mode 100644 docs/tr/agenteye/openclaw-capture.mdx delete mode 100644 docs/tr/agenteye/overview.mdx delete mode 100644 docs/tr/agenteye/python-sdk-skill.mdx delete mode 100644 docs/tr/agenteye/security.mdx rename docs/tr/{cli => }/audit.mdx (100%) create mode 100644 docs/tr/cli/backfill.mdx create mode 100644 docs/tr/cli/config.mdx create mode 100644 docs/tr/cli/flush.mdx create mode 100644 docs/tr/cli/harness.mdx create mode 100644 docs/tr/cli/migrate.mdx create mode 100644 docs/tr/cli/uninstall.mdx create mode 100644 docs/tr/cli/update.mdx rename docs/tr/{agenteye/api-keys.mdx => cloud/access.mdx} (98%) create mode 100644 docs/tr/cloud/agent-skills.mdx rename docs/tr/{agenteye => cloud}/alerts.mdx (76%) rename docs/tr/{agenteye => cloud}/assistant.mdx (82%) rename docs/tr/{agenteye => cloud}/audits.mdx (81%) create mode 100644 docs/tr/cloud/capture.mdx rename docs/tr/{agenteye => cloud}/cli-recipes.mdx (90%) rename docs/tr/{agenteye => cloud}/cli.mdx (91%) create mode 100644 docs/tr/cloud/connect.mdx rename docs/tr/{agenteye => cloud}/dashboards.mdx (68%) rename docs/tr/{agenteye/error-tracking.mdx => cloud/errors.mdx} (65%) rename docs/tr/{agenteye => cloud}/evaluations.mdx (65%) rename docs/tr/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (82%) rename docs/tr/{agenteye => cloud}/event-stream.mdx (78%) create mode 100644 docs/tr/cloud/fleet.mdx rename docs/tr/{agenteye => cloud}/incidents.mdx (83%) create mode 100644 docs/tr/cloud/managed-policies.mdx create mode 100644 docs/tr/cloud/overview.mdx rename docs/tr/{agenteye/telemetry.mdx => cloud/performance.mdx} (82%) rename docs/tr/{agenteye => cloud}/queries.mdx (78%) rename docs/tr/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (91%) create mode 100644 docs/tr/cloud/security.mdx rename docs/tr/{agenteye => cloud}/sessions.mdx (75%) create mode 100644 docs/tr/concepts.mdx create mode 100644 docs/tr/daemon.mdx rename docs/tr/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/tr/policies.mdx rename docs/tr/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/tr/reference/files.mdx create mode 100644 docs/vi/agent-support.mdx delete mode 100644 docs/vi/agenteye/cli-and-agents.mdx delete mode 100644 docs/vi/agenteye/cli-skill.mdx delete mode 100644 docs/vi/agenteye/codex-capture.mdx delete mode 100644 docs/vi/agenteye/concepts.mdx delete mode 100644 docs/vi/agenteye/evaluator-skill.mdx delete mode 100644 docs/vi/agenteye/hermes-capture.mdx delete mode 100644 docs/vi/agenteye/observability.mdx delete mode 100644 docs/vi/agenteye/openclaw-capture.mdx delete mode 100644 docs/vi/agenteye/python-sdk-skill.mdx delete mode 100644 docs/vi/agenteye/security.mdx rename docs/vi/{cli => }/audit.mdx (100%) create mode 100644 docs/vi/cli/backfill.mdx create mode 100644 docs/vi/cli/config.mdx create mode 100644 docs/vi/cli/flush.mdx create mode 100644 docs/vi/cli/harness.mdx create mode 100644 docs/vi/cli/migrate.mdx create mode 100644 docs/vi/cli/uninstall.mdx create mode 100644 docs/vi/cli/update.mdx rename docs/vi/{agenteye/api-keys.mdx => cloud/access.mdx} (95%) create mode 100644 docs/vi/cloud/agent-skills.mdx rename docs/vi/{agenteye => cloud}/alerts.mdx (77%) rename docs/vi/{agenteye => cloud}/assistant.mdx (84%) rename docs/vi/{agenteye => cloud}/audits.mdx (81%) create mode 100644 docs/vi/cloud/capture.mdx rename docs/vi/{agenteye => cloud}/cli-recipes.mdx (91%) rename docs/vi/{agenteye => cloud}/cli.mdx (91%) create mode 100644 docs/vi/cloud/connect.mdx rename docs/vi/{agenteye => cloud}/dashboards.mdx (71%) rename docs/vi/{agenteye/error-tracking.mdx => cloud/errors.mdx} (66%) rename docs/vi/{agenteye => cloud}/evaluations.mdx (68%) rename docs/vi/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (83%) rename docs/vi/{agenteye => cloud}/event-stream.mdx (79%) create mode 100644 docs/vi/cloud/fleet.mdx rename docs/vi/{agenteye => cloud}/incidents.mdx (86%) create mode 100644 docs/vi/cloud/managed-policies.mdx rename docs/vi/{agenteye => cloud}/overview.mdx (51%) rename docs/vi/{agenteye/telemetry.mdx => cloud/performance.mdx} (82%) rename docs/vi/{agenteye => cloud}/queries.mdx (78%) rename docs/vi/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (92%) create mode 100644 docs/vi/cloud/security.mdx rename docs/vi/{agenteye => cloud}/sessions.mdx (76%) create mode 100644 docs/vi/concepts.mdx create mode 100644 docs/vi/daemon.mdx rename docs/vi/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/vi/policies.mdx rename docs/vi/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/vi/reference/files.mdx create mode 100644 docs/zh/agent-support.mdx delete mode 100644 docs/zh/agenteye/cli-and-agents.mdx delete mode 100644 docs/zh/agenteye/cli-skill.mdx delete mode 100644 docs/zh/agenteye/codex-capture.mdx delete mode 100644 docs/zh/agenteye/concepts.mdx delete mode 100644 docs/zh/agenteye/evaluator-skill.mdx delete mode 100644 docs/zh/agenteye/hermes-capture.mdx delete mode 100644 docs/zh/agenteye/observability.mdx delete mode 100644 docs/zh/agenteye/openclaw-capture.mdx delete mode 100644 docs/zh/agenteye/python-sdk-skill.mdx delete mode 100644 docs/zh/agenteye/security.mdx rename docs/zh/{cli => }/audit.mdx (100%) create mode 100644 docs/zh/cli/backfill.mdx create mode 100644 docs/zh/cli/config.mdx create mode 100644 docs/zh/cli/flush.mdx create mode 100644 docs/zh/cli/harness.mdx create mode 100644 docs/zh/cli/migrate.mdx create mode 100644 docs/zh/cli/uninstall.mdx create mode 100644 docs/zh/cli/update.mdx rename docs/zh/{agenteye/api-keys.mdx => cloud/access.mdx} (95%) create mode 100644 docs/zh/cloud/agent-skills.mdx rename docs/zh/{agenteye => cloud}/alerts.mdx (74%) rename docs/zh/{agenteye => cloud}/assistant.mdx (84%) rename docs/zh/{agenteye => cloud}/audits.mdx (80%) create mode 100644 docs/zh/cloud/capture.mdx rename docs/zh/{agenteye => cloud}/cli-recipes.mdx (91%) rename docs/zh/{agenteye => cloud}/cli.mdx (91%) create mode 100644 docs/zh/cloud/connect.mdx rename docs/zh/{agenteye => cloud}/dashboards.mdx (66%) rename docs/zh/{agenteye/error-tracking.mdx => cloud/errors.mdx} (66%) rename docs/zh/{agenteye => cloud}/evaluations.mdx (67%) rename docs/zh/{agenteye/evaluation-suite.mdx => cloud/evaluators.mdx} (82%) rename docs/zh/{agenteye => cloud}/event-stream.mdx (85%) create mode 100644 docs/zh/cloud/fleet.mdx rename docs/zh/{agenteye => cloud}/incidents.mdx (84%) create mode 100644 docs/zh/cloud/managed-policies.mdx rename docs/zh/{agenteye => cloud}/overview.mdx (50%) rename docs/zh/{agenteye/telemetry.mdx => cloud/performance.mdx} (81%) rename docs/zh/{agenteye => cloud}/queries.mdx (82%) rename docs/zh/{agenteye/python-sdk.mdx => cloud/sdk.mdx} (96%) create mode 100644 docs/zh/cloud/security.mdx rename docs/zh/{agenteye => cloud}/sessions.mdx (79%) create mode 100644 docs/zh/concepts.mdx create mode 100644 docs/zh/daemon.mdx rename docs/zh/{architecture.mdx => how-it-works.mdx} (100%) create mode 100644 docs/zh/policies.mdx rename docs/zh/{getting-started.mdx => quickstart.mdx} (100%) create mode 100644 docs/zh/reference/files.mdx diff --git a/CHANGELOG.md b/CHANGELOG.md index a9e93431..af66cef4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,14 @@ # Changelog +## 1.0.1-beta.0 — 2026-08-12 + +### Docs + +- Rebuilt the documentation as one product instead of two. The site had two top-level tabs — "Enforcement" and "Observability" — which asked every reader to work out, before reading anything, which half of a single product they were in. There is now one navigation, ordered as a journey: install and guard a machine, see what happened on it, then govern the fleet from the cloud. The observability section is repositioned as **FailproofAI Cloud** and lives at `/cloud/*` (was `/agenteye/*`), with redirects from every old URL. (#684) +- Documented the integration surface that had no docs at all. `failproofai config --connect` was described only in `--help`: new pages cover [connecting a machine](https://docs.befailproof.ai/cloud/connect) (both capabilities, what leaves the machine, fleet provisioning, troubleshooting), [managed policies](https://docs.befailproof.ai/cloud/managed-policies) (deployments, digest verification, observe-mode rollout), [the fleet view](https://docs.befailproof.ai/cloud/fleet), and [session capture](https://docs.befailproof.ai/cloud/capture) across all 12 CLIs — replacing three per-CLI capture pages written for a separate collector. (#684) +- Added the pages a reader kept needing and could not find: [How it works](https://docs.befailproof.ai/how-it-works) (tool call → decision → dashboard, end to end, with the failure-mode table), [the failproofaid service](https://docs.befailproof.ai/daemon) (fail-closed, supervision, how the binary arrives, upgrades), [Supported agents](https://docs.befailproof.ai/agent-support) (per-CLI matrix of what a deny can actually block — the one fact that decides whether a policy does anything), [Policies](https://docs.befailproof.ai/policies) as a hub, one merged [Concepts](https://docs.befailproof.ai/concepts) glossary, and a [files and paths](https://docs.befailproof.ai/reference/files) reference. New CLI pages for `config`, `harness`, `backfill`, `flush`, and `uninstall`. (#684) +- Moved per-CLI hook-schema internals out of the Configuration page and into the support matrix, so configuration is about configuring again rather than a wall of vendor contracts. (#684) + ## 1.0.0 — 2026-08-12 The first stable release. Everything below this heading shipped across the diff --git a/README.md b/README.md index a565ee9a..b047debf 100644 --- a/README.md +++ b/README.md @@ -188,12 +188,14 @@ when something goes wrong. → [Dashboard guide](https://docs.befailproof.ai/das | | | |---|---| -| [Getting Started](https://docs.befailproof.ai/getting-started) | Installation and first steps | -| [Built-in Policies](https://docs.befailproof.ai/built-in-policies) | All 30 policies with parameters | +| [Quickstart](https://docs.befailproof.ai/quickstart) | Installation and first steps | +| [How it works](https://docs.befailproof.ai/how-it-works) | Tool call → decision → dashboard, end to end | +| [Built-in Policies](https://docs.befailproof.ai/built-in-policies) | All 39 policies with parameters | | [Custom Policies](https://docs.befailproof.ai/custom-policies) | Write your own | | [Configuration](https://docs.befailproof.ai/configuration) | Config scopes and merge rules | +| [Supported agents](https://docs.befailproof.ai/agent-support) | All 12 agent CLIs, and what each can block | | [Dashboard](https://docs.befailproof.ai/dashboard) | Session monitor and policy activity | -| [Architecture](https://docs.befailproof.ai/architecture) | How the hook system works | +| [FailproofAI Cloud](https://docs.befailproof.ai/cloud/overview) | Fleet-wide policy, observability, and evaluation | --- diff --git a/__tests__/scripts/translate-docs/mdx-translator.test.ts b/__tests__/scripts/translate-docs/mdx-translator.test.ts index e7e09164..17f6376e 100644 --- a/__tests__/scripts/translate-docs/mdx-translator.test.ts +++ b/__tests__/scripts/translate-docs/mdx-translator.test.ts @@ -59,10 +59,10 @@ function emptyCache(): TranslationCache { } describe("getEnglishMdxPages", () => { - it("includes AgentEye pages in automatic translation", () => { + it("includes cloud pages in automatic translation", () => { const pages = getEnglishMdxPages(); expect(pages.length).toBeGreaterThan(0); - expect(pages.some((page) => page.includes("/agenteye/"))).toBe(true); + expect(pages.some((page) => page.includes("/cloud/"))).toBe(true); }); }); diff --git a/__tests__/scripts/validate-mdx.test.ts b/__tests__/scripts/validate-mdx.test.ts index ece62123..80bda182 100644 --- a/__tests__/scripts/validate-mdx.test.ts +++ b/__tests__/scripts/validate-mdx.test.ts @@ -253,7 +253,7 @@ describe("findBrokenAssetRefs", () => { // Fixtures resolve against the real repo so the check is exercised with the // same two path conventions the docs actually use. const REPO = join(__dirname, "..", ".."); - const DOCS_PAGE = join(REPO, "docs", "agenteye", "alerts.mdx"); + const DOCS_PAGE = join(REPO, "docs", "cloud", "alerts.mdx"); const I18N_PAGE = join(REPO, "docs", "i18n", "README.ja.md"); it("flags the exact regression that broke every translated README", () => { @@ -281,19 +281,19 @@ describe("findBrokenAssetRefs", () => { }); it("resolves a leading slash against docs/, not the page directory", () => { - // Mintlify site-absolute form, used by every agenteye page. + // Mintlify site-absolute form, used by every cloud page. expect( findBrokenAssetRefs( DOCS_PAGE, - "![Alerts](/agenteye/images/alerts.png)\n", + "![Alerts](/cloud/images/alerts.png)\n", ), ).toEqual([]); const broken = findBrokenAssetRefs( DOCS_PAGE, - "![Nope](/agenteye/images/does-not-exist.png)\n", + "![Nope](/cloud/images/does-not-exist.png)\n", ); expect(broken).toHaveLength(1); - expect(broken[0].resolved).toBe("docs/agenteye/images/does-not-exist.png"); + expect(broken[0].resolved).toBe("docs/cloud/images/does-not-exist.png"); }); it("checks srcset candidates, not just src", () => { diff --git a/docs/agent-support.mdx b/docs/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/agenteye/cli-and-agents.mdx b/docs/agenteye/cli-and-agents.mdx deleted file mode 100644 index 67e77078..00000000 --- a/docs/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "Your entire Failproof AI Observability deployment, one command away." ---- - - -Your entire Failproof AI Observability deployment, one command away. Check production, cut an API key, or ack an incident without leaving your terminal, then script any of it into CI, or let a coding agent do it for you in plain English. - -```bash -pipx install agenteye -agenteye login --email you@example.com # a 6-digit code lands in your inbox -agenteye --json sessions --since 24h # every agent run from the last day, newest first -``` - -*The `agenteye` CLI talks to your dashboard. It is a different tool from the collector, which ships events to the server.* - -## Your whole deployment, one command away - -Stop tab-hopping to answer a quick question. The `agenteye` CLI reads your data and administers your org from a single binary, so a check that used to mean clicking through the dashboard becomes one line you can rerun, alias, or paste into a runbook. You get four surfaces: - -- **Read your data:** `sessions`, `events`, `evals`, and `errors`, filtered by time, agent, and environment. -- **Manage your org:** `keys`, `users`, `settings`, `alerts`, and `incidents`. -- **Run analytics:** saved SQL plus an ad-hoc `query` runner over your event data. -- **Ask the assistant:** `agent ask` reaches the same read-only analyst you chat with in the dashboard. - -Install it once with `pipx`, sign in with an emailed 6-digit code, and you are ready. The session lasts about a day; rerun `agenteye login` when it expires. Reach for it to spot-check production, provision a key, or triage a firing incident, all without opening a browser: - -```bash -agenteye errors --since 24h --aggregate # what is breaking, grouped by error type -agenteye incidents list --state firing # what is on fire right now -agenteye keys create ci --add events:add # a key that can only push events, secret shown once -``` - -One habit to know: global options like `--json` go before the command. `agenteye --json sessions` is right; `agenteye sessions --json` is not. - -## Script it, wire it into CI - -Every command takes `--json`, and that changes everything. Clean JSON goes to stdout while human status and warnings go to stderr, so a `--json` capture pipes straight into `jq` with no stray line to strip. That is what makes the CLI equally good for you at a prompt and for a coding agent parsing output: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -It is built to run unattended. Confirmation prompts auto-skip when no terminal is attached, so nothing hangs in a pipeline, and every command returns a meaningful exit code: `0` success, `4` not logged in, `5` missing a permission (the message names it, for example `alerts:write`), `3` dashboard unreachable. A script can branch on a `4` to reauthenticate or a `5` to tell you exactly what to ask an admin for, instead of failing blind. - -## Let a coding agent drive it in plain English - -Better yet, you should not have to remember any of these flags at all. The **CLI skill** is a small Agent Skill folder named `agenteye-cli` that teaches a coding agent such as Claude Code or Codex to drive the CLI from plain-English requests. Ask "is anything broken today?" and the agent picks the command, runs it as you, and answers in prose. - -For Claude Code, drop the `agenteye-cli` folder into `~/.claude/skills/` and it is auto-discovered. Failproof AI Observability provides the folder; there is nothing extra to install, because it only drives the CLI you already installed. Log in yourself first: the skill cannot complete the emailed-code login for you. - -Because the agent runs the CLI as you, it can do everything your login permits, reads and writes alike: create keys, change settings, resolve incidents. The CLI's "are you sure?" prompt does not fire for an agent, so the skill is written to state the exact command and wait for your OK before any change. You are the confirmation step. - -```text -you Why did session run-001 fail? - -agent Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -Reads stay instant, and every write pauses for you: - -```text -you Give CI a key that can only push events. - -agent This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you yes - -agent Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## Related - -- [CLI reference](/agenteye/cli): every command, flag, and JSON shape. -- [CLI recipes for agents](/agenteye/cli-recipes): copy-paste `jq` patterns and exit-code handling. -- [CLI agent skill](/agenteye/cli-skill): install and run the `agenteye-cli` skill. -- [AI assistant](/agenteye/assistant): the in-dashboard analyst that `agent ask` talks to. diff --git a/docs/agenteye/cli-skill.mdx b/docs/agenteye/cli-skill.mdx deleted file mode 100644 index 27baa29b..00000000 --- a/docs/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Failproof AI Observability CLI Agent Skill" -description: "Ask your coding agent \"is anything broken today?\" and let it answer from your live Failproof AI Observability data, with no commands to memorize." ---- - - -Ask your coding agent *"is anything broken today?"* and let it answer from your live Failproof AI Observability data, with no commands to memorize. The **Failproof AI Observability CLI skill** (`agenteye-cli`) is an *Agent Skill*: a small folder of instructions that a coding agent such as Claude Code or Codex loads on demand. It teaches the agent to operate your Observability deployment through the [`agenteye` CLI](/agenteye/cli) from plain-English requests like *"give CI a key that can only push events"* or *"ack the firing incident and assign it to me."* - -It is **not** a service or a separate binary; there is nothing to deploy. It rides on top of the CLI you have already installed: the agent shells out to `agenteye --json …`, parses the clean JSON, and answers you in prose. Everything it can do, you could do yourself by typing the same commands. - ---- - -## How it relates to the other Failproof AI Observability interfaces - -Failproof AI Observability gives you four ways to reach the same data and controls. They complement each other: - -| Interface | What it is | Where it runs | Reach for it when | -|---|---|---|---| -| **[CLI](/agenteye/cli)** | The command/flag reference for `agenteye` | Your terminal | You want to run or script a specific command | -| **[CLI recipes](/agenteye/cli-recipes)** | Copy-paste `jq`/pipeline patterns | Your terminal / scripts | You're wiring the CLI into automation | -| **CLI skill** (this doc) | A natural-language front door on the CLI | Your coding agent, on your workstation | You want to *just ask* and let the agent pick the command | -| **[Evaluator skill](/agenteye/evaluator-skill)** | A sibling skill that designs and builds your scoring service | Your coding agent, on your workstation | You want to *produce* eval scores rather than read them | -| **[Python SDK skill](/agenteye/python-sdk-skill)** | A sibling skill that instruments your agent so it emits telemetry at all | Your coding agent, on your workstation | You want your agent to *produce* the events this skill reads | -| **[In-dashboard AI assistant](/agenteye/assistant)** | A chat embedded in the dashboard | Server-side (in the dashboard) | You want in-dashboard Q&A over your data | - -The skill itself has no privileges of its own; it just turns your words into CLI calls that run as you: - -```mermaid -flowchart TD - YOU["you: 'ack the firing incident'"] --> AGENT["coding agent (Claude Code / Codex)
loads the agenteye-cli skill"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|your authenticated CLI session| API["Observability dashboard API"] -``` - -### vs. the in-dashboard AI assistant: an important distinction - -These are two different tools with very different blast radii: - -- The **in-dashboard AI assistant** ([AI assistant](/agenteye/assistant)) is a chat embedded in the dashboard, backed by the agent service. It is **read-only plus approval-gated authoring**: it can draft saved queries and dashboards, but every write pauses for your explicit click-approval, and it never deletes. It is gated by the `agent:use` permission and only ever sees data for the org you're viewing. -- The **CLI skill** runs on *your* workstation inside *your* coding agent and drives the `agenteye` CLI as **you**. It can perform the CLI's **full surface, including mutations** (create/rotate/disable API keys, change org settings, resolve incidents, delete saved queries), bounded only by the permissions of your CLI login. Treat it exactly as carefully as you would treat running those commands by hand. - ---- - -## Prerequisites - -1. The **`agenteye` CLI installed** and on `PATH` (see the [CLI](/agenteye/cli) reference: `pipx install agenteye`). -2. Your **dashboard URL** set (`AGENTEYE_DASHBOARD_URL`, or the agent passes `--base-url`). -3. A **logged-in session**: run `agenteye login` yourself first. The skill **cannot** complete the emailed one-time-code login for you; it will tell you to run `agenteye login` if the session is missing or expired (CLI exit code `4`). - ---- - -## Where to get it - -The skill is published in Failproof AI's public skills collection: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -Nothing about it is gated — the repository is public and the skill needs no credential of its own, because it only drives the **public** `agenteye` CLI against *your* dashboard, using the session *you* logged in with. You do not need to ask anyone for it. - -Note it ships as its own folder and is **not** inside the `pipx install agenteye` package, so don't look for it there. - -## Installing the skill - -The quickest path is the [`skills`](https://skills.sh) CLI, which fetches the folder and drops it where your agent looks: - -```bash -# Claude Code, this project only -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# every project (installs to ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex instead -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -Then manage it like any other skill: - -```bash -npx skills list -a claude-code # what's installed -npx skills update agenteye-cli # pull the latest version -npx skills remove agenteye-cli # remove it -``` - -Prefer to install by hand? An Agent Skill is just a folder containing a `SKILL.md` (plus optional references), so copying it works too: - -- **Claude Code**: put the `agenteye-cli/` folder in `~/.claude/skills/` (every project) or `/.claude/skills/` (that repo only). Claude Code auto-discovers it — verify with the `/skills` list, or simply ask a question that matches its description. -- **Codex (OpenAI)**: Codex reads the same `SKILL.md`. The bundled `agents/openai.yaml` sets `allow_implicit_invocation: true`, so Codex auto-selects the skill when a task matches; otherwise invoke it explicitly as `$agenteye-cli`. - ---- - -## Safety: mutations do NOT prompt when an agent runs the CLI - -> **Warning:** Read this before letting an agent make changes. - -The `agenteye` CLI normally asks *"are you sure?"* before a destructive action. It **auto-skips that confirmation whenever it is not attached to a terminal (which is exactly how a coding agent runs it), and `--json` skips it too.** So the safety prompt will **not** fire for the agent. - -The skill is written to compensate: it is instructed to state the exact command it will run and get your explicit **OK before any state change**. Keep that discipline. When you drive Failproof AI Observability through an agent, *you* are the confirmation step. The state-changing commands to watch for: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- the writing `incidents` subcommands: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -Everything under **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) is read-only and changes nothing. - -Because the agent acts as **you**, it can only do what your login is permitted to do; permissions are resolved **per org** (see [API keys](/agenteye/api-keys)). A command you lack permission for returns exit code `5` with the exact permission named, so the agent can tell you precisely what to ask an admin for rather than failing opaquely. - ---- - -## What you can ask it - -Here's what a real round-trip looks like, a read, then a change that waits for your OK: - -```text -you ▸ Is anything broken in the last day? - -agent ▸ Running: agenteye --json errors --since 24h --aggregate - 3 error types in the last 24h. The biggest is TimeoutError - (18 hits, all on the checkout-agent), then RateLimited (5). - Want the failing sessions for TimeoutError? - -you ▸ Give CI a key that can only push events. - -agent ▸ This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you ▸ yes - -agent ▸ Done. Key "ci" created with events:add only. - The secret is shown only once, so store it now. I can't reprint it. -``` - -The skill maps each plain-English intent to the right `agenteye` command, discovering valid values first (`list `, `whoami`) so it doesn't guess, and stating the exact command before any change. More examples: - -- *"Is anything broken / failing in the last 24 hours?"* → `errors --since 24h --aggregate`, then a breakdown. -- *"Why did session `run-001` fail?"* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *"How is quality trending this week?"* → `evals --aggregate --since 7d`, then drill into low-scoring runs. -- *"Give CI a key that can only push events."* → `keys create ci --add events:add` (it states the command, then creates it and captures the one-time secret). -- *"Who has access? Make Dana read-only."* → `users list` → `users update dana@… --permission-set read-only` (after confirming with you). -- *"Ack the firing incident and assign it to me."* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`. - -For the exact commands, flags, and JSON shapes behind these, see the [CLI](/agenteye/cli) reference and [CLI recipes for agents](/agenteye/cli-recipes). - ---- - -## Next steps - -- **[CLI](/agenteye/cli)**: full command and flag reference for `agenteye`. -- **[CLI recipes for agents](/agenteye/cli-recipes)**: copy-paste `jq` patterns and exit-code handling. -- **[Evaluator agent skill](/agenteye/evaluator-skill)**: the sibling skill, for building the evaluator whose scores `agenteye evals` reads. -- **[Python SDK agent skill](/agenteye/python-sdk-skill)**: the sibling skill, for instrumenting an agent so it emits the telemetry `agenteye` reads. -- **[AI assistant](/agenteye/assistant)**: the in-dashboard assistant (not to be confused with this terminal skill). -- **[API keys](/agenteye/api-keys)**: the per-org permission model that bounds what the skill can do. diff --git a/docs/agenteye/codex-capture.mdx b/docs/agenteye/codex-capture.mdx deleted file mode 100644 index c68b61a5..00000000 --- a/docs/agenteye/codex-capture.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "Codex session capture" -description: "Tail your team's local OpenAI Codex sessions into AgentEye as ordinary sessions and events — with no change to how they run Codex." ---- - -Your engineers already run OpenAI Codex every day. Codex session capture brings those coding sessions into AgentEye as ordinary sessions and events, so you can search, replay, and evaluate them next to everything else you observe. It complements the [Python SDK](/agenteye/python-sdk): the SDK instruments agents you write, while this captures the Codex work your team already does — with no change to how they run it. - -A small background collector reads Codex's local session transcripts as they are written and ships them to AgentEye. One collector per machine captures every local Codex surface at once — there is no per-surface setup. - -The same collector captures other agents too — see [OpenClaw](/agenteye/openclaw-capture) and [Hermes](/agenteye/hermes-capture). Enable each one you run; a single collector can capture several at once. - ---- - -## What it captures - -Every Codex surface that runs **locally** produces the same on-disk session transcripts, and the collector picks up all of them: - -- the Codex **CLI** and `codex exec` -- the **VS Code / IDE extension** -- the **desktop app**, when it runs a session locally - -Each Codex session becomes an AgentEye [session](/agenteye/sessions); its user and assistant messages, reasoning, tool calls, tool results, and token usage become the matching [events](/agenteye/event-stream). The surface each session came from (CLI, IDE, or desktop) is recorded, so you can tell them apart. - -> **Cloud sessions are not captured.** The desktop app increasingly runs sessions in the Codex cloud and keeps only their metadata on the machine — there is no local transcript to read. Only locally-executed sessions are captured. - ---- - -## Turn it on - -Capture is off until you enable it. Install the collector with an API key that has the `events:add` permission (see [API keys](/agenteye/api-keys)), and turn on Codex capture: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -That installs the collector, registers it as a background service, and starts capturing. Confirm it is running: - -```bash -agenteye-collector health -``` - -On first run, your existing Codex sessions are backfilled once and new activity then streams within seconds. Codex's own files are only ever read — never modified, moved, or deleted — and each session is shipped exactly once, even across restarts. - ---- - -## Where it shows up - -Captured sessions appear in **Sessions**, and their events in the **Events** stream, the same as any other agent you observe — so [session replay](/agenteye/sessions), [search](/agenteye/queries), [evaluations](/agenteye/evaluations), and [alerts](/agenteye/alerts) all work on them. Filter by the Codex agent to see them on their own. - ---- - -## Privacy - -Codex transcripts contain the full session — including command output, file contents, and anything Codex read or wrote — and can contain secrets. Captured sessions are shipped as-is, so enable capture only on machines and for teams where centralizing that content in AgentEye is appropriate, and give the collector a key scoped to `events:add` only. See [Security](/agenteye/security) for how your data is kept isolated. diff --git a/docs/agenteye/concepts.mdx b/docs/agenteye/concepts.mdx deleted file mode 100644 index c62d46b0..00000000 --- a/docs/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "Concepts" -description: "The vocabulary behind Failproof AI Observability — events, sessions, evaluations, audits, findings, and incidents — defined in one place." ---- - - -This page defines the vocabulary Failproof AI Observability uses. If a term in another guide is unfamiliar, it's defined here. You don't need to read it end to end: skim it, or jump back when you hit a word you want pinned down. - ---- - -## The data model - -**Event** -The smallest unit of data. One event records a single step your agent took: a `tool_use`, a `model_request`, a `hook_completed`, an `error`, and so on. Your agent emits events through the [Python SDK](/agenteye/python-sdk); they show up live on the **Events** page. - -**Session** -One agent run, identified by a `session_id`. A session is all the events that share that id, rolled up into a single row on the **Sessions** page and drawn as an execution graph on its detail page. A session usually starts with `agent_start` and ends with `agent_end`. - -**Agent** -A named actor inside a run, identified by an `agent_id`. A run can involve several agents: a planner that spawns a summarizer sub-agent, for example. Sub-agents carry a `parent_id`, which is what lets Failproof AI Observability draw them on their own lanes in the execution graph. - -**Environment** -A label for where the run happened: `production`, `staging`, `dev`. You set it once when you configure the SDK. Almost every dashboard page can filter by environment. - -**Context-window fill** -The percentage of a model's context window a response consumed. Failproof AI Observability stamps it on `model_response` events for models it recognizes, so prompt growth and impending compaction are visible right in the event stream. - ---- - -## Quality - -**Evaluation** -A quality score for a finished session, produced by a scoring service you run. Evaluations are opt-in: until you connect an evaluator, sessions are recorded but not scored. Each evaluation can carry several named scores (for example `helpfulness`, `factuality`, `tool_efficiency`), each with a short reasoning note. See [Evaluation suite](/agenteye/evaluation-suite). - -**Score key** -The name of one dimension an evaluator reports, such as `helpfulness`. Alerts and audits can watch a specific score key over time. - -**Evaluator** -Your scoring service. Failproof AI Observability POSTs a finished run's transcript to it and stores the scores it returns. It does not ship a default evaluator; the scoring logic is yours. - ---- - -## Finding and fixing failures - -**Hook** -A guardrail or side-effect your agent framework runs around a step: a content-safety check, PII redaction, a budget guard. Hooks emit `hook_triggered` / `hook_completed` events with an `outcome` (allow, deny, modify), and get their own observe page. - -**Alert rule** -A rule that fires when a metric crosses a threshold you set: error rate, p95 latency, token cost, or an evaluator score. When a rule fires, it opens an incident and notifies your chosen channels (email, Slack, webhook, in-dashboard). See [Alerts](/agenteye/alerts). - -**Incident** -An open issue created when an alert rule fires. Incidents have a lifecycle (acknowledge, assign, resolve) and an activity timeline that records every action. You can also open one manually. - -**Audit** -A recurring investigation (hourly to weekly) that mines your logs *across* sessions for failure patterns you haven't written a rule for: error clusters, low scores, latency outliers, tool-call loops, and runs that never finished. Where an alert watches a metric you already know about, an audit tells you what to look at next. See [Audits](/agenteye/audits). - -**Finding** -One ranked, evidence-backed result from an audit run. A finding names a pattern, links to the exact sessions behind it, and carries a triage lifecycle (acknowledge, resolve, mute, dismiss). Failproof AI Observability deduplicates findings run-over-run so a known pattern updates instead of piling up. - -**The AI assistant** -The in-dashboard chat that answers questions about your agents in plain English, over your own data. It is read-only by default; anything it creates (a saved query, a dashboard) is approval-gated, and it can never delete. See [AI assistant](/agenteye/assistant). - ---- - -## Running it - -**Organization (tenant)** -An isolated workspace. One Failproof AI Observability instance can host many organizations, each with its own users, keys, and data. Every dashboard URL is scoped under your org slug (`//…`). - -**Collector** -`agenteye-collector`, the lightweight daemon that runs on each agent machine, batches the events the SDK writes to disk, and ships them to the server. - -**API key** -A scoped token that authenticates a client against the server. Keys carry granular permissions (for example `events:add` for the collector, read-only scopes for a dashboard key). See [API keys](/agenteye/api-keys). - -**Server** -The ingest and API service. It ingests events, stores operational state in your databases, and serves the dashboard and CLI. - -**Dashboard** -The web UI. Every page is scoped to an organization and reads through the server's API. - ---- - -## Next steps - -- [Overview](/agenteye/overview): how these pieces fit together. -- [Observability](/agenteye/observability): the observe surfaces (Events, Sessions, Models, Tools, Hooks, Errors). diff --git a/docs/agenteye/evaluator-skill.mdx b/docs/agenteye/evaluator-skill.mdx deleted file mode 100644 index 4b4bd8f5..00000000 --- a/docs/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: "Failproof AI Observability Evaluator Agent Skill" -description: "Go from \"I think our agent is sometimes bad\" to a deployed scoring service, with your coding agent doing both the deciding and the building." ---- - - -Go from *"I think our agent is sometimes bad"* to a deployed scoring service, with your coding agent doing both the deciding and the building. The **Failproof AI Observability evaluator skill** (`agenteye-evaluator`) is an *Agent Skill*: a small folder of instructions that a coding agent such as Claude Code or Codex loads on demand. It teaches the agent to work out which quality dimensions are worth tracking for *your* agent, then write, test, and deploy the [evaluator service](/agenteye/evaluation-suite) that scores them. - -It is **not** a hosted scorer, a registry you upload to, or a plugin system. Your evaluator stays your own HTTP service on your own infrastructure, exactly as described in the [Evaluation suite](/agenteye/evaluation-suite) guide. The skill only teaches your agent to build it well, so everything it does, you could do yourself by writing the same code. - ---- - -## The hard part is deciding what to score - -The SDK surface is small — a decorator and two models — and an agent can write that from the [contract](/agenteye/evaluation-suite#http-contract) alone. That's not where evaluators fail. They fail because they score the wrong thing, and an evaluator that scores the wrong thing is worse than none: it produces a dashboard everyone learns to ignore. - -So most of the skill is the part before any code exists. It has the agent interview you (*"describe a run that went well; now one that went badly"*), then pull your real sessions through the [`agenteye` CLI](/agenteye/cli) and read them end to end. Those two halves usually disagree, and the gap is the point: what you intend to measure versus what your transcripts can actually support. A dimension only survives if it is **computable** from the events and **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches nothing and gets cut. - -What comes back is a proposal of 2-4 dimensions with the reasoning attached, for you to sign off on before a line is written. - -```mermaid -flowchart TD - YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent (Claude Code / Codex)
loads the agenteye-evaluator skill"] - AGENT -->|"interview: what does good vs bad look like?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["your real sessions
what actually happens"] - DATA --> DIMS["2-4 dimensions, you sign off"] - DIMS --> SVC["your evaluator service
agenteye-evaluator SDK"] - SVC --> SCORES["scores land in the dashboard
and agenteye evals"] -``` - ---- - -## How it relates to the other evaluation pieces - -Four docs cover scoring, and they hand off to each other in order: - -| Page | What it is | Reach for it when | -|---|---|---| -| **[Evaluations](/agenteye/evaluations)** | The feature: scores on the sessions grid, dashboards, re-evaluate | You want to know what automatic scoring gets you | -| **[Evaluation suite](/agenteye/evaluation-suite)** | The HTTP contract, the SDK, the server env vars | You're implementing or debugging the evaluator yourself | -| **Evaluator skill** (this doc) | A natural-language front door on designing *and* building the scorer | You want to go from "I want evals" to a running service | -| **[CLI skill](/agenteye/cli-skill)** | A natural-language front door on the `agenteye` CLI | You want to *read* the scores you already have | -| **[Python SDK skill](/agenteye/python-sdk-skill)** | A natural-language front door on instrumenting your agent | Your agent isn't emitting sessions yet — there is nothing to score | - -### vs. the CLI skill: build versus read - -The two skills are deliberately non-overlapping, and installing both is the normal setup — the agent picks between them based on what you ask: - -- **`agenteye-evaluator`** (this doc) builds the thing that *produces* scores. Its job ends when scores land for the first time. -- **[`agenteye-cli`](/agenteye/cli-skill)** reads scores that already exist (`agenteye evals`). *"Did quality drop this week?"* is its question, not this skill's. - ---- - -## Prerequisites - -1. The **`agenteye` CLI installed and logged in** (`pipx install agenteye`, then `agenteye login`). The skill leans on it twice: to pull the real sessions it designs against, and to confirm your scores landed at the end. Your login needs `events:read`, plus `evaluations:read` for that final check. As with the CLI skill, it **cannot** complete the emailed one-time-code login for you. -2. **Somewhere for the evaluator to live.** It gets built into an image and run as a long-running service, so it needs a real repo, not a scratch file. Evaluators often live in their own repo, separate from the agent being scored — the skill looks for an existing one and asks before scaffolding a new one. -3. **The `agenteye-evaluator` SDK wheel** — read the next section before your agent starts typing `pip` commands. - ---- - -## Where to get it - -The skill is published in Failproof AI's public skills collection: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -The repository is public and the skill needs no credential of its own — it only drives the `agenteye` CLI with the session *you* logged in with, and writes code in *your* repo. Note it ships as its own folder and is **not** inside the `pipx install agenteye` package, so don't look for it there. - -## Installing the skill - -The quickest path is the [`skills`](https://skills.sh) CLI, which fetches the folder and drops it where your agent looks: - -```bash -# Claude Code, this project only -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# every project (installs to ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codex instead -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -Then manage it like any other skill: - -```bash -npx skills list -a claude-code # what's installed -npx skills update agenteye-evaluator # pull the latest version -npx skills remove agenteye-evaluator # remove it -``` - -Prefer to install by hand? An Agent Skill is just a folder containing a `SKILL.md` (plus optional references), so copying it works too: - -- **Claude Code**: put the `agenteye-evaluator/` folder in `~/.claude/skills/` (every project) or `/.claude/skills/` (that repo only). Claude Code auto-discovers it — verify with the `/skills` list, or just ask for evals. -- **Codex (OpenAI)**: Codex reads the same `SKILL.md`. The bundled `agents/openai.yaml` sets `allow_implicit_invocation: true`, so Codex auto-selects the skill when a task matches; otherwise invoke it explicitly as `$agenteye-evaluator`. - ---- - -## The SDK is not on public PyPI - -> **Warning:** Read this before letting an agent install the SDK. - -The skill is public; the SDK it drives is not. `agenteye-evaluator` ships only as a private release artifact, and unlike `agenteye`, the name is **unclaimed on public PyPI** — so a bare `pip install agenteye-evaluator` could pull a stranger's package into the service that reads your production transcripts. That's a supply-chain problem, not a typo. - -The skill knows this and works down an install ladder instead, stopping at the first rung that applies: the monorepo source if you're inside the AgentEye repo, otherwise the private release wheel from GitHub Releases (needs access), and if neither is reachable it **stops and tells you to ask your Failproof AI contact for the wheel** rather than improvising. - -So if your agent proposes a bare `pip install agenteye-evaluator` from public PyPI, that is the tell that the skill never loaded. Stop there and check it's installed. - ---- - -## What you can ask it - -A real round-trip starts with a vague ask and ends with a signed-off design, not with code: - -```text -you ▸ I want evals for our support bot. I think it's sometimes bad. - -agent ▸ Before I write anything: describe a run that went well, - and one that went badly. - -you ▸ Good: it answers from the docs. Bad: it makes stuff up, - or gives up and says "contact support". - -agent ▸ Running: agenteye --json sessions --status error --since 7d - Reading 5 sessions, including two you'd call bad. - - Your words say factuality and deflection. The transcripts - agree on deflection (2 of 5 end with a support hand-off, - and it's plain in model_response.content), but factuality - needs a ground truth your events don't carry. - - So I'd score three things: - - deflection rules, cheap, separates your two runs - - tool_efficiency rules, catches the retry loop in run-004 - - groundedness LLM judge: is every claim in the docs it retrieved? - - "factuality" would need ground truth we don't have; groundedness - is the closest thing your transcripts can actually support. - OK with those three names? Renaming later splits the history. - -you ▸ yes - -agent ▸ No evaluator in this repo. Should I scaffold one here, or do - you have one elsewhere? -``` - -From there it writes the rules-based dimensions first (free, instant, deterministic), tests them against a real captured session including the empty and never-finished ones that crash naive evaluators, and only reaches for an LLM judge on the subjective dimension. It knows the [dispatcher's limits](/agenteye/evaluation-suite#configuring-the-server) — a 30s request timeout and 8 concurrent calls deployment-wide — so if the judge won't reliably fit, it goes async with `JobPending` rather than letting your judge get cancelled and retried five times at five times the cost. - -Then it deploys, sets the two server env vars, and confirms with `agenteye --json evals --session-id ` that scores actually landed. Scores landing is the only proof. - ---- - -## What to watch for - -- **Dimension names are close to permanent.** Score keys are arbitrary strings and the platform trends whatever you send, which means nothing downstream corrects a bad choice. Rename later and the history splits: old sessions keep the old key and the trend breaks. This is why the skill gets explicit sign-off before writing code — take that prompt seriously. -- **Fixtures are real production transcripts.** Designing against real sessions means pulling them to disk, and they can contain customer data. The skill asks before committing them to git; if in doubt, keep `fixtures/` out of the repo and have each developer pull their own. -- **The agent writes and deploys a service that reads every transcript.** It acts as you, bounded by your CLI login's permissions, but review the evaluator like any other code that touches production data. - ---- - -## Next steps - -- **[Evaluation suite](/agenteye/evaluation-suite)**: the HTTP contract, the SDK, and the server env vars the skill configures. -- **[Evaluations](/agenteye/evaluations)**: where the scores show up once they land. -- **[CLI skill](/agenteye/cli-skill)**: the sibling skill, for reading results rather than building the scorer. -- **[CLI](/agenteye/cli)**: the command reference behind the session data the skill designs against. diff --git a/docs/agenteye/hermes-capture.mdx b/docs/agenteye/hermes-capture.mdx deleted file mode 100644 index a6d00220..00000000 --- a/docs/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Hermes session capture" -description: "Bring your team's Hermes gateway sessions — Slack, Telegram, CLI, and scheduled runs — into AgentEye as ordinary sessions and events." ---- - -[Hermes](https://hermes-agent.nousresearch.com) answers your team from wherever they already work — Slack, Telegram, the CLI, scheduled runs. Hermes session capture brings all of it into AgentEye as ordinary sessions and events, so the assistant your team talks to every day is as observable as the agents you write yourself. - -A small background collector reads Hermes's local session store as it is written and ships sessions to AgentEye. It works the same way as [Codex](/agenteye/codex-capture) and [OpenClaw](/agenteye/openclaw-capture) capture, and one collector can capture several at once. - ---- - -## What it captures - -Every Hermes session on the machine is captured, whichever channel it came from. Each one becomes an AgentEye [session](/agenteye/sessions); its user and assistant messages, tool calls, and tool results become the matching [events](/agenteye/event-stream). - -The channel a session started from — Slack, Telegram, CLI, or a scheduled run — is recorded on the session, so you can tell them apart and filter to one at a time. Alongside it come the model the session ran on, the chat and person it was started from, and, when a session spawned another, the link back to its parent. - -Sessions appear as soon as Hermes starts them, whether or not anything has been said yet, and a turn's reply and its tool calls stay in the order they actually happened. When a session ends you also get why it ended, what it cost, and how many tokens it used. - ---- - -## Turn it on - -Capture is off until you enable it. Install the collector with an API key that has the `events:add` permission (see [API keys](/agenteye/api-keys)), and turn on Hermes capture: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -That installs the collector, registers it as a background service, and starts capturing. Confirm it is running: - -```bash -agenteye-collector health -``` - -Capturing more than one agent on the same machine? Add each one's flag to the same command — for example `--hermes-enabled --codex-enabled`. - -On first run, your existing Hermes sessions are backfilled once and new activity then streams within seconds. Hermes's own data is only ever read — never modified or deleted — and each message is shipped once, even across restarts. - -`health` also tells you whether everything the collector captured actually reached AgentEye. If a batch could not be delivered it is kept and retried rather than discarded, and the check reports unhealthy while anything is still outstanding — so "healthy" means your data arrived, not merely that the process is alive. - ---- - -## Where it shows up - -Captured sessions appear in **Sessions**, and their events in the **Events** stream, the same as any other agent you observe — so [session replay](/agenteye/sessions), [search](/agenteye/queries), [evaluations](/agenteye/evaluations), and [alerts](/agenteye/alerts) all work on them. Filter by the Hermes agent to see them on their own. - ---- - -## Privacy - -Hermes sessions contain the full transcript — including command output, file contents, and anything the agent read or wrote — and can contain secrets. Captured sessions are shipped as-is, so enable capture only where centralizing that content in AgentEye is appropriate, and give the collector a key scoped to `events:add` only. See [Security](/agenteye/security) for how your data is kept isolated. diff --git a/docs/agenteye/observability.mdx b/docs/agenteye/observability.mdx deleted file mode 100644 index 5581e00c..00000000 --- a/docs/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Observe" -description: "The observe surfaces are where you watch what your agents are doing right now and drill into any single run." ---- - - -The observe surfaces are where you watch what your agents are doing right now and drill into any single run. Everything here is live, scoped to your organization, and filterable by date range, environment, agent, and session, so you go from "something feels off" to the exact run in seconds. - -![The live Event Stream, colour-coded by type and filterable by environment, agent, and session](/agenteye/images/events-stream.png) - -Four surfaces, each with its own page: - -- **[Event stream](/agenteye/event-stream)**: the live, per-step trail of every run across every agent, newest first. Your org home and first stop for triage. -- **[Sessions and execution graph](/agenteye/sessions)**: those events rolled up into one row per run, plus a git-style picture of how each run unfolded. -- **[Performance metrics](/agenteye/telemetry)**: latency heat-maps and p50/p95/p99 vitals for your models, tools, and hooks, so a tail spike stands out from the median. -- **[Error tracking](/agenteye/error-tracking)**: one triage surface for everything that went wrong, one click from a firing alert to the run that broke. - -## Related - -- [Evaluations](/agenteye/evaluations): score every run for quality. -- [Alerts](/agenteye/alerts): turn any threshold into a paging rule. -- [Audits](/agenteye/audits): let Failproof AI Observability find failure patterns across sessions for you. -- [CLI and agents](/agenteye/cli-and-agents): the same observability from your terminal. diff --git a/docs/agenteye/openclaw-capture.mdx b/docs/agenteye/openclaw-capture.mdx deleted file mode 100644 index 7a076268..00000000 --- a/docs/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "OpenClaw session capture" -description: "Tail your team's local OpenClaw sessions into AgentEye as ordinary sessions and events — with no change to how OpenClaw runs." ---- - -If your team runs [OpenClaw](https://docs.openclaw.ai), OpenClaw session capture brings those sessions into AgentEye as ordinary sessions and events, so you can search, replay, and evaluate them next to everything else you observe. It complements the [Python SDK](/agenteye/python-sdk): the SDK instruments agents you write, while this captures the OpenClaw work your team already does — with no change to how they run it. - -A small background collector reads OpenClaw's local session transcripts as they are written and ships them to AgentEye. It works the same way as [Codex capture](/agenteye/codex-capture), and one collector can capture both at once. - ---- - -## What it captures - -Every agent configured in a machine's OpenClaw setup is captured by that machine's collector — there is no per-agent setup. - -Each OpenClaw session becomes an AgentEye [session](/agenteye/sessions); its user and assistant messages, tool calls, and tool results become the matching [events](/agenteye/event-stream). - ---- - -## Turn it on - -Capture is off until you enable it. Install the collector with an API key that has the `events:add` permission (see [API keys](/agenteye/api-keys)), and turn on OpenClaw capture: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -That installs the collector, registers it as a background service, and starts capturing. Confirm it is running: - -```bash -agenteye-collector health -``` - -Capturing more than one agent on the same machine? Add each one's flag to the same command — for example `--openclaw-enabled --codex-enabled`. - -On first run, your existing OpenClaw sessions are backfilled once and new activity then streams within seconds. OpenClaw's own files are only ever read — never modified, moved, or deleted — and each session is shipped exactly once, even across restarts. - ---- - -## Where it shows up - -Captured sessions appear in **Sessions**, and their events in the **Events** stream, the same as any other agent you observe — so [session replay](/agenteye/sessions), [search](/agenteye/queries), [evaluations](/agenteye/evaluations), and [alerts](/agenteye/alerts) all work on them. Filter by the OpenClaw agent to see them on their own. - ---- - -## Privacy - -OpenClaw transcripts contain the full session — including command output, file contents, and anything the agent read or wrote — and can contain secrets. Captured sessions are shipped as-is, so enable capture only on machines and for teams where centralizing that content in AgentEye is appropriate, and give the collector a key scoped to `events:add` only. See [Security](/agenteye/security) for how your data is kept isolated. diff --git a/docs/agenteye/overview.mdx b/docs/agenteye/overview.mdx deleted file mode 100644 index c979f1d9..00000000 --- a/docs/agenteye/overview.mdx +++ /dev/null @@ -1,108 +0,0 @@ ---- -title: "Failproof AI: Observe Agents for Failures" -description: "Failproof AI Observability is a self-hosted platform for observing, evaluating, and improving your AI agents in production." ---- - - -Failproof AI Observability is a self-hosted platform for observing, evaluating, and improving your AI agents in production. It records everything your agents do (every tool call, model request, hook, and error), scores the quality of each run, and surfaces the failures you didn't know to look for, all in a dashboard you run inside your own infrastructure. - -If you ship AI agents and you're tired of guessing why a run went wrong, this is the page to start on. It explains what Failproof AI Observability gives you and how the pieces fit together, before you install anything. - -> **Failproof AI Observability is an enterprise product from Failproof AI.** Want to see it in action? Request a demo: email [nikita@befailproof.ai](mailto:nikita@befailproof.ai). - -![A Failproof AI Observability session drawn as a git-style execution graph beside its event timeline, with a per-run breakdown of tools, models, and hooks in the right rail](/agenteye/images/session-detail.png) - -*Every agent run is drawn as a git-style execution graph (left) beside its event timeline. Parallel sub-agents each get their own lane; the right rail breaks down the tools, models, hooks, and token spend for the run.* - ---- - -## See it in action - -Two short videos show the two things teams reach for first: tracing a run, and finding failures automatically. - -
- -
- -*Agent tracing: follow a single run step by step, from goal to tools to final answer.* - -
- -
- -*Failproof Audit: let Failproof AI Observability mine your logs across sessions and tell you what to fix.* - ---- - -## Why teams use it - -- **See what your agent actually did.** Every run becomes a readable, git-style execution graph: which tools ran in parallel, which sub-agents branched off, where it stalled, and what it spent. -- **Catch quality regressions automatically.** Connect a small scoring service and Failproof AI Observability scores every finished run, so a drop in helpfulness or a spike in hallucinations shows up on its own. -- **Find failures you didn't write a rule for.** Recurring audits mine your logs across sessions for error clusters, latency outliers, low scores, and stuck runs, then hand you ranked, evidence-backed findings. -- **Get paged when it matters.** Threshold rules fire on error rate, latency, cost, or evaluator scores and open incidents you can acknowledge, assign, and resolve. -- **Ask questions in plain English.** An in-dashboard AI assistant answers "how is quality trending in prod this week?" over your own data. Any change it makes is approval-gated. -- **Keep your data.** Failproof AI Observability is self-hosted: events, prompts, and analytics stay in infrastructure you control. - ---- - -## What you get - -Failproof AI Observability is organized around three ideas (**observe**, **analyze**, and **admin**), mirrored in the dashboard's left sidebar. - -**Observe** (the raw truth of what happened): - -- **[Event stream](/agenteye/event-stream)**: the live, per-step trail of every run (tool calls, model calls, hooks, errors). -- **[Sessions](/agenteye/sessions)**: those events rolled up into one row per run, each ready to be scored, with a git-style execution graph. -- **[Performance metrics](/agenteye/telemetry)**: per-surface latency heat-maps and p50/p95/p99 vitals for models, tools, and hooks, so a tail spike stands out from the median. -- **[Error tracking](/agenteye/error-tracking)**: one triage surface for everything that went wrong, one click from a firing alert. - -![The Tools observe page: a latency heat-map, a percentile band, and a tool-distribution bar over 24 time bins](/agenteye/images/tools.png) - -*Each observe surface pairs a sparkline and p50/p95/p99 vitals with a latency heat-map and a percentile band. Shown here: Tools.* - -**Analyze** (turn activity into answers): - -- **[Queries](/agenteye/queries)** and **[dashboards](/agenteye/dashboards)**: saved SQL over your events and evaluations, charted into shared, org-scoped dashboards. -- **[Evaluations](/agenteye/evaluations)**: quality scores produced by your own evaluator service, with per-score reasoning. -- **[Audits](/agenteye/audits)**: recurring investigations that surface failure patterns across sessions. -- **[Alerts](/agenteye/alerts)** and **[incidents](/agenteye/incidents)**: threshold rules that page you, plus an incident workflow to triage them. - -**Interfaces** (reach your data your way): - -- **[CLI](/agenteye/cli-and-agents)**: drive your whole deployment from the terminal or a script, and let a coding agent do it for you in plain English. -- **[AI assistant](/agenteye/assistant)**: ask questions about your agents in plain English, right inside the dashboard. -- **REST API**: everything the dashboard and CLI do is backed by a REST API you can call directly with a scoped [API key](/agenteye/api-keys) — ingest events, query sessions and evaluations, and manage dashboards, alerts, audits, users, and keys, so you can wire Failproof AI Observability into your own tooling. - -**Admin** (run it for your team): - -- **[API keys](/agenteye/api-keys)**: scoped tokens for the collector, the dashboard, and the assistant. -- **Users**: passwordless, email-based sign-in with an allowlist. -- **Settings**: per-org configuration, including model context-window overrides. - ---- - -## How the pieces fit - -Data flows in one direction, from your agent code to the dashboard: your agent (via the Python SDK) emits events to the agenteye-collector, which ships them to the server, which serves the dashboard. Two optional services round it out — a scoring service (evaluations) and an AI assistant service (the in-dashboard chat). - -- **Python SDK**: you add a few `agenteye.event.*` calls to your agent; events are buffered locally. -- **agenteye-collector**: a lightweight daemon on each agent machine that batches events and ships them to the server. -- **Server**: ingests your events, keeps operational state in your own databases, and serves the REST API that the dashboard, CLI, and your own integrations all use. -- **Dashboard**: where you explore everything. -- **Optional services**: a scoring service (evaluations), and an AI assistant service (the in-dashboard chat). - -For the vocabulary used throughout the docs (*event, session, evaluation, audit, finding, incident*), see [Concepts](/agenteye/concepts). - ---- - -## Getting Failproof AI Observability - -Failproof AI Observability is an enterprise product from Failproof AI, and it works alongside Failproof AI Enforcement — the policy and guardrail product — under the Failproof AI brand. It runs entirely in your own environment. If you don't have access to the packages yet, request a demo and we'll get you set up: email [nikita@befailproof.ai](mailto:nikita@befailproof.ai). - ---- - -## Next steps - -- [Concepts](/agenteye/concepts): the Failproof AI Observability vocabulary in one place. -- [Observability](/agenteye/observability): follow what your agents do, run by run. -- [Security](/agenteye/security): how Failproof AI Observability keeps your data isolated and in your control. diff --git a/docs/agenteye/python-sdk-skill.mdx b/docs/agenteye/python-sdk-skill.mdx deleted file mode 100644 index 8d102cfb..00000000 --- a/docs/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,131 +0,0 @@ ---- -title: "Failproof AI Observability Python SDK Agent Skill" -description: "Go from an uninstrumented agent to events you can see, with your coding agent finding the instrumentation points, writing them, and proving they landed." ---- - -Tell your coding agent *"add Failproof AI Observability to this agent"* and let it read your loop, work out where the instrumentation belongs, write it, and verify the events before it calls the job done. - -The **Python SDK skill** (`agenteye-python-sdk`) is an *Agent Skill*: a folder of instructions that a coding agent such as Claude Code or Codex loads on demand when a task matches it. It teaches the agent to use the [Python SDK](/agenteye/python-sdk) — it is not a library, and it changes nothing about how the SDK works. - -## Instrumentation is easy to write and easy to get quietly wrong - -The SDK is small: thirteen event methods, all keyword-only. A coding agent can read the [Python SDK](/agenteye/python-sdk) reference and produce plausible instrumentation in a minute. - -The catch is that this SDK does not raise when you get it wrong, and wrong instrumentation looks exactly like right instrumentation until someone opens a dashboard and finds it empty. The mistakes that cost real time are all silences: - -| The mistake | What you see | -|---|---| -| No `agent_start` | Every event lands. Zero sessions. | -| Environment never set | Everything works, filed under `dev`. | -| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | -| A typo'd field name | Accepted and stored as a new field. | -| Events emitted from a thread pool | Silently dropped. | - -None of these raise. None show up in tests. Every one is in the skill, stated as a contract with the check that catches it. - -## What it does, in order - -The skill runs the same three steps a careful engineer would: - -1. **Plan.** It reads your agent loop and asks the two questions only you can answer: what counts as one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). It gets those agreed before writing code, because changing them later splits your history and breaks the trends. -2. **Write.** It binds identity once per run rather than threading it through every call site, and it picks a concurrency-safe shape — a detail that matters, because the obvious shortcut silently mixes two overlapping runs into one session. -3. **Verify.** It runs your agent and reads the resulting event files, checking that `agent_start` is present, the environment is right, and one run produced one session. - -That third step is the one people skip. The SDK writes events to local files, so a complete integration can be proven on a laptop with no server, no API key, and no network — which is exactly why the skill insists on doing it. - -## How it relates to the other skills - -Three skills, one clean split: - -| Skill | Reach for it when | What it touches | -|---|---|---| -| **Python SDK skill** (this page) | You want your agent to *emit* telemetry — "add observability", "why isn't my agent showing up?" | Writes code in your agent's repo. Reads nothing. | -| **[Evaluator skill](/agenteye/evaluator-skill)** | You want to *score* runs — "what should we even measure?" | Writes code in your repo; reads telemetry | -| **[CLI skill](/agenteye/cli-skill)** | You want to *read* what happened, or operate your deployment | Drives the CLI as you, including changes | - -They hand off in that order: this skill gets events flowing, the evaluator scores them, the CLI reads them back. There is nothing to evaluate and nothing to read until your agent emits sessions, so if you are starting from scratch, start here. - -## Prerequisites - -1. **Python 3.10+** and the agent codebase you want to instrument. -2. **The SDK.** It is distributed to customers as a private wheel rather than from a public index — your onboarding covers how to get it and install it. The skill knows the install path and will ask you rather than guess if it cannot find it. -3. **Nothing else.** No dashboard login, no API key, no network. The skill verifies against the event files the SDK writes, so it can finish and prove its work offline. - -## Where to get it - -The skill lives in the public [`FailproofAI/skills`](https://github.com/FailproofAI/skills) collection: - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -Add `-g` to install it for every project instead of just the current one, and `--copy` if your environment does not follow symlinks. For Codex, pass `-a codex`. - -## Installing it by hand - -Agent Skills are folders containing a `SKILL.md` plus references. If you would rather not use the installer: - -- **Claude Code**: copy the `agenteye-python-sdk/` folder into `~/.claude/skills/` (every project) or `/.claude/skills/` (that repo only). Claude Code discovers it automatically — check the `/skills` list, or just ask something that matches it. -- **Codex**: Codex reads the same `SKILL.md`. The bundled `agents/openai.yaml` sets `allow_implicit_invocation: true`, so it is auto-selected when a task matches; otherwise invoke it as `$agenteye-python-sdk`. - -Run your agent **in the repository holding the code you want instrumented** — the skill reads your agent loop before it proposes anything. - -## What a session looks like - -```text -you ▸ Add Failproof AI Observability to this agent. - -agent ▸ Read your loop — one tool dispatcher in tools.py, one Anthropic - wrapper in llm.py. Two questions before I write anything. - - What counts as one run? You handle requests concurrently, so I'd - use the request id you already have in ctx.request_id — then a - session here and a request in your logs are the same string. - - And who are the actors? I see a planner path and a researcher - path. Label them separately, or is that one agent to you? - -you ▸ Request id is right. They're one agent, call it "assistant". - -agent ▸ Done. Wrapped the dispatcher and the LLM client; agent_start and - agent_end bracket handle_request, including the error path. Used - a context-local for identity — your dispatcher runs tools in a - thread pool, and the simpler approach would have mixed concurrent - runs into one session without complaining. - - Verified against a real run: 11 events across 4 files — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, one session - per request, durations paired. - - One thing I did NOT do: your worker pool in batch.py submits - straight to the executor, so events from there would be dropped. - Want me to fix those too? -``` - -The pattern to notice: it read the code before proposing, asked only the questions you can answer, reused an id you already had, chose the concurrency-safe shape *because* it saw a thread pool, and **verified by reading the actual events** rather than declaring success — then flagged the one place it knew would fail quietly. - -## What you can ask it - -- *"Why isn't my agent showing up on the dashboard?"* → walks the ladder: are events being written, is `agent_start` there, is the environment right, is the collector reading the same place. -- *"Everything's landing under dev."* → the environment was never set, or was reset by a later call. -- *"Add token tracking."* → finds your LLM wrapper and records model, stop reason, and usage. -- *"Instrument the sub-agents too."* → one session, distinct agent labels, nested under their parent. -- *"Write tests for the instrumentation."* → points the SDK at a temporary directory and asserts on the events it wrote. - -## What to watch for - -**Let it verify.** The step that makes this skill worth using is the last one — running your agent and reading the events back. An agent that writes instrumentation and stops has done the easy half, and the half that fails silently is the other one. - -**Agree the names before the code.** `session_id` and `agent_id` are the axes every surface groups by. Renaming them later splits the history: old runs keep the old labels and your trends break. The skill will ask; the answer is worth a minute's thought. - -**If your agent proposes installing the SDK from a public index, the skill did not load.** The SDK is distributed privately. That proposal is a reliable tell that your coding agent is guessing rather than following the skill — stop it there and check the skill is installed. - -Beyond that its blast radius is small: it writes code in your working directory and event files where you tell it. It reads nothing from your deployment and changes nothing about it. - -## Next steps - -- **[Python SDK](/agenteye/python-sdk)**: the complete event reference — every event type and field — behind what this skill automates. -- **[Sessions](/agenteye/sessions)**: what your instrumentation produces once events land. -- **[Evaluator Agent Skill](/agenteye/evaluator-skill)**: the next step once runs are landing — scoring them. -- **[CLI Agent Skill](/agenteye/cli-skill)**: reading your telemetry back. diff --git a/docs/agenteye/security.mdx b/docs/agenteye/security.mdx deleted file mode 100644 index 65380357..00000000 --- a/docs/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "Security" -description: "Failproof AI Observability is built to sit close to your production agents, which means it sees your prompts, tool inputs, and outputs." ---- - - -Failproof AI Observability is built to sit close to your production agents, which means it sees your prompts, tool inputs, and outputs. This page explains how it keeps that data isolated, controlled, and in your hands. If you're evaluating Failproof AI Observability for a security review, start here. - ---- - -## Your data stays in your environment - -Failproof AI Observability is self-hosted. Events, prompts, model responses, and analytics are stored in your own databases, in your own environment. Nothing is sent to a third-party SaaS for storage, and your data stays in your own cloud account. - ---- - -## Tenant isolation - -One Failproof AI Observability instance can host many organizations, and each is isolated at the storage layer — enforced by the database, not just the UI: - -- An organization's operational data (users, keys, dashboards, saved queries) is scoped to that org, and cross-org reads are blocked by the database itself. -- Every ingested event is stamped with its owning org, so one organization's events can never be read by another. - -Every dashboard route is scoped under an org slug (`//…`). - ---- - -## Sign-in - -Failproof AI Observability uses passwordless, email-based sign-in. There is no password to phish or leak. A user requests a one-time code (or a one-click magic link), which is emailed to them and expires quickly. Sign-in is gated by an **allowlist**: only email addresses (or domains) you permit can authenticate. - -![The Failproof AI Observability sign-in screen, which sends a single-use code to your email](/agenteye/images/login.png) - ---- - -## Scoped access with API keys - -Every client authenticates with an API key that carries granular, least-privilege permissions. A collector needs only `events:add`; a dashboard or assistant key can be read-only; destructive actions (delete, regenerate) are separate grants you choose to include. - -![The API keys page: each key's permission grants, colour-coded by read, write, and destructive scope](/agenteye/images/api-keys.png) - -Keep the admin bootstrap key for setup, and issue narrow keys for everything else. See [API keys](/agenteye/api-keys). - ---- - -## A read-only, approval-gated assistant - -The in-dashboard [AI assistant](/agenteye/assistant) answers questions over your data, but it is constrained by design: - -- It is **read-only by default**: its SQL runs through a guard that permits only `SELECT`/`WITH` queries, single-statement, with a row cap. -- Anything it creates (a saved query, a dashboard) is **approval-gated**: you review and approve every write before it happens. -- It **can never delete**. - -So a teammate can ask "which agents errored most this week?" and act on the answer, without the assistant being able to change or remove your data on its own. - ---- - -## In transit - -All traffic runs over HTTPS. You terminate TLS with your own certificates, so collector-to-server and browser-to-server traffic is encrypted in transit. - ---- - -## Next steps - -- [Overview](/agenteye/overview): how Failproof AI Observability fits together. -- [API keys](/agenteye/api-keys): scope access for the collector, dashboard, and assistant. -- [Observability](/agenteye/observability): what Failproof AI Observability captures from your agents. diff --git a/docs/ar/agent-support.mdx b/docs/ar/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/ar/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/ar/agenteye/cli-and-agents.mdx b/docs/ar/agenteye/cli-and-agents.mdx deleted file mode 100644 index 860252b6..00000000 --- a/docs/ar/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,81 +0,0 @@ ---- ---- -title: "واجهة سطر الأوامر" -description: "نشر Failproof AI Observability بالكامل، أمر واحد فقط." ---- - - -نشر Failproof AI Observability بالكامل، أمر واحد فقط. تحقق من بيئة الإنتاج، أنشئ مفتاح API، أو أقرّ حادثة دون مغادرة جهازك الطرفي، ثم قم بإجراء أي من ذلك في خط أنابيب CI، أو اترك لوكيل الترميز القيام به بلغة إنجليزية عادية. - -```bash -pipx install agenteye -agenteye login --email you@example.com # a 6-digit code lands in your inbox -agenteye --json sessions --since 24h # every agent run from the last day, newest first -``` - -*واجهة سطر الأوامر `agenteye` تتواصل مع لوحة التحكم. إنها أداة مختلفة عن مجمّع البيانات، الذي يرسل الأحداث إلى الخادم.* - -## نشرك بالكامل، أمر واحد فقط - -توقف عن القفز بين علامات التبويب للإجابة على سؤال سريع. واجهة سطر الأوامر `agenteye` تقرأ بيانات نظامك وتدير مؤسستك من ملف تنفيذي واحد، لذا فإن الفحص الذي كان يعني النقر عبر لوحة التحكم يصبح سطر واحد يمكنك إعادة تشغيله أو إنشاء اختصار له أو لصقه في دليل التشغيل. تحصل على أربع واجهات: - -- **اقرأ بيانات نظامك:** `sessions` و `events` و `evals` و `errors`، مصفاة حسب الوقت والوكيل والبيئة. -- **أدر مؤسستك:** `keys` و `users` و `settings` و `alerts` و `incidents`. -- **قم بتشغيل التحليلات:** SQL المحفوظ بالإضافة إلى مشغل `query` مخصص على بيانات الأحداث لديك. -- **اسأل المساعد:** `agent ask` يصل إلى نفس محلل القراءة فقط الذي تتحدث معه في لوحة التحكم. - -ثبّته مرة واحدة باستخدام `pipx`، وسجّل الدخول برمز 6 أرقام يُرسل بالبريد الإلكتروني، وأنت جاهز. تستمر الجلسة حوالي يوم واحد؛ أعد تشغيل `agenteye login` عند انتهاء صلاحيتها. استخدمه للتحقق من الإنتاج أو توفير مفتاح أو فرز حادثة نشطة، كل ذلك دون فتح متصفح: - -```bash -agenteye errors --since 24h --aggregate # what is breaking, grouped by error type -agenteye incidents list --state firing # what is on fire right now -agenteye keys create ci --add events:add # a key that can only push events, secret shown once -``` - -عادة واحدة يجب معرفتها: الخيارات العامة مثل `--json` تأتي قبل الأمر. `agenteye --json sessions` صحيح؛ `agenteye sessions --json` غير صحيح. - -## قم بكتابة نص، ادمجه في CI - -كل أمر يقبل `--json`، وهذا يغير كل شيء. JSON نظيف يذهب إلى stdout بينما حالة النظام والتحذيرات تذهب إلى stderr، لذا فإن التقاط `--json` ينبوب مباشرة إلى `jq` بدون سطر شاذ لإزالته. هذا هو ما يجعل واجهة سطر الأوامر جيدة بنفس القدر لك في المحث ولوكيل ترميز يحلل النتيجة: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -إنها مبنية للعمل دون إشراف. تخطي المطالبات الأكيدة تلقائياً عندما لا يكون هناك محطة نصية مرفقة، لذا لا شيء يتعطل في خط الأنابيب، وكل أمر يعيد رمز خروج ذي معنى: `0` نجاح، `4` لم تقم بتسجيل الدخول، `5` تفتقد صلاحية (الرسالة تسميها، على سبيل المثال `alerts:write`)، `3` لوحة التحكم غير قابلة للوصول. يمكن للنص أن يتفرع على `4` لإعادة المصادقة أو على `5` لإخبارك بالضبط بما يجب أن تطلبه من مسؤول، بدلاً من الفشل العمياني. - -## اترك وكيل ترميز يتحكم به باللغة الإنجليزية العادية - -الأفضل من ذلك، لا يجب أن تتذكر أي من هذه الأعلام على الإطلاق. **مهارة واجهة سطر الأوامر** عبارة عن مجلد مهارة وكيل صغير يُسمى `agenteye-cli` يعلم وكيل ترميز مثل Claude Code أو Codex كيفية تشغيل واجهة سطر الأوامر من طلبات باللغة الإنجليزية البسيطة. اسأل "هل هناك أي شيء معطل اليوم؟" والوكيل يختار الأمر، ويقوم بتشغيله باسمك، ويجيب بشكل نثري. - -بالنسبة إلى Claude Code، اسحب مجلد `agenteye-cli` إلى `~/.claude/skills/` وسيتم اكتشافه تلقائياً. يوفر Failproof AI Observability المجلد؛ لا يوجد شيء إضافي للتثبيت، لأنه يقود فقط واجهة سطر الأوامر التي ثبتها بالفعل. قم بتسجيل الدخول بنفسك أولاً: المهارة لا يمكنها إكمال تسجيل الدخول برمز البريد الإلكتروني من أجلك. - -نظراً لأن الوكيل يقوم بتشغيل واجهة سطر الأوامر باسمك، فيمكنه القيام بكل شيء تسمح به عملية تسجيل الدخول الخاصة بك، القراءة والكتابة على حد سواء: إنشاء مفاتيح، تغيير الإعدادات، حل الحوادث. لا تظهر مطالبة "هل أنت متأكد؟" في واجهة سطر الأوامر لوكيل، لذا تمت كتابة المهارة لتوضيح الأمر الدقيق والانتظار لموافقتك قبل أي تغيير. أنت خطوة التأكيد. - -```text -you Why did session run-001 fail? - -agent Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -البيانات المقروءة تبقى فورية، وكل كتابة تتوقف من أجلك: - -```text -you Give CI a key that can only push events. - -agent This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you yes - -agent Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## ذات صلة - -- [مرجع واجهة سطر الأوامر](/ar/agenteye/cli): كل أمر وعلم وشكل JSON. -- [وصفات واجهة سطر الأوامر للوكلاء](/ar/agenteye/cli-recipes): أنماط `jq` التي يمكنك نسخها بسهولة ومعالجة رموز الخروج. -- [مهارة وكيل واجهة سطر الأوامر](/ar/agenteye/cli-skill): ثبّت وقم بتشغيل مهارة `agenteye-cli`. -- [مساعد ذكاء اصطناعي](/ar/agenteye/assistant): محلل لوحة التحكم الذي يتحدث معه `agent ask`. \ No newline at end of file diff --git a/docs/ar/agenteye/cli-skill.mdx b/docs/ar/agenteye/cli-skill.mdx deleted file mode 100644 index 499cc9af..00000000 --- a/docs/ar/agenteye/cli-skill.mdx +++ /dev/null @@ -1,160 +0,0 @@ ---- ---- -title: "مهارة عامل Failproof AI Observability CLI" -description: "اسأل وكيل الترميز الخاص بك \"هل حدث عطل ما اليوم؟\" ودعه يجيب من بيانات Failproof AI Observability المباشرة، بدون الحاجة لحفظ أوامر." ---- - - -اسأل وكيل الترميز الخاص بك *"هل حدث عطل ما اليوم؟"* ودعه يجيب من بيانات Failproof AI Observability المباشرة، بدون الحاجة لحفظ أوامر. **مهارة Failproof AI Observability CLI** (`agenteye-cli`) هي *مهارة عامل*: مجلد صغير يحتوي على تعليمات يحملها وكيل ترميز مثل Claude Code أو Codex عند الحاجة. تعلم الوكيل كيفية تشغيل نشر Observability الخاص بك من خلال [`agenteye` CLI](/ar/agenteye/cli) من طلبات باللغة الإنجليزية العادية مثل *"أعط CI مفتاح يمكنه فقط دفع الأحداث"* أو *"اعترف بالحادثة النشطة وعينها لي."* - -إنها **ليست** خدمة أو ملف تنفيذي منفصل؛ لا شيء للنشر. تعتمد على CLI الذي لديك بالفعل: يقوم الوكيل بتنفيذ `agenteye --json …`، ويحلل JSON النظيف، ويجيبك بنص عادي. كل شيء يمكنه القيام به، يمكنك القيام به بنفسك بكتابة نفس الأوامر. - ---- - -## كيف يرتبط بواجهات Failproof AI Observability الأخرى - -Failproof AI Observability يعطيك أربع طرق للوصول إلى نفس البيانات والتحكم. تكمل بعضها بعضاً: - -| الواجهة | ما هي | حيث تعمل | استخدمها عندما | -|---|---|---|---| -| **[CLI](/ar/agenteye/cli)** | مرجع الأوامر والخيارات لـ `agenteye` | محطة طرفية | تريد تشغيل أو كتابة أمر معين | -| **[وصفات CLI](/ar/agenteye/cli-recipes)** | أنماط `jq`/أنابيب جاهزة للنسخ | محطة طرفية / نصوص برمجية | تريد دمج CLI في أتمتة | -| **مهارة CLI** (هذا المستند) | باب أمامي بلغة طبيعية على CLI | وكيل ترميز، على محطة العمل الخاصة بك | تريد فقط أن تسأل ودع الوكيل يختار الأمر | -| **[مهارة المقيّم](/ar/agenteye/evaluator-skill)** | مهارة شقيقة تصمم وتبني خدمة التسجيل الخاصة بك | وكيل ترميز، على محطة العمل الخاصة بك | تريد **إنتاج** درجات التقييم بدلاً من قراءتها | -| **[مهارة Python SDK](/ar/agenteye/python-sdk-skill)** | مهارة شقيقة تجهز وكيلك لإصدار بيانات تلميترية | وكيل ترميز، على محطة العمل الخاصة بك | تريد من وكيلك **إنتاج** الأحداث التي تقرأها هذه المهارة | -| **[مساعد AI في لوحة المعلومات](/ar/agenteye/assistant)** | دردشة مضمنة في لوحة المعلومات | من جهة الخادم (في لوحة المعلومات) | تريد أسئلة وأجوبة داخل لوحة المعلومات حول البيانات | - -المهارة نفسها ليس لديها امتيازات خاصة بها؛ فهي تحول كلماتك ببساطة إلى استدعاءات CLI يتم تشغيلها باسمك: - -```mermaid -flowchart TD - YOU["أنت: 'اعترف بالحادثة النشطة'"] --> AGENT["وكيل ترميز (Claude Code / Codex)
يحمل مهارة agenteye-cli"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|جلسة CLI موثقة لديك| API["واجهة برمجية لوحة معلومات Observability"] -``` - -### مقابل مساعد AI في لوحة المعلومات: تمييز مهم - -هذان أداتان مختلفتان جداً بنطاقات انفجار مختلفة جداً: - -- **مساعد AI في لوحة المعلومات** ([مساعد AI](/ar/agenteye/assistant)) هو دردشة مضمنة في لوحة المعلومات، مدعومة بخدمة الوكيل. إنها **قراءة فقط بالإضافة إلى تأليف محمي بموافقة**: يمكنها صياغة الاستعلامات والمحاور المحفوظة، لكن كل عملية كتابة تتوقف لموافقتك الصريحة بالنقر، ولا تحذف أبداً. يتم حمايتها بواسطة إذن `agent:use` وترى فقط البيانات للمؤسسة التي تعرضها. -- **مهارة CLI** تعمل على *محطة العمل الخاصة بك* داخل *وكيل ترميز خاص بك* وتشغل CLI `agenteye` بـ **أنت**. يمكنها تنفيذ **السطح الكامل للـ CLI، بما في ذلك التغييرات** (إنشاء/تدوير/تعطيل مفاتيح API، تغيير إعدادات المؤسسة، حل الحوادث، حذف الاستعلامات المحفوظة)، محدودة فقط بأذونات تسجيل دخول CLI الخاص بك. تعامل معها بنفس الحذر الذي ستتعامل به إذا قمت بتشغيل تلك الأوامر يدوياً. - ---- - -## المتطلبات الأساسية - -1. **`agenteye` CLI مثبتة** وعلى `PATH` (انظر [مرجع CLI](/ar/agenteye/cli): `pipx install agenteye`). -2. **عنوان URL لوحة المعلومات الخاصة بك** محدد (`AGENTEYE_DASHBOARD_URL`، أو يمرر الوكيل `--base-url`). -3. **جلسة مسجلة الدخول**: قم بتشغيل `agenteye login` بنفسك أولاً. المهارة **لا يمكنها** إكمال تسجيل الدخول برمز لمرة واحدة عبر البريد الإلكتروني نيابة عنك؛ ستخبرك أن تشغل `agenteye login` إذا كانت الجلسة مفقودة أو منتهية الصلاحية (رمز خروج CLI `4`). - ---- - -## حيث تحصل عليها - -تُنشر المهارة في مجموعة المهارات العامة لـ Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -لا شيء محمي في ذلك — المستودع عام والمهارة لا تحتاج إلى بيانات اعتماد خاصة بها، لأنها تشغل فقط `agenteye` CLI **العام** ضد لوحة المعلومات *الخاصة بك*، باستخدام الجلسة *التي سجلت بها الدخول*. لا تحتاج إلى طلب إذن من أحد. - -لاحظ أنها تأتي كمجلد خاص بها و**ليست** داخل حزمة `pipx install agenteye`، لذا لا تبحث عنها هناك. - -## تثبيت المهارة - -أسرع طريق هي [`skills`](https://skills.sh) CLI، التي تجلب المجلد وتضعه حيث ينظر وكيلك: - -```bash -# Claude Code، هذا المشروع فقط -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# كل مشروع (التثبيت في ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex بدلاً من ذلك -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -ثم أدرها مثل أي مهارة أخرى: - -```bash -npx skills list -a claude-code # ما هو مثبت -npx skills update agenteye-cli # اسحب أحدث إصدار -npx skills remove agenteye-cli # أزلها -``` - -تفضل التثبيت يدوياً؟ مهارة عامل ما هي إلا مجلد يحتوي على `SKILL.md` (بالإضافة إلى مراجع اختيارية)، لذا نسخها يعمل أيضاً: - -- **Claude Code**: ضع مجلد `agenteye-cli/` في `~/.claude/skills/` (كل مشروع) أو `/.claude/skills/` (ذلك المستودع فقط). Claude Code يكتشفه تلقائياً — تحقق من قائمة `/skills`، أو ببساطة اسأل سؤالاً يطابق وصفه. -- **Codex (OpenAI)**: يقرأ Codex نفس `SKILL.md`. يعيّن `agents/openai.yaml` المضمن `allow_implicit_invocation: true`، لذا يختار Codex المهارة تلقائياً عندما تطابق المهمة؛ وإلا قم باستدعاؤها بشكل صريح كـ `$agenteye-cli`. - ---- - -## الأمان: التغييرات لا تطلب موافقة عندما يشغل الوكيل CLI - -> **تحذير:** اقرأ هذا قبل السماح لوكيل بإجراء تغييرات. - -CLI `agenteye` عادة ما يسأل *"هل أنت متأكد؟"* قبل إجراء تدميري. إنه **يتخطى هذا التأكيد تلقائياً كلما لم يكن متصلاً بمحطة طرفية (وهذا بالضبط كيفية تشغيل الوكيل له)، و `--json` يتخطاه أيضاً.** لذا فإن موجه الأمان لن **ينطلق** للوكيل. - -تمت كتابة المهارة للتعويض: تم تعليمها بيان الأمر الدقيق الذي ستشغله والحصول على موافقتك الصريحة **OK قبل أي تغيير في الحالة**. حافظ على هذا النظام. عندما تشغل Failproof AI Observability من خلال وكيل، *أنت* خطوة التأكيد. أوامر تغيير الحالة التي يجب مراقبتها: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- أوامر الكتابة في `incidents`: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -كل شيء تحت **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) للقراءة فقط ولا يغير شيء. - -لأن الوكيل يتصرف ب **أنت**، يمكنه فقط القيام بما يسمح به تسجيل دخولك؛ يتم حل الأذونات **لكل منظمة** (انظر [مفاتيح API](/ar/agenteye/api-keys)). الأمر الذي لا تملك إذناً له يرجع رمز خروج `5` مع الإذن المحدد بالضبط، لذا يمكن للوكيل أن يخبرك بالضبط ما الذي يجب عليك طلبه من المسؤول بدلاً من الفشل بشكل غير واضح. - ---- - -## ما الذي يمكنك أن تسأله - -إليك ما يبدو عليه جولة ذهاب وإياب حقيقية، قراءة ثم تغيير ينتظر موافقتك: - -```text -أنت ▸ هل حدث عطل ما في اليوم الأخير؟ - -وكيل ▸ تشغيل: agenteye --json errors --since 24h --aggregate - 3 أنواع أخطاء في آخر 24 ساعة. الأكبر هو TimeoutError - (18 نقرة، جميعها على checkout-agent)، ثم RateLimited (5). - تريد الجلسات الفاشلة لـ TimeoutError؟ - -أنت ▸ أعط CI مفتاح يمكنه فقط دفع الأحداث. - -وكيل ▸ هذا ينشئ مفتاح API. سأشغل: - agenteye keys create ci --add events:add - OK للمتابعة؟ - -أنت ▸ نعم - -وكيل ▸ تم. تم إنشاء المفتاح "ci" مع events:add فقط. - يتم عرض السر مرة واحدة فقط، لذا قم بتخزينه الآن. لا يمكنني إعادة طباعته. -``` - -تعيّن المهارة كل نية باللغة الإنجليزية العادية إلى أمر `agenteye` الصحيح، واكتشف القيم الصحيحة أولاً (`list `, `whoami`) حتى لا تخمن، وحدد الأمر الدقيق قبل أي تغيير. المزيد من الأمثلة: - -- *"هل حدث عطل / فشل في آخر 24 ساعة؟"* → `errors --since 24h --aggregate`، ثم تفصيل. -- *"لماذا فشلت الجلسة `run-001`؟"* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *"كيف تتجه الجودة هذا الأسبوع؟"* → `evals --aggregate --since 7d`، ثم الحفر في التشغيلات منخفضة التسجيل. -- *"أعط CI مفتاح يمكنه فقط دفع الأحداث."* → `keys create ci --add events:add` (يحدد الأمر، ثم ينشئه ويأسر السر لمرة واحدة). -- *"من لديه حق الوصول؟ اجعل Dana للقراءة فقط."* → `users list` → `users update dana@… --permission-set read-only` (بعد التأكيد معك). -- *"اعترف بالحادثة النشطة وعينها لي."* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`. - -للأوامر والخيارات والأشكال JSON الدقيقة خلف هذا، انظر [مرجع CLI](/ar/agenteye/cli) و[وصفات CLI للوكلاء](/ar/agenteye/cli-recipes). - ---- - -## الخطوات التالية - -- **[CLI](/ar/agenteye/cli)**: مرجع أمر وخيار كامل لـ `agenteye`. -- **[وصفات CLI للوكلاء](/ar/agenteye/cli-recipes)**: أنماط `jq` جاهزة للنسخ ومعالجة رموز الخروج. -- **[مهارة وكيل المقيّم](/ar/agenteye/evaluator-skill)**: المهارة الشقيقة، لبناء المقيّم الذي تقرأه `agenteye evals`. -- **[مهارة وكيل Python SDK](/ar/agenteye/python-sdk-skill)**: المهارة الشقيقة، لتجهيز وكيل حتى يصدر البيانات التي يقرأها `agenteye`. -- **[مساعد AI](/ar/agenteye/assistant)**: مساعد لوحة المعلومات (لا تخلطها مع مهارة المحطة الطرفية هذه). -- **[مفاتيح API](/ar/agenteye/api-keys)**: نموذج الأذونات لكل منظمة الذي يحدد ما يمكن للمهارة القيام به. \ No newline at end of file diff --git a/docs/ar/agenteye/codex-capture.mdx b/docs/ar/agenteye/codex-capture.mdx deleted file mode 100644 index 4e4c0a24..00000000 --- a/docs/ar/agenteye/codex-capture.mdx +++ /dev/null @@ -1,56 +0,0 @@ ---- ---- -title: "التقاط جلسات Codex" -description: "استخدم جلسات فريقك المحلية من OpenAI Codex في AgentEye كجلسات وأحداث عادية — دون أي تغيير في طريقة تشغيل Codex." ---- - -يقوم المهندسون لديك بتشغيل OpenAI Codex يومياً بالفعل. يوفر التقاط جلسات Codex القدرة على نقل جلسات الترميز تلك إلى AgentEye كجلسات وأحداث عادية، بحيث يمكنك البحث فيها وإعادة تشغيلها وتقييمها جنباً إلى جنب مع كل ما تلاحظه آخر. يكمل هذا [Python SDK](/ar/agenteye/python-sdk): حيث يقوم SDK بتوظيف الوكلاء الذين تكتبهم، بينما يقوم هذا بالتقاط عمل Codex الذي يقوم به فريقك بالفعل — دون أي تغيير في طريقة تشغيله. - -مجمع خلفي صغير يقرأ نسخ جلسات Codex المحلية كما يتم كتابتها وينقلها إلى AgentEye. مجمع واحد لكل جهاز يلتقط كل سطح Codex محلي في نفس الوقت — لا توجد عملية إعداد لكل سطح. - -يلتقط نفس المجمع وكلاء آخرين أيضاً — انظر [OpenClaw](/ar/agenteye/openclaw-capture) و [Hermes](/ar/agenteye/hermes-capture). فعّل كل واحد تقوم بتشغيله؛ يمكن لمجمع واحد أن يلتقط عدة منها في نفس الوقت. - ---- - -## ما يتم التقاطه - -كل سطح Codex يعمل **محلياً** ينتج نفس نسخ الجلسات على القرص، والمجمع يلتقط كل منها: - -- واجهة سطر أوامر Codex **CLI** و `codex exec` -- **ملحق VS Code / IDE** -- **تطبيق سطح المكتب**، عند تشغيل جلسة محلياً - -تصبح كل جلسة Codex [جلسة](/ar/agenteye/sessions) AgentEye؛ رسائل المستخدم والمساعد، والتفكير، واستدعاءات الأدوات، ونتائج الأدوات، واستخدام الرموز تصبح [أحداث](/ar/agenteye/event-stream) مطابقة. يتم تسجيل السطح الذي أتت منه كل جلسة (CLI أو IDE أو سطح مكتب)، حتى تتمكن من تمييزها. - -> **لا يتم التقاط جلسات السحابة.** يقوم تطبيق سطح المكتب بشكل متزايد بتشغيل الجلسات في سحابة Codex ويحتفظ فقط بالبيانات الوصفية الخاصة بها على الجهاز — لا توجد نسخة محلية لقراءتها. يتم التقاط الجلسات المنفذة محلياً فقط. - ---- - -## تشغيله - -الالتقاط معطل حتى تقوم بتفعيله. ثبت المجمع بمفتاح API له إذن `events:add` (انظر [مفاتيح API](/ar/agenteye/api-keys))، وفعّل التقاط Codex: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -يقوم هذا بتثبيت المجمع وتسجيله كخدمة خلفية وبدء الالتقاط. تأكد من أنه قيد التشغيل: - -```bash -agenteye-collector health -``` - -عند التشغيل الأول، يتم ملء جلسات Codex الموجودة لديك مرة واحدة وينتقل النشاط الجديد خلال ثوان. ملفات Codex نفسها تُقرأ فقط — لا تُعدّل أو تُنقل أو تُحذف — وكل جلسة تُنقل بالضبط مرة واحدة، حتى عبر إعادات التشغيل. - ---- - -## حيث يظهر - -تظهر الجلسات المقتناة في **Sessions**، وأحداثها في تيار **Events**، وبنفس طريقة أي وكيل آخر تراقبه — لذا [إعادة تشغيل الجلسة](/ar/agenteye/sessions) و [البحث](/ar/agenteye/queries) و [التقييمات](/ar/agenteye/evaluations) و [التنبيهات](/ar/agenteye/alerts) تعمل جميعها عليها. قم بالتصفية حسب وكيل Codex لرؤيتها بمفردها. - ---- - -## الخصوصية - -تحتوي نسخ Codex على الجلسة الكاملة — بما في ذلك مخرجات الأوامر وملتويات الملفات وأي شيء قراءه أو كتبه Codex — ويمكن أن تحتوي على أسرار. يتم نقل الجلسات المقتناة كما هي، لذا فعّل الالتقاط فقط على الأجهزة والفرق التي يكون فيها تجميع هذا المحتوى في AgentEye مناسباً، وامنح المجمع مفتاحاً مقتصراً على `events:add` فقط. انظر [Security](/ar/agenteye/security) لمعرفة كيف يتم الحفاظ على عزل بيانات التشفير الخاصة بك. \ No newline at end of file diff --git a/docs/ar/agenteye/concepts.mdx b/docs/ar/agenteye/concepts.mdx deleted file mode 100644 index 333e1fcc..00000000 --- a/docs/ar/agenteye/concepts.mdx +++ /dev/null @@ -1,88 +0,0 @@ ---- ---- -title: "المفاهيم" -description: "المصطلحات المستخدمة في Failproof AI Observability — الأحداث والجلسات والتقييمات والتدقيقات والنتائج والحوادث — معرّفة في مكان واحد." ---- - - -تحدد هذه الصفحة المصطلحات التي يستخدمها Failproof AI Observability. إذا كان هناك مصطلح غير مألوف في دليل آخر، فهو معرّف هنا. لا تحتاج إلى قراءة الصفحة كاملة: يمكنك تصفحها أو العودة إليها عند مصادفة كلمة تريد توضيحها. - ---- - -## نموذج البيانات - -**Event (الحدث)** -أصغر وحدة بيانات. يسجل حدث واحد خطوة واحدة اتخذها وكيلك: `tool_use` أو `model_request` أو `hook_completed` أو `error` وغيرها. ينبعث وكيلك الأحداث عبر [Python SDK](/ar/agenteye/python-sdk)؛ تظهر مباشرة على صفحة **Events**. - -**Session (الجلسة)** -تشغيل واحد للوكيل، يتم تعريفه بواسطة `session_id`. الجلسة هي جميع الأحداث التي تشترك في نفس المعرّف، مدمجة في صف واحد على صفحة **Sessions** وموضحة كرسم بياني تنفيذي في صفحة التفاصيل الخاصة بها. عادة ما تبدأ الجلسة بـ `agent_start` وتنتهي بـ `agent_end`. - -**Agent (الوكيل)** -فاعل مُسمّى داخل التشغيل، يتم تعريفه بواسطة `agent_id`. يمكن أن يشتمل التشغيل على عدة وكلاء: على سبيل المثال، مخطط ينتج وكيل فرعي للتلخيص. يحمل الوكلاء الفرعيون `parent_id`، وهذا هو ما يسمح لـ Failproof AI Observability برسمهم على مساراتهم الخاصة في الرسم البياني التنفيذي. - -**Environment (البيئة)** -تصنيف للمكان الذي حدث فيه التشغيل: `production` أو `staging` أو `dev`. تعيّنها مرة واحدة عند تكوين SDK. يمكن لكل صفحة لوحة تحكم تقريباً التصفية حسب البيئة. - -**Context-window fill (ملء نافذة السياق)** -نسبة مئوية من نافذة السياق للنموذج التي استهلكتها الاستجابة. يضيف Failproof AI Observability الطابع الزمني لها على أحداث `model_response` للنماذج التي يتعرف عليها، بحيث يكون نمو المطالبة والانضغاط الوشيك مرئياً مباشرة في تدفق الأحداث. - ---- - -## الجودة - -**Evaluation (التقييم)** -درجة جودة لجلسة منتهية، ينتجها خدمة تسجيل تديرها. التقييمات اختيارية: حتى تقوم بربط مُقيّم، يتم تسجيل الجلسات لكن لا يتم تقديرها. يمكن لكل تقييم أن يحمل عدة درجات مسمّاة (على سبيل المثال `helpfulness` و `factuality` و `tool_efficiency`)، كل منها مع ملاحظة قصيرة للتفكير. انظر [Evaluation suite](/ar/agenteye/evaluation-suite). - -**Score key (مفتاح الدرجة)** -اسم بُعد واحد يبلغ عنه المُقيّم، مثل `helpfulness`. يمكن للتنبيهات والتدقيقات مراقبة مفتاح درجة معين بمرور الوقت. - -**Evaluator (المُقيّم)** -خدمة التسجيل الخاصة بك. يرسل Failproof AI Observability نسخة نصية من التشغيل المنتهي إليها ويخزن الدرجات التي ترجعها. لا توفر مُقيّماً افتراضياً؛ منطق التسجيل خاص بك. - ---- - -## العثور على الأخطاء وإصلاحها - -**Hook (الخطاف)** -حماية أو تأثير جانبي يقوم إطار عمل وكيلك بتشغيله حول خطوة: فحص سلامة المحتوى أو إخفاء معلومات التعريف الشخصية أو حماية الميزانية. تنبعث الخطافات من أحداث `hook_triggered` / `hook_completed` مع `outcome` (allow أو deny أو modify)، وتحصل على صفحة ملاحظة خاصة بها. - -**Alert rule (قاعدة التنبيه)** -قاعدة تُطلق عندما تتجاوز مقياس حداً تعيّنه: معدل الخطأ أو كمون p95 أو تكلفة الرموز أو درجة المُقيّم. عند تفعيل القاعدة، تفتح حادثة وتُعلم القنوات المختارة لديك (البريد الإلكتروني أو Slack أو webhook أو داخل لوحة التحكم). انظر [Alerts](/ar/agenteye/alerts). - -**Incident (الحادثة)** -مشكلة مفتوحة يتم إنشاؤها عند تفعيل قاعدة التنبيه. للحوادث دورة حياة (الإقرار والتعيين والحل) وخط زمني للنشاط يسجل كل إجراء. يمكنك أيضاً فتح واحدة يدويّاً. - -**Audit (التدقيق)** -تحقيق متكرر (كل ساعة إلى أسبوعياً) يفحص السجلات *عبر* الجلسات للبحث عن أنماط الفشل التي لم تكتب قاعدة لها: تجمعات الأخطاء والدرجات المنخفضة والقيم الشاذة للكمون وحلقات استدعاء الأدوات والتشغيلات التي لم تنتهِ أبداً. حيث يراقب التنبيه مقياساً تعرفه بالفعل، يخبرك التدقيق بما يجب أن تنظر إليه بعد ذلك. انظر [Audits](/ar/agenteye/audits). - -**Finding (النتيجة)** -نتيجة واحدة مرتبة مدعومة بالأدلة من تشغيل التدقيق. تسمي النتيجة نمطاً وتربط بالجلسات الدقيقة وراءها وتحمل دورة حياة الفرز (الإقرار والحل والكتم والرفض). يقوم Failproof AI Observability بإزالة تكرار النتائج من تشغيل إلى آخر بحيث ينتج النمط المعروف تحديثاً بدلاً من التراكم. - -**The AI assistant (مساعد الذكاء الاصطناعي)** -الدردشة داخل لوحة التحكم التي تجيب على أسئلة حول وكلائك بلغة إنجليزية عادية، على بياناتك الخاصة. هو للقراءة فقط بشكل افتراضي؛ أي شيء ينشئه (استعلام محفوظ أو لوحة تحكم) مُوافق عليه، ولا يمكنه أبداً الحذف. انظر [AI assistant](/ar/agenteye/assistant). - ---- - -## تشغيله - -**Organization (tenant) (المنظمة)** -مساحة عمل معزولة. يمكن لمثيل واحد من Failproof AI Observability استضافة عدة منظمات، كل منها مع المستخدمين والمفاتيح والبيانات الخاصة بها. كل عنوان URL لوحة التحكم مُحدد النطاق تحت رمز المنظمة الخاص بك (`//…`). - -**Collector (المجمع)** -`agenteye-collector`، الديمون الخفيف الذي يعمل على كل جهاز وكيل، يجمع الأحداث التي يكتبها SDK إلى القرص، وينقلها إلى الخادم. - -**API key (مفتاح API)** -رمز مُحدد النطاق يوثّق عميل ضد الخادم. تحمل المفاتيح أذونات دقيقة (على سبيل المثال `events:add` للمجمع، نطاقات للقراءة فقط لمفتاح لوحة التحكم). انظر [API keys](/ar/agenteye/api-keys). - -**Server (الخادم)** -خدمة البلع والـ API. تبتلع الأحداث وتخزن الحالة التشغيلية في قواعد البيانات الخاصة بك وتخدم لوحة التحكم والـ CLI. - -**Dashboard (لوحة التحكم)** -واجهة المستخدم على الويب. كل صفحة مُحددة النطاق لمنظمة وتقرأ من خلال API الخادم. - ---- - -## الخطوات التالية - -- [Overview](/ar/agenteye/overview): كيف تتناسب هذه الأجزاء معاً. -- [Observability](/ar/agenteye/observability): سطح الملاحظة (Events و Sessions و Models و Tools و Hooks و Errors). \ No newline at end of file diff --git a/docs/ar/agenteye/evaluator-skill.mdx b/docs/ar/agenteye/evaluator-skill.mdx deleted file mode 100644 index 12e610be..00000000 --- a/docs/ar/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,168 +0,0 @@ ---- ---- -title: "مهارة وكيل Failproof AI Observability Evaluator" -description: "انتقل من \"أعتقد أن وكيلنا سيء أحياناً\" إلى خدمة تقييم مُنتشرة، مع قيام وكيل البرمجة بكل من القرار والبناء." ---- - - -انتقل من *"أعتقد أن وكيلنا سيء أحياناً"* إلى خدمة تقييم مُنتشرة، مع قيام وكيل البرمجة بكل من القرار والبناء. **مهارة Failproof AI Observability evaluator** (`agenteye-evaluator`) هي *مهارة وكيل*: مجلد صغير من التعليمات يحمّله وكيل برمجة مثل Claude Code أو Codex عند الحاجة. تعلّم الوكيل كيفية تحديد أي أبعاد جودة تستحق التتبع لـ *وكيلك*، ثم كتابة واختبار ونشر [خدمة المُقيّم](/ar/agenteye/evaluation-suite) التي تقيّمها. - -إنها **ليست** محدد درجات مستضاف، ولا سجل تحمّل عليه، ولا نظام إضافات. يبقى المُقيّم خدمة HTTP خاصة بك على البنية الأساسية الخاصة بك، بالضبط كما هو موضح في دليل [مجموعة التقييم](/ar/agenteye/evaluation-suite). تعلّم المهارة وكيلك فقط ليبنيها بشكل جيد، لذلك كل ما تفعله يمكنك أن تفعله بنفسك بكتابة الكود ذاته. - ---- - -## الجزء الصعب هو تحديد ما يجب تقييمه - -سطح SDK صغير — ديكوريتور ونموذجان — والوكيل يمكنه كتابة ذلك من [العقد](/ar/agenteye/evaluation-suite#http-contract) وحده. هذا ليس حيث يفشل المُقيّمون. يفشلون لأنهم يقيّمون الشيء الخطأ، والمُقيّم الذي يقيّم الشيء الخطأ أسوأ من لا شيء: فهو ينتج لوحة معلومات يتعلم الجميع تجاهلها. - -لذلك معظم المهارة هي الجزء قبل وجود أي كود. يحتوي على الوكيل الذي يقابلك (*"اصف تشغيلاً سار بشكل جيد؛ الآن واحداً سار بشكل سيء"*) ثم يسحب جلساتك الفعلية من خلال [`agenteye` CLI](/ar/agenteye/cli) ويقرأها من البداية إلى النهاية. هذان النصفان عادة ما يختلفان، والفجوة هي النقطة: ما تنوي قياسه مقابل ما يمكن لنصوصك فعلاً دعمه. يبقى البعد فقط إذا كان **قابلاً للحساب** من الأحداث و**تمييزياً** — إذا حقق 0.9 على جلستك الجيدة والسيئة معاً، فهو لا يعلم شيئاً ويتم حذفه. - -ما يعود عليك هو اقتراح 2-4 أبعاد مع التفكير المرفق، لتصديق عليها قبل كتابة سطر واحد. - -```mermaid -flowchart TD - YOU["أنت: 'أريد تقييمات لـ support bot الخاص بي'"] --> AGENT["وكيل البرمجة (Claude Code / Codex)
يحمّل مهارة agenteye-evaluator"] - AGENT -->|"مقابلة: كيف يبدو الجيد مقابل السيء؟"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["جلساتك الفعلية
ما يحدث فعلاً"] - DATA --> DIMS["2-4 أبعاد، أنت توافق"] - DIMS --> SVC["خدمة المُقيّم الخاصة بك
agenteye-evaluator SDK"] - SVC --> SCORES["الدرجات تهبط في لوحة المعلومات
و agenteye evals"] -``` - ---- - -## كيفية ارتباطها بأجزاء التقييم الأخرى - -أربعة مستندات تغطي التقييم، وتسلمها لبعضها البعض بالترتيب: - -| الصفحة | ما هي | استخدمها عندما | -|---|---|---| -| **[التقييمات](/ar/agenteye/evaluations)** | الميزة: درجات على شبكة الجلسات، لوحات المعلومات، إعادة تقييم | تريد معرفة ما يحصل عليه التقييم التلقائي | -| **[مجموعة التقييم](/ar/agenteye/evaluation-suite)** | عقد HTTP، SDK، متغيرات بيئة الخادم | تقوم بتطبيق أو تصحيح المُقيّم بنفسك | -| **مهارة المُقيّم** (هذا المستند) | باب باللغة الطبيعية لتصميم *وبناء* المقيّم | تريد الانتقال من "أريد تقييمات" إلى خدمة قيد التشغيل | -| **[مهارة CLI](/ar/agenteye/cli-skill)** | باب باللغة الطبيعية على `agenteye` CLI | تريد *قراءة* الدرجات التي لديك بالفعل | -| **[مهارة Python SDK](/ar/agenteye/python-sdk-skill)** | باب باللغة الطبيعية على جهاز وكيلك | وكيلك لا ينبت جلسات حتى الآن — لا يوجد شيء لتقييمه | - -### مقابل مهارة CLI: البناء مقابل القراءة - -المهارتان متعمداً غير متداخلتان، والتثبيت كليهما هو الإعداد الطبيعي — يختار الوكيل بينهما بناءً على ما تطلبه: - -- **`agenteye-evaluator`** (هذا المستند) يبني الشيء الذي *ينتج* الدرجات. تنتهي وظيفته عندما تهبط الدرجات للمرة الأولى. -- **[`agenteye-cli`](/ar/agenteye/cli-skill)** يقرأ درجات موجودة بالفعل (`agenteye evals`). *"هل انخفضت الجودة هذا الأسبوع؟"* هو سؤاله، وليس سؤال هذه المهارة. - ---- - -## المتطلبات الأساسية - -1. **`agenteye` CLI مثبت وقيد التسجيل** (`pipx install agenteye`، ثم `agenteye login`). تعتمد المهارة عليها مرتين: لسحب الجلسات الفعلية التي تصممها، والتأكيد من أن درجاتك هبطت في النهاية. يحتاج تسجيلك إلى `events:read`, بالإضافة إلى `evaluations:read` للتحقق النهائي. كما هو الحال مع مهارة CLI، **لا يمكنها** إكمال تسجيل دخول الرمز أحادي الاستخدام عبر البريد الإلكتروني نيابة عنك. -2. **مكان للمُقيّم ليعيش فيه.** يتم بناؤه في صورة ويعمل كخدمة طويلة الأجل، لذا فهو يحتاج إلى ريبو حقيقي، وليس ملف مؤقت. غالباً ما تعيش المُقيّمون في ريبو خاصة بهم، منفصلة عن الوكيل الذي يتم تقييمه — تبحث المهارة عن ريبو موجود وتطلب قبل إنشاء ريبو جديد. -3. **عجلة SDK `agenteye-evaluator`** — اقرأ القسم التالي قبل أن يبدأ وكيلك بكتابة أوامر `pip`. - ---- - -## أين تحصل عليها - -تُنشر المهارة في مجموعة المهارات العامة في Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -المستودع عام والمهارة لا تحتاج إلى بيانات اعتماد خاصة بها — فهي فقط تشغيل `agenteye` CLI مع جلسة *أنت* قيد التسجيل، وتكتب كوداً في *ريبوك* الخاص. لاحظ أنها تُشحن كمجلد خاص بها وهي **ليست** داخل حزمة `pipx install agenteye`، لذا لا تبحث عنها هناك. - -## تثبيت المهارة - -أسرع طريق هي CLI [`skills`](https://skills.sh)، الذي يحضر المجلد وينزله حيث يبحث وكيلك: - -```bash -# Claude Code، هذا المشروع فقط -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# كل مشروع (يثبت إلى ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codex بدلاً من ذلك -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -ثم أدره مثل أي مهارة أخرى: - -```bash -npx skills list -a claude-code # ما هو مثبت -npx skills update agenteye-evaluator # اسحب أحدث إصدار -npx skills remove agenteye-evaluator # أزله -``` - -تفضل التثبيت يدوياً؟ مهارة وكيل هي مجرد مجلد يحتوي على `SKILL.md` (بالإضافة إلى مراجع اختيارية)، لذا نسخه يعمل أيضاً: - -- **Claude Code**: ضع مجلد `agenteye-evaluator/` في `~/.claude/skills/` (كل مشروع) أو `/.claude/skills/` (هذا الريبو فقط). Claude Code يكتشفه تلقائياً — تحقق مع قائمة `/skills`، أو اطلب فقط تقييمات. -- **Codex (OpenAI)**: يقرأ Codex نفس `SKILL.md`. يعيّن `agents/openai.yaml` المرفق `allow_implicit_invocation: true`، لذا يختار Codex تلقائياً المهارة عندما تطابق المهمة؛ وإلا استدعِها بشكل صريح كـ `$agenteye-evaluator`. - ---- - -## SDK ليس على PyPI العام - -> **تحذير:** اقرأ هذا قبل السماح لوكيل بتثبيت SDK. - -المهارة عامة؛ SDK الذي تقوده ليس كذلك. `agenteye-evaluator` يُشحن فقط كقطعة إصدار خاصة، وخلافاً لـ `agenteye`، الاسم **غير مطالب به على PyPI العام** — لذا `pip install agenteye-evaluator` بسيط قد يسحب حزمة شخص غريب إلى الخدمة التي تقرأ نصوصك الإنتاجية. هذه مشكلة سلسلة التوريد، وليست خطأ إملائي. - -تعرف المهارة هذا وتعمل لأسفل سلم التثبيت بدلاً من ذلك، متوقفة عند أول درجة تنطبق: مصدر أحادي الريبو إذا كنت داخل ريبو AgentEye، وإلا عجلة الإصدار الخاصة من GitHub Releases (تحتاج إلى وصول)، وإذا لم تكن متاحة فإنها **توقف وتخبرك بطلب عجلة من جهة Failproof AI الخاصة بك** بدلاً من الارتجال. - -لذا إذا اقترح وكيلك `pip install agenteye-evaluator` بسيطة من PyPI العام، هذا يشير إلى أن المهارة لم تحمّل أبداً. توقف هناك وتحقق من تثبيتها. - ---- - -## ما يمكنك طلبه - -جولة ذهاب وإياب حقيقية تبدأ بطلب غامض وتنتهي بتصميم موقّع، وليس بكود: - -```text -أنت ▸ أريد تقييمات لـ support bot الخاص بنا. أعتقد أنه سيء أحياناً. - -وكيل ▸ قبل أن أكتب أي شيء: اصف تشغيلاً سار بشكل جيد، - وواحداً سار بشكل سيء. - -أنت ▸ جيد: يجيب من المستندات. سيء: يختلق أشياء، - أو يستسلم ويقول "اتصل بالدعم". - -وكيل ▸ تشغيل: agenteye --json sessions --status error --since 7d - قراءة 5 جلسات، بما في ذلك اثنتان كنت ستسميهما سيئة. - - كلماتك تقول دقة وتحويل. تتفق النصوص على التحويل - (اثنتان من 5 تنتهي مع تحويل دعم، وهو واضح في - model_response.content)، لكن الدقة تحتاج حقيقة - أرضية لا يحملها حدثك. - - إذن كنت أقيّم ثلاثة أشياء: - - deflection قواعد، رخيص، يفصل تشغيلاتك - - tool_efficiency قواعد، يمسك حلقة الإعادة في run-004 - - groundedness قاضي LLM: هل كل مطالبة في المستندات التي سحبتها؟ - - كان "factuality" يحتاج حقيقة أرضية ليس لدينا؛ groundedness - هو أقرب شيء يمكن لنصوصك فعلاً دعمه. - بخير مع تلك الأسماء الثلاثة؟ إعادة التسمية لاحقاً تقسم التاريخ. - -أنت ▸ نعم - -وكيل ▸ لا يوجد مُقيّم في هذا الريبو. هل يجب أن أنشئ واحداً هنا، أم - لديك واحد في مكان آخر؟ -``` - -من هناك يكتب الأبعاد القائمة على القواعد أولاً (مجاني، فوري، حتمي)، يختبرها مقابل جلسة حقيقية مأخوذة بما في ذلك الفارغة والتي لم تنتهِ أبداً التي تحطم المُقيّمين الساذجين، ولا تصل فقط إلى قاضي LLM على البعد الموضوعي. يعرف حدود [المرسل](/ar/agenteye/evaluation-suite#configuring-the-server) — مهلة 30 ثانية وثمانية استدعاءات متزامنة بنطاق النشر — لذا إذا لم يناسب القاضي بشكل موثوق، يذهب غير متزامن مع `JobPending` بدلاً من السماح لقاضيك بالإلغاء وإعادة المحاولة خمس مرات بخمسة أضعاف التكلفة. - -ثم ينشر، يعيّن متغيري بيئة الخادم، ويؤكد مع `agenteye --json evals --session-id ` أن الدرجات هبطت فعلاً. هبوط الدرجات هو الدليل الوحيد. - ---- - -## ما يجب الانتباه له - -- **أسماء الأبعاد قريبة من الدائمة.** مفاتيح الدرجات سلاسل اختيارية والمنصة تتجه أينما أرسلت، مما يعني لا شيء يصحح لاحقاً خياراً سيئاً. أعد التسمية لاحقاً وينقسم التاريخ: الجلسات القديمة تحتفظ بالمفتاح القديم وينقطع الاتجاه. هذا هو السبب في حصول المهارة على موافقة صريحة قبل كتابة الكود — خذ هذا الحث بجدية. -- **الدعائم هي نصوص إنتاجية حقيقية.** يعني التصميم مقابل جلسات حقيقية سحبها إلى الديسك، ويمكنها أن تحتوي بيانات العملاء. تطلب المهارة قبل التعهد بهم إلى git؛ إذا كنت غير متأكد، احفظ `fixtures/` خارج الريبو وليترك كل مطور سحب الخاص به. -- **الوكيل يكتب وينشر خدمة تقرأ كل نص.** يتصرف كأنك، محدود بأذونات تسجيل دخول CLI الخاصة بك، لكن مراجعة المُقيّم مثل أي كود آخر يلمس بيانات الإنتاج. - ---- - -## الخطوات التالية - -- **[مجموعة التقييم](/ar/agenteye/evaluation-suite)**: عقد HTTP، SDK، ومتغيرات بيئة الخادم التي تقوم المهارة بتكوينها. -- **[التقييمات](/ar/agenteye/evaluations)**: حيث تظهر الدرجات مرة تهبط. -- **[مهارة CLI](/ar/agenteye/cli-skill)**: المهارة الشقيقة، لقراءة النتائج بدلاً من بناء المُقيّم. -- **[CLI](/ar/agenteye/cli)**: مرجع الأوامر خلف بيانات الجلسة التي تصممها المهارة. \ No newline at end of file diff --git a/docs/ar/agenteye/hermes-capture.mdx b/docs/ar/agenteye/hermes-capture.mdx deleted file mode 100644 index c65c2e61..00000000 --- a/docs/ar/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,54 +0,0 @@ ---- ---- -title: "التقاط جلسات Hermes" -description: "أحضر جلسات بوابة Hermes الخاصة بفريقك — Slack و Telegram و CLI والتشغيلات المجدولة — إلى AgentEye كجلسات وأحداث عادية." ---- - -[Hermes](https://hermes-agent.nousresearch.com) يجيب فريقك من أي مكان يعملون فيه بالفعل — Slack و Telegram و CLI والتشغيلات المجدولة. يجلب التقاط جلسات Hermes كل شيء إلى AgentEye كجلسات وأحداث عادية، بحيث يكون المساعد الذي يتحدث معه فريقك يومياً قابلاً للملاحظة مثل الوكلاء الذين تكتبهم بنفسك. - -يقرأ جامع خفيف محلي مخزن جلسات Hermes المحلي أثناء كتابته وينقل الجلسات إلى AgentEye. يعمل بنفس الطريقة التي تعمل بها عمليات التقاط [Codex](/ar/agenteye/codex-capture) و [OpenClaw](/ar/agenteye/openclaw-capture)، ويمكن لجامع واحد أن يلتقط عدة في نفس الوقت. - ---- - -## ما الذي يتم التقاطه - -يتم التقاط كل جلسة Hermes على الجهاز، بغض النظر عن القناة التي جاءت منها. تصبح كل واحدة منها [جلسة](/ar/agenteye/sessions) AgentEye؛ رسائل المستخدم والمساعد وعمليات استدعاء الأدوات ونتائج الأدوات تصبح [الأحداث](/ar/agenteye/event-stream) المطابقة. - -يتم تسجيل القناة التي بدأت منها الجلسة — Slack أو Telegram أو CLI أو تشغيل مجدول — على الجلسة، بحيث يمكنك التمييز بينها والتصفية إلى واحدة في كل مرة. بجانبها يأتي النموذج الذي قامت الجلسة عليه وبيانات الدردشة والشخص الذي بدأت منه، وعندما تولد جلسة أخرى، الارتباط بالجلسة الأب. - -تظهر الجلسات حالما يبدأها Hermes، سواء تم قول أي شيء أم لا، وتبقى إجابة الدور واستدعاءات أدواته بالترتيب الذي حدثت فيه بالفعل. عندما تنتهي جلسة، تحصل أيضاً على سبب إنهاؤها وتكلفتها وعدد الرموز التي استخدمتها. - ---- - -## فعّله - -التقاط معطل حتى تفعّله. ثبّت الجامع باستخدام مفتاح API له صلاحية `events:add` (انظر [مفاتيح API](/ar/agenteye/api-keys))، وفعّل التقاط Hermes: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -هذا يثبّت الجامع ويسجله كخدمة خلفية ويبدأ التقاط. تأكد من أنه يعمل: - -```bash -agenteye-collector health -``` - -تلتقط أكثر من وكيل واحد على نفس الجهاز؟ أضف علم كل منها لنفس الأمر — على سبيل المثال `--hermes-enabled --codex-enabled`. - -عند التشغيل الأول، يتم ملء جلسات Hermes الموجودة لديك مرة واحدة والنشاط الجديد يبدأ في البث خلال ثوانٍ. بيانات Hermes الخاصة بها تُقرأ فقط — لا تُعدّل أو تُحذف — وكل رسالة تُنقل مرة واحدة، حتى عبر عمليات إعادة التشغيل. - -يخبرك `health` أيضاً ما إذا كان كل شيء قام الجامع بالتقاطه وصل فعلاً إلى AgentEye. إذا تعذر تسليم دفعة، يتم الاحتفاظ بها وإعادة محاولتها بدلاً من التخلص منها، والفحص يبلّغ عن حالة غير صحيحة طالما أن أي شيء قيد الانتظار — لذا فإن "صحيح" يعني وصول بياناتك، وليس فقط أن العملية حية. - ---- - -## حيث يظهر - -تظهر الجلسات المقبوضة في **Sessions**، وأحداثها في تيار **Events**، تماماً مثل أي وكيل آخر تراقبه — لذلك [إعادة تشغيل الجلسات](/ar/agenteye/sessions) و [البحث](/ar/agenteye/queries) و [التقييمات](/ar/agenteye/evaluations) و [التنبيهات](/ar/agenteye/alerts) تعمل جميعها عليها. صفّ حسب وكيل Hermes لرؤيتها بمفردها. - ---- - -## الخصوصية - -تحتوي جلسات Hermes على النص الكامل — بما في ذلك مخرجات الأوامر ومحتويات الملفات وأي شيء قرأه الوكيل أو كتبه — وقد تحتوي على أسرار. يتم نقل الجلسات المقبوضة كما هي، لذا فعّل التقاط فقط حيث يكون تركيز هذا المحتوى في AgentEye مناسباً، وأعطِ الجامع مفتاحاً محدود النطاق بـ `events:add` فقط. انظر [الأمان](/ar/agenteye/security) لمعرفة كيفية الحفاظ على بياناتك معزولة. \ No newline at end of file diff --git a/docs/ar/agenteye/observability.mdx b/docs/ar/agenteye/observability.mdx deleted file mode 100644 index ad82447a..00000000 --- a/docs/ar/agenteye/observability.mdx +++ /dev/null @@ -1,24 +0,0 @@ ---- ---- -title: "مراقبة" -description: "أسطح المراقبة هي حيث تشاهد ما يفعله وكلاؤك الآن وتتعمق في أي تشغيل واحد." ---- - - -أسطح المراقبة هي حيث تشاهد ما يفعله وكلاؤك الآن وتتعمق في أي تشغيل واحد. كل شيء هنا مباشر، وفي نطاق مؤسستك، وقابل للتصفية حسب نطاق التاريخ والبيئة والوكيل والجلسة، لذا تنتقل من "هناك شيء ما يبدو غريباً" إلى التشغيل الدقيق في ثوانٍ. - -![تدفق الأحداث المباشر، مرمز بألوان حسب النوع وقابل للتصفية حسب البيئة والوكيل والجلسة](/agenteye/images/events-stream.png) - -أربعة أسطح، لكل منها صفحته الخاصة: - -- **[تدفق الأحداث](/ar/agenteye/event-stream)**: مسار مباشر خطوة تلو الخطوة لكل تشغيل عبر كل وكيل، الأحدث أولاً. منزل مؤسستك والمحطة الأولى للفرز. -- **[الجلسات والرسم البياني للتنفيذ](/ar/agenteye/sessions)**: تلك الأحداث مدمجة في صف واحد لكل تشغيل، بالإضافة إلى صورة بأسلوب git لكيفية تطور كل تشغيل. -- **[مقاييس الأداء](/ar/agenteye/telemetry)**: خرائط حرارية للكمون وحيويات p50/p95/p99 لنماذجك وأدواتك وخطافاتك، لذا تبرز ارتفاعات الذيل عن المتوسط. -- **[تتبع الأخطاء](/ar/agenteye/error-tracking)**: سطح فرز واحد لكل ما حدث خطأ، نقرة واحدة من تنبيه مُطلق إلى التشغيل الذي انكسر. - -## مرتبط - -- [التقييمات](/ar/agenteye/evaluations): قيّم كل تشغيل من حيث الجودة. -- [التنبيهات](/ar/agenteye/alerts): حول أي حد إلى قاعدة استدعاء. -- [عمليات التدقيق](/ar/agenteye/audits): اترك Failproof AI Observability تجد أنماط الفشل عبر الجلسات لك. -- [واجهة سطر الأوامر والوكلاء](/ar/agenteye/cli-and-agents): نفس القابلية للمراقبة من محطتك الطرفية. \ No newline at end of file diff --git a/docs/ar/agenteye/openclaw-capture.mdx b/docs/ar/agenteye/openclaw-capture.mdx deleted file mode 100644 index 091183a6..00000000 --- a/docs/ar/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,50 +0,0 @@ ---- ---- -title: "التقاط جلسات OpenClaw" -description: "قم بتتبع جلسات OpenClaw المحلية لفريقك في AgentEye كجلسات وأحداث عادية — دون أي تغيير في طريقة تشغيل OpenClaw." ---- - -إذا كان فريقك يستخدم [OpenClaw](https://docs.openclaw.ai)، فإن التقاط جلسات OpenClaw يجلب تلك الجلسات إلى AgentEye كجلسات وأحداث عادية، بحيث يمكنك البحث عنها وإعادة تشغيلها وتقييمها جنباً إلى جنب مع كل شيء آخر تلاحظه. يكمل هذا [Python SDK](/ar/agenteye/python-sdk): يقوم SDK بتطبيق أدوات على الوكلاء الذين تكتبهم، بينما هذا يلتقط عمل OpenClaw الذي يقوم به فريقك بالفعل — دون أي تغيير في طريقة تشغيله. - -يقرأ جامع خلفية صغير نصوص جلسات OpenClaw المحلية كما تُكتب وينقلها إلى AgentEye. يعمل بنفس الطريقة التي يعمل بها [التقاط Codex](/ar/agenteye/codex-capture)، ويمكن لجامع واحد أن يلتقط كليهما في نفس الوقت. - ---- - -## ما الذي يتم التقاطه - -يتم التقاط كل وكيل تم تكوينه في إعداد OpenClaw على جهاز ما بواسطة جامع ذلك الجهاز — لا يوجد إعداد لكل وكيل. - -تصبح كل جلسة OpenClaw [جلسة](/ar/agenteye/sessions) في AgentEye؛ رسائل المستخدم والمساعد وعمليات الأدوات ونتائج الأدوات تصبح [الأحداث](/ar/agenteye/event-stream) المطابقة. - ---- - -## تشغيله - -التقاط مطفأ حتى تقوم بتفعيله. قم بتثبيت الجامع باستخدام مفتاح API له صلاحية `events:add` (راجع [مفاتيح API](/ar/agenteye/api-keys))، وقم بتشغيل التقاط OpenClaw: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -يقوم هذا بتثبيت الجامع وتسجيله كخدمة خلفية وبدء التقاط. تأكد من أنه يعمل: - -```bash -agenteye-collector health -``` - -هل تلتقط أكثر من وكيل واحد على نفس الجهاز؟ أضف علم كل واحد منهم إلى نفس الأمر — على سبيل المثال `--openclaw-enabled --codex-enabled`. - -عند التشغيل الأول، يتم ملء جلسات OpenClaw الموجودة لديك مرة واحدة ثم يبدأ النشاط الجديد في البث خلال ثوانٍ. لا تُقرأ ملفات OpenClaw الخاصة بها أبداً — لا تُعدَّل أو تُنقل أو تُحذف — وتُرسل كل جلسة مرة واحدة بالضبط، حتى عند إعادة التشغيل. - ---- - -## حيث يظهر - -تظهر الجلسات المُلتقطة في **Sessions**، وأحداثها في تدفق **Events**، تماماً مثل أي وكيل آخر تلاحظه — لذا فإن [إعادة تشغيل الجلسة](/ar/agenteye/sessions) و[البحث](/ar/agenteye/queries) و[التقييمات](/ar/agenteye/evaluations) و[التنبيهات](/ar/agenteye/alerts) تعمل جميعها عليها. قم بالتصفية حسب وكيل OpenClaw لرؤيتها بمفردها. - ---- - -## الخصوصية - -تحتوي نصوص OpenClaw على الجلسة الكاملة — بما في ذلك مخرجات الأوامر ومحتويات الملفات وأي شيء قرأه الوكيل أو كتبه — وقد تحتوي على أسرار. يتم شحن الجلسات المُلتقطة كما هي، لذا قم بتفعيل التقاط فقط على الأجهزة والفرق حيث يكون من المناسب مركزية هذا المحتوى في AgentEye، وأعط الجامع مفتاحاً محدوداً بـ `events:add` فقط. راجع [Security](/ar/agenteye/security) لمعرفة كيفية حفاظ نظامك على بيانات معزولة. \ No newline at end of file diff --git a/docs/ar/agenteye/python-sdk-skill.mdx b/docs/ar/agenteye/python-sdk-skill.mdx deleted file mode 100644 index e61c8734..00000000 --- a/docs/ar/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,129 +0,0 @@ ---- -title: "مهارة Failproof AI Observability Python SDK للعامل" -description: "انتقل من عامل بدون أدوات مراقبة إلى أحداث يمكنك رؤيتها، حيث يعثر عاملك البرمجي على نقاط الأدوات، ويكتبها، ويثبت أنها عملت بنجاح." ---- - -أخبر عاملك البرمجي *"أضف Failproof AI Observability إلى هذا العامل"* واتركه يقرأ حلقتك، ويعرّف مكان إضافة الأدوات، ويكتبها، ويتحقق من الأحداث قبل إكمال المهمة. - -**مهارة Python SDK** (`agenteye-python-sdk`) هي *مهارة عامل*: مجلد يحتوي على تعليمات يحملها عامل برمجي مثل Claude Code أو Codex عند الحاجة عندما تطابق المهمة. تعلم العامل كيفية استخدام [Python SDK](/ar/agenteye/python-sdk) — لا تعتبر مكتبة، وليس لها أي تأثير على طريقة عمل SDK. - -## الأدوات سهلة الكتابة وسهل الخطأ فيها بهدوء - -SDK صغير: ثلاثة عشر طريقة حدث، جميعها بكلمات مفتاحية فقط. يمكن لعامل برمجي قراءة مرجع [Python SDK](/ar/agenteye/python-sdk) وإنتاج أدوات معقولة في دقيقة واحدة. - -المشكلة هي أن SDK هذا لا يرفع استثناءً عند الخطأ، والأدوات الخاطئة تبدو تماماً مثل الأدوات الصحيحة حتى يفتح أحدهم لوحة التحكم ويجدها فارغة. الأخطاء التي تستهلك وقتاً حقيقياً كلها صمتية: - -| الخطأ | ما تراه | -|---|---| -| لا يوجد `agent_start` | كل حدث يهبط. صفر جلسات. | -| لم يتم تعيين البيئة أبداً | كل شيء يعمل، مرفوع ضمن `dev`. | -| `outcome="failure"` | يظهر التشغيل أخضر — فقط `failed`, `error`, `timeout`, `rejected` يتم عدها. | -| اسم حقل به خطأ إملائي | مقبول ومخزن كحقل جديد. | -| أحداث انبعثت من مجموعة خيوط | تم حذفها بهدوء. | - -لا أحد منهم يرفع استثناءً. لا أحد يظهر في الاختبارات. كل واحد منهم في المهارة، موضح كعقد مع الفحص الذي يكتشفه. - -## ما تفعله، بالترتيب - -تنفذ المهارة نفس الخطوات الثلاث التي سيتخذها مهندس حذر: - -1. **التخطيط.** تقرأ حلقة العامل لديك وتطرح السؤالين الذين يمكن لك وحدك الإجابة عليهما: ما الذي يعتبر تشغيلاً واحداً (`session_id`)، وَمَن الممثلون المختلفون (`agent_id`). تحصل على الموافقة قبل كتابة الكود، لأن تغييرهما لاحقاً يقسم السجل ويكسر الاتجاهات. -2. **الكتابة.** تربط الهوية مرة واحدة لكل تشغيل بدلاً من تمريرها عبر كل موقع استدعاء، وتختار شكلاً آمناً للتزامن — تفصيل مهم، لأن الاختصار الواضح يمزج بهدوء تشغيلين متداخلين في جلسة واحدة. -3. **التحقق.** تشغل عاملك وتقرأ ملفات الأحداث الناتجة، تتحقق من وجود `agent_start`، والبيئة صحيحة، وتشغيل واحد ينتج جلسة واحدة. - -تلك الخطوة الثالثة هي التي يتخطاها الناس. SDK يكتب الأحداث في ملفات محلية، لذلك يمكن إثبات تكامل كامل على جهاز محمول بدون خادم، بدون مفتاح API، وبدون شبكة — وهذا بالضبط السبب في إصرار المهارة على القيام به. - -## كيفية ارتباطها بالمهارات الأخرى - -ثلاث مهارات، تقسيم نظيف واحد: - -| المهارة | استخدمها عندما | ما الذي تلمسه | -|---|---|---| -| **مهارة Python SDK** (هذه الصفحة) | تريد من عاملك أن *ينبعث* من بيانات المراقبة — "أضف المراقبة"، "لماذا لا يظهر عاملي؟" | تكتب الكود في مستودع عاملك. لا تقرأ أي شيء. | -| **[مهارة المُقيّم](/ar/agenteye/evaluator-skill)** | تريد *تصنيف* التشغيلات — "ما الذي يجب أن نقيسه حتى؟" | تكتب الكود في مستودعك؛ تقرأ بيانات المراقبة | -| **[مهارة CLI](/ar/agenteye/cli-skill)** | تريد *قراءة* ما حدث، أو تشغيل نشرك | تقود CLI كما أنت، بما في ذلك التغييرات | - -تمرر بهذا الترتيب: هذه المهارة تجعل الأحداث تتدفق، يقيّمها المُقيّم، CLI يقرأها مرة أخرى. لا يوجد شيء لتقييمه ولا شيء لقراءته حتى يصدر عاملك جلسات، لذا إذا كنت تبدأ من الصفر، ابدأ هنا. - -## المتطلبات الأساسية - -1. **Python 3.10+** ومستودع الكود للعامل الذي تريد إضافة أدوات له. -2. **SDK.** يتم توزيعه للعملاء كعجلة خاصة بدلاً من فهرس عام — يشرح الإعداد الخاص بك كيفية الحصول عليها وتثبيتها. تعرف المهارة مسار التثبيت وستطلب منك بدلاً من التخمين إذا لم تجده. -3. **لا شيء آخر.** لا يوجد تسجيل دخول لوحة التحكم، لا مفتاح API، لا شبكة. تتحقق المهارة من ملفات الأحداث التي يكتبها SDK، لذلك يمكنها الإنهاء والإثبات بدون اتصال. - -## أين تجده - -تعيش المهارة في مجموعة [`FailproofAI/skills`](https://github.com/FailproofAI/skills) العامة: - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -أضف `-g` لتثبيتها في كل مشروع بدلاً من المشروع الحالي فقط، و`--copy` إذا كانت بيئتك لا تتبع الروابط الرمزية. بالنسبة لـ Codex، مرر `-a codex`. - -## تثبيتها يدوياً - -مهارات العامل عبارة عن مجلدات تحتوي على `SKILL.md` بالإضافة إلى مراجع. إذا كنت تفضل عدم استخدام المثبت: - -- **Claude Code**: انسخ مجلد `agenteye-python-sdk/` إلى `~/.claude/skills/` (كل مشروع) أو `/.claude/skills/` (هذا المستودع فقط). يكتشفه Claude Code تلقائياً — تحقق من قائمة `/skills`، أو ببساطة اطلب شيئاً يطابقه. -- **Codex**: يقرأ Codex نفس `SKILL.md`. يعيّن `agents/openai.yaml` المدرج `allow_implicit_invocation: true`، لذا يتم تحديده تلقائياً عند مطابقة المهمة؛ وإلا استدعه كـ `$agenteye-python-sdk`. - -شغّل عاملك **في المستودع الذي يحتوي على الكود الذي تريد إضافة أدوات له** — تقرأ المهارة حلقة عاملك قبل أن تقترح أي شيء. - -## شكل الجلسة - -```text -you ▸ أضف Failproof AI Observability إلى هذا العامل. - -agent ▸ اقرأ حلقتك — موزع أدوات واحد في tools.py، غلاف Anthropic - واحد في llm.py. سؤالان قبل أن أكتب أي شيء. - - ما الذي يعتبر تشغيلاً واحداً؟ تتعامل مع الطلبات بالتزامن، لذلك سأستخدم - معرّف الطلب الذي لديك بالفعل في ctx.request_id — ثم جلسة هنا - وطلب في سجلاتك هما نفس السلسلة. - - ومَن الممثلون؟ أرى مسار مخطط ومسار باحث. صنّفهما بشكل منفصل، - أم هو عامل واحد بالنسبة لك؟ - -you ▸ معرّف الطلب صحيح. إنهما عامل واحد، سمّه "مساعد". - -agent ▸ تم. لفّ الموزع وعميل LLM؛ يضع agent_start و - agent_end أقواس handle_request، بما في ذلك مسار الخطأ. استخدمت - سياق محلي للهوية — يشغل موزعك الأدوات في مجموعة خيوط، - والنهج الأبسط كان سيمزج التشغيلات المتزامنة في جلسة واحدة بدون شكوى. - - تحقق من تشغيل حقيقي: 11 حدث عبر 4 ملفات — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, جلسة واحدة - لكل طلب، المدد المقترنة. - - شيء واحد لم أفعله: مجموعة العمال لديك في batch.py تقدم - مباشرة إلى المنفذ، لذا الأحداث من هناك ستُحذف. تريد مني أن أصلح تلك أيضاً؟ -``` - -النمط الذي يجب ملاحظته: لقد قرأ الكود قبل الاقتراح، وطرح فقط الأسئلة التي يمكنك الإجابة عليها، وأعاد استخدام معرّف كان لديك بالفعل، اختار الشكل الآمن للتزامن *لأنه* رأى مجموعة خيوط، و**تحقق من خلال قراءة الأحداث الفعلية** بدلاً من التصريح بالنجاح — ثم وضع علامة على المكان الوحيد الذي عرف أنه سيفشل بهدوء. - -## ما يمكنك طلبه - -- *"لماذا لا يظهر عاملي على لوحة التحكم؟"* → يسير على السلم: هل يتم كتابة الأحداث، هل يوجد `agent_start`، هل البيئة صحيحة، هل يقرأ المجمّع نفس المكان. -- *"كل شيء يهبط تحت dev."* → لم يتم تعيين البيئة، أو تم إعادة تعيينها بعد ذلك. -- *"أضف تتبع الرموز."* → يجد غلاف LLM لديك ويسجل النموذج، سبب التوقف، والاستخدام. -- *"أضف أدوات للعوامل الفرعية أيضاً."* → جلسة واحدة، تصنيفات عامل مختلفة، مدرجة تحت أبيهما. -- *"اكتب اختبارات للأدوات."* → وجّه SDK إلى دليل مؤقت ويؤكد على الأحداث التي كتبها. - -## ما يجب الانتباه له - -**دعها تتحقق.** الخطوة التي تجعل هذه المهارة تستحق الاستخدام هي الأخيرة — تشغيل عاملك وقراءة الأحداث مرة أخرى. عامل يكتب أدوات ويتوقف قد فعل النصف السهل، والنصف الذي يفشل بهدوء هو الآخر. - -**وافق على الأسماء قبل الكود.** `session_id` و`agent_id` هما المحاور التي تجمع بها كل سطح. إعادة تسميتها لاحقاً تقسم السجل: التشغيلات القديمة تحتفظ بالتصنيفات القديمة وتنكسر الاتجاهات. ستسأل المهارة؛ الإجابة تستحق دقيقة تفكير. - -**إذا اقترح عاملك تثبيت SDK من فهرس عام، لم تُحمّل المهارة.** يتم توزيع SDK بشكل خاص. ذلك الاقتراح هو مؤشر موثوق على أن عاملك البرمجي يخمّن بدلاً من اتباع المهارة — توقفه هناك وتحقق من تثبيت المهارة. - -وراء ذلك نطاق انفجاره صغير: يكتب الكود في دليل العمل وملفات الأحداث حيث تخبره. لا يقرأ من نشرك ولا يغير شيئاً عنه. - -## الخطوات التالية - -- **[Python SDK](/ar/agenteye/python-sdk)**: مرجع الحدث الكامل — كل نوع حدث وحقل — خلف ما تأتمت هذه المهارة. -- **[الجلسات](/ar/agenteye/sessions)**: ما تنتجه أدواتك مرة هبطت الأحداث. -- **[مهارة عامل المُقيّم](/ar/agenteye/evaluator-skill)**: الخطوة التالية بمجرد هبوط التشغيلات — تصنيفها. -- **[مهارة عامل CLI](/ar/agenteye/cli-skill)**: قراءة بيانات المراقبة مرة أخرى. \ No newline at end of file diff --git a/docs/ar/agenteye/security.mdx b/docs/ar/agenteye/security.mdx deleted file mode 100644 index 51a70785..00000000 --- a/docs/ar/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "الأمان" -description: "تم بناء Failproof AI Observability للعمل بالقرب من وكلائك الإنتاجيين، مما يعني أنها ترى موجهاتك ومدخلات الأدوات والمخرجات." ---- - - -تم بناء Failproof AI Observability للعمل بالقرب من وكلائك الإنتاجيين، مما يعني أنها ترى موجهاتك ومدخلات الأدوات والمخرجات. توضح هذه الصفحة كيفية الحفاظ على عزل هذه البيانات والتحكم فيها وإبقاؤها في يديك. إذا كنت تقيّم Failproof AI Observability لمراجعة أمان، فابدأ من هنا. - ---- - -## بيانات تبقى في بيئتك - -Failproof AI Observability مستضافة ذاتياً. يتم تخزين الأحداث والموجهات واستجابات النموذج والتحليلات في قواعد بيانات خاصة بك، في بيئتك الخاصة. لا يتم إرسال أي شيء إلى طرف ثالث SaaS للتخزين، وتبقى بيانات عملك في حساب السحابة الخاص بك. - ---- - -## عزل المستأجرين - -يمكن لمثيل واحد من Failproof AI Observability استضافة عدة منظمات، وكل منها معزولة على مستوى التخزين — مفروض من قبل قاعدة البيانات وليس من الواجهة فقط: - -- بيانات المنظمة التشغيلية (المستخدمون والمفاتيح لوحات التحكم والاستعلامات المحفوظة) يتم تحديد نطاقها لتلك المنظمة، وتحظر قاعدة البيانات نفسها القراءات عبر المنظمات. -- كل حدث مُدرج موسوم بمنظمته المالكة، لذا لا يمكن أبداً قراءة أحداث منظمة واحدة من قبل منظمة أخرى. - -كل مسار لوحة تحكم يتم تحديد نطاقه تحت شعار منظمة (`//…`). - ---- - -## تسجيل الدخول - -تستخدم Failproof AI Observability تسجيل دخول بدون كلمة مرور قائم على البريد الإلكتروني. لا توجد كلمة مرور يمكن اختراقها أو تسريبها. يطلب المستخدم رمزاً لمرة واحدة (أو رابط سحر بنقرة واحدة)، والذي يُرسل إليه عبر البريد الإلكتروني وينتهي صلاحيته بسرعة. يتم حماية تسجيل الدخول بواسطة **قائمة بيضاء**: فقط عناوين البريد الإلكتروني (أو النطاقات) التي تسمح بها يمكنها المصادقة. - -![شاشة تسجيل دخول Failproof AI Observability، التي ترسل رمزاً لمرة واحدة إلى بريدك الإلكتروني](/agenteye/images/login.png) - ---- - -## الوصول المحدود باستخدام مفاتيح API - -يقوم كل عميل بالمصادقة باستخدام مفتاح API يحمل أذونات دقيقة وذات امتيازات محدودة. يحتاج المجمِّع فقط إلى `events:add`؛ يمكن أن يكون مفتاح لوحة التحكم أو المساعد بقراءة فقط؛ الإجراءات الضارة (الحذف وإعادة التوليد) هي منح منفصلة تختار تضمينها. - -![صفحة مفاتيح API: منحات أذونات كل مفتاح، مرمّزة بألوان حسب نطاق القراءة والكتابة والتدمير](/agenteye/images/api-keys.png) - -احتفظ بمفتاح bootstrap الإداري للإعداد، واستخدم مفاتيح محدودة لكل شيء آخر. انظر [مفاتيح API](/ar/agenteye/api-keys). - ---- - -## مساعد بقراءة فقط وموافقة مبوابة - -يجيب [المساعد في لوحة التحكم](/ar/agenteye/assistant) على أسئلة حول بيانات عملك، لكنه مقيد بالتصميم: - -- أنه **بقراءة فقط افتراضياً**: SQL الخاص به يمر عبر حراس يسمح فقط باستعلامات `SELECT`/`WITH`، بيان واحد، مع حد أقصى للصفوف. -- أي شيء ينشئه (استعلام محفوظ، لوحة تحكم) هو **موافقة مبوابة**: تراجع وتوافق على كل عملية كتابة قبل حدوثها. -- أنه **لا يمكنه أبداً الحذف**. - -لذا يمكن لزميل في الفريق أن يسأل "أي وكلاء أخطؤوا أكثر هذا الأسبوع؟" والتصرف بناءً على الإجابة، دون أن يتمكن المساعد من تغيير أو إزالة بيانات عملك بمفرده. - ---- - -## في النقل - -كل حركة المرور تعمل عبر HTTPS. تقوم بإنهاء TLS باستخدام شهاداتك الخاصة، لذلك يتم تشفير حركة المرور من المجمِّع إلى الخادم ومن المتصفح إلى الخادم أثناء النقل. - ---- - -## الخطوات التالية - -- [نظرة عامة](/ar/agenteye/overview): كيف تتناسب Failproof AI Observability معاً. -- [مفاتيح API](/ar/agenteye/api-keys): تحديد نطاق الوصول للمجمِّع ولوحة التحكم والمساعد. -- [القابلية للملاحظة](/ar/agenteye/observability): ما تلتقطه Failproof AI Observability من وكلائك. \ No newline at end of file diff --git a/docs/ar/cli/audit.mdx b/docs/ar/audit.mdx similarity index 100% rename from docs/ar/cli/audit.mdx rename to docs/ar/audit.mdx diff --git a/docs/ar/cli/backfill.mdx b/docs/ar/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/ar/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/ar/cli/config.mdx b/docs/ar/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/ar/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/ar/cli/flush.mdx b/docs/ar/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/ar/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/ar/cli/harness.mdx b/docs/ar/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/ar/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/ar/cli/migrate.mdx b/docs/ar/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/ar/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/ar/cli/uninstall.mdx b/docs/ar/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/ar/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/ar/cli/update.mdx b/docs/ar/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/ar/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/ar/agenteye/api-keys.mdx b/docs/ar/cloud/access.mdx similarity index 95% rename from docs/ar/agenteye/api-keys.mdx rename to docs/ar/cloud/access.mdx index 05043f71..b433e352 100644 --- a/docs/ar/agenteye/api-keys.mdx +++ b/docs/ar/cloud/access.mdx @@ -1,9 +1,9 @@ --- title: "مفاتيح API" -description: "تتحكم مفاتيح API بمن وما يمكنه الوصول إلى خادم Failproof AI Observability، بحيث يمكن لأداة جمع البيانات إرسال الأحداث دون الحصول على صلاحيات القراءة أو الإدارة." +description: "تتحكم مفاتيح API بمن وما يمكنه الوصول إلى خادم FailproofAI Cloud، بحيث يمكن لأداة جمع البيانات إرسال الأحداث دون الحصول على صلاحيات القراءة أو الإدارة." --- -تتحكم مفاتيح API بمن وما يمكنه الوصول إلى خادم Failproof AI Observability، بحيث يمكن لأداة جمع البيانات إرسال الأحداث دون الحصول على صلاحيات القراءة أو الإدارة. يحمل كل مفتاح واحداً أو أكثر من الصلاحيات، وكل صلاحية تتحكم في مسارات خادم محددة؛ فأنت تمنح فقط ما تحتاجه المهمة. تنشئ معظم عمليات النشر ثلاثة أنواع من المفاتيح فقط. +تتحكم مفاتيح API بمن وما يمكنه الوصول إلى خادم FailproofAI Cloud، بحيث يمكن لأداة جمع البيانات إرسال الأحداث دون الحصول على صلاحيات القراءة أو الإدارة. يحمل كل مفتاح واحداً أو أكثر من الصلاحيات، وكل صلاحية تتحكم في مسارات خادم محددة؛ فأنت تمنح فقط ما تحتاجه المهمة. تنشئ معظم عمليات النشر ثلاثة أنواع من المفاتيح فقط. ## المفاتيح الثلاثة التي تحتاجها معظم عمليات النشر @@ -81,7 +81,7 @@ description: "تتحكم مفاتيح API بمن وما يمكنه الوصول تدعم هذه الصلاحيات صفحة لوحة التحكم **المستخدمون**، حيث يتم عرض النطاقات الممنوحة لكل عضو كرقائق: -![صفحة المستخدمون: بطاقة لكل مستخدم لوحة تحكم مع بريده الإلكتروني والصلاحيات الممنوحة والتحكم في التعديل/التعطيل](/agenteye/images/users.png) +![صفحة المستخدمون: بطاقة لكل مستخدم لوحة تحكم مع بريده الإلكتروني والصلاحيات الممنوحة والتحكم في التعديل/التعطيل](/cloud/images/users.png) ### الإعدادات التشغيلية @@ -90,7 +90,7 @@ description: "تتحكم مفاتيح API بمن وما يمكنه الوصول | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | عرض الإعدادات التشغيلية المدارة بلوحة التحكم وبيانات التعريف الخاصة بها؛ إدراج تجاوزات نافذة السياق حسب النموذج؛ وحل النافذة الفعالة للنموذج. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | تعديل الإعدادات التشغيلية وإضافة أو تغيير أو إزالة تجاوزات نافذة السياق حسب النموذج. تؤثر التغييرات على الأحداث الجديدة دون إعادة تشغيل الخادم. | -![صفحة الإعدادات: إعدادات تشغيلية مدارة بلوحة التحكم مثل عمليات تسجيل الدخول المسموحة وأعمار الجلسات / OTP، قابلة للتعديل دون إعادة تشغيل](/agenteye/images/settings.png) +![صفحة الإعدادات: إعدادات تشغيلية مدارة بلوحة التحكم مثل عمليات تسجيل الدخول المسموحة وأعمار الجلسات / OTP، قابلة للتعديل دون إعادة تشغيل](/cloud/images/settings.png) ### التنبيهات والحوادث @@ -246,7 +246,7 @@ curl -s -X POST http://your-server/keys//regenerate \ عند إنشاء مفتاح من لوحة التحكم لا تقدم السر؛ تنتج لوحة التحكم سراً قوياً لك وتعرضه **مرة واحدة** عند الإنشاء. انسخه فوراً وخزّنه بأمان؛ لا يتم عرضه أبداً مرة أخرى، تماماً كما هو الحال مع إعادة الإنشاء. لا يزال بإمكانك انتقاء صلاحيات المفتاح مباشرة، أو تغذيتها من مجموعة صلاحيات (انظر أدناه). -![صفحة مفاتيح API: بطاقة لكل مفتاح توضح اسمه والصلاحيات الممنوحة ووقت الإنشاء، مع إجراءات إعادة الإنشاء والتعطيل؛ يتم وضع علامة على المفاتيح المحمية مثل `admin`](/agenteye/images/api-keys.png) +![صفحة مفاتيح API: بطاقة لكل مفتاح توضح اسمه والصلاحيات الممنوحة ووقت الإنشاء، مع إجراءات إعادة الإنشاء والتعطيل؛ يتم وضع علامة على المفاتيح المحمية مثل `admin`](/cloud/images/api-keys.png) --- @@ -275,5 +275,5 @@ curl -s -X POST http://your-server/keys//regenerate \ ## الخطوات التالية -- [Python SDK](/ar/agenteye/python-sdk): كيفية مصادقة كود الوكيل عند إرسال الأحداث. -- [الأمان](/ar/agenteye/security): كيف يعمل تسجيل الدخول والتحكم في الوصول وعزل البيانات لكل منظمة. \ No newline at end of file +- [Python SDK](/ar/cloud/sdk): كيفية مصادقة كود الوكيل عند إرسال الأحداث. +- [الأمان](/ar/cloud/security): كيف يعمل تسجيل الدخول والتحكم في الوصول وعزل البيانات لكل منظمة. \ No newline at end of file diff --git a/docs/ar/cloud/agent-skills.mdx b/docs/ar/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/ar/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/ar/agenteye/alerts.mdx b/docs/ar/cloud/alerts.mdx similarity index 77% rename from docs/ar/agenteye/alerts.mdx rename to docs/ar/cloud/alerts.mdx index 7e2f9959..1472c6e7 100644 --- a/docs/ar/agenteye/alerts.mdx +++ b/docs/ar/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "اكتشف اللحظة التي يتجاوز فيها شيء ما --- -اكتشف اللحظة التي يتجاوز فيها شيء ما حدك، على القناة التي يراقبها فريقك بالفعل، بدلاً من سماعها من العميل. عيّن قاعدة مرة واحدة و Failproof AI Observability تفحصها وفقاً لجدول زمني، ثم ترسل إليك تنبيهاً عبر البريد الإلكتروني أو Slack أو webhook أو مباشرة في لوحة التحكم. +اكتشف اللحظة التي يتجاوز فيها شيء ما حدك، على القناة التي يراقبها فريقك بالفعل، بدلاً من سماعها من العميل. عيّن قاعدة مرة واحدة و FailproofAI Cloud تفحصها وفقاً لجدول زمني، ثم ترسل إليك تنبيهاً عبر البريد الإلكتروني أو Slack أو webhook أو مباشرة في لوحة التحكم. -![صفحة التنبيهات: شبكة من بطاقات قواعد التنبيهات، تعرض كل منها محفزها ونافذة التقييم والقنوات وشارة الخطورة (معلومات أو تحذير أو حرج)](/agenteye/images/alerts.png) +![صفحة التنبيهات: شبكة من بطاقات قواعد التنبيهات، تعرض كل منها محفزها ونافذة التقييم والقنوات وشارة الخطورة (معلومات أو تحذير أو حرج)](/cloud/images/alerts.png) *كل قاعدة تنبيه في نظرة واحدة: ما الذي تراقبه وعدد المرات والقنوات ومستوى الإلحاح.* ## اعرف عن المشاكل قبل مستخدميك @@ -22,9 +22,9 @@ description: "اكتشف اللحظة التي يتجاوز فيها شيء ما ## بناء القاعدة في نموذج، وليس JSON -تصف ما معنى أن يكون الشيء "معطلاً" في نموذج، و Failproof AI Observability تكتب القاعدة الأساسية لك. مواصفات JSON ليست سوى ما ينتجه هذا النموذج تحت الغطاء، حتى تتمكن من قراءتها لفهم قاعدة لكن نادراً ما تكتبها. +تصف ما معنى أن يكون الشيء "معطلاً" في نموذج، و FailproofAI Cloud تكتب القاعدة الأساسية لك. مواصفات JSON ليست سوى ما ينتجه هذا النموذج تحت الغطاء، حتى تتمكن من قراءتها لفهم قاعدة لكن نادراً ما تكتبها. -![نموذج التنبيه الجديد: الاسم والوصف وزر التفعيل واختيار المحفز يعرض عتبة المقياس والـ SQL المخصص ودرجة التقييم والتقييم المركب والشروط لكل حدث](/agenteye/images/alert-new.png) +![نموذج التنبيه الجديد: الاسم والوصف وزر التفعيل واختيار المحفز يعرض عتبة المقياس والـ SQL المخصص ودرجة التقييم والتقييم المركب والشروط لكل حدث](/cloud/images/alert-new.png) *اختر محفزاً والنموذج يعدّل الحقول المناسبة؛ الحفظ يكتب القاعدة.* المسار السعيد سريع: سمِّه، اختر **محفز** (ما يجب مراقبته)، عيّن **العتبة والنافذة** (مدى السوء وعلى مدى كم من الوقت)، أرفق قناة واحدة على الأقل، ثم **احفظ** و**اختبر** لإرسال إخطار تركيبي وتأكيد أن كل وجهة متصلة. تحت الغطاء ينتج عن هذا مواصفات صغيرة مثل: @@ -43,7 +43,7 @@ description: "اكتشف اللحظة التي يتجاوز فيها شيء ما | **التقييم المركب** | عدة فحوصات درجات تجتمع مع أي أو الكل أو على الأقل منطق N، لاكتشاف انحدار يظهر فقط عبر الدرجات | | **لكل حدث** | يصل حدث واحد مطابق: وكيل محدد أو نوع خطأ محدد أو جزء رسالة | -تحدق بالفعل في خلل على [صفحة الأخطاء](/ar/agenteye/error-tracking)؟ كل صف هناك به زر **+ تنبيه** يفتح نفس النموذج معبأ مسبقاً لاكتشاف هذا الخلل بالضبط مرة أخرى، حتى الحادثة التي قمت بفحصها للتو تصبح الحادثة التي ستنبهك في المرة القادمة. +تحدق بالفعل في خلل على [صفحة الأخطاء](/ar/cloud/errors)؟ كل صف هناك به زر **+ تنبيه** يفتح نفس النموذج معبأ مسبقاً لاكتشاف هذا الخلل بالضبط مرة أخرى، حتى الحادثة التي قمت بفحصها للتو تصبح الحادثة التي ستنبهك في المرة القادمة. **حيث تجده:** التنبيهات موجودة في `//alerts`. إنشاء وتحرير وحذف واختبار القواعد يتطلب `alerts:write`؛ `alerts:read` كافٍ للمراقبة. منتقي المستقبل يسرد أعضاء منظمتك حسب الاسم، حتى تتمكن من إنذار شخص ما دون مغادرة النموذج. @@ -53,11 +53,11 @@ description: "اكتشف اللحظة التي يتجاوز فيها شيء ما ## ما يحدث عندما ينطلق تنبيه -يفتح انتهاك **حادثة** وينبه قنواتك مرة واحدة. من هناك يعترف فريقك بها ويعين مالكاً ويناقشها ويحلها، كل ذلك ضد سجل نظيف ومنسوب. لهذا سير العمل في الفحص منزل خاص به: انظر [الحوادث](/ar/agenteye/incidents). +يفتح انتهاك **حادثة** وينبه قنواتك مرة واحدة. من هناك يعترف فريقك بها ويعين مالكاً ويناقشها ويحلها، كل ذلك ضد سجل نظيف ومنسوب. لهذا سير العمل في الفحص منزل خاص به: انظر [الحوادث](/ar/cloud/incidents). ## ذات صلة -- [الحوادث](/ar/agenteye/incidents): تتبع تنبيه منطلق من مفتوح إلى معترف به إلى محلول. -- [تتبع الأخطاء](/ar/agenteye/error-tracking): تجميع إخفاقات الوكيل وترقية واحد إلى تنبيه بنقرة واحدة. -- [لوحات التحكم](/ar/agenteye/dashboards): راقب اللوحات المشتركة التي تأتي منها العتبات التي تنبه عليها. -- [CLI والوكلاء](/ar/agenteye/cli-and-agents): أنشئ تنبيهات وأقرّ الحوادث من محطتك الطرفية أو أدخلها في CI. \ No newline at end of file +- [الحوادث](/ar/cloud/incidents): تتبع تنبيه منطلق من مفتوح إلى معترف به إلى محلول. +- [تتبع الأخطاء](/ar/cloud/errors): تجميع إخفاقات الوكيل وترقية واحد إلى تنبيه بنقرة واحدة. +- [لوحات التحكم](/ar/cloud/dashboards): راقب اللوحات المشتركة التي تأتي منها العتبات التي تنبه عليها. +- [CLI والوكلاء](/ar/cloud/cli): أنشئ تنبيهات وأقرّ الحوادث من محطتك الطرفية أو أدخلها في CI. \ No newline at end of file diff --git a/docs/ar/agenteye/assistant.mdx b/docs/ar/cloud/assistant.mdx similarity index 83% rename from docs/ar/agenteye/assistant.mdx rename to docs/ar/cloud/assistant.mdx index 25309003..78a3d0e7 100644 --- a/docs/ar/agenteye/assistant.mdx +++ b/docs/ar/cloud/assistant.mdx @@ -3,9 +3,9 @@ title: "مساعد ذكاء اصطناعي" description: "اطرح سؤالاً على بيانات وكيلك بلغة إنجليزية عادية واحصل على إجابة مرتبطة مباشرة بالأدلة." --- -اطرح سؤالاً على بيانات وكيلك بلغة إنجليزية عادية واحصل على إجابة مرتبطة مباشرة بالأدلة. لا حاجة لكتابة SQL، ولا حاجة للحفر في لوحات المعلومات — مساعد **Failproof AI Observability** هو أسرع طريقة لأي شخص في فريقك للحصول على إجابات حول وكلائك. +اطرح سؤالاً على بيانات وكيلك بلغة إنجليزية عادية واحصل على إجابة مرتبطة مباشرة بالأدلة. لا حاجة لكتابة SQL، ولا حاجة للحفر في لوحات المعلومات — مساعد **FailproofAI Cloud** هو أسرع طريقة لأي شخص في فريقك للحصول على إجابات حول وكلائك. -![مساعد Failproof AI Observability يجيب على سؤال بلغة إنجليزية عادية داخل لوحة المعلومات، يعرض جدول نشاط الوكيل المباشر، وتفصيل استخدام النموذج لكل وكيل، والخلاصات المكتوبة، مع عرض الاستعلامات التي أجراها بشكل مدمج](/agenteye/images/assistant.png) +![مساعد FailproofAI Cloud يجيب على سؤال بلغة إنجليزية عادية داخل لوحة المعلومات، يعرض جدول نشاط الوكيل المباشر، وتفصيل استخدام النموذج لكل وكيل، والخلاصات المكتوبة، مع عرض الاستعلامات التي أجراها بشكل مدمج](/cloud/images/assistant.png) *اطرح السؤال بلغة إنجليزية عادية واحصل على إجابة مبنية من بياناتك الخاصة. هنا يقسم أي الوكلاء الأكثر انشغالاً وأي نماذج يستخدمونها، ويعرض الاستعلامات التي أجراها حتى تتمكن من التحقق من كل رقم.* لا شيء لتعلمه. افتح الدردشة، اكتب ما تريد معرفته، واتبع الروابط التي يعطيها لك: @@ -35,7 +35,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a في صفحة **Queries** يذهب خطوة أبعد ويصبح مؤلف SQL: صف الاستعلام الذي تريده ("عرض معدل الخطأ حسب الوكيل لآخر 7 أيام") وسيحول SQL مباشرة إلى المحرر، فاتحاً عرض diff حتى تتمكن من **Accept** أو **Reject** التغيير قبل أن يتم تطبيقه. -![صفحة Observability Queries ومحررها SQL](/agenteye/images/query-lab.png) +![صفحة FailproofAI Cloud Queries ومحررها SQL](/cloud/images/query-lab.png) *صفحة Queries: هذا المحرر هو المكان الذي يحول فيه المساعد مسودة استعلام للقراءة فقط بالنسبة لك لقبولها أو رفضها.* كتابة SQL بالسؤال هنا يستخدم إذن `queries:run`، وهو نفس الإذن خلف زر **Run** في المحرر. الدردشة في أي مكان آخر تحتاج `agent:use`. @@ -48,7 +48,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a - **كل كتابة تنتظر لك.** الاستعلامات المحفوظة ولوحات المعلومات يتم إنشاؤها فقط بعد نقرك Approve الصريح، ولا توجد إعدادات تطفئ هذه البوابة. - **لا يمكنه حذف أي شيء.** لا يتم الكشف عن أداة حذف والمساعد لا يحتفظ بإذن حذف. عمليات الحذف تبقى في يديك، في لوحة المعلومات. - **يبقى داخل مؤسستك.** المساعد يرى فقط المؤسسة التي تشاهدها حالياً. -- **أسئلتك تبقى لك.** الطلبات والإجابات تعيش في قاعدة بيانات Observability الخاصة بك؛ تسجيل تحليلات المنتج فقط بيانات وصفية الاستخدام، أبداً نص الطلب الخاص بك. +- **أسئلتك تبقى لك.** الطلبات والإجابات تعيش في قاعدة بيانات FailproofAI Cloud الخاصة بك؛ تسجيل تحليلات المنتج فقط بيانات وصفية الاستخدام، أبداً نص الطلب الخاص بك. ## مكان البحث عنها @@ -56,7 +56,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a ## ذات صلة -- [CLI والوكلاء](/ar/agenteye/cli-and-agents) -- [الاستعلامات](/ar/agenteye/queries) -- [لوحات المعلومات](/ar/agenteye/dashboards) -- [مجموعة التقييم](/ar/agenteye/evaluation-suite) \ No newline at end of file +- [CLI والوكلاء](/ar/cloud/cli) +- [الاستعلامات](/ar/cloud/queries) +- [لوحات المعلومات](/ar/cloud/dashboards) +- [مجموعة التقييم](/ar/cloud/evaluators) \ No newline at end of file diff --git a/docs/ar/agenteye/audits.mdx b/docs/ar/cloud/audits.mdx similarity index 81% rename from docs/ar/agenteye/audits.mdx rename to docs/ar/cloud/audits.mdx index 0b2b1efe..09f8a05c 100644 --- a/docs/ar/agenteye/audits.mdx +++ b/docs/ar/cloud/audits.mdx @@ -1,11 +1,11 @@ --- --- title: "التدقيق: محلل الموثوقية التلقائي الخاص بك" -description: "Failproof AI Observability يبحث عن الأعطال التي لم تكتب قاعدة لها ويسلمك قائمة مهام مرتبة ومدعومة بالأدلة حول ما يجب إصلاحه بالضبط." +description: "FailproofAI Cloud يبحث عن الأعطال التي لم تكتب قاعدة لها ويسلمك قائمة مهام مرتبة ومدعومة بالأدلة حول ما يجب إصلاحه بالضبط." --- -يبحث Failproof AI Observability عن الأعطال التي لم تكتب قاعدة لها ويسلمك قائمة مهام مرتبة ومدعومة بالأدلة حول ما يجب إصلاحه بالضبط. إنه مثل وجود محلل يمر عبر السجلات الخاصة بك كل ليلة، ثم يترك القائمة المختصرة على مكتبك في الصباح. +يبحث FailproofAI Cloud عن الأعطال التي لم تكتب قاعدة لها ويسلمك قائمة مهام مرتبة ومدعومة بالأدلة حول ما يجب إصلاحه بالضبط. إنه مثل وجود محلل يمر عبر السجلات الخاصة بك كل ليلة، ثم يترك القائمة المختصرة على مكتبك في الصباح.
@@ -13,7 +13,7 @@ description: "Failproof AI Observability يبحث عن الأعطال التي *جولة مدتها دقيقتان: من تشغيل مجدول إلى إصلاح يمكنك العمل عليه.* -![صفحة التدقيق: وظائف متكررة تفحص جلساتك بحثاً عن أنماط الفشل، كل منها مع جدول زمني وحساسية](/agenteye/images/audits.png) +![صفحة التدقيق: وظائف متكررة تفحص جلساتك بحثاً عن أنماط الفشل، كل منها مع جدول زمني وحساسية](/cloud/images/audits.png) *كل تدقيق هو وظيفة متكررة تستكشف جلساتك وتكتب توصيات مرتبة ومدعومة بالأدلة.* ## توقف عن التخمين بشأن ما يجب إصلاحه بعد ذلك @@ -49,7 +49,7 @@ description: "Failproof AI Observability يبحث عن الأعطال التي ## ذات صلة -- [التنبيهات](/ar/agenteye/alerts): احصل على إخطار في اللحظة التي يتم فيها تجاوز حد تعرفه بالفعل. -- [التقييمات](/ar/agenteye/evaluations): سجل كل عملية تشغيل حتى تظهر انحدارات الجودة من تلقاء نفسها. -- [تتبع الأخطاء](/ar/agenteye/error-tracking): جمّع واتبع الأخطاء التي يرميها الوكلاء الخاصون بك. -- [الحوادث](/ar/agenteye/incidents): تتبع المشكلة التي يكتشفها التدقيق حتى إصلاحها. \ No newline at end of file +- [التنبيهات](/ar/cloud/alerts): احصل على إخطار في اللحظة التي يتم فيها تجاوز حد تعرفه بالفعل. +- [التقييمات](/ar/cloud/evaluations): سجل كل عملية تشغيل حتى تظهر انحدارات الجودة من تلقاء نفسها. +- [تتبع الأخطاء](/ar/cloud/errors): جمّع واتبع الأخطاء التي يرميها الوكلاء الخاصون بك. +- [الحوادث](/ar/cloud/incidents): تتبع المشكلة التي يكتشفها التدقيق حتى إصلاحها. \ No newline at end of file diff --git a/docs/ar/cloud/capture.mdx b/docs/ar/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/ar/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/ar/agenteye/cli-recipes.mdx b/docs/ar/cloud/cli-recipes.mdx similarity index 90% rename from docs/ar/agenteye/cli-recipes.mdx rename to docs/ar/cloud/cli-recipes.mdx index b85fc60a..d2e4794b 100644 --- a/docs/ar/agenteye/cli-recipes.mdx +++ b/docs/ar/cloud/cli-recipes.mdx @@ -3,9 +3,9 @@ title: "وصفات سطر الأوامر للوكلاء" description: "انسخ والصق أنماط الاستعلام ووصفات jq التي تحول بيانات الجلسة والأحداث والتقييم إلى شيء يمكن لسكريبت أو وكيل ترميز أن يؤتمتنه." --- -اسحب بيانات الجلسة والأحداث والتقييم (وشغل إعادة التقييمات) مباشرة من سكريبت أو وكيل ترميز، مع JSON نظيف على stdout يتم توجيهه مباشرة إلى `jq`. هذه الوصفات تحول بيانات Failproof AI Observability إلى شيء يمكن لمستخدم المحطة الطرفية أو وكيل ترميز AI (Claude Code، Cursor) أن يستعلم عنه ويؤتمتنه، دون النقر عبر لوحة المعلومات. +اسحب بيانات الجلسة والأحداث والتقييم (وشغل إعادة التقييمات) مباشرة من سكريبت أو وكيل ترميز، مع JSON نظيف على stdout يتم توجيهه مباشرة إلى `jq`. هذه الوصفات تحول بيانات FailproofAI Cloud إلى شيء يمكن لمستخدم المحطة الطرفية أو وكيل ترميز AI (Claude Code، Cursor) أن يستعلم عنه ويؤتمتنه، دون النقر عبر لوحة المعلومات. -الأنماط أدناه جاهزة للنسخ واللصق في سطر أوامر Failproof AI Observability (`agenteye`). للتثبيت والمصادقة وقائمة الخيارات الكاملة، انظر [CLI](/ar/agenteye/cli)؛ شغّل `agenteye -h` أو `agenteye -h` للحصول على المساعدة المدمجة. +الأنماط أدناه جاهزة للنسخ واللصق في سطر أوامر FailproofAI Cloud (`agenteye`). للتثبيت والمصادقة وقائمة الخيارات الكاملة، انظر [CLI](/ar/cloud/cli)؛ شغّل `agenteye -h` أو `agenteye -h` للحصول على المساعدة المدمجة. ## القواعد الذهبية @@ -172,7 +172,7 @@ esac ## الخطوات التالية -- [CLI](/ar/agenteye/cli): التثبيت والمصادقة ومرجع الخيارات الكامل لكل أمر. -- [CLI agent skill](/ar/agenteye/cli-skill): احزم هذه الوصفات كمهارة يمكن لوكيل الترميز الخاص بك تحميلها. -- [مفاتيح API](/ar/agenteye/api-keys): أنشئ وحدد نطاق المفاتيح التي يستخدمها CLI و SDK والمجمع للمصادقة. -- [Python SDK](/ar/agenteye/python-sdk): أرسل الأحداث إلى Failproof AI Observability بحيث يكون هناك بيانات لهذه الوصفات للاستعلام عنها. \ No newline at end of file +- [CLI](/ar/cloud/cli): التثبيت والمصادقة ومرجع الخيارات الكامل لكل أمر. +- [CLI agent skill](/ar/cloud/agent-skills): احزم هذه الوصفات كمهارة يمكن لوكيل الترميز الخاص بك تحميلها. +- [مفاتيح API](/ar/cloud/access): أنشئ وحدد نطاق المفاتيح التي يستخدمها CLI و SDK والمجمع للمصادقة. +- [Python SDK](/ar/cloud/sdk): أرسل الأحداث إلى FailproofAI Cloud بحيث يكون هناك بيانات لهذه الوصفات للاستعلام عنها. \ No newline at end of file diff --git a/docs/ar/agenteye/cli.mdx b/docs/ar/cloud/cli.mdx similarity index 90% rename from docs/ar/agenteye/cli.mdx rename to docs/ar/cloud/cli.mdx index 5f59330b..d7e84154 100644 --- a/docs/ar/agenteye/cli.mdx +++ b/docs/ar/cloud/cli.mdx @@ -1,9 +1,9 @@ --- title: "واجهة سطر الأوامر (CLI)" -description: "قم بتشغيل كل عمليات Failproof AI Observability من المحطة الطرفية أو من نص برمجي: بدون الحاجة إلى لوحة التحكم." +description: "قم بتشغيل كل عمليات FailproofAI Cloud من المحطة الطرفية أو من نص برمجي: بدون الحاجة إلى لوحة التحكم." --- -قم بتشغيل كل عمليات Failproof AI Observability من المحطة الطرفية أو من نص برمجي: بدون الحاجة إلى لوحة التحكم. يستعلم CLI `agenteye` عن بيانات النظام (الجلسات وسجلات الأحداث والتقييمات) ويدير مؤسستك (مفاتيح API والمستخدمون والإعدادات والتنبيهات والحوادث والاستعلامات المحفوظة)، لذا استخدمه عندما تريد أتمتة فحص أو دمج الملاحظة في CI أو السماح لوكيل ترميز بفحص الإنتاج. يدعم كل أمر علم `--json`، لذلك يعمل بنفس الكفاءة سواء كنت في موجه الأوامر أو وكيل ترميز (Claude Code أو Cursor) يقوم بتنفيذ الأمر وتحليل النتيجة. +قم بتشغيل كل عمليات FailproofAI Cloud من المحطة الطرفية أو من نص برمجي: بدون الحاجة إلى لوحة التحكم. يستعلم CLI `agenteye` عن بيانات النظام (الجلسات وسجلات الأحداث والتقييمات) ويدير مؤسستك (مفاتيح API والمستخدمون والإعدادات والتنبيهات والحوادث والاستعلامات المحفوظة)، لذا استخدمه عندما تريد أتمتة فحص أو دمج الملاحظة في CI أو السماح لوكيل ترميز بفحص الإنتاج. يدعم كل أمر علم `--json`، لذلك يعمل بنفس الكفاءة سواء كنت في موجه الأوامر أو وكيل ترميز (Claude Code أو Cursor) يقوم بتنفيذ الأمر وتحليل النتيجة. باستخدام ملف ثنائي واحد يمكنك: @@ -68,7 +68,7 @@ agenteye --version agenteye --help ``` -> **ملاحظة:** SDK Python الخاص بـ Failproof AI Observability يستخدم أيضاً اسم توزيع `agenteye`. يحافظ تثبيت CLI باستخدام `pipx` أو `uv tool` (بدلاً من `pip install` في virtualenv مشترك) على عدم تضارب الاثنين. `pip install agenteye` عادي جيد فقط إذا لم يكن SDK مثبتاً في نفس البيئة. +> **ملاحظة:** SDK Python الخاص بـ FailproofAI Cloud يستخدم أيضاً اسم توزيع `agenteye`. يحافظ تثبيت CLI باستخدام `pipx` أو `uv tool` (بدلاً من `pip install` في virtualenv مشترك) على عدم تضارب الاثنين. `pip install agenteye` عادي جيد فقط إذا لم يكن SDK مثبتاً في نفس البيئة. --- @@ -90,7 +90,7 @@ agenteye logout # revoke the session and clear the stored token لا يخطئ `whoami` أبداً في جلسة مفقودة أو منتهية الصلاحية؛ بدلاً من ذلك يبلغ عن `logged_in: false`، لذا يمكن لنص برمجي أو وكيل التحقق من حالة المصادقة بأمان (لا يزال يمكن أن يخرج مع كود غير صفري إذا لم يتم تعيين عنوان URL أساسي أو كانت لوحة التحكم غير قابلة للوصول). -**المتطلبات:** يجب السماح لبريدك الإلكتروني بتسجيل الدخول إلى لوحة التحكم (اطلب من مسؤول Failproof AI Observability)، ويجب أن تكون لوحة التحكم قابلة للوصول على عنوان URL الأساسي الخاص بها (انظر [الإعدادات](#configuration)). إذا طلبت كوداً ولم يصل أي، فمن المحتمل أن بريدك الإلكتروني لم يتم تفعيله بعد للوصول إلى لوحة التحكم. +**المتطلبات:** يجب السماح لبريدك الإلكتروني بتسجيل الدخول إلى لوحة التحكم (اطلب من مسؤول FailproofAI Cloud)، ويجب أن تكون لوحة التحكم قابلة للوصول على عنوان URL الأساسي الخاص بها (انظر [الإعدادات](#configuration)). إذا طلبت كوداً ولم يصل أي، فمن المحتمل أن بريدك الإلكتروني لم يتم تفعيله بعد للوصول إلى لوحة التحكم. --- @@ -337,13 +337,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | مُصادق عليه، لكن حسابك يفتقد الأذن المطلوبة (الرسالة تسميها) | | 6 | لم يتم العثور على المورد المطلوب (على سبيل المثال، معرف جلسة أو حادثة غير معروف) | -وهذا يجعل CLI آمنة للنص البرمجي: يمكن لوكيل ترميز فرع على `4` لمطالبتك بإعادة المصادقة، أو `5` لسطح الأذن المفقودة. انظر [وصفات CLI للوكلاء](/ar/agenteye/cli-recipes) لأنماط معالجة أكواد الخروج وأشكال مخرجات JSON. +وهذا يجعل CLI آمنة للنص البرمجي: يمكن لوكيل ترميز فرع على `4` لمطالبتك بإعادة المصادقة، أو `5` لسطح الأذن المفقودة. انظر [وصفات CLI للوكلاء](/ar/cloud/cli-recipes) لأنماط معالجة أكواد الخروج وأشكال مخرجات JSON. --- ## الخطوات التالية -- **[وصفات CLI للوكلاء](/ar/agenteye/cli-recipes)**: أنماط استعلام نسخ لصق، `jq` سطر واحد، إسقاطات `--fields`، معالجة أكواد الخروج وأشكال مخرجات JSON، مكتوبة لوكلاء ترميز يقودون CLI. -- **[مهارة عامل CLI](/ar/agenteye/cli-skill)**: حزم هذا CLI كمهارة قابلة للتثبيت Claude Code / Codex بحيث يقود وكيل ترميز Failproof AI Observability من طلبات اللغة الطبيعية. -- **[مفاتيح API](/ar/agenteye/api-keys)**: نموذج الأذن خلف `keys create --add …`. -- **[مساعد الذكاء الاصطناعي](/ar/agenteye/assistant)**: تفعيل المساعد الذي يتحدث معه `agent ask`. \ No newline at end of file +- **[وصفات CLI للوكلاء](/ar/cloud/cli-recipes)**: أنماط استعلام نسخ لصق، `jq` سطر واحد، إسقاطات `--fields`، معالجة أكواد الخروج وأشكال مخرجات JSON، مكتوبة لوكلاء ترميز يقودون CLI. +- **[مهارة عامل CLI](/ar/cloud/agent-skills)**: حزم هذا CLI كمهارة قابلة للتثبيت Claude Code / Codex بحيث يقود وكيل ترميز FailproofAI Cloud من طلبات اللغة الطبيعية. +- **[مفاتيح API](/ar/cloud/access)**: نموذج الأذن خلف `keys create --add …`. +- **[مساعد الذكاء الاصطناعي](/ar/cloud/assistant)**: تفعيل المساعد الذي يتحدث معه `agent ask`. \ No newline at end of file diff --git a/docs/ar/cloud/connect.mdx b/docs/ar/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/ar/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/ar/agenteye/dashboards.mdx b/docs/ar/cloud/dashboards.mdx similarity index 72% rename from docs/ar/agenteye/dashboards.mdx rename to docs/ar/cloud/dashboards.mdx index 6c492a41..70097aa6 100644 --- a/docs/ar/agenteye/dashboards.mdx +++ b/docs/ar/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "حول بيانات الوكيل المباشرة إلى صورة حول بيانات الوكيل المباشرة إلى صورة موحدة تراقبها فريقك بالكامل. ثبّت الاستعلامات المهمة كرسوم بيانية، وسيفتح الجميع نفس الأرقام في لمحة واحدة، دون تشغيل استعلام واحد مرة أخرى. -![لوحة تحكم مبنية من الاستعلامات المحفوظة: رسم بياني خطي للأحداث في الساعة، ورسم بياني عمودي للأخطاء حسب النوع، ورسم بياني منطقة للكمون، وتفصيل الرموز حسب النموذج](/agenteye/images/dashboard-fleet.png) +![لوحة تحكم مبنية من الاستعلامات المحفوظة: رسم بياني خطي للأحداث في الساعة، ورسم بياني عمودي للأخطاء حسب النوع، ورسم بياني منطقة للكمون، وتفصيل الرموز حسب النموذج](/cloud/images/dashboard-fleet.png) *لوحة واحدة، أربع استعلامات محفوظة: الأحداث في الساعة، والأخطاء حسب النوع، والكمون، والرموز حسب النموذج.* @@ -25,15 +25,15 @@ description: "حول بيانات الوكيل المباشرة إلى صورة ## ثبّت الاستعلامات التي حفظتها بالفعل -كل بلاطة تبدأ كاستعلام محفوظ. بناء وحفظ الاستعلام الذي تهتم به في مكتبة [الاستعلامات](/ar/agenteye/queries) (الإعدادات المدمجة بالإضافة إلى إعداداتك الخاصة، فوق أحداثك وتقييماتك)، ثم ثبّته على لوحة تحكم كرسم بياني يناسب البيانات: **خط** للاتجاهات عبر الزمن، **عمود** للمقارنة بين الفئات، **منطقة** للحجم، أو **دائرة** لتفصيل النسبة. +كل بلاطة تبدأ كاستعلام محفوظ. بناء وحفظ الاستعلام الذي تهتم به في مكتبة [الاستعلامات](/ar/cloud/queries) (الإعدادات المدمجة بالإضافة إلى إعداداتك الخاصة، فوق أحداثك وتقييماتك)، ثم ثبّته على لوحة تحكم كرسم بياني يناسب البيانات: **خط** للاتجاهات عبر الزمن، **عمود** للمقارنة بين الفئات، **منطقة** للحجم، أو **دائرة** لتفصيل النسبة. لأن البلاطة ليست سوى استعلامك المحفوظ المعروض كرسم بياني، لا توجد حاجة للحفاظ على التزامن يدوياً. حدّث الاستعلام مرة واحدة وكل لوحة تحكم تستخدمه تتحدث أيضاً. ## راقب الجودة، ليس فقط الحجم -الحجم يخبرك أن الوكلاء مشغولون. الجودة تخبرك أنهم يقومون فعلاً بالعمل. وجّه لوحة تحكم نحو [درجات التقييم](/ar/agenteye/evaluations) الخاصة بك وستحصل على لوحة تتابع مدى جودة سير التشغيل عبر الزمن، لذا سيظهر انحدار الجودة كانخفاض على رسم بياني بدلاً من مفاجأة من عميل. +الحجم يخبرك أن الوكلاء مشغولون. الجودة تخبرك أنهم يقومون فعلاً بالعمل. وجّه لوحة تحكم نحو [درجات التقييم](/ar/cloud/evaluations) الخاصة بك وستحصل على لوحة تتابع مدى جودة سير التشغيل عبر الزمن، لذا سيظهر انحدار الجودة كانخفاض على رسم بياني بدلاً من مفاجأة من عميل. -![لوحة تحكم موجهة نحو الجودة مبنية من استعلامات التقييم المحفوظة](/agenteye/images/dashboard-quality.png) +![لوحة تحكم موجهة نحو الجودة مبنية من استعلامات التقييم المحفوظة](/cloud/images/dashboard-quality.png) *لوحة الجودة تبقي درجات التقييم في المقدمة والمركز، بجانب الأرقام التشغيلية مباشرة.* @@ -41,6 +41,6 @@ description: "حول بيانات الوكيل المباشرة إلى صورة ## ذات الصلة -- [الاستعلامات](/ar/agenteye/queries): بناء وحفظ الاستعلامات التي تصبح بلاطاتك. -- [التقييمات](/ar/agenteye/evaluations): سجّل عمليات التشغيل الخاصة بك حتى تتمكن من رسم الجودة عبر الزمن. -- [التنبيهات](/ar/agenteye/alerts): حول حد على أي من هذه المقاييس إلى صفحة. \ No newline at end of file +- [الاستعلامات](/ar/cloud/queries): بناء وحفظ الاستعلامات التي تصبح بلاطاتك. +- [التقييمات](/ar/cloud/evaluations): سجّل عمليات التشغيل الخاصة بك حتى تتمكن من رسم الجودة عبر الزمن. +- [التنبيهات](/ar/cloud/alerts): حول حد على أي من هذه المقاييس إلى صفحة. \ No newline at end of file diff --git a/docs/ar/agenteye/error-tracking.mdx b/docs/ar/cloud/errors.mdx similarity index 69% rename from docs/ar/agenteye/error-tracking.mdx rename to docs/ar/cloud/errors.mdx index f0db6694..086e490c 100644 --- a/docs/ar/agenteye/error-tracking.mdx +++ b/docs/ar/cloud/errors.mdx @@ -6,14 +6,14 @@ description: "اطّلع على كل الأخطاء التي ينتجها وكل اطّلع على كل الأخطاء التي ينتجها وكلاؤك في مكان واحد، مجمّعة بحيث تظهر الدفقة الصاخبة كمشكلة واحدة. تحصل على مسار بنقرة واحدة من "هناك شيء احمر" إلى التشغيل الدقيق الذي تعطّل، دون الحاجة للتمرير عبر تغذية مباشرة للعثور عليه. -![صفحة الأخطاء: رسم بياني يعرض الأخطاء عبر الزمن أعلاه، مع صفوف الأخطاء الحمراء المجمّعة، كل منها بزر "+تنبيه" بنقرة واحدة](/agenteye/images/errors.png) +![صفحة الأخطاء: رسم بياني يعرض الأخطاء عبر الزمن أعلاه، مع صفوف الأخطاء الحمراء المجمّعة، كل منها بزر "+تنبيه" بنقرة واحدة](/cloud/images/errors.png) *صفحة الأخطاء: رسم بياني يعرض الأخطاء عبر الزمن، مع انهيار الأخطاء المتكررة في صف واحد لكل حادثة.* ## كل خطأ، تم جمعه لك بالفعل عندما يتعطل الوكيل، لا يجب عليك التمرير عبر تدفق الأحداث المباشر على أمل اكتشاف الصفوف الحمراء قبل أن تختفي. تقوم صفحة **الأخطاء** بالجمع نيابة عنك. فهي تجمع كل شيء قد تعرضه لوحة المعلومات باللون الأحمر في سطح فحص واحد، بحيث يكون أول ما تراه هو ما يتعطل، وليس أين تذهب للبحث عنه. -وهي تعثر على أكثر من الواضح منها. إلى جانب أحداث `error` الصريحة، تطبيق Failproof AI Observability يسلّط الضوء على الأخطاء الصامتة أيضًا: أي `tool_result` أو `hook_completed` أو `agent_end` يحمل حمولته فشل يظهر هنا. أداة أرجعت خطأ، أو خطاف انتهى بشكل سيء، لا يمكن أن ينزلق بعيدًا عنك فقط لأنه لم يرمِ استثناء صاخبًا. +وهي تعثر على أكثر من الواضح منها. إلى جانب أحداث `error` الصريحة، تطبيق FailproofAI Cloud يسلّط الضوء على الأخطاء الصامتة أيضًا: أي `tool_result` أو `hook_completed` أو `agent_end` يحمل حمولته فشل يظهر هنا. أداة أرجعت خطأ، أو خطاف انتهى بشكل سيء، لا يمكن أن ينزلق بعيدًا عنك فقط لأنه لم يرمِ استثناء صاخبًا. عبر الأعلى، رسم بياني يحتسب الأخطاء عبر الزمن. نظرة واحدة تخبرك ما إذا كان هذا تسربًا ثابتًا في الخلفية أم ارتفاعًا بدأ قبل بضع دقائق، بحيث تعرف على الفور ما إذا كان يجب عليك إسقاط ما تفعله. @@ -23,19 +23,19 @@ description: "اطّلع على كل الأخطاء التي ينتجها وكل يمكن لتبعية مكسورة واحدة أن تطلق نفس الخطأ مئات المرات في الدقيقة. إذا تركت خامًا، فهي جدار من الخطوط المتشابهة جدًا التي تدفن الشيء الوحيد الذي تحتاج فعلاً إلى رؤيته. -يطبيق Failproof AI Observability ينهار الأخطاء المتكررة التي تشترك في نفس الجلسة ونوع الخطأ في صف واحد. الدفقة تقرأ كحادثة واحدة. ينتهي بك الحال بعد عد المشاكل، وليس سطور السجل، والإشارة التي تهم تبقى في الأعلى بدلاً من أن تغرق تحت وزنها الخاص. +يطبيق FailproofAI Cloud ينهار الأخطاء المتكررة التي تشترك في نفس الجلسة ونوع الخطأ في صف واحد. الدفقة تقرأ كحادثة واحدة. ينتهي بك الحال بعد عد المشاكل، وليس سطور السجل، والإشارة التي تهم تبقى في الأعلى بدلاً من أن تغرق تحت وزنها الخاص. ## من "هناك شيء احمر" إلى الحدث الدقيق انقر على أي صف للوصول مباشرة إلى جلسة هذا التشغيل، محددًا على الحدث الدقيق الذي فشل. لا نسخ معرفات الجلسة، لا التمرير للبحث عن اللحظة التي ساءت: تصل إليها مباشرة، مع الرسم البياني التنفيذي الكامل على بُعد نظرة واحدة بحيث يمكنك رؤية ما الذي قام به الوكيل في اللحظات قبل أن يتعطل. -إذا كان لديك `alerts:write`، فإن كل صف يحمل أيضًا زر **+ alert**. انقر عليه وتطبيق Observability يفتح قاعدة تنبيه جديدة مملوءة بالفعل للقبض على نفس الفشل مرة أخرى. الحادثة التي قمت بفحصها للتو تصبح الحادثة التي تنبهك في المرة القادمة، بدلاً من مفاجأتك مرتين. +إذا كان لديك `alerts:write`، فإن كل صف يحمل أيضًا زر **+ alert**. انقر عليه وتطبيق FailproofAI Cloud يفتح قاعدة تنبيه جديدة مملوءة بالفعل للقبض على نفس الفشل مرة أخرى. الحادثة التي قمت بفحصها للتو تصبح الحادثة التي تنبهك في المرة القادمة، بدلاً من مفاجأتك مرتين. **أين تجده:** صفحة **الأخطاء** توجد في قسم المراقبة من لوحة المعلومات، في `//errors`. ## ذات صلة -- [التنبيهات](/ar/agenteye/alerts): حول أي فشل إلى قاعدة نداء. -- [الحوادث](/ar/agenteye/incidents): تتبع التنبيه الناشط من الفتح إلى الحل. -- [الجلسات](/ar/agenteye/sessions): افتح التشغيل الكامل خلف أي خطأ. -- [المراجعات](/ar/agenteye/audits): دع تطبيق Observability يعثر على أنماط الفشل عبر تشغيلاتك نيابة عنك. \ No newline at end of file +- [التنبيهات](/ar/cloud/alerts): حول أي فشل إلى قاعدة نداء. +- [الحوادث](/ar/cloud/incidents): تتبع التنبيه الناشط من الفتح إلى الحل. +- [الجلسات](/ar/cloud/sessions): افتح التشغيل الكامل خلف أي خطأ. +- [المراجعات](/ar/cloud/audits): دع تطبيق FailproofAI Cloud يعثر على أنماط الفشل عبر تشغيلاتك نيابة عنك. \ No newline at end of file diff --git a/docs/ar/agenteye/evaluations.mdx b/docs/ar/cloud/evaluations.mdx similarity index 69% rename from docs/ar/agenteye/evaluations.mdx rename to docs/ar/cloud/evaluations.mdx index 9f292548..5a35114f 100644 --- a/docs/ar/agenteye/evaluations.mdx +++ b/docs/ar/cloud/evaluations.mdx @@ -4,15 +4,15 @@ description: "مشاكل الجودة تجدك الآن، بدلاً من سما --- -مشاكل الجودة تجدك الآن، بدلاً من سماعك عنها في شكوى من مستخدم. اربط خدمة التسجيل الخاصة بك مرة واحدة و Failproof AI Observability يقيّم كل عملية منتهية تلقائياً، بحيث ينخفاض في الفائدة أو ارتفاع حاد في الهلوسات يظهر من تلقاء نفسه، قبل أن يشعر به العميل. +مشاكل الجودة تجدك الآن، بدلاً من سماعك عنها في شكوى من مستخدم. اربط خدمة التسجيل الخاصة بك مرة واحدة و FailproofAI Cloud يقيّم كل عملية منتهية تلقائياً، بحيث ينخفاض في الفائدة أو ارتفاع حاد في الهلوسات يظهر من تلقاء نفسه، قبل أن يشعر به العميل. -![شبكة الجلسات مع عمود النقاط: كل عملية تحمل شارة حالة التقييم وشارات ملونة بالرمز (أحمر وأصفر وأخضر) للفائدة والدقة وكفاءة الأداة](/agenteye/images/sessions-list.png) +![شبكة الجلسات مع عمود النقاط: كل عملية تحمل شارة حالة التقييم وشارات ملونة بالرمز (أحمر وأصفر وأخضر) للفائدة والدقة وكفاءة الأداة](/cloud/images/sessions-list.png) *كل عملية في شبكة الجلسات تحمل نقاطها؛ الشارات الحمراء والصفراء والخضراء تجعل العمليات الضعيفة تبرز دون فتح نص واحد.* ## توقف عن أخذ عينات من العمليات يدويّاً -كنت تفحص عدداً قليلاً من العمليات وتأمل أن تكون البقية بخير. الآن كل جلسة مكتملة يتم تقييمها اللحظة التي تنتهي، على الأبعاد التي تهمك: الفائدة وكفاءة الأداة والدقة والأمان وأي معيار جودة لديك. أنت تعرّف مفاتيح النقاط؛ Failproof AI Observability يخزن وينظر ويعرض أي شيء يرسله المقيّم الخاص بك. لا عملية تتسلل بدون نقاط، وتتوقف عن معرفة الانحدار من تذكرة دعم. +كنت تفحص عدداً قليلاً من العمليات وتأمل أن تكون البقية بخير. الآن كل جلسة مكتملة يتم تقييمها اللحظة التي تنتهي، على الأبعاد التي تهمك: الفائدة وكفاءة الأداة والدقة والأمان وأي معيار جودة لديك. أنت تعرّف مفاتيح النقاط؛ FailproofAI Cloud يخزن وينظر ويعرض أي شيء يرسله المقيّم الخاص بك. لا عملية تتسلل بدون نقاط، وتتوقف عن معرفة الانحدار من تذكرة دعم. النقاط تظهر على شبكة الجلسات في **`//sessions`** (الشريط الجانبي → *مراقبة* → *جلسات*)، مجموعة شارات واحدة لكل صف. تريد فقط العمليات التي أخفقت؟ صفّي الشبكة حسب نطاق النقاط، على سبيل المثال الفائدة أقل من 0.5، واسحب العمليات التي تستحق القراءة بالضبط. يتطلب عرض النقاط صلاحية `evaluations:read`. @@ -20,7 +20,7 @@ description: "مشاكل الجودة تجدك الآن، بدلاً من سما الرقم يخبرك أن العملية كانت ضعيفة؛ صفحة الجلسة تخبرك لماذا. افتح أي عملية والسكة الجانبية اليمنى تبدأ بملخص العنوان الرئيسي، ثم تعرض شريطاً لكل بُعد مع المنطق الخاص بمقيّمك تحت كل واحد، حتى تنتقل من "هذا سجل 0.4 على الدقة" إلى الادعاء الدقيق الذي أخطأ فيه في ثوانٍ. -![السكة الجانبية اليمنى للجلسة: ملخص التقييم في الأعلى، ثم أشرطة النقاط لكل بُعد مع سطر من المنطق، بجانب خط الأحداث الكامل](/agenteye/images/session-detail.png) +![السكة الجانبية اليمنى للجلسة: ملخص التقييم في الأعلى، ثم أشرطة النقاط لكل بُعد مع سطر من المنطق، بجانب خط الأحداث الكامل](/cloud/images/session-detail.png) *عرض تفاصيل الجلسة: الملخص وأشرطة النقاط لكل بُعد والمنطق خلف كل نقاط، بجانب خط أحداث العملية.* @@ -30,7 +30,7 @@ description: "مشاكل الجودة تجدك الآن، بدلاً من سما عملية واحدة بنقاط منخفضة هي ضوضاء؛ مجموعة كاملة تنزلق هي إشارة. لوحات المعلومات المحفوظة تحول نقاطك إلى اتجاه يمكنك مراقبته بنظرة واحدة: متوسط الفائدة هذا الأسبوع مقابل الأسبوع الماضي، لكل وكيل، لكل بيئة. -![لوحة معلومات الجودة: أشرطة متوسط النقاط لكل بُعد مقيّم بجانب اتجاه عبر الزمن](/agenteye/images/dashboard-quality.png) +![لوحة معلومات الجودة: أشرطة متوسط النقاط لكل بُعد مقيّم بجانب اتجاه عبر الزمن](/cloud/images/dashboard-quality.png) *لوحة معلومات جودة محفوظة تعطي اتجاهاً لمفاتيح النقاط التي تعرضها، بحيث يكون الانجراف البطيء واضحاً قبل وقت طويل من أن يصبح حادثة.* @@ -38,14 +38,14 @@ description: "مشاكل الجودة تجدك الآن، بدلاً من سما ## اربط مقيّماً مرة واحدة -التسجيل اختياري ويبقى معطلاً تماماً حتى تشير Failproof AI Observability إلى مسجل. تقيم خدمة HTTP صغيرة واحدة (Observability تشحن مرجعاً عاملاً يمكنك نسخه)، وتعيين قيمتين على الخادم الخاص بك، وكل عملية من ذلك الحين فصاعداً يتم تقييمها لك. الإرشادات الكاملة والعقد التسجيل و SDK يعيشان في الدليل العميق. +التسجيل اختياري ويبقى معطلاً تماماً حتى تشير FailproofAI Cloud إلى مسجل. تقيم خدمة HTTP صغيرة واحدة (FailproofAI Cloud تشحن مرجعاً عاملاً يمكنك نسخه)، وتعيين قيمتين على الخادم الخاص بك، وكل عملية من ذلك الحين فصاعداً يتم تقييمها لك. الإرشادات الكاملة والعقد التسجيل و SDK يعيشان في الدليل العميق. -لا تعرف أي الأبعاد تستحق التسجيل في البداية؟ [مهارة وكيل المقيّم](/ar/agenteye/evaluator-skill) لديها وكيل الترميز الخاص بك ينقب عن ذلك ضد جلساتك الخاصة، ثم يبني وينشر الخدمة. +لا تعرف أي الأبعاد تستحق التسجيل في البداية؟ [مهارة وكيل المقيّم](/ar/cloud/agent-skills) لديها وكيل الترميز الخاص بك ينقب عن ذلك ضد جلساتك الخاصة، ثم يبني وينشر الخدمة. ## ذات صلة -- [مجموعة التقييم](/ar/agenteye/evaluation-suite): اربط مقيّمك وعقد التسجيل و SDK. -- [مهارة وكيل المقيّم](/ar/agenteye/evaluator-skill): دع وكيل الترميز يختار أبعاد النقاط الخاصة بك ويبني المقيّم. -- [الجلسات](/ar/agenteye/sessions): شبكة تشغيل تظهر بها النقاط. -- [لوحات المعلومات](/ar/agenteye/dashboards): احفظ وشارك اتجاهات الجودة عبر المنظمة. -- [عمليات التدقيق](/ar/agenteye/audits): ميزة الجودة التلقائية الأخرى لـ Observability، للتحقيقات عبر الجلسات. \ No newline at end of file +- [مجموعة التقييم](/ar/cloud/evaluators): اربط مقيّمك وعقد التسجيل و SDK. +- [مهارة وكيل المقيّم](/ar/cloud/agent-skills): دع وكيل الترميز يختار أبعاد النقاط الخاصة بك ويبني المقيّم. +- [الجلسات](/ar/cloud/sessions): شبكة تشغيل تظهر بها النقاط. +- [لوحات المعلومات](/ar/cloud/dashboards): احفظ وشارك اتجاهات الجودة عبر المنظمة. +- [عمليات التدقيق](/ar/cloud/audits): ميزة الجودة التلقائية الأخرى لـ FailproofAI Cloud، للتحقيقات عبر الجلسات. \ No newline at end of file diff --git a/docs/ar/agenteye/evaluation-suite.mdx b/docs/ar/cloud/evaluators.mdx similarity index 84% rename from docs/ar/agenteye/evaluation-suite.mdx rename to docs/ar/cloud/evaluators.mdx index fb5b05b1..a85d6a0d 100644 --- a/docs/ar/agenteye/evaluation-suite.mdx +++ b/docs/ar/cloud/evaluators.mdx @@ -1,19 +1,19 @@ --- title: "مجموعة التقييم" -description: "يمكن لـ Failproof AI Observability تسجيل كل جلسة وكيل مكتملة تلقائياً من حيث الجودة: أنت توفر خدمة تسجيل صغيرة، وتتعامل Observability مع الباقي." +description: "يمكن لـ FailproofAI Cloud تسجيل كل جلسة وكيل مكتملة تلقائياً من حيث الجودة: أنت توفر خدمة تسجيل صغيرة، وتتعامل FailproofAI Cloud مع الباقي." --- -يمكن لـ Failproof AI Observability تسجيل كل جلسة وكيل مكتملة تلقائياً من حيث الجودة: أنت توفر خدمة تسجيل صغيرة، وتتعامل Observability مع الباقي. استخدمها لتتبع الأبعاد التي تهمك (الفائدة، كفاءة الأدوات، الدقة، الأمان؛ اختر أنت)، اكتشف الانحدار مبكراً، وقارن الوكلاء أو البيئات في لمحة واحدة. التسجيل اختياري: لا يفعل خط الأنابيب شيئاً حتى تعيّن `EVALUATOR_ENDPOINT` على الخادم. +يمكن لـ FailproofAI Cloud تسجيل كل جلسة وكيل مكتملة تلقائياً من حيث الجودة: أنت توفر خدمة تسجيل صغيرة، وتتعامل FailproofAI Cloud مع الباقي. استخدمها لتتبع الأبعاد التي تهمك (الفائدة، كفاءة الأدوات، الدقة، الأمان؛ اختر أنت)، اكتشف الانحدار مبكراً، وقارن الوكلاء أو البيئات في لمحة واحدة. التسجيل اختياري: لا يفعل خط الأنابيب شيئاً حتى تعيّن `EVALUATOR_ENDPOINT` على الخادم. -> **ملاحظة:** أنت تحدد أبعاد النقاط. يمكن لمُقيّمك إرجاع أي مفاتيح رقمية يريدها؛ تخزن Observability وتتجه وتعرض كل ما تُرسله مرة أخرى. +> **ملاحظة:** أنت تحدد أبعاد النقاط. يمكن لمُقيّمك إرجاع أي مفاتيح رقمية يريدها؛ تخزن FailproofAI Cloud وتتجه وتعرض كل ما تُرسله مرة أخرى. ## لمحة سريعة -1. **اكتب مُسجّل.** أنشئ خدمة HTTP صغيرة تقرأ نسخة من جلسة وترجع نقاط. تشحن Observability مرجعاً يعمل يمكنك نسخه. انظر [كتابة مُقيّم مع SDK](#writing-an-evaluator-with-the-sdk). -2. **وجّه Observability إليه.** عيّن `EVALUATOR_ENDPOINT` (و`EVALUATOR_TOKEN` مشترك) على عملية الخادم. +1. **اكتب مُسجّل.** أنشئ خدمة HTTP صغيرة تقرأ نسخة من جلسة وترجع نقاط. تشحن FailproofAI Cloud مرجعاً يعمل يمكنك نسخه. انظر [كتابة مُقيّم مع SDK](#writing-an-evaluator-with-the-sdk). +2. **وجّه FailproofAI Cloud إليه.** عيّن `EVALUATOR_ENDPOINT` (و`EVALUATOR_TOKEN` مشترك) على عملية الخادم. 3. **راقب النقاط تصل.** كل جلسة مكتملة يتم تسجيلها تلقائياً؛ تظهر النتائج على صفحة تفاصيل الجلسة، شبكة الجلسات، والقوائم المحفوظة. -![عرض تفاصيل الجلسة مع ملخص التقييم، أشرطة نقاط لكل بعد، ونص التبرير في الشريط الأيمن](/agenteye/images/session-detail.png) +![عرض تفاصيل الجلسة مع ملخص التقييم، أشرطة نقاط لكل بعد، ونص التبرير في الشريط الأيمن](/cloud/images/session-detail.png) *بمجرد تكوين مُقيّم، يتم تسجيل كل عملية مكتملة وتظهر النتائج في الشريط الأيمن للجلسة: الملخص في الأعلى، ثم أشرطة نقاط لكل بعد مع التبرير.* @@ -23,7 +23,7 @@ description: "يمكن لـ Failproof AI Observability تسجيل كل جلسة ```mermaid flowchart LR - ING["ingest /events
agent_end"] --> SRV["Observability server"] + ING["ingest /events
agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -31,14 +31,14 @@ flowchart LR SRV --> RES["evaluations
terminal results"] ``` -عندما يُصدر Failproof AI Observability SDK حدث `agent_end` لجلسة، يجدول الخادم تقييماً. ثم يُرسل نسخة الحدث الكاملة إلى خدمة المُقيّم الخاصة بك، والتي يمكنها إما: +عندما يُصدر FailproofAI Cloud SDK حدث `agent_end` لجلسة، يجدول الخادم تقييماً. ثم يُرسل نسخة الحدث الكاملة إلى خدمة المُقيّم الخاصة بك، والتي يمكنها إما: - **إرجاع النتيجة مباشرة** مع `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`. تُلحق النتيجة بجدول تقييم الجلسة. `reasoning` و `summary` اختياريين. -- **تأجيل** مع `{"status":"pending", "job_id":"abc-123"}`. ثم تستدعي Observability `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` حتى يُرجع مُقيّمك `{"status":"done", ...}` أو `{"status":"error", "error":"..."}`. +- **تأجيل** مع `{"status":"pending", "job_id":"abc-123"}`. ثم تستدعي FailproofAI Cloud `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` حتى يُرجع مُقيّمك `{"status":"done", ...}` أو `{"status":"error", "error":"..."}`. - وتيرة الاستقصاء لكل وظيفة: قد تتضمن استجابة `pending` `next_poll_secs` للتجاوز؛ وإلا فتستخدم Observability قيمة `default_poll_interval_secs` من `GET /config`؛ وإلا يعود الخادم إلى `EVALUATOR_POLLING_INTERVAL_SECS` (افتراضي 10 ثانية). جميع القيم محصورة في [1 ثانية، 1 ساعة]. + وتيرة الاستقصاء لكل وظيفة: قد تتضمن استجابة `pending` `next_poll_secs` للتجاوز؛ وإلا فتستخدم FailproofAI Cloud قيمة `default_poll_interval_secs` من `GET /config`؛ وإلا يعود الخادم إلى `EVALUATOR_POLLING_INTERVAL_SECS` (افتراضي 10 ثانية). جميع القيم محصورة في [1 ثانية، 1 ساعة]. -يمكن أيضاً التقاط الجلسات التي لم تُصدر أبداً `agent_end` (على سبيل المثال، عملية وكيل منهارة): قد يُرجع `GET /config` الخاص بالمُقيّم `{"inactivity_timeout_secs": 1800}`، وستقيّم Observability أي جلسة خاملة لتلك المدة. عيّن الحقل إلى `null` أو احذفه لتعطيل هذا البديل. +يمكن أيضاً التقاط الجلسات التي لم تُصدر أبداً `agent_end` (على سبيل المثال، عملية وكيل منهارة): قد يُرجع `GET /config` الخاص بالمُقيّم `{"inactivity_timeout_secs": 1800}`، وستقيّم FailproofAI Cloud أي جلسة خاملة لتلك المدة. عيّن الحقل إلى `null` أو احذفه لتعطيل هذا البديل. خط الأنابيب عديم التأثير تماماً عندما يكون `EVALUATOR_ENDPOINT` غير محدد. @@ -46,7 +46,7 @@ flowchart LR يُعاد تفعيل بديل عدم النشاط على الجلسات المستأنفة أيضاً: إذا وصلت أحداث جديدة بعد تقييم نهائي سابق وذهبت الجلسة خاملة بعد `inactivity_timeout_secs`، يُدرج تقييم جديد في الطابور. -الأعطال العابرة (5xx، 429، انتهاءات المهلة الزمنية، أخطاء الشبكة) تُعاد محاولتها مع تراجع أسي حتى `EVALUATOR_MAX_ATTEMPTS`؛ استجابات 4xx نهائية. Observability آمن للتشغيل مع خوادم متعددة مقسمة أفقياً؛ يُقسم العمل بحيث لا تُرسل نفس الجلسة مرتين معاً. +الأعطال العابرة (5xx، 429، انتهاءات المهلة الزمنية، أخطاء الشبكة) تُعاد محاولتها مع تراجع أسي حتى `EVALUATOR_MAX_ATTEMPTS`؛ استجابات 4xx نهائية. FailproofAI Cloud آمن للتشغيل مع خوادم متعددة مقسمة أفقياً؛ يُقسم العمل بحيث لا تُرسل نفس الجلسة مرتين معاً. --- @@ -54,7 +54,7 @@ flowchart LR كل مسار مصادق يستخدم **مصادقة رمز الحامل**. يجب أن تكون نفس القيمة مُعدة على كلا الجانبين: -- خادم Observability: متغير env `EVALUATOR_TOKEN` +- خادم FailproofAI Cloud: متغير env `EVALUATOR_TOKEN` - خدمة المُقيّم: معدة بنفس الطريقة (يقرأ `EVALUATOR_TOKEN` SDK `agenteye-evaluator` حسب الاتفاقية) إذا كان `EVALUATOR_TOKEN` غير محدد، لا يُرسل الخادم رأس `Authorization`؛ قد يقبل المُقيّم طلبات مجهولة، وهذا جيد لشبكة داخلية فقط لكن غير موصى به على الإنترنت العام. @@ -125,7 +125,7 @@ flowchart LR لا يجب أن تُطبق عقد HTTP باليد. حزمة `agenteye-evaluator` Python توفر لك غلاف FastAPI مكتوب يتعامل مع المصادقة والتوجيه وأشكال الطلب/الاستجابة لك. -تشحن Failproof AI Observability أيضاً **مُقيّم مرجعي يعمل** يسجل `helpfulness` و `tool_efficiency` و `factuality` من شكل النسخة. انسخه كنقطة بداية وبدّل منطقك الخاص: قاضٍ LLM، محرك قواعد، أي شيء يناسب معيار الجودة لديك. +تشحن FailproofAI Cloud أيضاً **مُقيّم مرجعي يعمل** يسجل `helpfulness` و `tool_efficiency` و `factuality` من شكل النسخة. انسخه كنقطة بداية وبدّل منطقك الخاص: قاضٍ LLM، محرك قواعد، أي شيء يناسب معيار الجودة لديك. مُقيّم قابل للحياة الدنيا: @@ -148,7 +148,7 @@ def run(req: EvalRequest) -> EvalResponse: مثيل `app` يعمل تحت أي خادم ASGI، لذا `uvicorn module:app` يبدئه. -بالنسبة للمُقيّمين الذين يحتاجون تأجيل عمل مكلف، أرجع `JobPending` بدلاً من ذلك وسجل معالج `@app.job_lookup`؛ يستقصي خادم Observability `GET /evaluate/{job_id}` حتى تُرجع حالة نهائية أو تنقضي قيمة حد `EVALUATOR_MAX_POLL_DURATION_SECS` (افتراضي 1 ساعة). +بالنسبة للمُقيّمين الذين يحتاجون تأجيل عمل مكلف، أرجع `JobPending` بدلاً من ذلك وسجل معالج `@app.job_lookup`؛ يستقصي خادم FailproofAI Cloud `GET /evaluate/{job_id}` حتى تُرجع حالة نهائية أو تنقضي قيمة حد `EVALUATOR_MAX_POLL_DURATION_SECS` (افتراضي 1 ساعة). مرجع الـ API الكامل والنمط غير المتزامن وشماء الحدث موثقة في قراءة `agenteye-evaluator` SDK. @@ -156,7 +156,7 @@ def run(req: EvalRequest) -> EvalResponse: ## تشغيل مُقيّمك -المُقيّم هو **خدمتك** — لا تشحن Failproof AI Observability مُقيّماً افتراضياً، لذا تبني وتشغل أينما تشغل خدماتك. يعمل تحت أي خادم ASGI (على سبيل المثال `uvicorn my_evaluator:app`؛ خدم المسارات `/health` و `/config` و `/evaluate` من [عقد HTTP](#http-contract)، ثم وجّه الخادم إليه (انظر [تكوين الخادم](#configuring-the-server)). +المُقيّم هو **خدمتك** — لا تشحن FailproofAI Cloud مُقيّماً افتراضياً، لذا تبني وتشغل أينما تشغل خدماتك. يعمل تحت أي خادم ASGI (على سبيل المثال `uvicorn my_evaluator:app`؛ خدم المسارات `/health` و `/config` و `/evaluate` من [عقد HTTP](#http-contract)، ثم وجّه الخادم إليه (انظر [تكوين الخادم](#configuring-the-server)). بمجرد وصول المُقيّم، `GET /health` يُرجع `{"status":"ok"}`. بعد انتهاء الوكيل من البداية إلى النهاية، `GET /evaluations` على الخادم يُرجع صفاً مع `status: "done"` والنقاط التي أنتجها مُقيّمك. @@ -254,7 +254,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. - **`/sessions`**: شبكة جلسات قابلة للتصفية؛ عمود النقاط يعرض حالة تقييم كل جلسة ونقاطها في لمحة. - **`/dashboards`**: عروض صحة تقييم محفوظة (انظر [القوائس](#dashboards) أدناه). -![شبكة الجلسات مع حبوب حالة تقييم لكل جلسة وشارات نقاط ملونة (helpfulness، factuality، tool_efficiency، safety، coherence)](/agenteye/images/sessions-list.png) +![شبكة الجلسات مع حبوب حالة تقييم لكل جلسة وشارات نقاط ملونة (helpfulness، factuality، tool_efficiency، safety، coherence)](/cloud/images/sessions-list.png) *تعرض شبكة الجلسات حالة تقييم كل جلسة ونقاطها في لمحة؛ جعل الشارات الحمراء/الكهرمانية/الخضراء النقاط المنخفضة تبرز.* @@ -271,7 +271,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. يعرض كل بطاقة عدد الجلسات المطابقة، تفصيل done/error/timeout، متوسط كل نقطة مميزة، وخط اتجاه صغير. فتح لوحة يعرض اللوحات بحجم كامل؛ **تفتح في جلسات** توديعك في صفحة الجلسات المصفاة مسبقاً لتلك الشريحة تماماً. تُحسب المقاييس على جانب الخادم على المجموعة المطابقة بأكملها (عبر `GET /evaluations/aggregate`)، لذا تكون الأرقام دقيقة بدلاً من أخذ عينات. -![لوحة صحة تقييم مع متوسط أشرطة نقاط لكل بعد مقيّم، تفصيل أداة ok-vs-error، أفضل الأدوات واتجاه أحداث لكل ساعة](/agenteye/images/dashboard-quality.png) +![لوحة صحة تقييم مع متوسط أشرطة نقاط لكل بعد مقيّم، تفصيل أداة ok-vs-error، أفضل الأدوات واتجاه أحداث لكل ساعة](/cloud/images/dashboard-quality.png) **الصلاحيات:** العرض يحتاج كلاً من `dashboards:read` و `evaluations:read`؛ الإنشاء والتعديل يحتاج `dashboards:write`؛ الحذف يحتاج `dashboards:delete`. يستقبل المسؤول التمهيدي جميع هذه تلقائياً. @@ -293,7 +293,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. ## الخطوات التالية -- [مهارة وكيل المُقيّم](/ar/agenteye/evaluator-skill): اطلب من وكيل ترميز أن يصمم أبعادك ضد جلسات حقيقية وينشئ هذه الخدمة لك. -- [Python SDK](/ar/agenteye/python-sdk): أصدر أحداث `agent_end` التي تُثير التسجيل. -- [مفاتيح API](/ar/agenteye/api-keys): صلاحيات `evaluations:read` و `evaluations:trigger`. -- [عمليات التدقيق](/ar/agenteye/audits): ميزة جودة مؤتمتة أخرى من Observability، للمراجعة المستندة إلى السياسة. \ No newline at end of file +- [مهارة وكيل المُقيّم](/ar/cloud/agent-skills): اطلب من وكيل ترميز أن يصمم أبعادك ضد جلسات حقيقية وينشئ هذه الخدمة لك. +- [Python SDK](/ar/cloud/sdk): أصدر أحداث `agent_end` التي تُثير التسجيل. +- [مفاتيح API](/ar/cloud/access): صلاحيات `evaluations:read` و `evaluations:trigger`. +- [عمليات التدقيق](/ar/cloud/audits): ميزة جودة مؤتمتة أخرى من FailproofAI Cloud، للمراجعة المستندة إلى السياسة. \ No newline at end of file diff --git a/docs/ar/agenteye/event-stream.mdx b/docs/ar/cloud/event-stream.mdx similarity index 79% rename from docs/ar/agenteye/event-stream.mdx rename to docs/ar/cloud/event-stream.mdx index e427ef40..29ccdcb5 100644 --- a/docs/ar/agenteye/event-stream.mdx +++ b/docs/ar/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "في اللحظة التي يقوم بها وكيلك بشيء م في اللحظة التي يقوم بها وكيلك بشيء ما، ترى ذلك. تدفق الأحداث هو نبضك الحي لكل وكيل في الإنتاج: بدون انتظار، بدون البحث في السجلات، بدون التكهنات حول ما حدث للتو. -![تدفق الأحداث المباشر: صفوف الأحداث الملونة بالألوان تظهر في الوقت الفعلي، قابلة للتصفية حسب البيئة والوكيل والجلسة ونوع الحدث والبحث النصي](/agenteye/images/events-stream.png) +![تدفق الأحداث المباشر: صفوف الأحداث الملونة بالألوان تظهر في الوقت الفعلي، قابلة للتصفية حسب البيئة والوكيل والجلسة ونوع الحدث والبحث النصي](/cloud/images/events-stream.png) *كل حدث من كل وكيل في مؤسستك، الأحدث أولاً، يتحدث في الوقت الفعلي.* @@ -37,14 +37,14 @@ description: "في اللحظة التي يقوم بها وكيلك بشيء م تدفق الأحداث هو منزل مؤسستك. سجل الدخول وهو أول سطح تهبط عليه، في `//`، لذا يبدأ الفرز في اللحظة التي تصل فيها. -خلفه، يصدر وكلاؤك أحداثاً عبر SDK، ويشحن المجمّع إلى خادم Failproof AI Observability الخاص بك، والتدفق يتابعهم عندما يصلون إلى البنية التحتية التي تتحكم فيها. عندما تريد العرض المجمع بدلاً من المسار الأولي، تنهار أحداث كل تشغيل إلى صف واحد على الجلسات، على بعد نقرة واحدة. +خلفه، يصدر وكلاؤك أحداثاً عبر SDK، ويشحن المجمّع إلى خادم FailproofAI Cloud الخاص بك، والتدفق يتابعهم عندما يصلون إلى البنية التحتية التي تتحكم فيها. عندما تريد العرض المجمع بدلاً من المسار الأولي، تنهار أحداث كل تشغيل إلى صف واحد على الجلسات، على بعد نقرة واحدة. هذا هو مصدر الحقيقة الأولي الذي تبني عليه جميع أسطح الملاحظة الأخرى، لذا عندما يبدو الرقم خاطئاً في مكان آخر، التدفق هو المكان الذي تؤكد فيه ما حدث فعلاً. ## ذات الصلة -- [الجلسات](/ar/agenteye/sessions): نفس الأحداث مجمعة في صف واحد لكل تشغيل، مع رسم بياني للتنفيذ بنمط git. -- [القياس عن بعد](/ar/agenteye/telemetry): ما يرسله وكلاؤك وكيف تصل الأحداث إلى التدفق. -- [تتبع الأخطاء](/ar/agenteye/error-tracking): سطح فرز واحد لكل ما حدث بشكل خاطئ. -- [التنبيهات](/ar/agenteye/alerts): حول أي عتبة إلى قاعدة صفحة. -- [CLI والوكلاء](/ar/agenteye/cli-and-agents): نفس المسار الحي من المحطة الطرفية. \ No newline at end of file +- [الجلسات](/ar/cloud/sessions): نفس الأحداث مجمعة في صف واحد لكل تشغيل، مع رسم بياني للتنفيذ بنمط git. +- [القياس عن بعد](/ar/cloud/performance): ما يرسله وكلاؤك وكيف تصل الأحداث إلى التدفق. +- [تتبع الأخطاء](/ar/cloud/errors): سطح فرز واحد لكل ما حدث بشكل خاطئ. +- [التنبيهات](/ar/cloud/alerts): حول أي عتبة إلى قاعدة صفحة. +- [CLI والوكلاء](/ar/cloud/cli): نفس المسار الحي من المحطة الطرفية. \ No newline at end of file diff --git a/docs/ar/cloud/fleet.mdx b/docs/ar/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/ar/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/ar/agenteye/incidents.mdx b/docs/ar/cloud/incidents.mdx similarity index 91% rename from docs/ar/agenteye/incidents.mdx rename to docs/ar/cloud/incidents.mdx index 9be50d72..8e0558f3 100644 --- a/docs/ar/agenteye/incidents.mdx +++ b/docs/ar/cloud/incidents.mdx @@ -7,7 +7,7 @@ description: "عندما يطلق تنبيه ما، يمكن للجميع رؤي عندما يطلق تنبيه ما، السؤال الأول هو دائماً "من يتولى الأمر؟" الحوادث تجيب على ذلك: في اللحظة التي يحدث خرق ما، يمكن للجميع رؤية أن الحادثة مفتوحة، ومن يملك الملكية، وبالضبط ما حدث حتى الآن، مع سجل نظيف ومنسوب يمكنك تسليمه مباشرة إلى جلسة تحليل ما بعد الحادثة. -![صندوق وارد الحوادث: بطاقات حوادث مرتبطة بالتنبيهات ومفتوحة يدويًا، مجمعة حسب الحالة، كل منها مع شارة خطورة وشخص مسؤول](/agenteye/images/incidents.png) +![صندوق وارد الحوادث: بطاقات حوادث مرتبطة بالتنبيهات ومفتوحة يدويًا، مجمعة حسب الحالة، كل منها مع شارة خطورة وشخص مسؤول](/cloud/images/incidents.png) *يجمع الصندوق الحوادث المفتوحة حسب الحالة وينقيها حسب مستوى الخطورة والشخص المسؤول، لتري ما يحتاج تدخل بشري الآن.* ## اعرف من يتولى الأمر، بلمحة واحدة @@ -18,7 +18,7 @@ description: "عندما يطلق تنبيه ما، يمكن للجميع رؤي عندما تنتهي الحادثة، تكون لديك بالفعل التقرير. افتح أي حادثة وستحصل على دليل الخرق، والأشخاص المسؤولين والمشتركين، وخيط تعليقات للتنسيق في نفس المكان، وخط زمني نشاط منسوب وإضافي فقط. -![عرض تفاصيل الحادثة: التنبيه الأب وملخص الخرق، الأشخاص المسؤولين والمشتركين، خط زمني نشاط منسوب، وخيط تعليقات](/agenteye/images/incident-detail.png) +![عرض تفاصيل الحادثة: التنبيه الأب وملخص الخرق، الأشخاص المسؤولين والمشتركين، خط زمني نشاط منسوب، وخيط تعليقات](/cloud/images/incident-detail.png) *كل ما حدث، بالترتيب، كل سطر موقّع من قبل من قام به.* كل إجراء (مفتوح، معترف به، تم حله، وما إلى ذلك) يُكتب في هذا الخط الزمني ولا يُعدّل أبداً. كل إدخال منسوب: إلى المشغل الذي اتخذه، برسالة البريد الإلكتروني، أو إلى **automated** لأي شيء فعلته Failproof AI تلقائياً، مثل فتح الحادثة على الخرق. لا شيء مجهول ولا شيء ضائع، لذا فإن تحليل ما بعد الحادثة يكتب نفسه تقريباً. @@ -46,6 +46,6 @@ stateDiagram-v2 ## ذات صلة -- [التنبيهات](/ar/agenteye/alerts): القواعد التي تفتح هذه الحوادث عندما يحدث خرق للحد. -- [تتبع الأخطاء](/ar/agenteye/error-tracking): شاهد كل فشل في مكان واحد وارفعه إلى تنبيه. -- [التدقيق](/ar/agenteye/audits): محلل مجدول يجد الأخطاء التي لم تراقبها أي قاعدة. \ No newline at end of file +- [التنبيهات](/ar/cloud/alerts): القواعد التي تفتح هذه الحوادث عندما يحدث خرق للحد. +- [تتبع الأخطاء](/ar/cloud/errors): شاهد كل فشل في مكان واحد وارفعه إلى تنبيه. +- [التدقيق](/ar/cloud/audits): محلل مجدول يجد الأخطاء التي لم تراقبها أي قاعدة. \ No newline at end of file diff --git a/docs/ar/cloud/managed-policies.mdx b/docs/ar/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/ar/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/ar/agenteye/overview.mdx b/docs/ar/cloud/overview.mdx similarity index 51% rename from docs/ar/agenteye/overview.mdx rename to docs/ar/cloud/overview.mdx index 5ba3aa5b..cacda463 100644 --- a/docs/ar/agenteye/overview.mdx +++ b/docs/ar/cloud/overview.mdx @@ -1,16 +1,16 @@ --- --- title: "Failproof AI: مراقبة الوكلاء بحثاً عن الأعطال" -description: "Failproof AI Observability هي منصة ذاتية الاستضافة لمراقبة وتقييم وتحسين وكلائك الذكيين في بيئة الإنتاج." +description: "FailproofAI Cloud هي منصة ذاتية الاستضافة لمراقبة وتقييم وتحسين وكلائك الذكيين في بيئة الإنتاج." --- -Failproof AI Observability هي منصة ذاتية الاستضافة لمراقبة وتقييم وتحسين وكلائك الذكيين في بيئة الإنتاج. تسجل كل شيء يفعله وكلاؤك (كل استدعاء أداة، طلب نموذج، hook، وخطأ)، وتقيّم جودة كل تشغيل، وتكشف الأعطال التي لم تكن تعرف أنك بحاجة للبحث عنها، كل ذلك في لوحة تعمل داخل بنيتك التحتية الخاصة. +FailproofAI Cloud هي منصة ذاتية الاستضافة لمراقبة وتقييم وتحسين وكلائك الذكيين في بيئة الإنتاج. تسجل كل شيء يفعله وكلاؤك (كل استدعاء أداة، طلب نموذج، hook، وخطأ)، وتقيّم جودة كل تشغيل، وتكشف الأعطال التي لم تكن تعرف أنك بحاجة للبحث عنها، كل ذلك في لوحة تعمل داخل بنيتك التحتية الخاصة. -إذا كنت تطلق وكلاء ذكيين وتعبت من التخمين حول سبب فشل التشغيل، فهذه هي الصفحة المناسبة للبدء. تشرح ما يقدمه Failproof AI Observability وكيف تتناسب الأجزاء معاً، قبل تثبيت أي شيء. +إذا كنت تطلق وكلاء ذكيين وتعبت من التخمين حول سبب فشل التشغيل، فهذه هي الصفحة المناسبة للبدء. تشرح ما يقدمه FailproofAI Cloud وكيف تتناسب الأجزاء معاً، قبل تثبيت أي شيء. -> **Failproof AI Observability هو منتج للمؤسسات من Failproof AI.** هل تريد رؤيته قيد التشغيل؟ اطلب عرضاً توضيحياً: أرسل بريداً إلى [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +> **FailproofAI Cloud هو منتج للمؤسسات من Failproof AI.** هل تريد رؤيته قيد التشغيل؟ اطلب عرضاً توضيحياً: أرسل بريداً إلى [nikita@befailproof.ai](mailto:nikita@befailproof.ai). -![جلسة Failproof AI Observability مرسومة كرسم بياني تنفيذي بنمط git بجانب جدول الأحداث الخاص بها، مع تفصيل لكل تشغيل للأدوات والنماذج والـ hooks في العمود الأيمن](/agenteye/images/session-detail.png) +![جلسة FailproofAI Cloud مرسومة كرسم بياني تنفيذي بنمط git بجانب جدول الأحداث الخاص بها، مع تفصيل لكل تشغيل للأدوات والنماذج والـ hooks في العمود الأيمن](/cloud/images/session-detail.png) *يتم رسم كل تشغيل وكيل كرسم بياني تنفيذي بنمط git (على اليسار) بجانب جدول الأحداث الخاص به. يحصل كل وكيل فرعي متوازي على خطه الخاص؛ يقدم العمود الأيمن تفصيلاً للأدوات والنماذج والـ hooks واستهلاك الرموز للتشغيل.* @@ -30,52 +30,52 @@ Failproof AI Observability هي منصة ذاتية الاستضافة لمرا
-*Failproof Audit: اترك Failproof AI Observability تُنقِّب عن السجلات عبر الجلسات وأخبرك بما يجب إصلاحه.* +*Failproof Audit: اترك FailproofAI Cloud تُنقِّب عن السجلات عبر الجلسات وأخبرك بما يجب إصلاحه.* --- ## لماذا تستخدمه الفرق - **شاهد ما فعله وكيلك فعلاً.** كل تشغيل يصبح رسم بياني تنفيذي قابلاً للقراءة بنمط git: أي الأدوات تعمل بالتوازي، أي الوكلاء الفرعيين انقسموا، أين توقفت، وما الذي أنفقته. -- **اكتشف انحدارات الجودة تلقائياً.** اربط خدمة تقييم صغيرة و Failproof AI Observability ستقيّم كل تشغيل منتهٍ، بحيث ينعكس انخفاض الفائدة أو ارتفاع الهلوسة بنفسه. +- **اكتشف انحدارات الجودة تلقائياً.** اربط خدمة تقييم صغيرة و FailproofAI Cloud ستقيّم كل تشغيل منتهٍ، بحيث ينعكس انخفاض الفائدة أو ارتفاع الهلوسة بنفسه. - **اعثر على أعطال لم تكتب لها قاعدة.** تعمل عمليات التدقيق المتكررة على تنقيب السجلات عبر الجلسات بحثاً عن مجموعات الأخطاء ونقاط الكمون الشاذة والنتائج المنخفضة والتشغيلات المعلقة، ثم تسلمك النتائج المرتبة والمدعومة بالأدلة. - **احصل على تنبيه عند أهمية ذلك.** تطلق قواعد الحد الأدنى على معدل الخطأ والكمون والتكلفة أو نقاط المقيّم وتفتح حوادث يمكنك الإقرار بها وتعيينها وحلها. - **اطرح أسئلة باللغة الإنجليزية العادية.** يجيب مساعد ذكي داخل لوحة التحكم على سؤال مثل كيف تتجه الجودة في الإنتاج هذا الأسبوع على بيانات الخاصة بك. أي تغيير يقوم به يخضع لموافقة. -- **احتفظ ببيانات الخاص بك.** Failproof AI Observability ذاتية الاستضافة: تبقى الأحداث والتوجيهات والتحليلات في البنية التحتية التي تتحكم فيها. +- **احتفظ ببيانات الخاص بك.** FailproofAI Cloud ذاتية الاستضافة: تبقى الأحداث والتوجيهات والتحليلات في البنية التحتية التي تتحكم فيها. --- ## ما تحصل عليه -يتم تنظيم Failproof AI Observability حول ثلاث أفكار (**المراقبة** و**التحليل** و**الإدارة**)، مما يعكس الشريط الجانبي الأيسر للوحة التحكم. +يتم تنظيم FailproofAI Cloud حول ثلاث أفكار (**المراقبة** و**التحليل** و**الإدارة**)، مما يعكس الشريط الجانبي الأيسر للوحة التحكم. **المراقبة** (الحقيقة الخام لما حدث): -- **[تدفق الأحداث](/ar/agenteye/event-stream)**: مسار الحي، لكل خطوة، لكل تشغيل (استدعاءات أدوات، استدعاءات نموذج، hooks، أخطاء). -- **[الجلسات](/ar/agenteye/sessions)**: تلك الأحداث المجمعة في صف واحد لكل تشغيل، كل منها جاهز للتقييم، مع رسم بياني تنفيذي بنمط git. -- **[مقاييس الأداء](/ar/agenteye/telemetry)**: خرائط حرارية للكمون لكل سطح و p50/p95/p99 الحيويات للنماذج والأدوات والـ hooks، بحيث تبرز قمة الذيل عن الوسيط. -- **[تتبع الأخطاء](/ar/agenteye/error-tracking)**: سطح فحص واحد لكل شيء خاطئ، نقرة واحدة من تنبيه حار. +- **[تدفق الأحداث](/ar/cloud/event-stream)**: مسار الحي، لكل خطوة، لكل تشغيل (استدعاءات أدوات، استدعاءات نموذج، hooks، أخطاء). +- **[الجلسات](/ar/cloud/sessions)**: تلك الأحداث المجمعة في صف واحد لكل تشغيل، كل منها جاهز للتقييم، مع رسم بياني تنفيذي بنمط git. +- **[مقاييس الأداء](/ar/cloud/performance)**: خرائط حرارية للكمون لكل سطح و p50/p95/p99 الحيويات للنماذج والأدوات والـ hooks، بحيث تبرز قمة الذيل عن الوسيط. +- **[تتبع الأخطاء](/ar/cloud/errors)**: سطح فحص واحد لكل شيء خاطئ، نقرة واحدة من تنبيه حار. -![صفحة الملاحظات للأدوات: خريطة حرارية للكمون، وشريط حدود النسبة المئوية، وشريط توزيع الأدوات على 24 صندوق زمني](/agenteye/images/tools.png) +![صفحة الملاحظات للأدوات: خريطة حرارية للكمون، وشريط حدود النسبة المئوية، وشريط توزيع الأدوات على 24 صندوق زمني](/cloud/images/tools.png) *يجمع كل سطح ملاحظات بين خط رقيق و p50/p95/p99 الحيويات مع خريطة حرارية للكمون وشريط حدود النسبة المئوية. معروض هنا: الأدوات.* **التحليل** (تحويل النشاط إلى إجابات): -- **[الاستعلامات](/ar/agenteye/queries)** و**[لوحات التحكم](/ar/agenteye/dashboards)**: SQL المحفوظة على أحداثك والتقييمات الخاصة بك، المرسومة في لوحات تحكم مشتركة ومحدودة بالمنظمة. -- **[التقييمات](/ar/agenteye/evaluations)**: نقاط الجودة التي ينتجها خدمة المقيّم الخاصة بك، مع الأسباب لكل نقطة. -- **[عمليات التدقيق](/ar/agenteye/audits)**: تحقيقات متكررة تكشف أنماط الأعطال عبر الجلسات. -- **[التنبيهات](/ar/agenteye/alerts)** و**[الحوادث](/ar/agenteye/incidents)**: قواعد الحد الأدنى التي تنبهك، بالإضافة إلى سير عمل الحادثة لفحصها. +- **[الاستعلامات](/ar/cloud/queries)** و**[لوحات التحكم](/ar/cloud/dashboards)**: SQL المحفوظة على أحداثك والتقييمات الخاصة بك، المرسومة في لوحات تحكم مشتركة ومحدودة بالمنظمة. +- **[التقييمات](/ar/cloud/evaluations)**: نقاط الجودة التي ينتجها خدمة المقيّم الخاصة بك، مع الأسباب لكل نقطة. +- **[عمليات التدقيق](/ar/cloud/audits)**: تحقيقات متكررة تكشف أنماط الأعطال عبر الجلسات. +- **[التنبيهات](/ar/cloud/alerts)** و**[الحوادث](/ar/cloud/incidents)**: قواعد الحد الأدنى التي تنبهك، بالإضافة إلى سير عمل الحادثة لفحصها. **الواجهات** (الوصول إلى بيانات الخاصة بك بطريقتك): -- **[واجهة سطر الأوامر](/ar/agenteye/cli-and-agents)**: قيادة نشرك الكامل من الطرفية أو نص، والسماح لوكيل البرمجة بفعل ذلك باللغة الإنجليزية العادية. -- **[المساعد الذكي](/ar/agenteye/assistant)**: اطرح أسئلة حول وكلائك باللغة الإنجليزية العادية، مباشرة داخل لوحة التحكم. -- **REST API**: كل ما تفعله لوحة التحكم والـ CLI يدعمه REST API يمكنك استدعاؤه مباشرة باستخدام [مفتاح API](/ar/agenteye/api-keys) محدود النطاق — ابتلع الأحداث، استعلم عن الجلسات والتقييمات، وأدر لوحات التحكم والتنبيهات وعمليات التدقيق والمستخدمين والمفاتيح، حتى تتمكن من دمج Failproof AI Observability في أدواتك الخاصة. +- **[واجهة سطر الأوامر](/ar/cloud/cli)**: قيادة نشرك الكامل من الطرفية أو نص، والسماح لوكيل البرمجة بفعل ذلك باللغة الإنجليزية العادية. +- **[المساعد الذكي](/ar/cloud/assistant)**: اطرح أسئلة حول وكلائك باللغة الإنجليزية العادية، مباشرة داخل لوحة التحكم. +- **REST API**: كل ما تفعله لوحة التحكم والـ CLI يدعمه REST API يمكنك استدعاؤه مباشرة باستخدام [مفتاح API](/ar/cloud/access) محدود النطاق — ابتلع الأحداث، استعلم عن الجلسات والتقييمات، وأدر لوحات التحكم والتنبيهات وعمليات التدقيق والمستخدمين والمفاتيح، حتى تتمكن من دمج FailproofAI Cloud في أدواتك الخاصة. **الإدارة** (قم بتشغيله لفريقك): -- **[مفاتيح API](/ar/agenteye/api-keys)**: رموز محدودة النطاق لجامع البيانات ولوحة التحكم والمساعد. +- **[مفاتيح API](/ar/cloud/access)**: رموز محدودة النطاق لجامع البيانات ولوحة التحكم والمساعد. - **المستخدمون**: تسجيل الدخول بدون كلمة مرور على أساس البريد الإلكتروني مع قائمة بيضاء. - **الإعدادات**: تكوين لكل منظمة، بما في ذلك تجاوزات نافذة السياق للنموذج. @@ -91,18 +91,18 @@ Failproof AI Observability هي منصة ذاتية الاستضافة لمرا - **لوحة التحكم**: حيث تستكشف كل شيء. - **الخدمات الاختيارية**: خدمة تقييم (التقييمات)، وخدمة مساعد ذكي (الدردشة داخل لوحة التحكم). -للمفردات المستخدمة في جميع أنحاء المستندات (*event و session و evaluation و audit و finding و incident*)، انظر [المفاهيم](/ar/agenteye/concepts). +للمفردات المستخدمة في جميع أنحاء المستندات (*event و session و evaluation و audit و finding و incident*)، انظر [المفاهيم](/ar/concepts). --- -## الحصول على Failproof AI Observability +## الحصول على FailproofAI Cloud -Failproof AI Observability هو منتج للمؤسسات من Failproof AI، ويعمل جنباً إلى جنب مع Failproof AI Enforcement — منتج السياسة والحواجز الوقائية — تحت علامة Failproof AI. يعمل بالكامل في بيئتك الخاصة. إذا لم يكن لديك حق الوصول إلى الحزم بعد، اطلب عرضاً توضيحياً وسنحضرك للإعداد: أرسل بريداً إلى [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +FailproofAI Cloud هو منتج للمؤسسات من Failproof AI، ويعمل جنباً إلى جنب مع FailproofAI guardrails — منتج السياسة والحواجز الوقائية — تحت علامة Failproof AI. يعمل بالكامل في بيئتك الخاصة. إذا لم يكن لديك حق الوصول إلى الحزم بعد، اطلب عرضاً توضيحياً وسنحضرك للإعداد: أرسل بريداً إلى [nikita@befailproof.ai](mailto:nikita@befailproof.ai). --- ## الخطوات التالية -- [المفاهيم](/ar/agenteye/concepts): مفردات Failproof AI Observability في مكان واحد. -- [الملاحظة](/ar/agenteye/observability): تابع ما يفعله وكلاؤك، تشغيل تلو الآخر. -- [الأمان](/ar/agenteye/security): كيف يحتفظ Failproof AI Observability ببيانات الخاصة بك معزولة وتحت سيطرتك. \ No newline at end of file +- [المفاهيم](/ar/concepts): مفردات FailproofAI Cloud في مكان واحد. +- [الملاحظة](/ar/cloud/overview): تابع ما يفعله وكلاؤك، تشغيل تلو الآخر. +- [الأمان](/ar/cloud/security): كيف يحتفظ FailproofAI Cloud ببيانات الخاصة بك معزولة وتحت سيطرتك. \ No newline at end of file diff --git a/docs/ar/agenteye/telemetry.mdx b/docs/ar/cloud/performance.mdx similarity index 83% rename from docs/ar/agenteye/telemetry.mdx rename to docs/ar/cloud/performance.mdx index 8648e842..63c8aa11 100644 --- a/docs/ar/agenteye/telemetry.mdx +++ b/docs/ar/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "اكتشف اللحظة التي تبطئ فيها نماذجك أ اكتشف اللحظة التي تبطئ فيها نماذجك أو أدواتك أو خطافاتك أو تزيد الفواتير، واعترض قفزة الكمون النهائي قبل أن يشعر بها مستخدموك. ثلاث صفحات مخصصة تحول التوقيتات الخام إلى p50 و p95 و p99 يمكنك قراءتها في لمحة. -![صفحة النماذج تعرض خريطة حرارية للكمون، وشريط مئوي، وأرقام التوكن والتكلفة والنافذة السياقية لكل نموذج](/agenteye/images/models.png) +![صفحة النماذج تعرض خريطة حرارية للكمون، وشريط مئوي، وأرقام التوكن والتكلفة والنافذة السياقية لكل نموذج](/cloud/images/models.png) *صفحة النماذج: خريطة حرارية للكمون، وشريط مئوي، وأرقام التوكن والتكلفة المقدرة ومؤشر امتلاء النافذة السياقية لكل نموذج.* ## توقف عن السماح للمتوسطات بإخفاء أسوأ عملياتك @@ -24,13 +24,13 @@ description: "اكتشف اللحظة التي تبطئ فيها نماذجك أ صفحة النماذج (كما هو موضح أعلاه) تجيب على السؤالين اللذين تطرحهما الفاتورة دائمًا: أي نموذج وكم التكلفة. بالإضافة إلى عرض الكمون المشترك، فإنها تضيف **استهلاك التوكن لكل نموذج** و **التكلفة المقدرة** و **مؤشر امتلاء النافذة السياقية**، بحيث يكون نمو الطلب الجامح واقتراب الضغط مرئيًا قبل أن يفاجئك. -يتعرف Failproof AI Observability على معرّفات النماذج الشائعة تلقائيًا. إذا بدت نافذة غير صحيحة، أو كنت تشغل نموذجك الخاص، قم بتصحيحها أو أضف واحدة ضمن **الإعدادات** في **نوافذ السياق للنموذج** والقراءات المتعلقة بالامتلاء تتبع. +يتعرف FailproofAI Cloud على معرّفات النماذج الشائعة تلقائيًا. إذا بدت نافذة غير صحيحة، أو كنت تشغل نموذجك الخاص، قم بتصحيحها أو أضف واحدة ضمن **الإعدادات** في **نوافذ السياق للنموذج** والقراءات المتعلقة بالامتلاء تتبع. ## الأدوات: ميز البطيء عن المكسور يمكن أن تكون استدعاءة الأداة بطيئة، أو قد تفشل بهدوء، وتريد أن تعرف أيهما في ثوانٍ، وليس بعد البحث في السجلات. -![صفحة الأدوات تعرض خريطة الكمون الحرارية المشتركة وشريط المئويات بجانب تفصيل النجاح والفشل وشريط توزيع الأدوات](/agenteye/images/tools.png) +![صفحة الأدوات تعرض خريطة الكمون الحرارية المشتركة وشريط المئويات بجانب تفصيل النجاح والفشل وشريط توزيع الأدوات](/cloud/images/tools.png) *صفحة الأدوات: نفس الخريطة الحرارية والشريط المئوي، بالإضافة إلى تفصيل النجاح والفشل وشريط توزيع الأدوات.* إلى جانب عرض الكمون المشترك، تضيف صفحة الأدوات **تفصيل النجاح والفشل** و **شريط توزيع الأدوات**، بحيث ترى في لمحة الأدوات التي تعتمد عليها أكثر والتي تستنزف ميزانية الخطأ الخاصة بك. @@ -39,14 +39,14 @@ description: "اكتشف اللحظة التي تبطئ فيها نماذجك أ عندما يبطئ خطاف دورة حياة عملية ما، "الخطافات بطيئة" ليس شيئًا يمكنك العمل عليه. تأخذك صفحة الخطافات إلى الواحد الذي يهمك. -![صفحة الخطافات تعرض الكمون مقسم حسب اسم الخطاف وحدث التفعيل على خريطة الكمون الحرارية المشتركة والشريط المئوي](/agenteye/images/hooks.png) +![صفحة الخطافات تعرض الكمون مقسم حسب اسم الخطاف وحدث التفعيل على خريطة الكمون الحرارية المشتركة والشريط المئوي](/cloud/images/hooks.png) *صفحة الخطافات: الكمون مقسم حسب اسم الخطاف وحدث التفعيل.* فوق نفس خريطة الكمون الحرارية والشريط المئوي، تقسم صفحة الخطافات النشاط حسب **اسم الخطاف** و **حدث التفعيل**، بحيث تهبط على الخطاف الواحد وحدث التفعيل الواحد اللذين يحتاجان إلى انتباه. ## ذات صلة -- [دفق الأحداث](/ar/agenteye/event-stream): المسار الفوري الملون لكل حدث. -- [الجلسات](/ar/agenteye/sessions): قم بتجميع الأحداث في صف واحد لكل تشغيل وافتح رسم البياني الخاص به. -- [تتبع الأخطاء](/ar/agenteye/error-tracking): سطح تريج واحد لكل شيء يرسمه لوحة المعلومات باللون الأحمر. -- [لوحات المعلومات](/ar/agenteye/dashboards): طرق التجميع عبر أسطولك. \ No newline at end of file +- [دفق الأحداث](/ar/cloud/event-stream): المسار الفوري الملون لكل حدث. +- [الجلسات](/ar/cloud/sessions): قم بتجميع الأحداث في صف واحد لكل تشغيل وافتح رسم البياني الخاص به. +- [تتبع الأخطاء](/ar/cloud/errors): سطح تريج واحد لكل شيء يرسمه لوحة المعلومات باللون الأحمر. +- [لوحات المعلومات](/ar/cloud/dashboards): طرق التجميع عبر أسطولك. \ No newline at end of file diff --git a/docs/ar/agenteye/queries.mdx b/docs/ar/cloud/queries.mdx similarity index 78% rename from docs/ar/agenteye/queries.mdx rename to docs/ar/cloud/queries.mdx index 421195ac..c072ce01 100644 --- a/docs/ar/agenteye/queries.mdx +++ b/docs/ar/cloud/queries.mdx @@ -5,9 +5,9 @@ description: "اطرح أي سؤال حول بيانات وكيلك واحصل --- -اطرح أي سؤال حول بيانات وكيلك واحصل على إجابة في ثوان. يوفر لك Failproof AI Observability مكتبة من الاستعلامات المحفوظة والجاهزة للتشغيل على أحداثك وتقييماتك، لذلك تبدأ من مثال يعمل بدلاً من محرر SQL فارغ. +اطرح أي سؤال حول بيانات وكيلك واحصل على إجابة في ثوان. يوفر لك FailproofAI Cloud مكتبة من الاستعلامات المحفوظة والجاهزة للتشغيل على أحداثك وتقييماتك، لذلك تبدأ من مثال يعمل بدلاً من محرر SQL فارغ. -![مكتبة الاستعلامات المحفوظة: شبكة من الاستعلامات القابلة لإعادة الاستخدام، سواء كانت إعدادات مدمجة أو استعلامات مخصصة](/agenteye/images/queries.png) +![مكتبة الاستعلامات المحفوظة: شبكة من الاستعلامات القابلة لإعادة الاستخدام، سواء كانت إعدادات مدمجة أو استعلامات مخصصة](/cloud/images/queries.png) *مكتبة الاستعلامات المحفوظة لديك في `//queries`: الإعدادات المدمجة بجانب الاستعلامات التي حفظتها فريقك.* @@ -23,7 +23,7 @@ description: "اطرح أي سؤال حول بيانات وكيلك واحصل افتح أي استعلام وسيهبط في مؤلف SQL، حيث يمكنك تعديله ورؤية الإجابة على الفور: لا توجد عمليات تصدير، لا رحلات ذهاباً وإياباً، لا انتظار لشخص آخر. -![مؤلف استعلام SQL يقوم بتشغيل استعلام محفوظ، مع شريط جانبي للمخطط وشبكة نتائج حية](/agenteye/images/query-lab.png) +![مؤلف استعلام SQL يقوم بتشغيل استعلام محفوظ، مع شريط جانبي للمخطط وشبكة نتائج حية](/cloud/images/query-lab.png) *مؤلف SQL: استعلامك على اليسار، وشريط جانبي للمخطط حتى لا تخمن اسم عمود، وشبكة نتائج حية أدناه.* @@ -44,14 +44,14 @@ agenteye query list # same saved queries, from your terminal agenteye query run errs --arg prod # run one and print the rows (add --json to pipe it) ``` - انظر [CLI والوكلاء](/ar/agenteye/cli-and-agents) للحصول على مجموعة الأوامر الكاملة. + انظر [CLI والوكلاء](/ar/cloud/cli) للحصول على مجموعة الأوامر الكاملة. -- **من مساعد AI.** غير متأكد من كيفية صياغة SQL؟ اسأل [مساعد AI](/ar/agenteye/assistant) في لوحة المعلومات بلغة إنجليزية عادية وسيقوم بصياغة الاستعلام وحفظه في مكتبتك لك. +- **من مساعد AI.** غير متأكد من كيفية صياغة SQL؟ اسأل [مساعد AI](/ar/cloud/assistant) في لوحة المعلومات بلغة إنجليزية عادية وسيقوم بصياغة الاستعلام وحفظه في مكتبتك لك. يتم التحكم في تشغيل استعلام محفوظ بواسطة صلاحية `queries:run`، يتم فصله عن الأذونات لإنشاء أو حذف الاستعلامات، لذا يمكنك منح إمكانية الوصول للقراءة دون السماح للجميع بإعادة كتابة المكتبة. ## ذو الصلة -- [لوحات المعلومات](/ar/agenteye/dashboards): ثبت نتائج الاستعلامات في الرسوم البيانية المشتركة على مستوى المنظمة. -- [مساعد AI](/ar/agenteye/assistant): اطرح أسئلة باللغة الإنجليزية العادية واحصل على استعلام. -- [CLI والوكلاء](/ar/agenteye/cli-and-agents): شغّل واحفظ نفس الاستعلامات من محطتك الطرفية. \ No newline at end of file +- [لوحات المعلومات](/ar/cloud/dashboards): ثبت نتائج الاستعلامات في الرسوم البيانية المشتركة على مستوى المنظمة. +- [مساعد AI](/ar/cloud/assistant): اطرح أسئلة باللغة الإنجليزية العادية واحصل على استعلام. +- [CLI والوكلاء](/ar/cloud/cli): شغّل واحفظ نفس الاستعلامات من محطتك الطرفية. \ No newline at end of file diff --git a/docs/ar/agenteye/python-sdk.mdx b/docs/ar/cloud/sdk.mdx similarity index 92% rename from docs/ar/agenteye/python-sdk.mdx rename to docs/ar/cloud/sdk.mdx index e89710dd..4d4b6a2a 100644 --- a/docs/ar/agenteye/python-sdk.mdx +++ b/docs/ar/cloud/sdk.mdx @@ -3,11 +3,11 @@ title: "Python SDK" description: "شاهد بالضبط ما فعلته وكلاء الذكاء الاصطناعي الخاصة بك في الإنتاج: كل تشغيل للوكيل، استدعاء أداة، طلب نموذج، خطاف، وتدخل بشري." --- -شاهد بالضبط ما فعلته وكلاء الذكاء الاصطناعي الخاصة بك في الإنتاج: كل تشغيل للوكيل، استدعاء أداة، طلب نموذج، خطاف، وتدخل بشري. يسجل Failproof AI Observability Python SDK هذا المسار من داخل كود الوكيل الخاص بك حتى تتمكن من تصحيح الأخطاء والتدقيق وتقييم ما حدث. استخدمه كلما أردت أن يراقب Failproof AI Observability وكلاءك. +شاهد بالضبط ما فعلته وكلاء الذكاء الاصطناعي الخاصة بك في الإنتاج: كل تشغيل للوكيل، استدعاء أداة، طلب نموذج، خطاف، وتدخل بشري. يسجل FailproofAI Cloud Python SDK هذا المسار من داخل كود الوكيل الخاص بك حتى تتمكن من تصحيح الأخطاء والتدقيق وتقييم ما حدث. استخدمه كلما أردت أن يراقب FailproofAI Cloud وكلاءك. تحت الغطاء، يكتب SDK أحداثاً منظمة في ملفات JSONL محلية، وتلتقطها عملية جمع البيانات الخلفية وترسلها إلى المنصة تلقائياً. لا تحتاج إلى إدارة تلك الملفات بنفسك. -> **نصيحة:** جديد في Failproof AI Observability؟ هذه الصفحة هي مرجع أحداث SDK الكامل. +> **نصيحة:** جديد في FailproofAI Cloud؟ هذه الصفحة هي مرجع أحداث SDK الكامل.
@@ -25,7 +25,7 @@ description: "شاهد بالضبط ما فعلته وكلاء الذكاء ال python -c "import agenteye; print(agenteye.__version__)" ``` -هل تفضل السماح لوكيل ترميز بإجراء التكامل كله؟ [Python SDK Agent Skill](/ar/agenteye/python-sdk-skill) يعرف مسار التثبيت، ويخطط نقاط الأداة، ويكتبها، ويتحقق من وصول الأحداث. +هل تفضل السماح لوكيل ترميز بإجراء التكامل كله؟ [Python SDK Agent Skill](/ar/cloud/agent-skills) يعرف مسار التثبيت، ويخطط نقاط الأداة، ويكتبها، ويتحقق من وصول الأحداث. --- @@ -59,7 +59,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### أداة استدعاء حقيقية -في الممارسة العملية، تلف كود الوكيل الموجود لديك. ضع استدعاء نموذج بين `model_request` قبل و `model_response` بعده، بحيث يمتد الحدثان على الطلب الفعلي ويمكن لـ Failproof AI Observability أن يقرن بينهما: +في الممارسة العملية، تلف كود الوكيل الموجود لديك. ضع استدعاء نموذج بين `model_request` قبل و `model_response` بعده، بحيث يمتد الحدثان على الطلب الفعلي ويمكن لـ FailproofAI Cloud أن يقرن بينهما: ```python import anthropic @@ -98,7 +98,7 @@ agenteye.event.model_response( إليك ما تبدو عليه تلك الأحداث بمجرد وصولها إلى لوحة التحكم، مرمزة بالألوان حسب النوع وقابلة للتصفية حسب البيئة والوكيل والجلسة: -![تدفق الأحداث المباشر، مرمز بألوان حسب نوع الحدث وقابل للتصفية حسب البيئة والوكيل والجلسة](/agenteye/images/events-stream.png) +![تدفق الأحداث المباشر، مرمز بألوان حسب نوع الحدث وقابل للتصفية حسب البيئة والوكيل والجلسة](/cloud/images/events-stream.png) --- @@ -155,9 +155,9 @@ export AGENTEYE_ENVIRONMENT=production ## مرجع الأحداث -تأتي معظم الأحداث في أزواج البداية/النهاية التي تشترك في معرف الارتباط: يشترك `tool_use` و `tool_result` في `tool_call_id`، و `hook_triggered` و `hook_completed` يشتركان في `hook_id`، و `human_wait` و `human_input` يشتركان في `input_id`. أرسل حدث البداية، قم بالعمل، ثم أرسل حدث النهاية برفقة نفس المعرف. يطابق Failproof AI Observability الزوج ويحسب `duration_ms` لك، لذا لا تمرر `duration_ms` بنفسك. +تأتي معظم الأحداث في أزواج البداية/النهاية التي تشترك في معرف الارتباط: يشترك `tool_use` و `tool_result` في `tool_call_id`، و `hook_triggered` و `hook_completed` يشتركان في `hook_id`، و `human_wait` و `human_input` يشتركان في `input_id`. أرسل حدث البداية، قم بالعمل، ثم أرسل حدث النهاية برفقة نفس المعرف. يطابق FailproofAI Cloud الزوج ويحسب `duration_ms` لك، لذا لا تمرر `duration_ms` بنفسك. -![رسم بياني لتنفيذ جلسة على طراز git بجانب الخط الزمني للحدث، تم إعادة بناؤه من الأحداث المقترنة، مع لوحة تفصيل الأداة/النموذج/الخطاف](/agenteye/images/session-detail.png) +![رسم بياني لتنفيذ جلسة على طراز git بجانب الخط الزمني للحدث، تم إعادة بناؤه من الأحداث المقترنة، مع لوحة تفصيل الأداة/النموذج/الخطاف](/cloud/images/session-detail.png) تتطلب جميع طرق الأحداث هذين الحقلين: @@ -431,5 +431,5 @@ agenteye.event.tool_use( ## الخطوات التالية -- [تدفق الأحداث](/ar/agenteye/event-stream): شاهد هذه الأحداث تصل مباشرة، مرمزة بألوان وقابلة للتصفية حسب البيئة والوكيل والجلسة. -- [الجلسات](/ar/agenteye/sessions): شاهد كيف يعيد الأحداث المقترنة بناء كل تشغيل وكيل كرسم بياني للتنفيذ وخط زمني. \ No newline at end of file +- [تدفق الأحداث](/ar/cloud/event-stream): شاهد هذه الأحداث تصل مباشرة، مرمزة بألوان وقابلة للتصفية حسب البيئة والوكيل والجلسة. +- [الجلسات](/ar/cloud/sessions): شاهد كيف يعيد الأحداث المقترنة بناء كل تشغيل وكيل كرسم بياني للتنفيذ وخط زمني. \ No newline at end of file diff --git a/docs/ar/cloud/security.mdx b/docs/ar/cloud/security.mdx new file mode 100644 index 00000000..ca1c7629 --- /dev/null +++ b/docs/ar/cloud/security.mdx @@ -0,0 +1,68 @@ +--- +title: "الأمان" +description: "تم بناء FailproofAI Cloud للعمل بالقرب من وكلائك الإنتاجيين، مما يعني أنها ترى موجهاتك ومدخلات الأدوات والمخرجات." +--- + + +تم بناء FailproofAI Cloud للعمل بالقرب من وكلائك الإنتاجيين، مما يعني أنها ترى موجهاتك ومدخلات الأدوات والمخرجات. توضح هذه الصفحة كيفية الحفاظ على عزل هذه البيانات والتحكم فيها وإبقاؤها في يديك. إذا كنت تقيّم FailproofAI Cloud لمراجعة أمان، فابدأ من هنا. + +--- + +## بيانات تبقى في بيئتك + +FailproofAI Cloud مستضافة ذاتياً. يتم تخزين الأحداث والموجهات واستجابات النموذج والتحليلات في قواعد بيانات خاصة بك، في بيئتك الخاصة. لا يتم إرسال أي شيء إلى طرف ثالث SaaS للتخزين، وتبقى بيانات عملك في حساب السحابة الخاص بك. + +--- + +## عزل المستأجرين + +يمكن لمثيل واحد من FailproofAI Cloud استضافة عدة منظمات، وكل منها معزولة على مستوى التخزين — مفروض من قبل قاعدة البيانات وليس من الواجهة فقط: + +- بيانات المنظمة التشغيلية (المستخدمون والمفاتيح لوحات التحكم والاستعلامات المحفوظة) يتم تحديد نطاقها لتلك المنظمة، وتحظر قاعدة البيانات نفسها القراءات عبر المنظمات. +- كل حدث مُدرج موسوم بمنظمته المالكة، لذا لا يمكن أبداً قراءة أحداث منظمة واحدة من قبل منظمة أخرى. + +كل مسار لوحة تحكم يتم تحديد نطاقه تحت شعار منظمة (`//…`). + +--- + +## تسجيل الدخول + +تستخدم FailproofAI Cloud تسجيل دخول بدون كلمة مرور قائم على البريد الإلكتروني. لا توجد كلمة مرور يمكن اختراقها أو تسريبها. يطلب المستخدم رمزاً لمرة واحدة (أو رابط سحر بنقرة واحدة)، والذي يُرسل إليه عبر البريد الإلكتروني وينتهي صلاحيته بسرعة. يتم حماية تسجيل الدخول بواسطة **قائمة بيضاء**: فقط عناوين البريد الإلكتروني (أو النطاقات) التي تسمح بها يمكنها المصادقة. + +![شاشة تسجيل دخول FailproofAI Cloud، التي ترسل رمزاً لمرة واحدة إلى بريدك الإلكتروني](/cloud/images/login.png) + +--- + +## الوصول المحدود باستخدام مفاتيح API + +يقوم كل عميل بالمصادقة باستخدام مفتاح API يحمل أذونات دقيقة وذات امتيازات محدودة. يحتاج المجمِّع فقط إلى `events:add`؛ يمكن أن يكون مفتاح لوحة التحكم أو المساعد بقراءة فقط؛ الإجراءات الضارة (الحذف وإعادة التوليد) هي منح منفصلة تختار تضمينها. + +![صفحة مفاتيح API: منحات أذونات كل مفتاح، مرمّزة بألوان حسب نطاق القراءة والكتابة والتدمير](/cloud/images/api-keys.png) + +احتفظ بمفتاح bootstrap الإداري للإعداد، واستخدم مفاتيح محدودة لكل شيء آخر. انظر [مفاتيح API](/ar/cloud/access). + +--- + +## مساعد بقراءة فقط وموافقة مبوابة + +يجيب [المساعد في لوحة التحكم](/ar/cloud/assistant) على أسئلة حول بيانات عملك، لكنه مقيد بالتصميم: + +- أنه **بقراءة فقط افتراضياً**: SQL الخاص به يمر عبر حراس يسمح فقط باستعلامات `SELECT`/`WITH`، بيان واحد، مع حد أقصى للصفوف. +- أي شيء ينشئه (استعلام محفوظ، لوحة تحكم) هو **موافقة مبوابة**: تراجع وتوافق على كل عملية كتابة قبل حدوثها. +- أنه **لا يمكنه أبداً الحذف**. + +لذا يمكن لزميل في الفريق أن يسأل "أي وكلاء أخطؤوا أكثر هذا الأسبوع؟" والتصرف بناءً على الإجابة، دون أن يتمكن المساعد من تغيير أو إزالة بيانات عملك بمفرده. + +--- + +## في النقل + +كل حركة المرور تعمل عبر HTTPS. تقوم بإنهاء TLS باستخدام شهاداتك الخاصة، لذلك يتم تشفير حركة المرور من المجمِّع إلى الخادم ومن المتصفح إلى الخادم أثناء النقل. + +--- + +## الخطوات التالية + +- [نظرة عامة](/ar/cloud/overview): كيف تتناسب FailproofAI Cloud معاً. +- [مفاتيح API](/ar/cloud/access): تحديد نطاق الوصول للمجمِّع ولوحة التحكم والمساعد. +- [القابلية للملاحظة](/ar/cloud/overview): ما تلتقطه FailproofAI Cloud من وكلائك. \ No newline at end of file diff --git a/docs/ar/agenteye/sessions.mdx b/docs/ar/cloud/sessions.mdx similarity index 80% rename from docs/ar/agenteye/sessions.mdx rename to docs/ar/cloud/sessions.mdx index 48d6bfb0..3f72b2de 100644 --- a/docs/ar/agenteye/sessions.mdx +++ b/docs/ar/cloud/sessions.mdx @@ -7,7 +7,7 @@ description: "كل حدث من تشغيل، مجموع في صف واحد قاب توقف عن التخمين حول سبب فشل التشغيل. تجميع بيانات Failproof AI كل حدث من تشغيل في صف واحد قابل للقراءة، ثم يرسم التشغيل بالكامل كصورة بنمط git يمكنك قراءتها في ثوان، حتى تشاهد بالضبط ما فعله وكيلك، خطوة تلو الأخرى. -![قائمة الجلسات: صف واحد لكل تشغيل، عبر البيئات والوكلاء، مع شارات الحالة وشارات درجات التقييم](/agenteye/images/sessions-list.png) +![قائمة الجلسات: صف واحد لكل تشغيل، عبر البيئات والوكلاء، مع شارات الحالة وشارات درجات التقييم](/cloud/images/sessions-list.png) *صف واحد لكل تشغيل: شارة الحالة تخبرك كيف انتهى التشغيل للوهلة الأولى، وشارة درجة تظهر بجانبه بمجرد توصيل محيّم.* @@ -31,7 +31,7 @@ description: "كل حدث من تشغيل، مجموع في صف واحد قاب ## اقرأ التشغيل بالكامل كصورة -![رسم البياني للتنفيذ بنمط git بجانب الجدول الزمني للأحداث، مع لوحة تفصيل الأداة والنموذج والـ hook](/agenteye/images/session-detail.png) +![رسم البياني للتنفيذ بنمط git بجانب الجدول الزمني للأحداث، مع لوحة تفصيل الأداة والنموذج والـ hook](/cloud/images/session-detail.png) *رسم البياني للتنفيذ (اليسار) يجلس بجانب الجدول الزمني للأحداث؛ الشريط الأيمن يفصل الأدوات والنماذج والـ hooks وإنفاق الرموز للتشغيل.* @@ -39,7 +39,7 @@ description: "كل حدث من تشغيل، مجموع في صف واحد قاب يعطيك الشريط الأيمن التفصيل لكل تشغيل: أي أدوات ونماذج تم تشغيلها، أي hooks أُطلق، وما أنفقه التشغيل في الرموز. هذا هو الجواب على "لماذا كلف هذا التشغيل الكثير؟" أو "أي أداة هي البطيئة؟" يجلس بجانب الرسم البياني الذي سببه. -الأحداث الفردية قابلة للعنونة، بحيث يمكنك إعطاء شخص ما رابطاً إلى لحظة واحدة بدلاً من "الجلسة، حوالي ثلثي الطريق لأسفل". انسخ الرابط من أي حدث، أو اتبع واحداً من نتيجة [audit](/ar/agenteye/audits) أو خطأ، وستفتح الجلسة مع تحديد هذا الحدث والتمرير إليه. هذا ينطبق على التشغيلات الطويلة جداً أيضاً: الجدول الزمني يحمّل نافذة محدودة من أجل متصفحك، والرابط الذي يشير إلى ما وراء تلك النافذة يجد حدثه بدلاً من إسقاطك في البداية. إذا كان الحدث قد تقادم خارج نافذة الاحتفاظ بك، تخبرك الصفحة بذلك بدلاً من اختيار أي شيء بصمت. +الأحداث الفردية قابلة للعنونة، بحيث يمكنك إعطاء شخص ما رابطاً إلى لحظة واحدة بدلاً من "الجلسة، حوالي ثلثي الطريق لأسفل". انسخ الرابط من أي حدث، أو اتبع واحداً من نتيجة [audit](/ar/cloud/audits) أو خطأ، وستفتح الجلسة مع تحديد هذا الحدث والتمرير إليه. هذا ينطبق على التشغيلات الطويلة جداً أيضاً: الجدول الزمني يحمّل نافذة محدودة من أجل متصفحك، والرابط الذي يشير إلى ما وراء تلك النافذة يجد حدثه بدلاً من إسقاطك في البداية. إذا كان الحدث قد تقادم خارج نافذة الاحتفاظ بك، تخبرك الصفحة بذلك بدلاً من اختيار أي شيء بصمت. --- @@ -47,12 +47,12 @@ description: "كل حدث من تشغيل، مجموع في صف واحد قاب كل صفحة لوحة معلومات مرتبطة بمنظمتك (`//…`). الجلسات تعيش تحت **Observe** في الشريط الجانبي الأيسر، بجانب الأحداث، مع فلاتر نطاق التاريخ والبيئة والوكيل والجلسة عبر أعلى القائمة. كل صف هو نقرة واحدة من رسم البياني الكامل للتنفيذ. -لتشغيل شارات الدرجات وتصفية نطاق الدرجات، اتصل بمحيّم: انظر [Evaluations](/ar/agenteye/evaluations). +لتشغيل شارات الدرجات وتصفية نطاق الدرجات، اتصل بمحيّم: انظر [Evaluations](/ar/cloud/evaluations). --- ## ذات صلة -- [تدفق الأحداث](/ar/agenteye/event-stream): مسار كل خطوة الخام الذي يتم تجميع كل جلسة منه. -- [التقييمات](/ar/agenteye/evaluations): اتصل بمحيّم حتى يحصل كل تشغيل على شارة درجة يمكنك التصفية بها. -- [التلمترة](/ar/agenteye/telemetry): كيف ينتقل التشغيل من وكيلك إلى هذه الجلسات. \ No newline at end of file +- [تدفق الأحداث](/ar/cloud/event-stream): مسار كل خطوة الخام الذي يتم تجميع كل جلسة منه. +- [التقييمات](/ar/cloud/evaluations): اتصل بمحيّم حتى يحصل كل تشغيل على شارة درجة يمكنك التصفية بها. +- [التلمترة](/ar/cloud/performance): كيف ينتقل التشغيل من وكيلك إلى هذه الجلسات. \ No newline at end of file diff --git a/docs/ar/concepts.mdx b/docs/ar/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/ar/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/ar/daemon.mdx b/docs/ar/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/ar/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/ar/dashboard.mdx b/docs/ar/dashboard.mdx index a4d2e739..6a3aeb4f 100644 --- a/docs/ar/dashboard.mdx +++ b/docs/ar/dashboard.mdx @@ -70,7 +70,7 @@ failproofai 4. **كيفية التحسن** — قائمة صفوف هادئة، واحدة لكل سياسة موصى بها: اسم السياسة بالأبيض، وصف سطر واحد، أمر التثبيت + زر نسخ على الجانب الأيمن. يقرأ رأس القسم `enable all N → projected · ` (الدرجة التي ستصل إليها مع تطبيق كل إصلاح)، وزر `[install all]` الخاص به ينسخ الأمر المدمج `failproofai policy add a b c …` لكل سياسة موصى بها. 5. **العودة أفضل** — بطاقتان جنبًا إلى جنب. اليسار: ضع تذكيرًا (منتقي الإيقاع `3d` / `7d` / `14d` / `30d`; يستمر عبر `/api/auth/reminder` بمجرد المصادقة). اليمين: فتح امتيازات failproof — `invite a friend` يفتح نافذة تأخذ قائمة بريد إلكتروني للأصدقاء مفصولة بفواصل/مسافات/سطور جديدة (10 كحد أقصى لكل إرسال)، POST لهم إلى `/api/audit/invite`، الذي يحول إلى `POST /v0/invite` لخادم api. يرسل خادم api بريد إلكترونيًا واحدًا لكل مستقبل من `invite@failproof.ai` مع نسخة المرسل وتعيين `Reply-To`، بحيث يرى المستقبل من دعاهم والمرسل يحصل على نسخة في علبة الوارد الخاصة به. يتم توجيه المستخدمين المجهولين عبر `AuthDialog` أولاً بحيث يتم معرفة بريد المرسل قبل الدعوات. يعتبر الاستحقاق / تحقيق الامتيازات متابعة. -مدفوع بوقت تشغيل `failproofai audit` — انظر [Audit CLI](/ar/cli/audit) لمحرك المسح الأساسي والأعلام المدعومة وثوابت التخزين المؤقت لكل نسخة. تخزن لوحة التحكم النتيجة الأخيرة في `~/.failproofai/audit-dashboard.json` (الوضع `0600`، فتحة واحدة، تخزين جديد يكتب فوق) بحيث تكون الزيارات الثانية فورية؛ **يتم رفض كل من التخزين المؤقت لكل نسخة ونتيجة كاملة عند القراءة بمجرد أن تصبح أقدم من 7 أيام** بحيث لا تخدم لوحة التحكم بصمت نتيجة بعمر أسبوع — بعد انتهاء الصلاحية `/audit` يسقط إلى حالته الفارغة ويطالب بتشغيل جديد. انقر على `[ re-audit now ]` بالقرب من أسفل التقرير POST `/api/audit/run` مع `noCache: true` — إعادة التدقيق تتجاوز التخزين المؤقت لكل نسخة وتعيد مسح كل نسخة من الصفر بدلاً من صامتة إرجاع النتيجة المخزنة مؤقتًا — وتستطلع لوحة التحكم `/api/audit/status` بـ 1Hz حتى ينتهي التشغيل؛ ينقر شريط تقدم وردي لاصق إلى أعلى منفذ العرض أثناء التشغيل مع موقت انقضاء، والنتيجة الطازجة تدخل في مكانها عند النجاح (لا إعادة تحميل كامل الصفحة؛ تترك إعادة التدقيق الفاشلة التقرير السابق سليمًا). عند الفشل يتحول الشريط إلى الأحمر مع نسخ مفتاح قبالة `RerunError.kind` (`timeout` / `network` / `post_failed`). يتم سطح الحالة الفارغة (لا يوجد تخزين مؤقت أو منتهي الصلاحية) وحالة الصفر جلسات (يوجد التخزين المؤقت لكن المسح لم يجد نسخ) بشكل منفصل. +مدفوع بوقت تشغيل `failproofai audit` — انظر [Audit CLI](/ar/audit) لمحرك المسح الأساسي والأعلام المدعومة وثوابت التخزين المؤقت لكل نسخة. تخزن لوحة التحكم النتيجة الأخيرة في `~/.failproofai/audit-dashboard.json` (الوضع `0600`، فتحة واحدة، تخزين جديد يكتب فوق) بحيث تكون الزيارات الثانية فورية؛ **يتم رفض كل من التخزين المؤقت لكل نسخة ونتيجة كاملة عند القراءة بمجرد أن تصبح أقدم من 7 أيام** بحيث لا تخدم لوحة التحكم بصمت نتيجة بعمر أسبوع — بعد انتهاء الصلاحية `/audit` يسقط إلى حالته الفارغة ويطالب بتشغيل جديد. انقر على `[ re-audit now ]` بالقرب من أسفل التقرير POST `/api/audit/run` مع `noCache: true` — إعادة التدقيق تتجاوز التخزين المؤقت لكل نسخة وتعيد مسح كل نسخة من الصفر بدلاً من صامتة إرجاع النتيجة المخزنة مؤقتًا — وتستطلع لوحة التحكم `/api/audit/status` بـ 1Hz حتى ينتهي التشغيل؛ ينقر شريط تقدم وردي لاصق إلى أعلى منفذ العرض أثناء التشغيل مع موقت انقضاء، والنتيجة الطازجة تدخل في مكانها عند النجاح (لا إعادة تحميل كامل الصفحة؛ تترك إعادة التدقيق الفاشلة التقرير السابق سليمًا). عند الفشل يتحول الشريط إلى الأحمر مع نسخ مفتاح قبالة `RerunError.kind` (`timeout` / `network` / `post_failed`). يتم سطح الحالة الفارغة (لا يوجد تخزين مؤقت أو منتهي الصلاحية) وحالة الصفر جلسات (يوجد التخزين المؤقت لكن المسح لم يجد نسخ) بشكل منفصل. ### السياسات diff --git a/docs/ar/architecture.mdx b/docs/ar/how-it-works.mdx similarity index 100% rename from docs/ar/architecture.mdx rename to docs/ar/how-it-works.mdx diff --git a/docs/ar/introduction.mdx b/docs/ar/introduction.mdx index 96ecc271..3a816b1a 100644 --- a/docs/ar/introduction.mdx +++ b/docs/ar/introduction.mdx @@ -55,4 +55,4 @@ failproofai policies --install # فعّل السياسات (أو تخطَّ failproofai # شغّل لوحة التحكم ``` -انظر إلى دليل [ابدأ الآن](/ar/getting-started) للحصول على المسار الكامل. \ No newline at end of file +انظر إلى دليل [ابدأ الآن](/ar/quickstart) للحصول على المسار الكامل. \ No newline at end of file diff --git a/docs/ar/policies.mdx b/docs/ar/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/ar/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/ar/getting-started.mdx b/docs/ar/quickstart.mdx similarity index 100% rename from docs/ar/getting-started.mdx rename to docs/ar/quickstart.mdx diff --git a/docs/ar/reference/files.mdx b/docs/ar/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/ar/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/architecture.mdx b/docs/architecture.mdx deleted file mode 100644 index cf636389..00000000 --- a/docs/architecture.mdx +++ /dev/null @@ -1,332 +0,0 @@ ---- -title: Architecture -description: "How the hook handler, config loading, and policy evaluation work internally" -icon: sitemap ---- - -This document explains how failproofai works internally: how the hook system intercepts agent tool calls, how configuration is loaded and merged, how policies are evaluated, and how the dashboard monitors agent activity. - ---- - -## Overview - -failproofai has two independent subsystems: - -1. **Hook handler** - A fast CLI subprocess that Claude Code invokes on every agent tool call. Evaluates policies and returns a decision. -2. **Agent Monitor (Dashboard)** - A Next.js web application for monitoring agent sessions and managing policies. - -Both subsystems share configuration files in `~/.failproofai/` and the project's `.failproofai/` directory, but they run as separate processes and communicate only through the filesystem. - ---- - -## Hook handler - -### Integration with Claude Code - -When you run `failproofai policies --install`, it writes entries like this into `~/.claude/settings.json`: - -```json -{ - "hooks": { - "PreToolUse": [ - { - "matcher": "", - "hooks": [ - { - "type": "command", - "command": "failproofai --hook PreToolUse" - } - ] - } - ], - "PostToolUse": [ ... ] - } -} -``` - -Claude Code then invokes `failproofai --hook PreToolUse` as a subprocess before each tool call, passing a JSON payload on stdin. - -### Payload format - -```json -{ - "session_id": "abc123", - "transcript_path": "/home/user/.claude/projects/myproject/sessions/abc123.jsonl", - "cwd": "/home/user/myproject", - "permission_mode": "default", - "hook_event_name": "PreToolUse", - "tool_name": "Bash", - "tool_input": { "command": "sudo apt install nodejs" } -} -``` - -For `PostToolUse` events, the payload also contains `tool_result` with the tool's output. - -The handler enforces a 1 MB stdin limit. Payloads exceeding this are discarded and all policies implicitly allow. - -### Response format - -**Deny (PreToolUse):** -```json -{ - "hookSpecificOutput": { - "permissionDecision": "deny", - "permissionDecisionReason": "Blocked by failproofai: sudo command blocked" - } -} -``` - -**Deny (PostToolUse):** -```json -{ - "hookSpecificOutput": { - "additionalContext": "Blocked by failproofai because: API key detected in output" - } -} -``` - -**Instruct (any event except Stop):** -```json -{ - "hookSpecificOutput": { - "additionalContext": "Instruction from failproofai: Verify tests pass before committing." - } -} -``` - -**Stop event instruct:** -- Exit code: `2` -- Reason written to stderr (not stdout) - -**Allow:** -- Exit code: `0` -- Empty stdout - -**Allow with message:** - -`allow(message)` lets a policy send informational context back to Claude even when the operation is permitted. The hook handler writes the following JSON to **stdout** (not a config file — this is the handler's response to Claude Code, just like deny and instruct responses above): - -```json -// Written to stdout by the hook handler process -{ - "hookSpecificOutput": { - "additionalContext": "All CI checks passed on branch 'feat/my-feature'." - } -} -``` -- Exit code: `0` (operation is allowed) -- When multiple policies return `allow` with a message, their messages are joined with newlines into a single `additionalContext` string -- If no policy provides a message, stdout is empty (same as before) - -### Processing pipeline - -`src/hooks/handler.ts` implements the full pipeline: - -```text -stdin JSON - → parse payload (max 1 MB) - → extract session metadata (session_id, cwd, tool_name, tool_input, etc.) - → readMergedHooksConfig(cwd) ← merges project + local + global config - → register enabled builtin policies with resolved params - → load custom policies from customPoliciesPath (if set) - → register custom policies into policy registry - → evaluate all policies (builtins first, then custom) - → first deny short-circuits - → instruct decisions accumulate - → allow messages accumulate - → write JSON decision to stdout - → persist event to ~/.failproofai/hook-activity/current.jsonl - → exit -``` - -The entire process runs in under 100ms for typical payloads with no LLM calls. - ---- - -## Configuration loading - -`src/hooks/hooks-config.ts` implements three-scope config loading. - -```text -[1] {cwd}/.failproofai/policies-config.json ← project (highest priority) -[2] {cwd}/.failproofai/policies-config.local.json ← local -[3] ~/.failproofai/policies-config.json ← global (lowest priority) -``` - -Merge logic: -- `enabledPolicies` - deduplicated union across all three files -- `policyParams` - per-policy key, first file that defines it wins entirely -- `customPoliciesPath` - first file that defines it wins -- `llm` - first file that defines it wins - -The web dashboard uses `readHooksConfig()` (global only) for reading and writing, since it is not invoked with a project cwd. - ---- - -## Policy evaluation - -`src/hooks/policy-evaluator.ts` runs policies in order. - -For each policy: - -1. Look up the policy's `params` schema (if it has one). -2. Read `policyParams[policy.name]` from the merged config. -3. Merge user-provided values over schema defaults to produce `ctx.params`. -4. Call `policy.fn(ctx)` with the resolved context. -5. If the result is `deny`, stop immediately and return that decision. -6. If the result is `instruct`, accumulate the message and continue. -7. If the result is `allow`, continue to the next policy. - -After all policies run: -- If any `deny` was returned, emit the deny response. -- If any `instruct` returns were collected, emit a single instruct response with all messages joined. -- Otherwise, emit an allow response (empty stdout, exit 0). - ---- - -## Builtin policies - -`src/hooks/builtin-policies.ts` defines all 39 built-in policies as `BuiltinPolicyDefinition` objects: - -```typescript -interface BuiltinPolicyDefinition { - name: string; - description: string; - fn: (ctx: PolicyContext) => PolicyResult; - match: { - events: HookEventType[]; - tools?: string[]; - }; - defaultEnabled: boolean; - category: string; - beta?: boolean; - params?: PolicyParamsSchema; -} -``` - -Policies that accept `params` declare a `PolicyParamsSchema` with types and defaults for each parameter. The policy evaluator injects resolved values into `ctx.params` before calling `fn`. Policy functions read `ctx.params` without null-guarding because defaults are always applied first. - -Pattern matching inside policies uses parsed command tokens (argv), not raw string matching. This prevents bypass via shell operator injection (e.g. a pattern for `sudo systemctl status *` cannot be bypassed by appending `; rm -rf /` to the command). - ---- - -## Custom policies - -`src/hooks/custom-hooks-registry.ts` implements a `globalThis`-backed registry: - -```typescript -const REGISTRY_KEY = "__failproofai_custom_hooks__"; - -export const customPolicies = { - add(hook: CustomHook): void { ... } -}; - -export function getCustomHooks(): CustomHook[] { ... } -export function clearCustomHooks(): void { ... } // used in tests -``` - -`src/hooks/custom-hooks-loader.ts` loads the user's policy file: - -1. Read `customPoliciesPath` from config; skip if absent. -2. Resolve to absolute path; check file exists. -3. Rewrite all `from "failproofai"` imports to the actual dist path so `customPolicies` resolves to the same `globalThis` registry. -4. Recursively rewrite transitive local imports to ensure ESM compatibility. -5. Write temporary `.mjs` files and `import()` the entry file. -6. Call `getCustomHooks()` to retrieve registered hooks. -7. Clean up all temp files in a `finally` block. - -On any error (file not found, syntax error, import failure), the error is logged to `~/.failproofai/hook.log` and the loader returns an empty array. Built-in policies are unaffected. - -Custom policies are evaluated after all built-in policies. A custom policy `deny` still short-circuits further custom policies (but all built-ins have already run by that point). - ---- - -## Activity logging - -After each hook event, the handler appends a JSONL line to `~/.failproofai/hook-activity/current.jsonl`, which rotates into `page--.jsonl` once it reaches a page: - -```json -{ - "timestamp": "2026-04-06T12:34:56.789Z", - "sessionId": "abc123", - "eventType": "PreToolUse", - "toolName": "Bash", - "policyName": "block-sudo", - "decision": "deny", - "reason": "sudo command blocked by failproofai", - "durationMs": 12 -} -``` - -One line per policy that made a non-allow decision. Allow decisions are not logged (to keep the file small). - ---- - -## Dashboard architecture - -The dashboard is a **Next.js 16** application using the App Router with React Server Components and Server Actions. - -```text -app/ - layout.tsx ← Root layout (theme, telemetry, nav) - projects/page.tsx ← Server component: list all Claude projects - project/[name]/page.tsx ← Server component: list sessions in a project - project/[name]/session/ - [sessionId]/page.tsx ← Server component: render session viewer - policies/page.tsx ← Client component: policy management + activity log - actions/ - get-hooks-config.ts ← Read config + policy list - update-hooks-config.ts ← Toggle policy on/off - update-policy-params.ts ← Update policy parameters - get-hook-activity.ts ← Paginate/search activity log - install-hooks-web.ts ← Install/remove hooks from the browser - api/ - download/[project]/[session]/route.ts ← Per-CLI session export (JSONL or JSON) -``` - -**Data flow:** - -- Page components call `lib/projects.ts` and `lib/log-entries.ts` to read project/session data directly from the filesystem (no API layer for reads). -- The Policies page uses Server Actions for all mutations (toggle, params update, install/remove). -- The session viewer parses Claude's JSONL transcript format and renders a timeline of messages and tool calls. - -**Key design decisions:** - -- No database - all persistent state is in plain files (`~/.failproofai/`, `~/.claude/projects/`). -- Server Actions for mutations - no REST API needed for CRUD operations. -- React Server Components for read pages - faster initial load, no client bundle for data fetching. -- Client components only where interactivity is needed (policy toggles, activity search, log viewer). - ---- - -## File layout - -```text -failproofai/ -├── bin/ -│ └── failproofai.mjs # CLI router (hook / dashboard / install / etc.) -├── src/hooks/ -│ ├── handler.ts # Hook event pipeline -│ ├── builtin-policies.ts # 39 policy definitions -│ ├── policy-evaluator.ts # Policy execution engine -│ ├── policy-registry.ts # Policy registration and lookup -│ ├── policy-types.ts # TypeScript interfaces -│ ├── hooks-config.ts # Multi-scope config loading -│ ├── custom-hooks-registry.ts # globalThis-backed hook registry -│ ├── custom-hooks-loader.ts # ESM loader for user JS hooks -│ ├── manager.ts # install / remove / list operations -│ ├── install-prompt.ts # Interactive policy selection prompt -│ ├── hook-logger.ts # Logging to hook.log -│ ├── hook-activity-store.ts # Persist activity to hook-activity/ -│ └── llm-client.ts # LLM API client (for AI-powered policies) -├── app/ # Next.js dashboard (pages + server actions) -├── lib/ # Shared utilities -│ ├── projects.ts # Enumerate Claude projects from filesystem -│ ├── log-entries.ts # Parse Claude transcript JSONL format -│ ├── paths.ts # Resolve system paths -│ └── ... -├── components/ # Shared React UI components -├── contexts/ # React context providers (theme, auto-refresh, telemetry) -├── examples/ # Example custom hook files -└── __tests__/ # Unit and E2E tests -``` diff --git a/docs/cli/audit.mdx b/docs/audit.mdx similarity index 99% rename from docs/cli/audit.mdx rename to docs/audit.mdx index dce3f35d..92be05d4 100644 --- a/docs/cli/audit.mdx +++ b/docs/audit.mdx @@ -1,5 +1,5 @@ --- -title: Audit past sessions (beta) +title: "Audit your agents" description: "Count how often the agent did wasteful or risky things across past transcripts" --- diff --git a/docs/built-in-policies.mdx b/docs/built-in-policies.mdx index 29ac4f0a..9682f554 100644 --- a/docs/built-in-policies.mdx +++ b/docs/built-in-policies.mdx @@ -1,10 +1,17 @@ --- -title: Built-in Policies +title: "Built-in policies" description: "All 39 built-in policies that catch common agent failure modes" icon: shield --- -failproofai ships with 39 built-in policies that catch common agent failure modes. Each policy fires on a specific hook event type and tool name. Nineteen policies accept parameters that let you tune their behavior without writing code. Five workflow policies enforce a commit → push → PR → CI pipeline before Claude stops. +FailproofAI ships with 39 built-in policies that catch common agent failure modes. Each fires on a specific hook event and tool. Nineteen accept parameters, so you can tune them without writing code. Five workflow policies enforce a commit → push → PR → CI pipeline before the agent is allowed to finish. + +They work identically across all [12 supported agent CLIs](/agent-support) — event names, tool names, and tool inputs are normalized before any policy runs. Examples on this page say "Claude" where they quote a real message, but nothing here is Claude-specific. + + + Not sure which to turn on? `failproofai config` offers **Recommended** — 16 policies + chosen to cover the failures you cannot undo. See [Policies](/policies#turning-policies-on). + --- @@ -25,6 +32,11 @@ Policies are grouped into categories: | [Package managers](#package-managers) | prefer-package-manager | PreToolUse | | [Workflow](#workflow) | require-commit-before-stop, require-push-before-stop, require-pr-before-stop, require-no-conflicts-before-stop, require-ci-green-before-stop | Stop | + + The five **Workflow** policies need a turn-end gate, and two CLIs do not have one — + they never fire on Hermes or Goose. [Which CLI supports what →](/agent-support#what-can-actually-be-blocked-per-cli) + + - **`block-`** — stop the agent from proceeding. - **`warn-`** — give the agent additional context so it can self-correct. - **`sanitize-`** — scrub sensitive data from tool output before the agent sees it. diff --git a/docs/cli/backfill.mdx b/docs/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/cli/config.mdx b/docs/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/cli/dashboard.mdx b/docs/cli/dashboard.mdx index b458aa5c..9d7fc022 100644 --- a/docs/cli/dashboard.mdx +++ b/docs/cli/dashboard.mdx @@ -1,5 +1,5 @@ --- -title: View sessions +title: "failproofai (dashboard)" description: "Launch the dashboard to browse agent sessions and manage policies" --- diff --git a/docs/cli/flush.mdx b/docs/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/cli/harness.mdx b/docs/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/cli/hook.mdx b/docs/cli/hook.mdx index effc7efb..f1c07b5d 100644 --- a/docs/cli/hook.mdx +++ b/docs/cli/hook.mdx @@ -1,5 +1,5 @@ --- -title: Hook handler (internal) +title: "failproofai --hook" description: "The subprocess Claude Code calls on each tool event" --- diff --git a/docs/cli/install-policies.mdx b/docs/cli/install-policies.mdx index 00c23d3e..a0a1e024 100644 --- a/docs/cli/install-policies.mdx +++ b/docs/cli/install-policies.mdx @@ -1,5 +1,5 @@ --- -title: Install policies +title: "failproofai policies --install" description: "Enable policies so they run on every agent tool call" --- diff --git a/docs/cli/list-policies.mdx b/docs/cli/list-policies.mdx index da66a802..3b97dfc5 100644 --- a/docs/cli/list-policies.mdx +++ b/docs/cli/list-policies.mdx @@ -1,5 +1,5 @@ --- -title: List policies +title: "failproofai policies" description: "See which policies are enabled, their parameters, and custom policies" --- diff --git a/docs/cli/migrate.mdx b/docs/cli/migrate.mdx index fbf6435f..490ccf18 100644 --- a/docs/cli/migrate.mdx +++ b/docs/cli/migrate.mdx @@ -1,5 +1,5 @@ --- -title: Migrate the home directory +title: "failproofai migrate" description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" --- diff --git a/docs/cli/remove-policies.mdx b/docs/cli/remove-policies.mdx index 08f9f968..13817168 100644 --- a/docs/cli/remove-policies.mdx +++ b/docs/cli/remove-policies.mdx @@ -1,5 +1,5 @@ --- -title: Uninstall policies +title: "failproofai policies --uninstall" description: "Remove hook entries from Claude Code's settings" --- diff --git a/docs/cli/uninstall.mdx b/docs/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/cli/update.mdx b/docs/cli/update.mdx index 8d28ab47..7489a22c 100644 --- a/docs/cli/update.mdx +++ b/docs/cli/update.mdx @@ -1,5 +1,5 @@ --- -title: Update after an upgrade +title: "failproofai update" description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" --- diff --git a/docs/cli/version.mdx b/docs/cli/version.mdx index 5ce4c415..3d601112 100644 --- a/docs/cli/version.mdx +++ b/docs/cli/version.mdx @@ -1,5 +1,5 @@ --- -title: Check version +title: "failproofai --version" description: "Print the installed failproofai version" --- diff --git a/docs/agenteye/api-keys.mdx b/docs/cloud/access.mdx similarity index 89% rename from docs/agenteye/api-keys.mdx rename to docs/cloud/access.mdx index 55e1ca45..ed5d2e54 100644 --- a/docs/agenteye/api-keys.mdx +++ b/docs/cloud/access.mdx @@ -1,20 +1,38 @@ --- -title: "API Keys" -description: "API keys control who and what can reach your Failproof AI Observability server, so a collector can send events without ever gaining read or admin powers." +title: "Access and permissions" +description: "Scoped API keys, permission sets, and users — so a machine can report activity without ever gaining read or admin powers." +icon: key --- +Every client that reaches FailproofAI Cloud authenticates with a key that carries explicit +permissions, and each permission gates specific routes. You grant only the few a job needs. -API keys control who and what can reach your Failproof AI Observability server, so a collector can send events without ever gaining read or admin powers. Each key carries one or more permissions, and each permission gates specific server routes; you grant only the few a job needs. Most deployments create just three kinds of key. +Most teams create three kinds of key and never think about the catalogue below. -## The 3 keys most deployments need +## The three keys most teams need | Key | Permissions | Who uses it | |---|---|---| -| Collector key | `events:add` | The `agenteye-collector` on each agent machine, to send events. | -| Dashboard read key | `events:read`, `keys:read` | A read-only operator or integration that queries data without changing it. | -| Bootstrap admin key | all permissions | The operator who first brings the instance up (and the dashboard). Seeded from the `ADMIN_KEY` environment variable. See [Bootstrap admin key](#bootstrap-admin-key). | - -Start here. Reach for the full permission catalogue below only when you need a narrower, custom-scoped key. See also [Recommended key layout](#recommended-key-layout) and [Creating keys](#creating-keys). +| **Machine key** | `events:add`, `policies:pull` | Each machine you [connect](/cloud/connect). Reports what its agents did, and receives centrally-managed policy. Nothing else. | +| **Read-only key** | `events:read`, `evaluations:read` | A dashboard, a script, or an integration that queries data without changing it. | +| **Admin key** | all permissions | Bringing the deployment up and provisioning everything else. | + + + Split the machine key when the two halves belong to different trust levels. A key with + `events:add` alone reports activity but receives no policy; a key with `policies:pull` + alone receives policy but reports nothing. Both are supported states, and + `failproofai config --status` names which one a machine is in. + + + + **`policies:pull`** is the permission a machine needs to receive [managed + policies](/cloud/managed-policies). It is verified independently of `events:add` at + connect time, so a key carrying one and not the other fails with a precise reason instead + of half-working. + + +Reach for the full catalogue below only when you need something narrower. See also +[Recommended key layout](#recommended-key-layout) and [Creating keys](#creating-keys). --- @@ -82,7 +100,7 @@ The server enforces a fixed catalogue of permissions; each one gates specific HT These permissions back the dashboard's **Users** page, where each member's granted scopes are shown as chips: -![The Users page: a card per dashboard user with their email, granted permissions, and edit/disable controls](/agenteye/images/users.png) +![The Users page: a card per dashboard user with their email, granted permissions, and edit/disable controls](/cloud/images/users.png) ### Operational settings @@ -91,7 +109,7 @@ These permissions back the dashboard's **Users** page, where each member's grant | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | View dashboard-managed operational settings and their metadata; list per-model context-window overrides; and resolve the effective window for a model. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | Edit operational settings and add, change, or remove per-model context-window overrides. Changes affect new events without restarting the server. | -![The Settings page: dashboard-managed operational settings such as allowed sign-ins and session/OTP lifetimes, editable without a restart](/agenteye/images/settings.png) +![The Settings page: dashboard-managed operational settings such as allowed sign-ins and session/OTP lifetimes, editable without a restart](/cloud/images/settings.png) ### Alerts & incidents @@ -167,7 +185,7 @@ In a multi-org deployment, every key an org member creates (through this keys AP Use the admin key (or any key with `keys:create` permission) to create additional scoped keys. -### Collector key (ingest only) +### Machine key (report only) ```bash curl -s -X POST http://your-server/keys \ @@ -247,7 +265,7 @@ The **Keys** page in the dashboard provides a UI for all of the above operations When you create a key from the dashboard you do not supply the secret; the dashboard generates a strong secret for you and displays it **once** at creation. Copy it immediately and store it securely; it is never shown again, exactly as with a regenerate. You can still pick the key's permissions directly, or seed them from a permission set (see below). -![The API Keys page: a card per key showing its name, granted permissions, and creation time, with regenerate and disable actions; protected keys like `admin` are marked](/agenteye/images/api-keys.png) +![The API Keys page: a card per key showing its name, granted permissions, and creation time, with regenerate and disable actions; protected keys like `admin` are marked](/cloud/images/api-keys.png) --- @@ -256,7 +274,7 @@ When you create a key from the dashboard you do not supply the secret; the dashb | Key | Permissions | Used by | |---|---|---| | `admin` (bootstrap via `ADMIN_KEY` env var) | all | Ops/setup, and the dashboard (authenticates with `ADMIN_KEY`, proxies user requests with permission checks) | -| Per-host collector key | `events:add` | Collector on each agent machine | +| Per-host machine key | `events:add`, `policies:pull` | Each [connected machine](/cloud/connect) | | `dashboard-assistant` (bootstrap via `AGENT_API_KEY` env var) | `events:read`, `evaluations:read`, `dashboards:read`, `dashboards:write`, `queries:read`, `queries:write`, `queries:run` | AI assistant, seeded automatically, **protected**; can't be edited through the API | | Assistant telemetry key (optional) | `events:add` | AI assistant self-instrumentation, if enabled | @@ -276,5 +294,5 @@ You only need these if you are upgrading an existing instance; new deployments c ## Next steps -- [Python SDK](/agenteye/python-sdk): how your agent code authenticates when sending events. -- [Security](/agenteye/security): how sign-in, access control, and per-organization data isolation work. +- [Python SDK](/cloud/sdk): how your agent code authenticates when sending events. +- [Security](/cloud/security): how sign-in, access control, and per-organization data isolation work. diff --git a/docs/cloud/agent-skills.mdx b/docs/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/agenteye/alerts.mdx b/docs/cloud/alerts.mdx similarity index 74% rename from docs/agenteye/alerts.mdx rename to docs/cloud/alerts.mdx index 4c996a09..8f1e817f 100644 --- a/docs/agenteye/alerts.mdx +++ b/docs/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "Find out the moment something crosses your line, on the channel yo --- -Find out the moment something crosses your line, on the channel your team already watches, instead of hearing about it from a customer. Set a rule once and Failproof AI Observability checks it on a schedule, then pages you by email, Slack, webhook, or right in the dashboard. +Find out the moment something crosses your line, on the channel your team already watches, instead of hearing about it from a customer. Set a rule once and FailproofAI Cloud checks it on a schedule, then pages you by email, Slack, webhook, or right in the dashboard. -![The Alerts page: a grid of alert-rule cards, each showing its trigger, evaluation window, channels, and an info, warning, or critical severity badge](/agenteye/images/alerts.png) +![The Alerts page: a grid of alert-rule cards, each showing its trigger, evaluation window, channels, and an info, warning, or critical severity badge](/cloud/images/alerts.png) *Every alert rule at a glance: what it watches, how often, where it pages, and how urgent.* ## Hear about problems before your users do @@ -22,9 +22,9 @@ Attach any combination to a single rule, and its severity (info, warning, or cri ## Build the rule in a form, not JSON -You describe what "broken" means in a form, and Failproof AI Observability writes the underlying rule for you. The JSON spec is just what that form produces under the hood, so you can read it to understand a rule but you rarely type it. +You describe what "broken" means in a form, and FailproofAI Cloud writes the underlying rule for you. The JSON spec is just what that form produces under the hood, so you can read it to understand a rule but you rarely type it. -![The new-alert form: name and description, an enabled toggle, and a trigger picker offering metric threshold, custom SQL, evaluation score, compound eval, and per-event conditions](/agenteye/images/alert-new.png) +![The new-alert form: name and description, an enabled toggle, and a trigger picker offering metric threshold, custom SQL, evaluation score, compound eval, and per-event conditions](/cloud/images/alert-new.png) *Pick a trigger and the form swaps in the right fields; Save writes the rule.* The happy path is quick: name it, pick a **trigger** (what to watch), set the **threshold and window** (how bad, over how long), attach at least one **channel**, then **Save** and hit **Test** to fire a synthetic notification and confirm every destination is wired up. Under the hood that produces a small spec like: @@ -43,7 +43,7 @@ You are not limited to one kind of signal. Pick the trigger that matches how you | **Compound eval** | several score checks combine with any, all, or at-least-N logic, to catch a regression that only shows across scores | | **Per event** | a single matching event lands: a specific agent, a specific error type, or a message substring | -Already staring at a failure on the [Errors page](/agenteye/error-tracking)? Every row there has a **+ alert** button that opens this same form prefilled to catch that exact failure again, so the incident you just triaged becomes the one that pages you next time. +Already staring at a failure on the [Errors page](/cloud/errors)? Every row there has a **+ alert** button that opens this same form prefilled to catch that exact failure again, so the incident you just triaged becomes the one that pages you next time. **Where to find it:** Alerts live at `//alerts`. Creating, editing, deleting, and testing rules needs **`alerts:write`**; `alerts:read` is enough to look. The recipient picker lists your org's members by name, so you can page a person without leaving the form. @@ -53,11 +53,11 @@ One bad measurement should not wake you. The **M of N** noise filter controls ho ## What happens when an alert fires -A breach opens an **incident** and pages your channels once. From there your team acknowledges it, assigns an owner, talks it through, and resolves it, all against a clean, attributed record. That triage workflow has its own home: see [Incidents](/agenteye/incidents). +A breach opens an **incident** and pages your channels once. From there your team acknowledges it, assigns an owner, talks it through, and resolves it, all against a clean, attributed record. That triage workflow has its own home: see [Incidents](/cloud/incidents). ## Related -- [Incidents](/agenteye/incidents): track a firing alert from open to acknowledged to resolved. -- [Error tracking](/agenteye/error-tracking): group agent failures and promote one to an alert in a click. -- [Dashboards](/agenteye/dashboards): watch the shared boards the thresholds you alert on come from. -- [CLI and agents](/agenteye/cli-and-agents): create alerts and ack incidents from your terminal, or script them into CI. +- [Incidents](/cloud/incidents): track a firing alert from open to acknowledged to resolved. +- [Error tracking](/cloud/errors): group agent failures and promote one to an alert in a click. +- [Dashboards](/cloud/dashboards): watch the shared boards the thresholds you alert on come from. +- [CLI and agents](/cloud/cli): create alerts and ack incidents from your terminal, or script them into CI. diff --git a/docs/agenteye/assistant.mdx b/docs/cloud/assistant.mdx similarity index 83% rename from docs/agenteye/assistant.mdx rename to docs/cloud/assistant.mdx index 0ac3dbaf..1d87c07c 100644 --- a/docs/agenteye/assistant.mdx +++ b/docs/cloud/assistant.mdx @@ -1,12 +1,12 @@ --- -title: "AI Assistant" +title: "AI assistant" description: "Ask your agent data a question in plain English and get an answer that links straight to the evidence." --- -Ask your agent data a question in plain English and get an answer that links straight to the evidence. No SQL to write, no dashboards to dig through — the **Failproof AI Observability** assistant is the fastest way for anyone on your team to get answers about your agents. +Ask your agent data a question in plain English and get an answer that links straight to the evidence. No SQL to write, no dashboards to dig through — the **FailproofAI Cloud** assistant is the fastest way for anyone on your team to get answers about your agents. -![The Failproof AI Observability assistant answering a plain-English question inside the dashboard, showing a live Agent Activity table, a per-agent model-usage breakdown, and written takeaways, with the queries it ran shown inline](/agenteye/images/assistant.png) +![The FailproofAI Cloud assistant answering a plain-English question inside the dashboard, showing a live Agent Activity table, a per-agent model-usage breakdown, and written takeaways, with the queries it ran shown inline](/cloud/images/assistant.png) *Ask in plain English and get an answer built from your own data. Here it breaks down which agents are busiest and which models they use, and shows the queries it ran so you can verify every number.* There is nothing to learn. Open the chat, type what you want to know, and follow the links it hands back: @@ -36,7 +36,7 @@ When an answer is worth keeping, ask the assistant to save it. It drafts the SQL On the **Queries** page it goes a step further and becomes a SQL author: describe the query you want ("show error rate by agent for the last 7 days") and it streams SQL straight into the editor, opening a diff view so you can **Accept** or **Reject** the change before it lands. -![The Observability Queries page and its SQL editor](/agenteye/images/query-lab.png) +![The FailproofAI Cloud Queries page and its SQL editor](/cloud/images/query-lab.png) *The Queries page: this editor is where the assistant streams a draft, read-only query for you to accept or reject.* Authoring SQL by asking here uses the `queries:run` permission, the same one behind the editor's **Run** button. Chat everywhere else needs `agent:use`. @@ -49,7 +49,7 @@ You can open the assistant up to everyone without worrying about what it might t - **Every write waits for you.** Saved queries and dashboards are created only after your explicit Approve click, and there is no setting that turns that gate off. - **It can never delete anything.** No delete tool is exposed and the assistant holds no delete permission. Deletions stay in your hands, in the dashboard. - **It stays inside your org.** The assistant only ever sees the organization you are currently viewing. -- **Your questions stay yours.** Prompts and answers live in your own Observability database; product analytics records usage metadata only, never your prompt text. +- **Your questions stay yours.** Prompts and answers live in your own FailproofAI Cloud database; product analytics records usage metadata only, never your prompt text. ## Where to find it @@ -57,7 +57,7 @@ The assistant rides along on the right edge of every page under your org (`/ @@ -12,7 +12,7 @@ Failproof AI Observability goes looking for the failures you never wrote a rule *A two-minute tour: from a scheduled run to a fix you can act on.* -![The Audits page: recurring jobs that scan your sessions for failure patterns, each with a schedule and sensitivity](/agenteye/images/audits.png) +![The Audits page: recurring jobs that scan your sessions for failure patterns, each with a schedule and sensitivity](/cloud/images/audits.png) *Each audit is a recurring job that mines your sessions and writes up ranked, evidence-backed recommendations.* ## Stop guessing what to fix next @@ -48,7 +48,7 @@ Audits live in the dashboard at **`//audits`** (sidebar to *analyze* t ## Related -- [Alerts](/agenteye/alerts): get paged the moment a threshold you already know about is crossed. -- [Evaluations](/agenteye/evaluations): score every run so quality regressions surface on their own. -- [Error tracking](/agenteye/error-tracking): group and follow the errors your agents throw. -- [Incidents](/agenteye/incidents): track an issue an audit turns up through to its fix. +- [Alerts](/cloud/alerts): get paged the moment a threshold you already know about is crossed. +- [Evaluations](/cloud/evaluations): score every run so quality regressions surface on their own. +- [Error tracking](/cloud/errors): group and follow the errors your agents throw. +- [Incidents](/cloud/incidents): track an issue an audit turns up through to its fix. diff --git a/docs/cloud/capture.mdx b/docs/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/ru/agenteye/cli-recipes.mdx b/docs/cloud/cli-recipes.mdx similarity index 90% rename from docs/ru/agenteye/cli-recipes.mdx rename to docs/cloud/cli-recipes.mdx index 86affd35..ed77b385 100644 --- a/docs/ru/agenteye/cli-recipes.mdx +++ b/docs/cloud/cli-recipes.mdx @@ -1,12 +1,12 @@ --- -title: "CLI recipes for agents" +title: "CLI recipes" description: "Copy-paste query patterns and jq recipes that turn session, event, and evaluation data into something a script or coding agent can automate." --- -Pull session, event, and evaluation data (and trigger re-evaluations) straight from a script or coding agent, with clean JSON on stdout that pipes directly into `jq`. These recipes turn Failproof AI Observability's data into something a terminal user or an AI coding agent (Claude Code, Cursor) can query and automate, without clicking through the dashboard. +Pull session, event, and evaluation data (and trigger re-evaluations) straight from a script or coding agent, with clean JSON on stdout that pipes directly into `jq`. These recipes turn FailproofAI Cloud's data into something a terminal user or an AI coding agent (Claude Code, Cursor) can query and automate, without clicking through the dashboard. -The patterns below are copy-paste ready for the Failproof AI Observability CLI (`agenteye`). For installation, authentication, and the full option list see [CLI](/ru/agenteye/cli); run `agenteye -h` or `agenteye -h` for the built-in help. +The patterns below are copy-paste ready for the FailproofAI Cloud CLI (`agenteye`). For installation, authentication, and the full option list see [CLI](/cloud/cli); run `agenteye -h` or `agenteye -h` for the built-in help. ## Golden rules @@ -173,7 +173,7 @@ Each command's `--fields` accepts exactly its own item's field names. The set di ## Next steps -- [CLI](/ru/agenteye/cli): installation, authentication, and the full option reference for every command. -- [CLI agent skill](/ru/agenteye/cli-skill): package these recipes as a skill your coding agent can load. -- [API keys](/ru/agenteye/api-keys): create and scope the keys the CLI, SDK, and collector authenticate with. -- [Python SDK](/ru/agenteye/python-sdk): send events into Failproof AI Observability so there is data for these recipes to query. \ No newline at end of file +- [CLI](/cloud/cli): installation, authentication, and the full option reference for every command. +- [CLI agent skill](/cloud/agent-skills): package these recipes as a skill your coding agent can load. +- [API keys](/cloud/access): create and scope the keys the CLI, SDK, and collector authenticate with. +- [Python SDK](/cloud/sdk): send events into FailproofAI Cloud so there is data for these recipes to query. diff --git a/docs/agenteye/cli.mdx b/docs/cloud/cli.mdx similarity index 90% rename from docs/agenteye/cli.mdx rename to docs/cloud/cli.mdx index b61e9ad8..8f6707c7 100644 --- a/docs/agenteye/cli.mdx +++ b/docs/cloud/cli.mdx @@ -1,10 +1,10 @@ --- -title: "CLI" -description: "Drive all of Failproof AI Observability from the terminal or a script: no dashboard round-trips." +title: "Cloud CLI" +description: "Drive all of FailproofAI Cloud from the terminal or a script: no dashboard round-trips." --- -Drive all of Failproof AI Observability from the terminal or a script: no dashboard round-trips. The `agenteye` CLI queries your data (sessions, event logs, evaluations) and administers your org (API keys, users, settings, alerts, incidents, saved queries), so reach for it when you want to automate a check, wire Observability into CI, or let a coding agent inspect production. Every command supports a `--json` flag, so it works equally well for you at a prompt or for a coding agent (Claude Code, Cursor) shelling out and parsing the result. +Drive all of FailproofAI Cloud from the terminal or a script: no dashboard round-trips. The `agenteye` CLI queries your data (sessions, event logs, evaluations) and administers your org (API keys, users, settings, alerts, incidents, saved queries), so reach for it when you want to automate a check, wire FailproofAI Cloud into CI, or let a coding agent inspect production. Every command supports a `--json` flag, so it works equally well for you at a prompt or for a coding agent (Claude Code, Cursor) shelling out and parsing the result. With one binary you can: @@ -13,7 +13,11 @@ With one binary you can: - **Run analytics**: saved SQL and an ad-hoc query runner (`query`). - **Ask the AI assistant**: the same read-only analyst you chat with in the dashboard (`agent`). -> **Note:** This is the `agenteye` CLI, a different tool from the collector daemon (`agenteye-collector`). The CLI talks to your dashboard; the collector ships events to the server. + + Two CLIs, two jobs. **`failproofai`** configures and enforces guardrails on a machine + ([reference](/cli/config)). **`agenteye`** — this page — reads and administers your cloud + organization. You do not need it to connect a machine or to enforce policy. + --- @@ -69,7 +73,7 @@ agenteye --version agenteye --help ``` -> **Note:** The Failproof AI Observability Python SDK also uses the `agenteye` distribution name. Installing the CLI with `pipx` or `uv tool` (rather than `pip install` into a shared virtualenv) keeps the two from colliding. A plain `pip install agenteye` is fine only if the SDK is not installed in the same environment. +> **Note:** The FailproofAI Cloud Python SDK also uses the `agenteye` distribution name. Installing the CLI with `pipx` or `uv tool` (rather than `pip install` into a shared virtualenv) keeps the two from colliding. A plain `pip install agenteye` is fine only if the SDK is not installed in the same environment. --- @@ -91,7 +95,7 @@ agenteye logout # revoke the session and clear the stored token `whoami` never errors on a missing or expired session; it reports `logged_in: false` instead, so a script or agent can probe auth state safely (it can still exit non-zero if no base URL is set or the dashboard is unreachable). -**Requirements:** your email must be permitted to sign in to the dashboard (ask your Failproof AI Observability administrator), and the dashboard must be reachable at its base URL (see [Configuration](#configuration)). If you request a code and none arrives, your email is likely not yet enabled for dashboard access. +**Requirements:** your email must be permitted to sign in to the dashboard (ask your FailproofAI Cloud administrator), and the dashboard must be reachable at its base URL (see [Configuration](#configuration)). If you request a code and none arrives, your email is likely not yet enabled for dashboard access. --- @@ -338,13 +342,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | Authenticated, but your account lacks the required permission (the message names it) | | 6 | The requested resource was not found (e.g. unknown session or incident id) | -These make the CLI safe to script: a coding agent can branch on a `4` to prompt you to re-authenticate, or a `5` to surface the missing permission. See [CLI recipes for agents](/agenteye/cli-recipes) for exit-code-handling patterns and JSON output shapes. +These make the CLI safe to script: a coding agent can branch on a `4` to prompt you to re-authenticate, or a `5` to surface the missing permission. See [CLI recipes for agents](/cloud/cli-recipes) for exit-code-handling patterns and JSON output shapes. --- ## Next steps -- **[CLI recipes for agents](/agenteye/cli-recipes)**: copy-paste query patterns, `jq` one-liners, `--fields` projections, exit-code handling, and JSON output shapes, written for coding agents driving the CLI. -- **[CLI agent skill](/agenteye/cli-skill)**: package this CLI as an installable Claude Code / Codex *skill* so a coding agent drives Failproof AI Observability from plain-English requests. -- **[API keys](/agenteye/api-keys)**: the permission model behind `keys create --add …`. -- **[AI assistant](/agenteye/assistant)**: enabling the assistant that `agent ask` talks to. +- **[CLI recipes for agents](/cloud/cli-recipes)**: copy-paste query patterns, `jq` one-liners, `--fields` projections, exit-code handling, and JSON output shapes, written for coding agents driving the CLI. +- **[CLI agent skill](/cloud/agent-skills)**: package this CLI as an installable Claude Code / Codex *skill* so a coding agent drives FailproofAI Cloud from plain-English requests. +- **[API keys](/cloud/access)**: the permission model behind `keys create --add …`. +- **[AI assistant](/cloud/assistant)**: enabling the assistant that `agent ask` talks to. diff --git a/docs/cloud/connect.mdx b/docs/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/agenteye/dashboards.mdx b/docs/cloud/dashboards.mdx similarity index 71% rename from docs/agenteye/dashboards.mdx rename to docs/cloud/dashboards.mdx index ecf562c7..5dd5bbdb 100644 --- a/docs/agenteye/dashboards.mdx +++ b/docs/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "Turn your live agent data into one shared picture your whole team Turn your live agent data into one shared picture your whole team watches. Pin the queries that matter as charts, and everyone opens the same numbers at a glance, without re-running a single query. -![A dashboard built from saved queries: an events-per-hour line, an errors-by-type bar, a latency area chart, and tokens-by-model](/agenteye/images/dashboard-fleet.png) +![A dashboard built from saved queries: an events-per-hour line, an errors-by-type bar, a latency area chart, and tokens-by-model](/cloud/images/dashboard-fleet.png) *One board, four saved queries: events per hour, errors by type, latency, and tokens by model.* @@ -25,15 +25,15 @@ You'll find your boards at `//dashboards`. ## Pin the queries you already saved -Every tile starts as a saved query. Build and save the query you care about in the [Queries](/agenteye/queries) library (built-in presets plus your own, over your events and evaluations), then pin it to a dashboard as the chart that fits the data: a **line** for trends over time, a **bar** for comparing categories, an **area** for volume, or a **pie** for a share breakdown. +Every tile starts as a saved query. Build and save the query you care about in the [Queries](/cloud/queries) library (built-in presets plus your own, over your events and evaluations), then pin it to a dashboard as the chart that fits the data: a **line** for trends over time, a **bar** for comparing categories, an **area** for volume, or a **pie** for a share breakdown. Because a tile is just your saved query rendered as a chart, there's nothing to keep in sync by hand. Update the query once and every dashboard that uses it updates too. ## Watch quality, not just volume -Volume tells you the agents are busy. Quality tells you they're actually doing the job. Point a dashboard at your [evaluation scores](/agenteye/evaluations) and you get a board that tracks how well runs are going over time, so a quality regression shows up as a dip on a chart instead of a surprise from a customer. +Volume tells you the agents are busy. Quality tells you they're actually doing the job. Point a dashboard at your [evaluation scores](/cloud/evaluations) and you get a board that tracks how well runs are going over time, so a quality regression shows up as a dip on a chart instead of a surprise from a customer. -![A quality-focused dashboard built from saved evaluation queries](/agenteye/images/dashboard-quality.png) +![A quality-focused dashboard built from saved evaluation queries](/cloud/images/dashboard-quality.png) *A quality board keeps your evaluation scores front and center, right beside the operational numbers.* @@ -41,6 +41,6 @@ Keep an operations board and a quality board side by side and your team has one ## Related -- [Queries](/agenteye/queries): build and save the queries that become your tiles. -- [Evaluations](/agenteye/evaluations): score your runs so you can chart quality over time. -- [Alerts](/agenteye/alerts): turn a threshold on any of these metrics into a page. +- [Queries](/cloud/queries): build and save the queries that become your tiles. +- [Evaluations](/cloud/evaluations): score your runs so you can chart quality over time. +- [Alerts](/cloud/alerts): turn a threshold on any of these metrics into a page. diff --git a/docs/agenteye/error-tracking.mdx b/docs/cloud/errors.mdx similarity index 64% rename from docs/agenteye/error-tracking.mdx rename to docs/cloud/errors.mdx index 4929a03e..00340aed 100644 --- a/docs/agenteye/error-tracking.mdx +++ b/docs/cloud/errors.mdx @@ -1,19 +1,19 @@ --- -title: "Error Tracking" +title: "Errors" description: "See every failure your agents produce in one place, grouped so a noisy burst reads as a single problem." --- See every failure your agents produce in one place, grouped so a noisy burst reads as a single problem. You get a one-click path from "something is red" to the exact run that broke, without scrolling a live feed to find it. -![The Errors page: a histogram of failures over time above grouped red error rows, each with a one-click "+ alert" button](/agenteye/images/errors.png) +![The Errors page: a histogram of failures over time above grouped red error rows, each with a one-click "+ alert" button](/cloud/images/errors.png) *The Errors page: a histogram of failures over time, with repeat failures collapsed into one row per incident.* ## Every failure, already collected for you When an agent breaks, you should not have to scroll a live event stream hoping to catch the red rows before they scroll away. The **Errors** page does the collecting for you. It pulls together everything the dashboard would paint red into one triage surface, so the first thing you see is what is failing, not where to go looking for it. -And it catches more than the obvious ones. Alongside explicit `error` events, Failproof AI Observability surfaces the quiet failures too: any `tool_result`, `hook_completed`, or `agent_end` whose payload carries a failure shows up here. A tool that returned an error, or a hook that exited badly, no longer slips past you just because nothing threw a loud exception. +And it catches more than the obvious ones. Alongside explicit `error` events, FailproofAI Cloud surfaces the quiet failures too: any `tool_result`, `hook_completed`, or `agent_end` whose payload carries a failure shows up here. A tool that returned an error, or a hook that exited badly, no longer slips past you just because nothing threw a loud exception. Across the top, a histogram plots errors over time. One look tells you whether this is a steady background trickle or a spike that started a few minutes ago, so you know right away whether to drop what you are doing. @@ -23,19 +23,19 @@ Like every observe surface, the Errors page is scoped to your organization and f A single broken dependency can fire the same error hundreds of times a minute. Left raw, that is a wall of near-identical lines that buries the one thing you actually need to see. -Failproof AI Observability collapses repeat failures that share the same session and error type into a single row. A burst reads as one incident. You end up counting problems, not log lines, and the signal that matters stays on top instead of being drowned out by its own volume. +FailproofAI Cloud collapses repeat failures that share the same session and error type into a single row. A burst reads as one incident. You end up counting problems, not log lines, and the signal that matters stays on top instead of being drowned out by its own volume. ## From "something is red" to the exact event Click any row to land straight inside that run's session, positioned on the exact event that failed. No copying session IDs, no scrolling to hunt for the moment it went wrong: you arrive right on it, with the full execution graph one glance away so you can see what the agent did in the moments before it broke. -If you have `alerts:write`, every row also carries a **+ alert** button. Click it and Observability opens a new alert rule already filled in to catch that same failure again. The incident you just triaged becomes the one that pages you next time, instead of surprising you twice. +If you have `alerts:write`, every row also carries a **+ alert** button. Click it and FailproofAI Cloud opens a new alert rule already filled in to catch that same failure again. The incident you just triaged becomes the one that pages you next time, instead of surprising you twice. **Where to find it:** the **Errors** page lives in the observe section of the dashboard, at `//errors`. ## Related -- [Alerts](/agenteye/alerts): turn any failure into a paging rule. -- [Incidents](/agenteye/incidents): track a firing alert from open to resolved. -- [Sessions](/agenteye/sessions): open the full run behind any error. -- [Audits](/agenteye/audits): let Observability find failure patterns across your runs for you. +- [Alerts](/cloud/alerts): turn any failure into a paging rule. +- [Incidents](/cloud/incidents): track a firing alert from open to resolved. +- [Sessions](/cloud/sessions): open the full run behind any error. +- [Audits](/cloud/audits): let FailproofAI Cloud find failure patterns across your runs for you. diff --git a/docs/agenteye/evaluations.mdx b/docs/cloud/evaluations.mdx similarity index 67% rename from docs/agenteye/evaluations.mdx rename to docs/cloud/evaluations.mdx index 9aaf5e4c..80aa63eb 100644 --- a/docs/agenteye/evaluations.mdx +++ b/docs/cloud/evaluations.mdx @@ -4,15 +4,15 @@ description: "Quality problems find you now, instead of you hearing about them i --- -Quality problems find you now, instead of you hearing about them in a user complaint. Connect your own scoring service once and Failproof AI Observability grades every finished run automatically, so a drop in helpfulness or a spike in hallucinations shows up on its own, before a customer feels it. +Quality problems find you now, instead of you hearing about them in a user complaint. Connect your own scoring service once and FailproofAI Cloud grades every finished run automatically, so a drop in helpfulness or a spike in hallucinations shows up on its own, before a customer feels it. -![The Sessions grid with a score column: each run carries an evaluation status pill and colour-coded helpfulness, factuality, and tool-efficiency badges](/agenteye/images/sessions-list.png) +![The Sessions grid with a score column: each run carries an evaluation status pill and colour-coded helpfulness, factuality, and tool-efficiency badges](/cloud/images/sessions-list.png) *Every run on the sessions grid carries its scores; red, amber, and green badges make the weak runs jump out without you opening a single transcript.* ## Stop sampling runs by hand -You used to spot-check a handful of runs and hope the rest were fine. Now every completed session is scored the moment it finishes, on the dimensions you care about: helpfulness, tool efficiency, factuality, safety, whatever your quality bar is. You define the score keys; Failproof AI Observability stores, trends, and displays whatever your evaluator sends back. No run slips through unscored, and you stop learning about a regression from a support ticket. +You used to spot-check a handful of runs and hope the rest were fine. Now every completed session is scored the moment it finishes, on the dimensions you care about: helpfulness, tool efficiency, factuality, safety, whatever your quality bar is. You define the score keys; FailproofAI Cloud stores, trends, and displays whatever your evaluator sends back. No run slips through unscored, and you stop learning about a regression from a support ticket. The scores ride along on the sessions grid at **`//sessions`** (sidebar → *observe* → *sessions*), one badge cluster per row. Want just the runs that fell short? Filter the grid by score range, say helpfulness below 0.5, and pull up exactly the runs worth reading. Viewing scores needs the `evaluations:read` permission. @@ -20,7 +20,7 @@ The scores ride along on the sessions grid at **`//sessions`** (sideba A number tells you a run was weak; the session page tells you why. Open any run and the right rail leads with the headline summary, then shows a bar per dimension with your evaluator's own reasoning under each one, so you go from "this scored 0.4 on factuality" to the exact claim it got wrong in seconds. -![A session's right rail: the evaluation summary on top, then per-dimension score bars each with a line of reasoning, beside the full event timeline](/agenteye/images/session-detail.png) +![A session's right rail: the evaluation summary on top, then per-dimension score bars each with a line of reasoning, beside the full event timeline](/cloud/images/session-detail.png) *The session detail view: summary, per-dimension score bars, and the reasoning behind each score, right next to the run's event timeline.* @@ -30,7 +30,7 @@ Shipped a sharper evaluator, or looking at a run that crashed before it could be One run scoring low is noise; a whole cohort sliding is a signal. Saved dashboards turn your scores into a trend you can watch at a glance: average helpfulness this week against last, per agent, per environment. -![A quality dashboard: average-score bars per evaluator dimension alongside a trend over time](/agenteye/images/dashboard-quality.png) +![A quality dashboard: average-score bars per evaluator dimension alongside a trend over time](/cloud/images/dashboard-quality.png) *A saved quality dashboard trends the score keys you feature, so a slow drift is obvious long before it becomes an incident.* @@ -38,14 +38,14 @@ Dashboards live at **`//dashboards`** (sidebar → *analyze* → *dash ## Connect an evaluator once -Scoring is opt-in and stays completely off until you point Failproof AI Observability at a scorer. You stand up one small HTTP service (Observability ships a working reference you can copy), set two values on your server, and every run from then on is scored for you. The full walkthrough, the scoring contract, and the SDK live in the deep guide. +Scoring is opt-in and stays completely off until you point FailproofAI Cloud at a scorer. You stand up one small HTTP service (FailproofAI Cloud ships a working reference you can copy), set two values on your server, and every run from then on is scored for you. The full walkthrough, the scoring contract, and the SDK live in the deep guide. -Not sure which dimensions are worth scoring in the first place? The [evaluator agent skill](/agenteye/evaluator-skill) has your coding agent work that out against your own sessions, then build and deploy the service. +Not sure which dimensions are worth scoring in the first place? The [evaluator agent skill](/cloud/agent-skills) has your coding agent work that out against your own sessions, then build and deploy the service. ## Related -- [Evaluation suite](/agenteye/evaluation-suite): connect your evaluator, the scoring contract, and the SDK. -- [Evaluator agent skill](/agenteye/evaluator-skill): let a coding agent pick your score dimensions and build the evaluator. -- [Sessions](/agenteye/sessions): the run-by-run grid where scores appear. -- [Dashboards](/agenteye/dashboards): save and share quality trends across your org. -- [Audits](/agenteye/audits): Observability's other automatic quality feature, for cross-session investigations. +- [Evaluation suite](/cloud/evaluators): connect your evaluator, the scoring contract, and the SDK. +- [Evaluator agent skill](/cloud/agent-skills): let a coding agent pick your score dimensions and build the evaluator. +- [Sessions](/cloud/sessions): the run-by-run grid where scores appear. +- [Dashboards](/cloud/dashboards): save and share quality trends across your org. +- [Audits](/cloud/audits): FailproofAI Cloud's other automatic quality feature, for cross-session investigations. diff --git a/docs/agenteye/evaluation-suite.mdx b/docs/cloud/evaluators.mdx similarity index 85% rename from docs/agenteye/evaluation-suite.mdx rename to docs/cloud/evaluators.mdx index 157469be..7dbd9a99 100644 --- a/docs/agenteye/evaluation-suite.mdx +++ b/docs/cloud/evaluators.mdx @@ -1,20 +1,40 @@ --- -title: "Evaluation Suite" -description: "Failproof AI Observability can automatically score every finished agent run for quality: you supply a small scoring service, and Observability handles the rest." +title: "Evaluators" +description: "Connect a scoring service once and every finished run is graded automatically — on the dimensions you define, with your own reasoning attached." +icon: ruler --- +FailproofAI Cloud scores every finished agent run for quality. You supply a small scoring +service; the platform handles scheduling, retries, storage, and display. Use it to track the +dimensions you actually care about, catch regressions early, and compare agents or +environments at a glance. -Failproof AI Observability can automatically score every finished agent run for quality: you supply a small scoring service, and Observability handles the rest. Use it to track the dimensions you care about (helpfulness, tool efficiency, factuality, safety; you choose), catch regressions early, and compare agents or environments at a glance. Scoring is opt-in: the pipeline does nothing until you set `EVALUATOR_ENDPOINT` on the server. +Scoring is **opt-in** and completely inert until you point the platform at an evaluator. -> **Note:** You define the score dimensions. Your evaluator can return any numeric keys it likes; Observability stores, trends, and displays whatever you send back. + + **You define the dimensions.** Your evaluator returns whatever numeric keys it likes — + `helpfulness`, `tool_efficiency`, `factuality`, `resolved`, anything — and the platform + stores, trends, and displays them. There is no fixed schema to conform to, and no default + evaluator that quietly measures the wrong thing on your behalf. + + + + Not sure which dimensions are worth scoring? The [evaluator agent + skill](/cloud/agent-skills#agenteye-evaluator--decide-what-to-score-then-build-the-scorer) + has your coding agent work that out against your own sessions before writing any code — + which is where evaluators usually go wrong. + ## At a glance -1. **Write a scorer.** Stand up a small HTTP service that reads a session transcript and returns scores. Observability ships a working reference you can copy. See [Writing an evaluator with the SDK](#writing-an-evaluator-with-the-sdk). -2. **Point Observability at it.** Set `EVALUATOR_ENDPOINT` (and a shared `EVALUATOR_TOKEN`) on the server process. -3. **Watch the scores land.** Every completed session is scored automatically; results show up on the session detail page, the sessions grid, and saved dashboards. +1. **Write a scorer.** A small HTTP service that reads a session transcript and returns + scores. A working reference ships with the SDK — copy it and swap in your logic. See + [Writing an evaluator with the SDK](#writing-an-evaluator-with-the-sdk). +2. **Point the platform at it.** Set `EVALUATOR_ENDPOINT` and a shared `EVALUATOR_TOKEN`. +3. **Watch the scores land.** Every completed session is scored automatically, and results + appear on the session detail page, the sessions grid, and saved dashboards. -![A session detail view with the evaluation summary, per-dimension score bars, and reasoning text in the right rail](/agenteye/images/session-detail.png) +![A session detail view with the evaluation summary, per-dimension score bars, and reasoning text in the right rail](/cloud/images/session-detail.png) *Once an evaluator is configured, each completed run is scored and the results appear in the session's right rail: the summary on top, then per-dimension score bars with reasoning.* @@ -24,7 +44,7 @@ Failproof AI Observability can automatically score every finished agent run for ```mermaid flowchart LR - ING["ingest /events
agent_end"] --> SRV["Observability server"] + ING["ingest /events
agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -32,26 +52,26 @@ flowchart LR SRV --> RES["evaluations
terminal results"] ``` -When the Observability SDK emits an `agent_end` event for a session, the server +When the FailproofAI Cloud SDK emits an `agent_end` event for a session, the server schedules an evaluation. It then POSTs the full event transcript to your evaluator service, which can either: - **Return the result inline** with `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`. The result is appended to the session's evaluation timeline. `reasoning` and `summary` are optional. -- **Defer** with `{"status":"pending", "job_id":"abc-123"}`. Observability then +- **Defer** with `{"status":"pending", "job_id":"abc-123"}`. FailproofAI Cloud then calls `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` until your evaluator returns `{"status":"done", ...}` or `{"status":"error", "error":"..."}`. The polling cadence is per-job: a `pending` response may include - `next_poll_secs` to override; otherwise Observability uses the + `next_poll_secs` to override; otherwise FailproofAI Cloud uses the `default_poll_interval_secs` value from `GET /config`; otherwise the server falls back to `EVALUATOR_POLLING_INTERVAL_SECS` (default 10s). All values are clamped to [1s, 1h]. Sessions that never emit `agent_end` (for example, a crashed agent process) can also be picked up: the evaluator's `GET /config` may return -`{"inactivity_timeout_secs": 1800}`, and Observability will evaluate any session +`{"inactivity_timeout_secs": 1800}`, and FailproofAI Cloud will evaluate any session that has gone idle for that long. Set the field to `null` or omit it to disable this fallback. @@ -74,9 +94,8 @@ past `inactivity_timeout_secs`, a fresh evaluation is enqueued. Transient failures (5xx, 429, timeouts, network errors) are retried with exponential backoff up to `EVALUATOR_MAX_ATTEMPTS`; 4xx responses are -terminal. Observability is safe to run with multiple horizontally-scaled server -instances; work is partitioned so the same session is never dispatched -twice concurrently. +terminal. A single session is never evaluated twice concurrently, however the deployment is +scaled — so your evaluator never receives duplicate work for the same run. --- @@ -85,7 +104,7 @@ twice concurrently. Every authenticated route uses **bearer token auth**. The same value must be configured on both sides: -- Observability server: env var `EVALUATOR_TOKEN` +- FailproofAI Cloud server: env var `EVALUATOR_TOKEN` - Evaluator service: configured the same way (the `agenteye-evaluator` SDK reads `EVALUATOR_TOKEN` by convention) @@ -169,7 +188,7 @@ You don't have to implement the HTTP contract by hand. The `agenteye-evaluator` Python package gives you a typed FastAPI wrapper that handles auth, routing, and the request/response shapes for you. -Failproof AI Observability also ships a **working reference evaluator** that +FailproofAI Cloud also ships a **working reference evaluator** that scores `helpfulness`, `tool_efficiency`, and `factuality` from the shape of the transcript. Copy it as a starting point and swap in your own logic: an LLM judge, a rule engine, whatever fits your quality bar. @@ -196,7 +215,7 @@ def run(req: EvalRequest) -> EvalResponse: The `app` instance runs under any ASGI server, so `uvicorn module:app` starts it. For evaluators that need to defer expensive work, return `JobPending` -instead and register a `@app.job_lookup` handler; the Observability server +instead and register a `@app.job_lookup` handler; the FailproofAI Cloud server polls `GET /evaluate/{job_id}` until you return a terminal status or the `EVALUATOR_MAX_POLL_DURATION_SECS` cap (default 1 h) elapses. @@ -207,7 +226,7 @@ The full API reference, async pattern, and event schema are documented in the ## Running your evaluator -The evaluator is **your service** — Failproof AI Observability does not ship a +The evaluator is **your service** — FailproofAI Cloud does not ship a default evaluator, so you build and run it wherever you run your own services. It runs under any ASGI server (for example `uvicorn my_evaluator:app`); serve the `/health`, `/config`, and `/evaluate` routes from the @@ -326,7 +345,7 @@ The bootstrap admin (`ADMIN_KEY`, `ADMIN_EMAIL`) automatically receives these. session's evaluation status and scores at a glance. - **`/dashboards`**: saved eval-health views (see [Dashboards](#dashboards) below). -![The Sessions grid with per-session evaluation status pills and colour-coded score badges (helpfulness, factuality, tool_efficiency, safety, coherence)](/agenteye/images/sessions-list.png) +![The Sessions grid with per-session evaluation status pills and colour-coded score badges (helpfulness, factuality, tool_efficiency, safety, coherence)](/cloud/images/sessions-list.png) *The sessions grid shows each run's evaluation status and scores at a glance; red/amber/green badges make low scores jump out.* @@ -354,7 +373,7 @@ sessions page pre-filtered to exactly that slice. Metrics are computed server-side over the whole matching set (via `GET /evaluations/aggregate`), so the numbers are exact rather than sampled. -![An eval-health dashboard with average-score bars per evaluator dimension, a tool ok-vs-error breakdown, top tools, and an events-per-hour trend](/agenteye/images/dashboard-quality.png) +![An eval-health dashboard with average-score bars per evaluator dimension, a tool ok-vs-error breakdown, top tools, and an events-per-hour trend](/cloud/images/dashboard-quality.png) **Permissions:** viewing needs both `dashboards:read` and `evaluations:read`; creating and editing needs `dashboards:write`; deleting needs `dashboards:delete`. @@ -396,7 +415,7 @@ longer than the default. ## Next steps -- [Evaluator agent skill](/agenteye/evaluator-skill): have a coding agent design your dimensions against real sessions and build this service for you. -- [Python SDK](/agenteye/python-sdk): emit the `agent_end` events that trigger scoring. -- [API keys](/agenteye/api-keys): the `evaluations:read` and `evaluations:trigger` permissions. -- [Audits](/agenteye/audits): Observability's other automated quality feature, for policy-based review. +- [Evaluator agent skill](/cloud/agent-skills): have a coding agent design your dimensions against real sessions and build this service for you. +- [Python SDK](/cloud/sdk): emit the `agent_end` events that trigger scoring. +- [API keys](/cloud/access): the `evaluations:read` and `evaluations:trigger` permissions. +- [Audits](/cloud/audits): FailproofAI Cloud's other automated quality feature, for policy-based review. diff --git a/docs/agenteye/event-stream.mdx b/docs/cloud/event-stream.mdx similarity index 77% rename from docs/agenteye/event-stream.mdx rename to docs/cloud/event-stream.mdx index ed764591..6e9ce155 100644 --- a/docs/agenteye/event-stream.mdx +++ b/docs/cloud/event-stream.mdx @@ -1,12 +1,12 @@ --- -title: "Event Stream" +title: "Event stream" description: "The moment your agent does something, you see it." --- The moment your agent does something, you see it. The Event Stream is your live pulse on every agent in production: no waiting, no grepping logs, no guessing what just happened. -![The live Event Stream: colour-coded event rows tailing in real time, filterable by environment, agent, session, event type, and free text](/agenteye/images/events-stream.png) +![The live Event Stream: colour-coded event rows tailing in real time, filterable by environment, agent, session, event type, and free text](/cloud/images/events-stream.png) *Every event from every agent in your org, newest first, updating as it happens.* @@ -37,14 +37,14 @@ Free-text search cuts straight to a message, a tool name, or an id you already h The Event Stream is your org home. Sign in and it is the first surface you land on, at `//`, so triage starts the second you arrive. -Behind it, your agents emit events through the SDK, the collector ships them to your Failproof AI Observability server, and the stream tails them as they arrive in infrastructure you control. When you want the rolled-up view instead of the raw trail, each run's events collapse into a single row on Sessions, one click away. +Behind it, your agents emit events through the SDK, the collector ships them to your FailproofAI Cloud server, and the stream tails them as they arrive in infrastructure you control. When you want the rolled-up view instead of the raw trail, each run's events collapse into a single row on Sessions, one click away. This is the raw source of truth that every other observe surface builds on, so when a number looks wrong elsewhere, the stream is where you confirm what actually happened. ## Related -- [Sessions](/agenteye/sessions): the same events rolled up into one row per run, with a git-style execution graph. -- [Telemetry](/agenteye/telemetry): what your agents send and how events reach the stream. -- [Error tracking](/agenteye/error-tracking): one triage surface for everything that went wrong. -- [Alerts](/agenteye/alerts): turn any threshold into a paging rule. -- [CLI and agents](/agenteye/cli-and-agents): the same live trail from your terminal. +- [Sessions](/cloud/sessions): the same events rolled up into one row per run, with a git-style execution graph. +- [Telemetry](/cloud/performance): what your agents send and how events reach the stream. +- [Error tracking](/cloud/errors): one triage surface for everything that went wrong. +- [Alerts](/cloud/alerts): turn any threshold into a paging rule. +- [CLI and agents](/cloud/cli): the same live trail from your terminal. diff --git a/docs/cloud/fleet.mdx b/docs/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/agenteye/images/alert-new.png b/docs/cloud/images/alert-new.png similarity index 100% rename from docs/agenteye/images/alert-new.png rename to docs/cloud/images/alert-new.png diff --git a/docs/agenteye/images/alerts.png b/docs/cloud/images/alerts.png similarity index 100% rename from docs/agenteye/images/alerts.png rename to docs/cloud/images/alerts.png diff --git a/docs/agenteye/images/api-keys.png b/docs/cloud/images/api-keys.png similarity index 100% rename from docs/agenteye/images/api-keys.png rename to docs/cloud/images/api-keys.png diff --git a/docs/agenteye/images/assistant.png b/docs/cloud/images/assistant.png similarity index 100% rename from docs/agenteye/images/assistant.png rename to docs/cloud/images/assistant.png diff --git a/docs/agenteye/images/audits.png b/docs/cloud/images/audits.png similarity index 100% rename from docs/agenteye/images/audits.png rename to docs/cloud/images/audits.png diff --git a/docs/agenteye/images/dashboard-fleet.png b/docs/cloud/images/dashboard-fleet.png similarity index 100% rename from docs/agenteye/images/dashboard-fleet.png rename to docs/cloud/images/dashboard-fleet.png diff --git a/docs/agenteye/images/dashboard-quality.png b/docs/cloud/images/dashboard-quality.png similarity index 100% rename from docs/agenteye/images/dashboard-quality.png rename to docs/cloud/images/dashboard-quality.png diff --git a/docs/agenteye/images/errors.png b/docs/cloud/images/errors.png similarity index 100% rename from docs/agenteye/images/errors.png rename to docs/cloud/images/errors.png diff --git a/docs/agenteye/images/events-stream.png b/docs/cloud/images/events-stream.png similarity index 100% rename from docs/agenteye/images/events-stream.png rename to docs/cloud/images/events-stream.png diff --git a/docs/agenteye/images/hooks.png b/docs/cloud/images/hooks.png similarity index 100% rename from docs/agenteye/images/hooks.png rename to docs/cloud/images/hooks.png diff --git a/docs/agenteye/images/incident-detail.png b/docs/cloud/images/incident-detail.png similarity index 100% rename from docs/agenteye/images/incident-detail.png rename to docs/cloud/images/incident-detail.png diff --git a/docs/agenteye/images/incidents.png b/docs/cloud/images/incidents.png similarity index 100% rename from docs/agenteye/images/incidents.png rename to docs/cloud/images/incidents.png diff --git a/docs/agenteye/images/login.png b/docs/cloud/images/login.png similarity index 100% rename from docs/agenteye/images/login.png rename to docs/cloud/images/login.png diff --git a/docs/agenteye/images/models.png b/docs/cloud/images/models.png similarity index 100% rename from docs/agenteye/images/models.png rename to docs/cloud/images/models.png diff --git a/docs/agenteye/images/queries.png b/docs/cloud/images/queries.png similarity index 100% rename from docs/agenteye/images/queries.png rename to docs/cloud/images/queries.png diff --git a/docs/agenteye/images/query-lab.png b/docs/cloud/images/query-lab.png similarity index 100% rename from docs/agenteye/images/query-lab.png rename to docs/cloud/images/query-lab.png diff --git a/docs/agenteye/images/session-detail.png b/docs/cloud/images/session-detail.png similarity index 100% rename from docs/agenteye/images/session-detail.png rename to docs/cloud/images/session-detail.png diff --git a/docs/agenteye/images/sessions-list.png b/docs/cloud/images/sessions-list.png similarity index 100% rename from docs/agenteye/images/sessions-list.png rename to docs/cloud/images/sessions-list.png diff --git a/docs/agenteye/images/settings.png b/docs/cloud/images/settings.png similarity index 100% rename from docs/agenteye/images/settings.png rename to docs/cloud/images/settings.png diff --git a/docs/agenteye/images/tools.png b/docs/cloud/images/tools.png similarity index 100% rename from docs/agenteye/images/tools.png rename to docs/cloud/images/tools.png diff --git a/docs/agenteye/images/users.png b/docs/cloud/images/users.png similarity index 100% rename from docs/agenteye/images/users.png rename to docs/cloud/images/users.png diff --git a/docs/agenteye/images/video-audit.jpg b/docs/cloud/images/video-audit.jpg similarity index 100% rename from docs/agenteye/images/video-audit.jpg rename to docs/cloud/images/video-audit.jpg diff --git a/docs/agenteye/images/video-tracing.jpg b/docs/cloud/images/video-tracing.jpg similarity index 100% rename from docs/agenteye/images/video-tracing.jpg rename to docs/cloud/images/video-tracing.jpg diff --git a/docs/agenteye/incidents.mdx b/docs/cloud/incidents.mdx similarity index 85% rename from docs/agenteye/incidents.mdx rename to docs/cloud/incidents.mdx index 51bebde6..21a96146 100644 --- a/docs/agenteye/incidents.mdx +++ b/docs/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "When an alert fires, everyone can see the incident is open, who ow When an alert fires, the first question is always "who's on it?" Incidents answer it: the moment something breaches, everyone can see the incident is open, who owns it, and exactly what has happened so far, with a clean, attributed record you can hand straight to a post-mortem. -![The Incidents inbox: alert-linked and manually opened incident cards, grouped by state, each with a severity badge and an assignee](/agenteye/images/incidents.png) +![The Incidents inbox: alert-linked and manually opened incident cards, grouped by state, each with a severity badge and an assignee](/cloud/images/incidents.png) *The inbox groups open incidents by state and filters by severity and assignee, so you see what needs a human now.* ## Know who has it, at a glance @@ -17,10 +17,10 @@ No more "is anyone looking at this?" in a chat thread. A breach opens an inciden When the incident is over, you already have the write-up. Open any incident and you get the breach evidence, its assignees and subscribers, a comment thread for coordinating in place, and an append-only activity timeline. -![An incident detail view: the parent alert and breach summary, assignees and subscribers, an attributed activity timeline, and a comment thread](/agenteye/images/incident-detail.png) +![An incident detail view: the parent alert and breach summary, assignees and subscribers, an attributed activity timeline, and a comment thread](/cloud/images/incident-detail.png) *Everything that happened, in order, each line signed by whoever did it.* -Every action (opened, acknowledged, resolved, and so on) is written to that timeline and never edited away. Each entry is attributed: to the operator who took it, by email, or to **automated** for anything Failproof AI Observability did on its own, like opening the incident on the breach. Nothing is anonymous and nothing is lost, so the post-mortem more or less writes itself. +Every action (opened, acknowledged, resolved, and so on) is written to that timeline and never edited away. Each entry is attributed: to the operator who took it, by email, or to **automated** for anything FailproofAI Cloud did on its own, like opening the incident on the breach. Nothing is anonymous and nothing is lost, so the post-mortem more or less writes itself. ## How an incident moves @@ -45,6 +45,6 @@ Incidents live at `//incidents`. Viewing needs **`incidents:read`**; o ## Related -- [Alerts](/agenteye/alerts): the rules that open these incidents when a threshold breaches. -- [Error tracking](/agenteye/error-tracking): see every failure in one place and promote one to an alert. -- [Audits](/agenteye/audits): the scheduled analyst that finds the failures no rule was watching. +- [Alerts](/cloud/alerts): the rules that open these incidents when a threshold breaches. +- [Error tracking](/cloud/errors): see every failure in one place and promote one to an alert. +- [Audits](/cloud/audits): the scheduled analyst that finds the failures no rule was watching. diff --git a/docs/cloud/managed-policies.mdx b/docs/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/cloud/overview.mdx b/docs/cloud/overview.mdx new file mode 100644 index 00000000..8fb83816 --- /dev/null +++ b/docs/cloud/overview.mdx @@ -0,0 +1,183 @@ +--- +title: "FailproofAI Cloud" +description: "One place to govern every agent your organization runs — deploy policy to the whole fleet, replay any run, score quality automatically, and get paged when it breaks." +--- + +Guardrails on one laptop are useful. Guardrails across a company are a different problem: +you cannot see whether the rule you wrote last week is actually running on the twelve +machines that need it, you cannot tell which agent burned an afternoon on a loop, and you +find out about the leaked key when someone mentions it in standup. + +**FailproofAI Cloud is the answer to that problem.** Connect a machine with one command and +two things start flowing: policy comes *down* from a dashboard your team controls, and +everything your agents did goes *up* to it. + +![A FailproofAI Cloud session drawn as a git-style execution graph beside its event timeline, with a per-run breakdown of tools, models, and hooks in the right rail](/cloud/images/session-detail.png) + +*Every agent run, drawn as a git-style execution graph beside its event timeline. Parallel +sub-agents get their own lanes; the right rail breaks down tools, models, hooks, and token +spend for the run.* + +--- + +## What connecting gets you + + + + + Write a rule once, assign it, and every connected machine picks it up on its next poll — + digest-verified before it runs. Roll it out in observe-only mode first and watch what it + *would* have blocked before it blocks anything. + + + + Which hosts are connected, which deployment each is actually on, and which are running + agents with no guardrails at all. "Did it roll out?" becomes a page you look at, not a + question you ask in chat. + + + + Every session from every machine becomes a readable execution graph: what ran in + parallel, which sub-agent stalled, where it went off course, and what it spent. + + + + Connect your own scoring service and every finished run is graded. A drop in helpfulness + or a spike in hallucinations shows up on its own, before a customer feels it. + + + + Scheduled investigations mine your sessions across runs for error clusters, drift, tool + misuse, and goal failures — then hand you ranked findings with the evidence attached. + + + + Thresholds on error rate, latency, cost, or evaluator scores open incidents you can + acknowledge, assign, and resolve — with an attributed timeline your post-mortem writes + itself from. + + + + +--- + +## See it in action + +
+ +
+ +*Agent tracing: follow a single run step by step, from goal to tools to final answer.* + +
+ +
+ +*Audits: let FailproofAI Cloud mine your sessions and tell you what to fix.* + +--- + +## Connect in one command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is the whole integration for a machine already running FailproofAI. No second agent +to install, no per-project setup, no code change. The [connect guide](/cloud/connect) +covers machine ids, environments, what leaves the machine, and how to connect a fleet +without a human at each keyboard. + +Don't have a key yet? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + + + **Already writing your own agents in Python?** The [SDK](/cloud/sdk) instruments them + directly, so agents you build show up next to the coding sessions your team runs. The + two are complementary: the SDK covers agents you write, the machine connection covers + the agent CLIs your team already uses. + + +--- + +## What the dashboard gives you + +Organized around three ideas, mirroring the left sidebar. + +### Observe — the raw truth of what happened + +- **[Event stream](/cloud/event-stream)** — the live, per-step trail of every run, across + every machine, newest first. Your triage starting point. +- **[Sessions](/cloud/sessions)** — those events rolled into one row per run, each with a + git-style execution graph. +- **[Performance](/cloud/performance)** — latency heat-maps and p50/p95/p99 vitals for + models, tools, and hooks, so a tail spike stands out from the median. +- **[Errors](/cloud/errors)** — one triage surface for everything that went wrong, grouped + so a noisy burst reads as a single problem. + +![The Tools page: a latency heat-map, a percentile band, and a tool-distribution bar over 24 time bins](/cloud/images/tools.png) + +### Analyze — turn activity into answers + +- **[Queries](/cloud/queries)** and **[dashboards](/cloud/dashboards)** — saved SQL over + your events and evaluations, charted into shared boards. +- **[Evaluations](/cloud/evaluations)** — quality scores from your own evaluator, with + per-score reasoning. +- **[Audits](/cloud/audits)** — recurring investigations that surface patterns across + sessions. +- **[Alerts](/cloud/alerts)** and **[incidents](/cloud/incidents)** — thresholds that page + you, plus the workflow to triage what they open. + +### Govern and administer + +- **[Managed policies](/cloud/managed-policies)** — the guardrails your fleet enforces, + assigned from here. +- **[Fleet](/cloud/fleet)** — every machine, its deployment, and its coverage. +- **[Access](/cloud/access)** — scoped API keys, users, and permission sets. +- **[Security](/cloud/security)** — isolation, sign-in, and what the assistant can and + cannot do. + +--- + +## Reach it however you work + +- **[The dashboard](https://app.befailproof.ai)** — every page scoped to your organization. +- **[The `agenteye` CLI](/cloud/cli)** — read your data and administer your org from a + terminal or a script. Every command takes `--json`. +- **[The AI assistant](/cloud/assistant)** — ask questions about your agents in plain + English, inside the dashboard, with links to the evidence behind every answer. +- **[Agent skills](/cloud/agent-skills)** — hand the CLI to a coding agent and let it + answer "is anything broken today?" for you. +- **A REST API** — everything the dashboard and CLI do is backed by it. Call it with a + [scoped key](/cloud/access) to wire FailproofAI Cloud into your own tooling. + +--- + +## Getting access + +FailproofAI Cloud is the commercial half of FailproofAI. The guardrails are open source and +free forever; the cloud is what a team buys when one machine becomes twenty. + +Start at [befailproof.ai/get-started](https://befailproof.ai/get-started/), or +[talk to us](https://cal.com/nikita-agarwal-exosphere/30-minute-chat-failproof-ai) about a +self-hosted deployment inside your own infrastructure. + +--- + +## Next steps + + + + + One command, two capabilities, and exactly what gets sent. + + + + Every term in these docs, defined once. + + + + Isolation, sign-in, and data control. + + + diff --git a/docs/agenteye/telemetry.mdx b/docs/cloud/performance.mdx similarity index 78% rename from docs/agenteye/telemetry.mdx rename to docs/cloud/performance.mdx index e6180878..e7262567 100644 --- a/docs/agenteye/telemetry.mdx +++ b/docs/cloud/performance.mdx @@ -1,12 +1,12 @@ --- -title: "Performance Metrics" +title: "Performance" description: "See the instant your models, tools, or hooks slow down or run up a bill, and catch a tail-latency spike before your users ever feel it." --- See the instant your models, tools, or hooks slow down or run up a bill, and catch a tail-latency spike before your users ever feel it. Three dedicated pages turn raw timings into p50, p95, and p99 you can read at a glance. -![The Models page showing a latency heat-map, a percentile band, and per-model token, cost, and context-window figures](/agenteye/images/models.png) +![The Models page showing a latency heat-map, a percentile band, and per-model token, cost, and context-window figures](/cloud/images/models.png) *The Models page: a latency heat-map, a percentile band, and per-model tokens, estimated cost, and context-window fill.* ## Stop letting averages hide your worst runs @@ -24,13 +24,13 @@ A shared hover crosshair links the heat-map and the band, so a tail spike lines The Models page (shown up top) answers the two questions a bill always raises: which model, and how much. On top of the shared latency view, it adds **per-model token consumption**, **estimated cost**, and **context-window fill**, so runaway prompt growth and an impending compaction are visible before they surprise you. -Failproof AI Observability recognizes common model IDs automatically. If a window looks wrong, or you run a private model of your own, correct it or add one under **Settings**, in **model context windows**, and the fill readouts follow. +FailproofAI Cloud recognizes common model IDs automatically. If a window looks wrong, or you run a private model of your own, correct it or add one under **Settings**, in **model context windows**, and the fill readouts follow. ## Tools: tell the slow apart from the broken A tool call can be slow, or it can be quietly failing, and you want to know which one in seconds, not after digging through logs. -![The Tools page showing the shared latency heat-map and percentile band beside a success and failure breakdown and a tool-distribution bar](/agenteye/images/tools.png) +![The Tools page showing the shared latency heat-map and percentile band beside a success and failure breakdown and a tool-distribution bar](/cloud/images/tools.png) *The Tools page: the same heat-map and percentile band, plus a success and failure breakdown and a tool-distribution bar.* Alongside the shared latency view, the Tools page adds a **success and failure breakdown** and a **tool-distribution bar**, so you see at a glance which tools you lean on most and which are eating your error budget. @@ -39,14 +39,14 @@ Alongside the shared latency view, the Tools page adds a **success and failure b When a lifecycle hook drags a run, "hooks are slow" is not something you can act on. The Hooks page gets you to the one that matters. -![The Hooks page showing latency broken down by hook name and trigger event over the shared heat-map and percentile band](/agenteye/images/hooks.png) +![The Hooks page showing latency broken down by hook name and trigger event over the shared heat-map and percentile band](/cloud/images/hooks.png) *The Hooks page: latency broken down by hook name and trigger event.* Over the same latency heat-map and percentile band, the Hooks page breaks activity down by **hook name** and **trigger event**, so you land on the single hook and the single event that need attention. ## Related -- [Event stream](/agenteye/event-stream): the live, colour-coded trail of every event. -- [Sessions](/agenteye/sessions): roll events up into one row per run and open its execution graph. -- [Error tracking](/agenteye/error-tracking): one triage surface for everything the dashboard paints red. -- [Dashboards](/agenteye/dashboards): roll-up views across your fleet. +- [Event stream](/cloud/event-stream): the live, colour-coded trail of every event. +- [Sessions](/cloud/sessions): roll events up into one row per run and open its execution graph. +- [Error tracking](/cloud/errors): one triage surface for everything the dashboard paints red. +- [Dashboards](/cloud/dashboards): roll-up views across your fleet. diff --git a/docs/agenteye/queries.mdx b/docs/cloud/queries.mdx similarity index 78% rename from docs/agenteye/queries.mdx rename to docs/cloud/queries.mdx index 33ae39cc..0e051f8f 100644 --- a/docs/agenteye/queries.mdx +++ b/docs/cloud/queries.mdx @@ -4,9 +4,9 @@ description: "Ask any question of your agent data and get an answer in seconds." --- -Ask any question of your agent data and get an answer in seconds. Failproof AI Observability gives you a library of saved, ready-to-run queries over your events and evaluations, so you start from a working example instead of a blank SQL editor. +Ask any question of your agent data and get an answer in seconds. FailproofAI Cloud gives you a library of saved, ready-to-run queries over your events and evaluations, so you start from a working example instead of a blank SQL editor. -![The saved-queries library: a grid of reusable queries, both built-in presets and custom ones](/agenteye/images/queries.png) +![The saved-queries library: a grid of reusable queries, both built-in presets and custom ones](/cloud/images/queries.png) *Your saved-queries library at `//queries`: built-in presets sitting alongside the queries your team has saved.* @@ -22,7 +22,7 @@ Find it at `//queries`. Open any query and it lands in the SQL composer, where you can adjust it and see the answer immediately: no export, no round-trip, no waiting on someone else. -![The SQL query composer running a saved query, with a schema sidebar and a live result grid](/agenteye/images/query-lab.png) +![The SQL query composer running a saved query, with a schema sidebar and a live result grid](/cloud/images/query-lab.png) *The SQL composer: your query on the left, a schema sidebar so you never guess a column name, and a live result grid below.* @@ -43,14 +43,14 @@ agenteye query list # the same saved queries, from your terminal agenteye query run errs --arg prod # run one and print the rows (add --json to pipe it) ``` - See [CLI and agents](/agenteye/cli-and-agents) for the full command set. + See [CLI and agents](/cloud/cli) for the full command set. -- **From the AI assistant.** Not sure how to phrase the SQL? Ask the in-dashboard [AI assistant](/agenteye/assistant) in plain English and it will draft the query and save it to your library for you. +- **From the AI assistant.** Not sure how to phrase the SQL? Ask the in-dashboard [AI assistant](/cloud/assistant) in plain English and it will draft the query and save it to your library for you. Running a saved query is gated by the `queries:run` permission, kept separate from the permissions to create or delete queries, so you can grant read access without letting everyone rewrite the library. ## Related -- [Dashboards](/agenteye/dashboards): pin query results into shared, org-wide charts. -- [AI assistant](/agenteye/assistant): ask questions in plain English and get a query back. -- [CLI and agents](/agenteye/cli-and-agents): run and save the same queries from your terminal. +- [Dashboards](/cloud/dashboards): pin query results into shared, org-wide charts. +- [AI assistant](/cloud/assistant): ask questions in plain English and get a query back. +- [CLI and agents](/cloud/cli): run and save the same queries from your terminal. diff --git a/docs/agenteye/python-sdk.mdx b/docs/cloud/sdk.mdx similarity index 90% rename from docs/agenteye/python-sdk.mdx rename to docs/cloud/sdk.mdx index c8657a17..dc23e278 100644 --- a/docs/agenteye/python-sdk.mdx +++ b/docs/cloud/sdk.mdx @@ -4,11 +4,22 @@ description: "See exactly what your AI agents did in production: every agent run --- -See exactly what your AI agents did in production: every agent run, tool call, model request, hook, and human intervention. The Failproof AI Observability Python SDK records that trail from inside your agent code so you can debug, audit, and evaluate what happened. Use it whenever you want Failproof AI Observability to observe your agents. +See exactly what your AI agents did in production: every agent run, tool call, model +request, hook, and human intervention. The FailproofAI Cloud Python SDK records that trail +from inside **agents you write yourself**, so you can debug, audit, and evaluate what +happened. -Under the hood, the SDK writes structured events to local JSONL files, and the collector daemon picks them up and ships them to the platform automatically. You do not manage those files yourself. + + **Do you need this?** If you want to observe the agent CLIs your team already runs — + Claude Code, Codex, Cursor and the rest — you don't. [Connect the + machine](/cloud/connect) and those sessions are captured with no code change. The SDK is + for agents *you build*: your own Python loops, services, and pipelines. + -> **Tip:** New to Failproof AI Observability? This page is the complete SDK event reference. +Events are buffered in your process, written to local files, and shipped for you. You never +manage those files, and if delivery is interrupted they wait on disk rather than being lost. + +This page is the complete event reference.
@@ -26,7 +37,7 @@ Once it is installed, confirm you have it: python -c "import agenteye; print(agenteye.__version__)" ``` -Prefer to let a coding agent do the whole integration? The [Python SDK Agent Skill](/agenteye/python-sdk-skill) knows the install path, plans the instrumentation points, writes them, and verifies the events land. +Prefer to let a coding agent do the whole integration? The [Python SDK Agent Skill](/cloud/agent-skills) knows the install path, plans the instrumentation points, writes them, and verifies the events land. --- @@ -60,7 +71,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### Instrumenting a real call -In practice you wrap your existing agent code. Bracket a model call with `model_request` before and `model_response` after, so the two events span the real request and Failproof AI Observability can pair them: +In practice you wrap your existing agent code. Bracket a model call with `model_request` before and `model_response` after, so the two events span the real request and FailproofAI Cloud can pair them: ```python import anthropic @@ -99,7 +110,7 @@ Wrap tool calls the same way with `tool_use` and `tool_result`, reusing one `too Here is what those events look like once they reach the dashboard, colour-coded by type and filterable by environment, agent, and session: -![The live Events stream, colour-coded by event type and filterable by environment, agent, and session](/agenteye/images/events-stream.png) +![The live Events stream, colour-coded by event type and filterable by environment, agent, and session](/cloud/images/events-stream.png) --- @@ -156,9 +167,9 @@ That makes redaction your choice and your responsibility. If a prompt or tool pa ## Event Reference -Most events come in start/end pairs that share a correlation ID: `tool_use` and `tool_result` share a `tool_call_id`, `hook_triggered` and `hook_completed` share a `hook_id`, and `human_wait` and `human_input` share an `input_id`. Emit the start event, do the work, then emit the end event with the same ID. Failproof AI Observability matches the pair and computes `duration_ms` for you, so you never pass `duration_ms` yourself. +Most events come in start/end pairs that share a correlation ID: `tool_use` and `tool_result` share a `tool_call_id`, `hook_triggered` and `hook_completed` share a `hook_id`, and `human_wait` and `human_input` share an `input_id`. Emit the start event, do the work, then emit the end event with the same ID. FailproofAI Cloud matches the pair and computes `duration_ms` for you, so you never pass `duration_ms` yourself. -![A session's git-style execution graph beside its event timeline, reconstructed from the paired events, with the tool/model/hook breakdown panel](/agenteye/images/session-detail.png) +![A session's git-style execution graph beside its event timeline, reconstructed from the paired events, with the tool/model/hook breakdown panel](/cloud/images/session-detail.png) All event methods require these two fields: @@ -432,5 +443,5 @@ Each file is written atomically: the SDK writes to a temporary file and then ren ## Next steps -- [Event stream](/agenteye/event-stream): watch these events arrive live, colour-coded and filterable by environment, agent, and session. -- [Sessions](/agenteye/sessions): see how the paired events reconstruct each agent run as an execution graph and timeline. +- [Event stream](/cloud/event-stream): watch these events arrive live, colour-coded and filterable by environment, agent, and session. +- [Sessions](/cloud/sessions): see how the paired events reconstruct each agent run as an execution graph and timeline. diff --git a/docs/cloud/security.mdx b/docs/cloud/security.mdx new file mode 100644 index 00000000..8cd28caf --- /dev/null +++ b/docs/cloud/security.mdx @@ -0,0 +1,187 @@ +--- +title: Security and data handling +description: "What FailproofAI sees, where it is stored, who can reach it, and what stays on your machine — written for the person doing the review." +icon: shield-check +--- + +FailproofAI sits close to your agents, which means it sees prompts, tool inputs, and +outputs. This page is the straight answer to what that means, for the person who has to +sign off on it. + +--- + +## Nothing leaves a machine until you connect it + +The guardrails are local. Policy evaluation, the decision log, session replay, and the +[audit](/audit) all run on the machine, against files already on it. There is no account to +create and no network call on the enforcement path. + +That changes only when you run `failproofai config --connect`, and the CLI states plainly — +at the moment you connect, not in a footnote — what starts flowing: + +| Stream | Contents | Turn it off | +|---|---|---| +| Policy decisions | Which policy fired, on which tool, in which session, with what verdict | Don't connect, or disconnect | +| Session transcripts | The full session: prompts, model responses, file contents, command output | `--connect … --no-transcripts` | + +`failproofai config --status` always reports which mode is in effect. +[Full detail →](/cloud/connect#what-leaves-this-machine) + +--- + +## Where your data is stored + +FailproofAI Cloud runs two ways, and the choice is yours: + +- **Hosted**, at `app.befailproof.ai`. The fastest path: create a key, connect a machine, + done. +- **Self-hosted**, inside your own infrastructure. Events, prompts, model responses, and + analytics live in your own databases, in your own environment, and nothing is sent to a + third party for storage. + +Regulated environment, data-residency requirement, or an air-gapped network? Self-hosting +is the supported answer — [talk to +us](https://cal.com/nikita-agarwal-exosphere/30-minute-chat-failproof-ai). + +--- + +## Tenant isolation + +One deployment can host many organizations, and each is isolated **at the storage layer**, +enforced by the database rather than only by the UI: + +- An organization's operational data — users, keys, machines, policies, dashboards, saved + queries — is scoped to that organization, and cross-organization reads are blocked + underneath the application. +- Every ingested event is stamped with its owning organization, so one organization's + events can never be read by another. + +Every dashboard route is scoped under an organization slug (`//…`). + +--- + +## Sign-in + +Sign-in is **passwordless and email-based** — there is no password to phish, reuse, or +leak. A user requests a one-time code (or a one-click link), which is emailed and expires +quickly. + +Access is gated by an **allowlist**: only the email addresses or domains you permit can +authenticate at all. + +![The FailproofAI Cloud sign-in screen, which sends a single-use code to your email](/cloud/images/login.png) + +--- + +## Least-privilege API keys + +Every non-human client authenticates with a key that carries **granular permissions**, and +the intended pattern is narrow keys per job: + +| Job | Permissions it actually needs | +|---|---| +| A machine that reports activity | `events:add` | +| A machine that receives policy | `policies:pull` | +| A read-only integration | `events:read` and nothing else | + +Destructive actions — disable, regenerate, delete — are **separate grants** you choose to +include rather than side effects of a broad role. Keys are shown once at creation and +stored only as a hash, so a leaked list of keys is not a leaked set of credentials. + +Two permissions can never be held by an API key at all — instance administration and +editing another key's permissions. A bearer key may create keys; it may never re-scope +existing ones. + +![The API keys page: each key's grants, colour-coded by read, write, and destructive scope](/cloud/images/api-keys.png) + +[Full permission catalogue →](/cloud/access) + +--- + +## The AI assistant is constrained by construction + +The [in-dashboard assistant](/cloud/assistant) answers questions over your data, and its +limits are structural rather than prompt-based: + +- **Read-only by default.** Its SQL runs through a guard that permits only `SELECT` / + `WITH`, single-statement, with a row cap and a timeout. +- **It only sees what you can see.** Answers are scoped to your own read permissions — it + never widens your data surface. +- **Every write waits for you.** A saved query or dashboard it drafts is created only after + your explicit approval click. There is no setting that turns that gate off. +- **It can never delete.** No delete tool is exposed, and the assistant holds no delete + permission. +- **It stays inside one organization** — the one you are currently viewing. +- **Your questions stay yours.** Prompts and answers live in your own deployment's + database; product analytics records usage metadata only, never prompt text. + + + The [CLI agent skills](/cloud/agent-skills) are a different thing with a different blast + radius: they run on your workstation and drive the CLI **as you**, including writes and + deletes. Know which one you are handing to whom. + + +--- + +## In transit + +All traffic runs over HTTPS. On a self-hosted deployment you terminate TLS with your own +certificates, so both machine-to-server and browser-to-server traffic are encrypted. + +The CLI refuses to send a machine token to a non-`https` host — the single exception being +`localhost`, where there is no network to intercept. + +--- + +## Credentials on a machine + +Cloud tokens live in an **owner-only** file (`0600`) inside `~/.failproofai/`, and the +directory around it is tightened to match. They are deliberately **not** placed in the +background service's definition file: that file is installed world-readable, so a token +there would hand an organization-scoped key to every local user on the box. + +Practical consequences, all good ones: connecting, rotating a token, and disconnecting need +no root, and an already-running service can be connected without reinstalling anything. + +--- + +## Redaction is your choice, and it is honoured + +Two independent controls: + +- **Locally**, the [sanitize policies](/built-in-policies#secrets-sanitizers) scrub JWTs, + API keys, connection strings, private keys, and bearer tokens out of tool output *before + the model reads them*. On by default under Recommended. +- **In the SDK**, only the fields you explicitly pass are recorded. Nothing is read from + your process or captured implicitly, and any field you leave unset is not written at all. + If a payload contains something you would rather not store, strip it before you pass it. + + + Neither control makes a transcript safe by assumption. A transcript is the whole session + — treat it as sensitive, and use `--no-transcripts` on machines where centralizing that + content is not appropriate. + + +--- + +## Related + + + + + Every permission, what it gates, and the three keys most teams need. + + + + Exactly what a connected machine sends and receives. + + + + What is captured from your agent CLIs, and how to narrow it. + + + + How enforcement fails closed, and how credentials are stored. + + + diff --git a/docs/agenteye/sessions.mdx b/docs/cloud/sessions.mdx similarity index 71% rename from docs/agenteye/sessions.mdx rename to docs/cloud/sessions.mdx index 974d49ac..7baada7e 100644 --- a/docs/agenteye/sessions.mdx +++ b/docs/cloud/sessions.mdx @@ -1,12 +1,12 @@ --- -title: "Sessions & Execution Graph" +title: "Sessions" description: "Every event from a run, rolled into one readable row and drawn as a git-style execution graph you can read in seconds." --- -Stop guessing why a run failed. Failproof AI Observability rolls every event from a run into one readable row, then draws the whole run as a git-style picture you can read in seconds, so you see exactly what your agent did, step by step. +Stop guessing why a run failed. FailproofAI Cloud rolls every event from a run into one readable row, then draws the whole run as a git-style picture you can read in seconds, so you see exactly what your agent did, step by step. -![The Sessions list: one row per run, across environments and agents, with status pills and evaluation score badges](/agenteye/images/sessions-list.png) +![The Sessions list: one row per run, across environments and agents, with status pills and evaluation score badges](/cloud/images/sessions-list.png) *One row per run: the status pill tells you how the run ended at a glance, and a score badge rides along once an evaluator is connected.* @@ -30,7 +30,7 @@ Once you connect an evaluator, every completed run is scored automatically and i ## Read the whole run as a picture -![A session's git-style execution graph beside its event timeline, with the tool, model, and hook breakdown panel](/agenteye/images/session-detail.png) +![A session's git-style execution graph beside its event timeline, with the tool, model, and hook breakdown panel](/cloud/images/session-detail.png) *The execution graph (left) sits beside the event timeline; the right rail breaks down the tools, models, hooks, and token spend for the run.* @@ -38,7 +38,7 @@ Click any session to open its execution graph: a git-style view of how agents, t The right rail gives you the per-run breakdown: which tools and models ran, which hooks fired, and what the run spent in tokens. That is the answer to "why did this run cost so much?" or "which tool is the slow one?" sitting right next to the graph that caused it. -Individual events are addressable, so you can hand someone a link to one moment rather than "the session, about two thirds down". Copy the link from any event, or follow one from an [audit](/agenteye/audits) finding or an error, and the session opens with that event selected and scrolled to. This holds for very long runs too: the timeline loads a bounded window for the sake of your browser, and a link pointing past that window still finds its event rather than dropping you at the start. If the event has aged out of your retention window, the page tells you that instead of quietly selecting nothing. +Individual events are addressable, so you can hand someone a link to one moment rather than "the session, about two thirds down". Copy the link from any event, or follow one from an [audit](/cloud/audits) finding or an error, and the session opens with that event selected and scrolled to. This holds for very long runs too: the timeline loads a bounded window for the sake of your browser, and a link pointing past that window still finds its event rather than dropping you at the start. If the event has aged out of your retention window, the page tells you that instead of quietly selecting nothing. --- @@ -46,12 +46,12 @@ Individual events are addressable, so you can hand someone a link to one moment Every dashboard page is scoped to your org (`//…`). Sessions lives under **Observe** in the left sidebar, next to Events, with the date range, environment, agent, and session filters across the top of the list. Every row is one click from its full execution graph. -To turn on the score badges and score-range filtering, connect an evaluator: see [Evaluations](/agenteye/evaluations). +To turn on the score badges and score-range filtering, connect an evaluator: see [Evaluations](/cloud/evaluations). --- ## Related -- [Event stream](/agenteye/event-stream): the raw, per-step trail every session is rolled up from. -- [Evaluations](/agenteye/evaluations): connect an evaluator so each run gets a score badge you can filter by. -- [Telemetry](/agenteye/telemetry): how runs get from your agent into these sessions. +- [Event stream](/cloud/event-stream): the raw, per-step trail every session is rolled up from. +- [Evaluations](/cloud/evaluations): connect an evaluator so each run gets a score badge you can filter by. +- [Telemetry](/cloud/performance): how runs get from your agent into these sessions. diff --git a/docs/concepts.mdx b/docs/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/configuration.mdx b/docs/configuration.mdx index 12e9641d..3086f6bd 100644 --- a/docs/configuration.mdx +++ b/docs/configuration.mdx @@ -1,44 +1,55 @@ --- title: Configuration -description: "Config file format, three-scope system, and merge rules" +description: "The config files, the three scopes, how they merge, and every field you can set." icon: gear --- -failproofai uses JSON configuration files to control which policies are active, how they behave, and where custom policies are loaded from. Configuration is designed to be easy to share with your team - commit it to your repo and every developer gets the same agent safety net. +FailproofAI keeps two kinds of configuration, and they answer different questions: + +| File | Question it answers | Who writes it | +|---|---|---| +| `policies-config.json` | **Which policies run, and how?** | You, the dashboard, or `failproofai policy add` | +| Each agent CLI's own settings file | **When does FailproofAI get called at all?** | `failproofai config` / `policies --install` | + +You edit the first one freely. The second is managed for you — see [Supported +agents](/agent-support) for what gets written where. --- -## Configuration scopes +## The three scopes -There are three configuration scopes, evaluated in priority order: +Policy configuration is loaded from three places and merged, in priority order: -| Scope | File path | Purpose | -|-------|-----------|---------| +| Scope | Path | Purpose | +|---|---|---| | **project** | `.failproofai/policies-config.json` | Per-repo settings, committed to version control | | **local** | `.failproofai/policies-config.local.json` | Personal per-repo overrides, gitignored | -| **global** | `~/.failproofai/policies-config.json` | User-level defaults across all projects | +| **global** | `~/.failproofai/policies-config.json` | Your defaults across every project | -When failproofai receives a hook event, it loads and merges all three files that exist for the current working directory. +All three that exist for the current working directory are read on every hook event. +Changes take effect immediately — nothing to restart. ### Merge rules -**`enabledPolicies`** - the union of all three scopes. A policy enabled at any level is active. +**`enabledPolicies` — deduplicated union.** A policy enabled at any scope is on. ```text project: ["block-sudo"] local: ["block-rm-rf"] global: ["block-sudo", "sanitize-api-keys"] -resolved: ["block-sudo", "block-rm-rf", "sanitize-api-keys"] ← deduplicated union +resolved: ["block-sudo", "block-rm-rf", "sanitize-api-keys"] ``` -**`policyParams`** - first scope that defines params for a given policy wins entirely. There is no deep merging of values within a policy's params. +**`policyParams` — first scope that defines a policy wins, entirely.** There is no deep +merge inside one policy's parameter block, which keeps the resolved value something you +can predict by reading one file rather than three. ```text project: block-sudo → { allowPatterns: ["sudo apt-get update"] } global: block-sudo → { allowPatterns: ["sudo systemctl status"] } -resolved: { allowPatterns: ["sudo apt-get update"] } ← project wins, global ignored +resolved: { allowPatterns: ["sudo apt-get update"] } ← project wins outright ``` ```text @@ -46,22 +57,21 @@ project: (no block-sudo entry) local: (no block-sudo entry) global: block-sudo → { allowPatterns: ["sudo systemctl status"] } -resolved: { allowPatterns: ["sudo systemctl status"] } ← falls through to global +resolved: { allowPatterns: ["sudo systemctl status"] } ← falls through to global ``` -**`customPoliciesPaths` / `customPoliciesPath`** - first scope that defines either form wins. +**`customPoliciesPaths` / `customPoliciesPath` — first scope that defines either form wins.** -**`disabledCustomPolicies`** - union across all scopes. The dashboard writes a -source-qualified ID here when you switch off an individual policy from an -explicit or convention policy file. Policies not listed remain enabled by -default; IDs include the source file so same-named policies in multiple files -can be controlled independently. +**`disabledCustomPolicies` — union across all scopes.** The dashboard writes a +source-qualified id here when you switch off an individual policy from a custom or +convention file. Policies not listed stay enabled; the id includes its source file, so +same-named policies in different files are controlled independently. -**`llm`** - first scope that defines it wins. +**`llm` — first scope that defines it wins.** --- -## Config file format +## File format ```json { @@ -92,11 +102,9 @@ can be controlled independently. { "regex": "myco_[A-Za-z0-9]{32}", "label": "MyCo API key" } ] }, - "warn-large-file-write": { - "thresholdKb": 512 - } + "warn-large-file-write": { "thresholdKb": 512 } }, - "customPoliciesPath": "/home/alice/myproject/my-policies.js" + "customPoliciesPaths": ["/home/alice/myproject/my-policies.js"] } ``` @@ -106,174 +114,139 @@ can be controlled independently. ### `enabledPolicies` -Type: `string[]` - -List of policy names to enable. Names must match exactly the policy identifiers shown by `failproofai policies`. See [Built-in Policies](/built-in-policies) for the full list. +`string[]` — the policy names to enable. Names must match exactly what `failproofai +policies` prints. See [Built-in policies](/built-in-policies) for the catalogue. -Policies not in `enabledPolicies` are inactive, even if they have entries in `policyParams`. +A policy not listed here is inactive, even if it has an entry in `policyParams`. ### `policyParams` -Type: `Record>` - -Per-policy parameter overrides. The outer key is the policy name; the inner keys are policy-specific. Each policy documents its available parameters in [Built-in Policies](/built-in-policies). - -If a policy has parameters but you don't specify them, the policy's built-in defaults are used. Users who do not configure `policyParams` at all get identical behavior to previous versions. - -Unknown keys inside a policy's params block are silently ignored at hook-fire time but flagged as warnings when you run `failproofai policies`. - -#### `hint` (cross-cutting) +`Record>` — per-policy parameter overrides, keyed by policy +name. Each policy documents its own parameters. -Type: `string` (optional) +Omit a parameter and the policy's built-in default applies. Unknown keys inside a +policy's block are ignored at evaluation time and flagged as warnings when you run +`failproofai policies` — so a typo surfaces when you look, not never. -A message appended to the reason when a policy returns `deny` or `instruct`. Use it to give Claude actionable guidance without modifying the policy itself. +#### `hint` — accepted by every policy -Works with any policy type — built-in, custom (`custom/`), project convention (`.failproofai-project/`), or user convention (`.failproofai-user/`). +`string` — appended to the reason whenever a policy denies or instructs. It is the +zero-code way to make a generic rule speak your team's language: ```json { "policyParams": { - "block-force-push": { - "hint": "Try creating a fresh branch instead." - }, + "block-force-push": { "hint": "Branch off and open a PR instead." }, "block-sudo": { "allowPatterns": ["sudo apt-get"], - "hint": "Use apt-get directly without sudo." + "hint": "Use apt-get directly, without sudo." }, - "custom/my-policy": { - "hint": "Ask the user for approval first." - } + "custom/my-policy": { "hint": "Ask the user for approval first." } } } ``` -When `block-force-push` denies, Claude sees: *"Force-pushing is blocked. Try creating a fresh branch instead."* +The agent then reads *"Force-pushing is blocked. Branch off and open a PR instead."* -Non-string values and empty strings are silently ignored. If `hint` is not set, behavior is unchanged (backward-compatible). +Works on built-in, custom (`custom/…`), project-convention (`.failproofai-project/…`), and +user-convention (`.failproofai-user/…`) policies. Non-string and empty values are ignored. -### `customPoliciesPath` +### `customPoliciesPaths` -Type: `string` (absolute path) +`string[]` — absolute paths to your own policy files, loaded in order. Set for you by +`failproofai policies --install --custom ` (repeat the flag for several files). The +older single-value `customPoliciesPath` still works. -Path to a JavaScript file containing custom hook policies. This is set automatically by `failproofai policies --install --custom ` (the path is resolved to absolute before being stored). +Files are loaded fresh on every hook event — no caching, so editing a policy takes effect +on the next tool call. -The file is loaded fresh on every hook event - there is no caching. See [Custom Policies](/custom-policies) for authoring details. +### `llm` -### Convention-based policies +`object`, optional — connection details for policies that make model calls. Not needed for +most setups. -In addition to the explicit `customPoliciesPath`, failproofai automatically discovers and loads policy files from `.failproofai/policies/` directories: +```json +{ "llm": { "model": "claude-sonnet-4-6", "apiKey": "sk-ant-..." } } +``` -| Level | Directory | Scope | -|-------|-----------|-------| -| Project | `.failproofai/policies/` | Shared with team via version control | -| User | `~/.failproofai/policies/` | Personal, applies to all projects | +--- - - Drop your policies straight into `~/.failproofai/policies/`. The - `cloud-policies/` folder beside them holds policies your organisation deployed - to this machine — discovery does not descend into subdirectories, so it is - never scanned, and nothing you put in `policies/` can collide with it. - - If you are upgrading from a version that used - `~/.failproofai/policies/custom-policies/`, everything in that folder — your - policy files, any `lib/` of helpers they import, and any data files they read - — is moved back up automatically the first time you run a `failproofai` - command, and the command tells you what it moved. - +## Convention policies need no configuration at all -**File matching:** Only files matching `*policies.{js,mjs,ts}` are loaded (e.g. `security-policies.mjs`, `workflow-policies.js`). Other files in the directory are ignored. +Alongside the explicit paths above, FailproofAI discovers policy files by location: -**No config needed:** Convention policies require no entries in `policies-config.json`. Just drop files into the directory and they're picked up on the next hook event. +| Level | Directory | Shared how | +|---|---|---| +| Project | `.failproofai/policies/` | Committed to git — the whole team gets it | +| User | `~/.failproofai/policies/` | Personal, applies to every project | -**Union loading:** Both project and user convention directories are scanned. All matching files from both levels are loaded (unlike `customPoliciesPath` which uses first-scope-wins). +- **Matching:** only files named `*policies.{js,mjs,ts}` are loaded. Everything else in the + directory is ignored. +- **Union, not first-wins:** both directories are scanned and all matching files load. +- **No config entry needed.** Drop the file in; it is picked up on the next hook event. -See [Custom Policies](/custom-policies) for more details and examples. + + Put your own files directly in `~/.failproofai/policies/`. The `cloud-policies/` folder + beside them holds policies your organization deployed to this machine — discovery does + not descend into subdirectories, so the two can never collide. + -### `llm` +[Authoring guide →](/custom-policies) + +--- + +## Machine settings -Type: `object` (optional) +Policy configuration is about rules. A second file — `~/.failproofai/config.json` — holds +settings about *this machine*: whether the daemon is configured, what the collector sends, +extra session-capture paths, telemetry, and the audit schedule. -LLM client configuration for policies that make AI calls. Not required for most setups. +You rarely edit it by hand; `failproofai config`, `failproofai harness`, and the connect +flow write it. Two blocks are worth knowing: ```json { - "llm": { - "model": "claude-sonnet-4-6", - "apiKey": "sk-ant-..." - } + "audit": { "auto": true, "interval_days": 7 }, + "collector": { "sessions": true, "hooks": true, "environment": "production" } } ``` ---- - -## Managing configuration from the CLI - -The `policies --install` and `policies --uninstall` commands write to your agent CLI's hook settings file (the hook entry points), while `policies-config.json` is the file you manage directly. The two are separate: - -- **Agent CLI settings** — tells the agent to call `failproofai --hook ` on each tool use: - - **Claude Code**: `~/.claude/settings.json` (user), `/.claude/settings.json` (project), `/.claude/settings.local.json` (local) - - **OpenAI Codex**: `~/.codex/hooks.json` (user), `/.codex/hooks.json` (project) — Codex doesn't have a `local` scope - - **GitHub Copilot CLI _(beta)_**: `~/.copilot/hooks/failproofai.json` (user), `/.github/hooks/failproofai.json` (project) — Copilot has no `local` scope. Hook entries use Copilot's OS-keyed `bash`/`powershell` command fields with `timeoutSec`; the file carries a top-level `version: 1` marker. Copilot CLI support is **beta** while we verify the `events.jsonl` record schema (which the public docs do not specify) against more real-world sessions. **VS Code Copilot Chat agent mode (Preview)** reads hook configs from `.github/hooks/*.json`, `~/.copilot/hooks/*.json`, and `~/.claude/settings.json` (governed by the `chat.hookFilesLocations` setting) using the same Claude-shaped `{hookSpecificOutput:{permissionDecision:"deny",…}}` contract — the exact paths this `copilot` integration and the `claude` integration (`~/.claude/settings.json`) already write, so `failproofai policies --install --cli copilot` (or `--cli claude`) **already enforces in VS Code agent mode** with no separate `vscode` integration needed (confirmed live from VS Code's discovery logs). - - **Cursor Agent _(beta)_**: `~/.cursor/hooks.json` (user), `/.cursor/hooks.json` (project) — Cursor has no `local` scope. Hook entries use the Claude-shaped `{type, command, timeout}` form (no `bash`/`powershell` split), but stored under camelCase event keys (`preToolUse`, `beforeSubmitPrompt`, …) in a flat array per Cursor's [hooks schema](https://cursor.com/docs/hooks); the file carries a top-level `version: 1` marker. The handler canonicalizes camelCase → PascalCase via `CURSOR_EVENT_MAP` so existing built-in policies fire unchanged. Cursor Agent support is **beta** while we verify Cursor's transcript on-disk format (not specified in the public docs) against more real-world installs. - - **OpenCode _(beta)_**: `~/.config/opencode/opencode.json` + `~/.config/opencode/plugins/failproofai.mjs` (user), `/.opencode/opencode.json` + `/.opencode/plugins/failproofai.mjs` (project) — OpenCode has no `local` scope. Unlike the other five CLIs, OpenCode has **no external-command hook system**: it loads in-process JS/TS plugins explicitly registered via the `plugin: []` array in `opencode.json` (auto-discovery from `.opencode/plugins/` is **not** how plugins load on opencode v1.14.33). Install drops a small generated plugin shim that subprocess-calls the failproofai binary and translates the binary's Claude-shape JSON response back into plugin semantics: `throw new Error()` for tool-event deny (cancels the tool call), `client.session.prompt(...)` for instruct AND for `Stop` / `SubagentStop` deny (submits the deny reason as the next user message — the only force-retry channel since `session.idle` is notification-only and throwing from it is a no-op), and no-op for allow. The shim canonicalizes both tool names (lowercase → PascalCase via `OPENCODE_TOOL_MAP`) and tool-input arg keys (camelCase → snake_case via `OPENCODE_TOOL_INPUT_MAP` for `Read` / `Write` / `Edit`, e.g. `filePath` → `file_path`, `oldString` → `old_string`) before forwarding to the binary, so path-checking builtins like `block-read-outside-cwd`, `block-env-files`, and `block-secrets-write` fire unchanged on OpenCode tool calls. Sessions live in opencode's SQLite DB at `~/.local/share/opencode/opencode.db`; the dashboard's session viewer reads them via `opencode db --format json` and `opencode export `. OpenCode support is **beta** while we verify behavior across versions and against more real-world sessions. See the [OpenCode plugins docs](https://opencode.ai/docs/plugins/). - - **Pi _(beta)_**: `~/.pi/agent/settings.json` (user), `/.pi/settings.json` (project) — Pi has no `local` scope. Pi loads TypeScript extension packages at startup; the settings file is a flat string array `{"packages": ["./relative/path", …]}`. failproofai writes a single packages-array entry pointing at its bundled `pi-extension/` directory. The extension internally subscribes to Pi's `tool_call` / `user_bash` / `input` / `session_start` events and shells out to `failproofai --hook --cli pi`; the handler canonicalizes underscore_lower_snake_case → PascalCase via `PI_EVENT_MAP` so existing built-in policies fire unchanged. Tool input args are also canonicalized via `PI_TOOL_INPUT_MAP` (Pi's Read / Write / Edit deliver `path` rather than `file_path`; mapping the top-level key lets `block-env-files` and `block-secrets-write` fire — `block-read-outside-cwd` already had a `path` fallback). Pi support is **beta** while Pi's extension API and session-log layout stabilize. - - **Hermes (hermes-agent)**: `~/.hermes/config.yaml` (**user scope only** — Hermes has no project/local config). Hermes is a Slack/Telegram **gateway**, so one install intercepts tool calls from every platform (Slack/Telegram/cli/cron) **and** internal subagents. Hook entries are a `{command, timeout}` pair (timeout in **seconds**) under a `hooks:` map keyed by Hermes's snake_case events (`pre_tool_call` / `post_tool_call` / `on_session_start` / `on_session_end` / `subagent_stop`); the handler canonicalizes events via `HERMES_EVENT_MAP` and tool names via `HERMES_TOOL_MAP` so built-in policies fire unchanged. The config is edited through a comment-preserving YAML `Document` round-trip so the operator's other settings survive, and install sets `hooks_auto_accept: true` so the headless gateway (no TTY) runs the hooks without a consent prompt. The evaluator emits Hermes's `{"decision":"block","reason"}` stdout contract (Hermes ignores exit codes). **Limitations:** Hermes has no turn-end `Stop` event, so the `require-*-before-stop` builtins never fire for it (inapplicable, not broken); `instruct` degrades to allow-with-logged-note (no additional-context channel); and output-secret redaction (`sanitize-*`) can't rewrite tool output over the shell-hook contract. Hermes is **also** an offline **audit** source — the dashboard reads its gateway sessions directly from `~/.hermes/state.db`. - - **OpenClaw (openclaw gateway)**: `~/.openclaw/openclaw.json` (**user scope only** — OpenClaw has no project/local config). Like Hermes, OpenClaw is a self-hosted multi-channel **gateway**, so one install intercepts tool calls from every channel and its internal subagents. Enforcement runs through OpenClaw's **in-process plugin hooks** (its file-based internal hooks are observation-only and cannot block), so — like OpenCode/Pi — failproofai ships a static `openclaw-plugin/` package that async-spawns the failproofai binary and translates the verdict. Install registers the shipped plugin dir in `openclaw.json`'s `plugins.load.paths[]` and enables it under `plugins.entries.failproofai` (with `hooks.allowConversationAccess: true`, required for the raw-conversation hooks). The evaluator emits a flat `{permission, reason}` verdict and the shim maps it to each hook's native return shape: `before_tool_call → {block:true, blockReason}` (**PreToolUse**), `before_agent_run → {outcome:"block", reason}` (**UserPromptSubmit**), and `before_agent_finalize → {action:"revise", reason}` (**Stop** — a real turn-end gate, so the `require-*-before-stop` builtins **enforce** on OpenClaw, unlike Hermes). Events and tool names canonicalize binary-side via `OPENCLAW_EVENT_MAP` / `OPENCLAW_TOOL_MAP` (`exec→Bash`, `read→Read`, …) so built-in policies fire unchanged; the shim fails open on any spawn/parse/timeout error. OpenClaw is **also** an offline **audit** source — the dashboard reads its JSONL sessions at `~/.openclaw/agents//sessions/.jsonl`. - - **Factory Droid (`droid`)**: `~/.factory/hooks.json` (user), `/.factory/hooks.json` (project) — Factory has no `local` scope. droid ships a Claude-style external-command hook system, but with two quirks verified live against droid v0.171.0: (1) event names live at the **top level** of `hooks.json` — there is **no `"hooks"` wrapper** (droid rejects one); tool events (`PreToolUse`/`PostToolUse`) carry `"matcher": "*"`, non-tool events omit it. (2) Deny is driven by hook **exit code 2 + stderr**, not a JSON decision — the evaluator's `factory` branch returns exit 2 for tool/prompt events and `{decision:"block", reason}` only on the turn-end `Stop` event (droid's sole force-retry channel). Events are already PascalCase (no event map) and the payload is Claude snake_case; only tool names are canonicalized via `FACTORY_TOOL_MAP` (`Execute→Bash`, `Create→Write`, `FetchUrl→WebFetch`, …). Factory is **also** an offline **audit** source — the dashboard reads its on-disk JSONL sessions at `~/.factory/sessions//.jsonl`. - - **Devin CLI (`devin`, Cognition)**: `~/.config/devin/config.json` (user), `/.devin/config.json` (project) — Devin has no `local` scope. Devin is a **pure Claude-clone** verified live against devin v3000.1.27: it uses the standard Claude `"hooks"`-wrapper schema (writes are merge-preserving so the config file's other keys — `org_id`, `theme_mode`, … — survive), already-PascalCase event names (no event map, no handler branch), and a Claude snake_case stdin payload (no normalization). The evaluator's `devin` branch denies with `{"decision":"block","reason"}` JSON on stdout at exit 0 for **every** event (verified — the block overrode `--permission-mode dangerous`); on the turn-end `Stop` event the reason carries the MANDATORY-ACTION force-retry wording so the `require-*-before-stop` builtins enforce. Only tool names are canonicalized via `DEVIN_TOOL_MAP` (`exec→Bash`; `tool_input.command` is already canonical). Devin is **also** an offline **audit** source — the dashboard reads its SQLite sessions at `~/.local/share/devin/cli/sessions.db` (each `sessions` row carries a real `working_directory`, so sessions group by project cwd like Claude). - - **Antigravity CLI (`agy`)**: `~/.gemini/config/hooks.json` (user), `/.agents/hooks.json` (project) — Antigravity has no `local` scope. Unlike Factory/Devin, Antigravity has its **own** contract (not a Claude-clone), verified live against agy v1.1.2. `hooks.json` uses a **named-hook** schema: the top-level key is a hook *name* (`"failproofai"`) whose value is an event→handlers map — tool events (`PreToolUse`/`PostToolUse`) wrap handlers in `{matcher:"*", hooks:[…]}`, while `PreInvocation`/`Stop` are **flat** handler arrays (other named hooks are preserved). The stdin payload is **camelCase protojson** (`toolCall:{name,args}`, `conversationId`, `workspacePaths`, `transcriptPath`) — failproofai normalizes it to snake_case before policies run, and maps `run_command`'s PascalCase args (`CommandLine`/`Cwd`) via `ANTIGRAVITY_TOOL_INPUT_MAP`. The evaluator's `antigravity` branch uses Antigravity's **own** response shapes: `{decision:"deny", reason}` blocks a tool/prompt (exit 0), `{decision:"continue", reason}` on the turn-end `Stop` re-enters the loop (so the `require-*-before-stop` builtins enforce), and `{injectSteps:[{ephemeralMessage}]}` injects an instruction on `PreInvocation` (→ `UserPromptSubmit`). Tool names canonicalize via `ANTIGRAVITY_TOOL_MAP` (`run_command→Bash`, `view_file→Read`, …). Antigravity is **also** an offline **audit** source — the dashboard reads its plain-JSONL transcripts at `~/.gemini/antigravity-cli/brain//.system_generated/logs/transcript_full.jsonl` (conversation index in `conversation_summaries.db`). - - **Goose (codename goose, Block)**: `~/.agents/plugins/failproofai/hooks/hooks.json` (user), `/.agents/plugins/failproofai/hooks/hooks.json` (project) — Goose has no `local` scope. Enforcement uses Goose's **hooks** system, the cross-agent **Open Plugins** spec: the installer just drops the `failproofai` plugin dir and Goose auto-discovers it at startup (self-registering it into `~/.config/goose/config.yaml`). The `hooks.json` uses an Open Plugins schema **with** a top-level `"hooks"` wrapper, and the matcher is **omitted** on every event — a bare `"*"` is an invalid regex that matches nothing (verified live against goose v1.43.0). Event names are already PascalCase (no event map); the stdin payload uses `event`/`working_dir`, which the handler normalizes to `hook_event_name`/`cwd`. The evaluator's `goose` branch denies with `{"decision":"block","reason"}` JSON on stdout at exit 0, honored on the **`PreToolUse`** event only (shipped in goose ≥ v1.37.0) — which fires for the shell tool **and inside delegated subagents**, so it is the single sufficient deny point; any other hook error fails **open**. Goose has **no `Stop` event**, so the `require-*-before-stop` builtins don't apply (as with Hermes). Tool names canonicalize via `GOOSE_TOOL_MAP` (`shell→Bash`, `write→Write`, `todo__todo_write→TodoWrite`, …) and path keys via `GOOSE_TOOL_INPUT_MAP` (`path`/`source` → `file_path`). Goose is **also** an offline **audit** source — the dashboard reads its SQLite sessions at `~/.local/share/goose/sessions/sessions.db` (each `sessions` row carries a real `working_dir`, so sessions group by project cwd like Devin; `--no-session` scratch runs are filtered). -- **`policies-config.json`** — tells failproofai which policies to evaluate and with what params (shared across all agent CLIs) - -Pass `--cli claude|codex|copilot|cursor|opencode|pi|hermes|openclaw|factory|devin|antigravity|goose` to target a specific agent (space-separated or repeated for any subset): - -```bash -failproofai policies --install --cli codex --scope project -failproofai policies --install --cli copilot --scope project -failproofai policies --install --cli cursor --scope project -failproofai policies --install --cli opencode --scope project -failproofai policies --install --cli pi --scope project -failproofai policies --install --cli hermes --scope user -failproofai policies --install --cli openclaw --scope user -failproofai policies --install --cli factory --scope project -failproofai policies --install --cli devin --scope project -failproofai policies --install --cli antigravity --scope project -failproofai policies --install --cli goose --scope project -failproofai policies --install --cli claude codex copilot cursor opencode pi hermes openclaw factory devin antigravity goose -``` - -When `--cli` is omitted, `failproofai` detects which agent CLIs are installed (`which claude` / `which codex` / `which copilot` / `which cursor-agent` / `which opencode` / `which pi` / `which hermes` / `which openclaw` / `which droid` / `which devin` / `which agy` / `which goose`): +| Key | Meaning | +|---|---| +| `audit.auto` | Run the [audit](/audit) on a schedule. Off unless you set it, because it reads the contents of every transcript on the machine. | +| `audit.interval_days` | Days between scans. Clamped to 1–90; anything invalid falls back to 7. | +| `collector.sessions` | Ship session transcripts to the cloud. Set by `--no-transcripts` at connect time. | +| `collector.hooks` | Ship policy decisions to the cloud. | +| `collector.environment` | The environment label stamped on everything this machine reports. | -- **One CLI detected** — auto-selects that CLI without prompting. -- **Multiple CLIs detected** in an interactive terminal — shows an arrow-key single-select prompt grouped into a `Detected (N)` section (with an `Install for all N detected` aggregate row + each detected CLI individually) and a `Not installed (M) · install hooks ahead of time` section listing every undetected supported CLI as a forward-install option (↑↓ to move, Enter to select, ^C to quit). The uninstall flow shows only the Detected section. -- **Multiple CLIs detected** in a non-interactive run (CI, no TTY) — installs for all detected CLIs without prompting. -- **None detected** — falls back to `claude`, with a warning that no agent binary was found in PATH; the hook command is still written so it activates as soon as you install one. +[Full file-layout reference →](/reference/files) -You can edit `policies-config.json` directly at any time; changes take effect immediately on the next hook event with no restart needed. +--- ## Upgrades keep your configuration -A new version of failproofai may organise `~/.failproofai/` differently. When it does, the first command after the upgrade migrates the directory, and **your configuration is carried across, not reset**: +A new version may organize `~/.failproofai/` differently. When it does, the first command +after the upgrade migrates the directory and **carries your configuration across**: | Kept | Rebuilt | |---|---| -| Your policy selection and params (`policies-config.json`) | The audit cache | -| Your settings, including `daemon.configured` and extra capture paths (`config.json`) | Cloud-managed policy deployments — re-fetched and digest-verified on the next poll | -| Your cloud enrolment (`credentials.json`) | Daemon scratch state | -| Your own policy files in `policies/`, and the helpers they import | | -| The decision log the dashboard reads, and events not yet delivered | | - -Keys written by a *newer* failproofai are preserved as well, rather than being dropped by an older reader — so moving between versions does not silently discard settings either direction. +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Daemon scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered | | -You do **not** need to re-run setup afterwards: a migrated machine enforces exactly as it did before, which is what makes an upgrade safe on machines with nobody sitting at them. Every migration is recorded in `~/.failproofai/migrations/applied.json`, and the irreplaceable files are copied to `~/.failproofai/migrations/backup-layout/` before anything runs. +Keys written by a *newer* version are preserved rather than dropped by an older reader, so +moving between versions does not silently discard settings in either direction. You do not +need to re-run setup: a migrated machine enforces exactly as it did before. -See [`failproofai update`](/cli/update) for the one-line upgrade, and [`failproofai migrate`](/cli/migrate) — including `--dry-run` — for the details. +See [`failproofai update`](/cli/update) and [`failproofai migrate`](/cli/migrate). --- -## Example: project-level config with team defaults +## Example: a team standard, committed -Commit `.failproofai/policies-config.json` to your repo: +`.failproofai/policies-config.json`, checked in: ```json { @@ -286,10 +259,15 @@ Commit `.failproofai/policies-config.json` to your repo: ], "policyParams": { "block-push-master": { - "protectedBranches": ["main", "release", "hotfix"] + "protectedBranches": ["main", "release", "hotfix"], + "hint": "Open a PR from a feature branch." } } } ``` -Each developer can then create `.failproofai/policies-config.local.json` (gitignored) for personal overrides without affecting teammates. +Each developer can add `.failproofai/policies-config.local.json` (gitignored) for personal +overrides without touching anyone else's setup. + +Running this across many machines? [Deploy it from the cloud](/cloud/managed-policies) +instead and skip the git round-trip entirely. diff --git a/docs/custom-policies.mdx b/docs/custom-policies.mdx index f33a28ed..1149bb00 100644 --- a/docs/custom-policies.mdx +++ b/docs/custom-policies.mdx @@ -1,10 +1,16 @@ --- -title: Custom Policies +title: "Custom policies" description: "Write your own policies in JavaScript - enforce conventions, prevent drift, detect failures, integrate with external systems" icon: code --- -Custom policies let you write rules for any agent behavior: enforce project conventions, prevent drift, gate destructive operations, detect stuck agents, or integrate with Slack, approval workflows, and more. They use the same hook event system and `allow`, `deny`, `instruct` decisions as built-in policies. +Custom policies let you write rules for any agent behavior: enforce project conventions, prevent drift, gate destructive operations, detect stuck agents, or integrate with Slack, approval workflows, and more. They use the same hook event system and `allow`, `deny`, `instruct` decisions as [built-in policies](/built-in-policies). + + + New here? [Policies](/policies) covers the three decisions and where policies come from. + This page is the authoring reference. To roll your own rules out across a fleet without + a git round-trip, see [cloud-managed policies](/cloud/managed-policies). + --- @@ -198,9 +204,10 @@ customPolicies.add({ Policies are evaluated in this order: 1. Built-in policies (in definition order) -2. Explicit custom policies from `customPoliciesPath` (in `.add()` order) -3. Convention policies from project `.failproofai/policies/` (files alphabetical, `.add()` order within) -4. Convention policies from user `~/.failproofai/policies/` (files alphabetical, `.add()` order within) +2. [Cloud-managed policies](/cloud/managed-policies), if this machine is connected (digest-verified before each load) +3. Explicit custom policies from `customPoliciesPaths` (in `.add()` order) +4. Convention policies from project `.failproofai/policies/` (files alphabetical, `.add()` order within) +5. Convention policies from user `~/.failproofai/policies/` (files alphabetical, `.add()` order within) The first `deny` short-circuits all subsequent policies. All `instruct` messages are accumulated and delivered together. diff --git a/docs/daemon.mdx b/docs/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/dashboard.mdx b/docs/dashboard.mdx index 7e921a81..8488b1ad 100644 --- a/docs/dashboard.mdx +++ b/docs/dashboard.mdx @@ -1,151 +1,192 @@ --- -title: Dashboard -description: "Monitor agent sessions, review tool calls, and manage policies" +title: "Local dashboard" +description: "See what your agents did while you were away — every session, every tool call, and every policy decision — from a dashboard that runs on your own machine." icon: chart-line --- -The failproofai dashboard is a local web application for monitoring your AI agent sessions and managing policies. See what your agents did while you were away. - ---- - -## Starting the dashboard - ```bash failproofai ``` -Opens at `http://localhost:8020`. +Opens at `http://localhost:8020`. No account, no network, no configuration — it reads what +is already on the machine. -The dashboard reads local project, session, and failproofai configuration data directly from the filesystem. Optional authenticated features, such as audit reminders and invitations, send the information needed for those requests (including email addresses) to remote APIs. +This is the answer to the question you have every morning after leaving an agent running: +*what did it actually do?* --- -## Pages +## Projects + +Every project any supported agent CLI has touched on this machine, in one list — Claude +Code, Codex, Copilot, Cursor, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin, +Antigravity, and Goose. -### Projects +A project used by several CLIs is one row with several badges, so you see the work rather +than the tooling. Filter by CLI with the dropdown; the URL remembers your choice +(`?cli=claude`), so a filtered view is shareable. -Lists all Claude Code, OpenAI Codex, GitHub Copilot CLI _(beta)_, Cursor Agent _(beta)_, OpenCode _(beta)_, Pi _(beta)_, Hermes, OpenClaw, Factory Droid, Devin, Antigravity, and Goose projects found on your machine. Claude projects are discovered from `~/.claude/projects/` (or the path set by `CLAUDE_PROJECTS_PATH`); Codex projects are discovered by scanning every transcript under `~/.codex/sessions///
/*.jsonl` and grouping by the `cwd` recorded in each session's first record; Copilot CLI projects are discovered by scanning each `~/.copilot/session-state//workspace.yaml` (configurable via `COPILOT_HOME`) and grouping by its `cwd` field; Cursor Agent projects are discovered by scanning per-session metadata under `~/.cursor/agent-sessions//` (configurable via `CURSOR_HOME`, with `conversations/` and `sessions/` probed as fallbacks) for a `cwd` scalar in `meta.json` / `session.json` / `workspace.yaml`; OpenCode projects are discovered by querying its SQLite DB at `~/.local/share/opencode/opencode.db` via `opencode db --format json` (we read the `session` and `project` tables and group by `project_id`); Pi projects are discovered by scanning per-session JSONL transcripts under `~/.pi/agent/sessions//_.jsonl` (configurable via `PI_SESSIONS_DIR`) and pulling the `cwd` from each session's first record; Hermes gateway sessions are read directly from the SQLite store of every profile — `~/.hermes/state.db` plus `~/.hermes/profiles//state.db` (overridable via `HERMES_HOME`, or `HERMES_DB_PATH` for a single database) — and grouped into `hermes--` projects by profile and `source` (Slack/Telegram/cli/cron — gateway sessions have no cwd); OpenClaw gateway sessions are read from `~/.openclaw/agents//sessions/*.jsonl` and grouped into `openclaw--` projects by agent and channel (also cwd-less); Factory Droid projects are discovered from the JSONL transcripts at `~/.factory/sessions//*.jsonl` and grouped by cwd; Devin projects from its SQLite DB at `~/.local/share/devin/cli/sessions.db` (grouped by each session's `working_directory`); Antigravity projects from the JSONL transcripts at `~/.gemini/antigravity-cli/brain//…/transcript_full.jsonl` and grouped by cwd; and Goose projects from its SQLite DB at `~/.local/share/goose/sessions/sessions.db` (grouped by each session's `working_dir`). A project that has been used by multiple CLIs renders as a single row with all matching badges. Use the **CLI** dropdown above the table to filter by a specific agent CLI; the URL preserves your selection as `?cli=claude|codex|copilot|cursor|opencode|pi|hermes|openclaw|factory|devin|antigravity|goose`. +Gateways — **Hermes** and **OpenClaw** — have no working directory to group by, so they +render as a collapsible tree instead: profile (or agent) at the top, its channels +underneath. Folder rows roll up the session count and latest activity of everything below, +collapsed folders are remembered between visits, and a search expands whatever it matches. -Hermes and OpenClaw are user-scoped and have no working directory to group by, so they render as a **collapsible folder tree** — profile (or agent) at the top level, its channels beneath — while every cwd-based CLI stays a flat row. Folder rows roll up the session count and most recent activity of everything under them, collapsed folders are remembered between visits, and a keyword search expands whatever it matches. +--- + +## Sessions -Each project shows: -- Project name (derived from the folder path) -- A CLI badge — `Claude Code` (orange), `OpenAI Codex` (purple), `GitHub Copilot` (blue), `Cursor Agent` (emerald), `OpenCode` (amber), `Pi` (pink), and/or `Hermes` (indigo) -- Date of most recent session activity +Inside a project, one row per run: the session id, when it started and ended, how many tool +calls it made, and how many policies fired on it. -Click a project to see its sessions. +That last number is the one to scan. A session with 40 tool calls and 11 policy hits is a +different story from one with 40 and none. -### Sessions +Filter by date range, search by session id, and page through. Click any row to open it. -Lists all sessions within a project. Each session shows: -- Session ID -- Start and end timestamps -- Number of tool calls -- Hook activity count (policies that fired) +--- -Use the date range filter and session ID search to narrow the list. Sessions are paginated. +## The session viewer -Click a session to open the session viewer. +![The local session viewer: run stats across the top, a per-tool breakdown, then the full timeline of prompts, tool calls with their inputs and outputs, and the policy decisions that fired on them](/images/local-session-viewer.png) -### Session viewer +*One run, end to end: the stats bar, the tools it used, and every step it took.* -The session viewer answers the key question for autonomous agents: what did the agent do, and did it stay on track? A CLI badge beside the header indicates whether the session is a Claude Code, OpenAI Codex, GitHub Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin, Antigravity, or Goose transcript. It shows a timeline of everything that happened in a session: +The timeline of everything that happened, in order: -- **Messages** - Claude's text responses and user prompts -- **Tool calls** - Every tool Claude invoked, with its input and output -- **Policy activity** - For each tool call, which policies fired and what decision they returned +- **Messages** — the model's text and your prompts. +- **Tool calls** — every tool invoked, with its input and its output. +- **Policy activity** — for each call, which policies fired and what each decided. -The stats bar at the top shows session duration, total tool calls, and a summary of hook decisions (allow / deny / instruct counts). +The stats bar across the top gives you the run at a glance: duration, total tool calls, and +the allow / deny / instruct split. -Click the **Download Logs** button to export the session. For Claude Code, Codex, Copilot, Cursor, and Pi sessions you get the original on-disk JSONL transcript byte-for-byte; for OpenCode (whose sessions live in SQLite, not on disk) you get a JSON document mirroring the underlying `session` / `messages` / `parts` tables. +**Download Logs** exports the session. For CLIs that write transcripts to disk you get the +original file, byte for byte; for CLIs that keep sessions in a database you get a JSON +export mirroring the same content. + +--- -### Audit +## Audit -A personality-driven report of how your agent has actually been behaving across past sessions. Runs the same scan as the `failproofai audit` CLI but renders it as a single-screen shareable poster + four below-the-fold sections: +A read of how your agents have *actually* been behaving, across every session already on +this machine — an archetype, a 0–100 score, a ranked list of what slipped through, and a +copy-pasteable fix for each one. -1. **Poster** — fills the first viewport. Self-contained PNG-capture region with the failproof_ai wordmark + audit label · archetype index (`№ NN of 08`) + audit date · numeric score (0–100) + percentile rank pill (`top 15%`) · the archetype name (one of `the optimist`, `the cowboy`, `the explorer`, `the goldfish`, `the paranoid architect`, `the precision builder`, `the hammer`, `the ghost`) + 3-keyword strip · `// only N% of agents are this archetype` rarity line · 8×8 pixel sigil tile · `audit yours → failproof.ai` footer. Three share buttons sit just outside the capture box: `post your archetype` (X intent), `share on linkedin`, `download poster`. Capture runs through `html-to-image` so the PNG matches the on-screen render pixel-for-pixel (dashed borders, SVG logo mask, gradients, font metrics — all preserved). -2. **Strengths** — calm ✓ row list of behaviors your agent already does right, derived from the live audit data (clean tool-call rate, no direct pushes to main, zero credential leaks, zero retry storms) — each surfaced only when the relevant policy has a clean record across the audit window. -3. **Quirks** — table of what slipped through, ranked by severity: `when · what slipped + the policy that would've caught it · severity pill · seen`, where the recurrence reads `new` (once), `N× seen` (2–9 times), or `recurring` (10+). -4. **How to improve** — calm row list, one per prescribed policy: policy name in white, one-line description, install command + copy button on the right side. The section header reads `enable all N → projected · ` (the score you'd reach with every fix applied), and its `[install all]` button copies the combined `failproofai policy add a b c …` command for every prescribed policy. -5. **Come back better** — two side-by-side cards. Left: set a reminder (`3d` / `7d` / `14d` / `30d` cadence picker; persists through `/api/auth/reminder` once authed). Right: unlock failproof perks — `invite a friend` opens a modal that takes a comma/space/newline-separated list of friend emails (max 10 per send), POSTs them to `/api/audit/invite`, which forwards to the api-server's `POST /v0/invite`. The api-server sends one email per recipient from `invite@failproof.ai` with the sender Cc'd and `Reply-To` set, so the recipient sees who invited them and the sender gets a copy in their inbox. Anonymous users get routed through the `AuthDialog` first so the sender's email is known before invites go out. Entitlement / perks fulfillment is a follow-up. +It is the fastest way to find out which policies you should have turned on. +[Full guide →](/audit) -Driven by the `failproofai audit` runtime — see [Audit CLI](/cli/audit) for the underlying scan engine, supported flags, and per-transcript cache invariants. The dashboard caches the latest result at `~/.failproofai/audit-dashboard.json` (mode `0600`, single slot, new runs overwrite) so revisits are instant; **both the per-transcript and whole-result caches are rejected on read once they're older than 7 days** so the dashboard never silently serves a week-old result — past the TTL `/audit` falls through to its empty state and prompts a fresh run. Clicking `[ re-audit now ]` near the bottom of the report POSTs `/api/audit/run` with `noCache: true` — re-audit bypasses the per-transcript cache and re-scans every transcript from scratch rather than silently returning the cached result — and the dashboard polls `/api/audit/status` at 1Hz until the run finishes; a sticky pink progress strip pins to the top of the viewport during the run with an elapsed timer, and the fresh result swaps in place on success (no full-page reload; a failed re-audit leaves the prior report intact). On failure the strip turns red with copy keyed off the `RerunError.kind` (`timeout` / `network` / `post_failed`). Empty state (no cache or expired) and zero-sessions state (cache exists but the scan found no transcripts) are surfaced separately. +--- -### Policies +## Policies -A two-tab page for managing policies and reviewing activity. +Two tabs, and between them this is where most people manage FailproofAI day to day. - - Multi-select which agent CLIs failproofai protects from a single panel — Claude Code, OpenAI Codex, GitHub Copilot, Cursor Agent, OpenCode, Pi, and Hermes all have a row with install status (`Active` / `Detected` / `Inactive`), the user-scope settings path, and a brand-colored accent. Check or uncheck the CLIs you want and click `Apply changes` to install/uninstall the diff in one step. CLIs whose binary is detected on PATH are pre-checked. - - Toggle individual policies on or off with a single click (writes to `~/.failproofai/policies-config.json` — shared across every installed CLI) - - Expand a policy to configure its parameters (for policies that support `policyParams`) - - Set a custom policies file path + - **Pick which agent CLIs to protect**, from one panel. Each supported CLI has a row with + its install status (`Active` / `Detected` / `Inactive`) and its settings path. Check the + ones you want and apply the whole diff in one step. CLIs found on your `PATH` are + pre-checked. + - **Toggle any policy** on or off with a click. Writes to + `~/.failproofai/policies-config.json`, shared across every installed CLI. + - **Expand a policy** to edit its parameters — allowlists, protected branches, thresholds + — without touching a file. + - **Point at your own policy files.** - - Full paginated history of every hook event that has fired across all sessions - - Filter by decision, event type, CLI (Claude Code / OpenAI Codex / GitHub Copilot _(beta)_ / Cursor Agent _(beta)_ / OpenCode _(beta)_ / Pi _(beta)_ / Hermes / OpenClaw / Factory Droid / Devin / Antigravity / Goose), policy name, or session ID - - Each row shows: timestamp, policy name, decision, CLI badge (orange = Claude Code, purple = OpenAI Codex, blue = GitHub Copilot, emerald = Cursor Agent, amber = OpenCode, pink = Pi, indigo = Hermes, teal = OpenClaw, rose = Factory Droid, violet = Devin, cyan = Antigravity, lime = Goose), tool name, session ID, and the reason for deny/instruct decisions - - Click a session ID to open its transcript — the viewer auto-detects which CLI fired the hook (Claude `~/.claude/projects/…`, Codex `~/.codex/sessions/…`, Copilot CLI `~/.copilot/session-state//events.jsonl`, Cursor Agent `~/.cursor/agent-sessions//events.jsonl`, OpenCode `~/.local/share/opencode/opencode.db`, Pi `~/.pi/agent/sessions//.jsonl`, Hermes `~/.hermes/state.db`, OpenClaw `~/.openclaw/agents//sessions/*.jsonl`, Factory Droid `~/.factory/sessions//.jsonl`, Devin `~/.local/share/devin/cli/sessions.db`, Antigravity `~/.gemini/antigravity-cli/brain//…/transcript_full.jsonl`, Goose `~/.local/share/goose/sessions/sessions.db`) and renders the matching CLI badge in the header + - The **full paginated history** of every hook decision across every session. + - **Filter** by decision, event type, CLI, policy name, or session id. + - Each row shows the timestamp, policy, decision, CLI badge, tool, session, and the reason + for a deny or instruct. + - **Click a session id** to open its transcript — the viewer works out which CLI produced + it and renders the matching badge. --- -## Auto-refresh +## Watching a long run -The dashboard has an auto-refresh toggle in the top navigation. When enabled, the current page refreshes periodically to show new sessions and policy activity as they appear. Essential for monitoring long-running autonomous agent sessions. +Turn on **auto-refresh** in the top navigation and the current page updates as new sessions +and decisions land. That is what makes this usable as a live monitor for an agent you left +running, rather than only a post-mortem tool. --- -## Disabling pages - -If you only need some parts of the dashboard, set `FAILPROOFAI_DISABLE_PAGES` to a comma-separated list of page names: +## Options ```bash -FAILPROOFAI_DISABLE_PAGES=policies failproofai +FAILPROOFAI_DISABLE_PAGES=policies failproofai # hide pages you don't want +CLAUDE_PROJECTS_PATH=/custom/path failproofai # non-standard project location ``` -Valid values: `policies`, `projects`, `audit`. - ---- - -## Configuring the projects path +`FAILPROOFAI_DISABLE_PAGES` accepts a comma-separated list of `policies`, `projects`, and +`audit`. -By default, the dashboard reads from the standard Claude Code projects directory. Override it for custom setups: - -```bash -CLAUDE_PROJECTS_PATH=/custom/path/to/projects failproofai -``` +For transcripts in an unusual location, [`failproofai harness add-path`](/cli/harness) is +the general answer — it works for every CLI, not just Claude Code, and the daemon uses it +too. [All environment variables →](/cli/environment-variables) --- -## Accessing from a non-localhost host +## Accessing it from another host -When running the dashboard in **dev mode** (`npm run dev`) and accessing it from a hostname other than `localhost` - for example, a custom domain, a remote IP, or a tunneled URL - you may see a warning like: +Running the dashboard in **dev mode** (`npm run dev`) and opening it from a hostname other +than `localhost` — a custom domain, a remote IP, a tunnel — trips Next.js's cross-origin +guard on its hot-reload socket: ```text ⚠ Blocked cross-origin request to Next.js dev resource /_next/webpack-hmr from "dashboard.example.com". ``` -This is Next.js blocking cross-origin access to its HMR (hot module reload) websocket, which is a dev-only feature. To allow your host, use the `--allowed-origins` flag: +Allow your host: ```bash -npm run dev -- --allowed-origins dashboard.example.com +npm run dev -- --allowed-origins dashboard.example.com,192.168.1.5 +# or +FAILPROOFAI_ALLOWED_DEV_ORIGINS=dashboard.example.com npm run dev ``` -For multiple hosts or IPs, pass a comma-separated list: + + Dev mode only. Running `failproofai` normally has no hot-reload socket and no cross-origin + issue. + -```bash -npm run dev -- --allowed-origins dashboard.example.com,192.168.1.5 -``` +--- + +## When one machine isn't enough -You can also set the `FAILPROOFAI_ALLOWED_DEV_ORIGINS` environment variable instead: +This dashboard reads one machine. The moment you care about what agents did across your +team, you want [FailproofAI Cloud](/cloud/overview) — the same picture for every machine at +once, plus [central policy](/cloud/managed-policies), [quality scores](/cloud/evaluations), +and [alerts](/cloud/alerts). ```bash -FAILPROOFAI_ALLOWED_DEV_ORIGINS=dashboard.example.com npm run dev +failproofai config --connect https://app.befailproof.ai --token ``` - -This only applies to dev mode. When running `failproofai` (production mode), there is no HMR websocket and no cross-origin dev resource issue. - +The local dashboard keeps working exactly as before. + +--- + +## Related + + + + + A scored report of the habits your agents already have. + + + + What you are toggling on that page. + + + + Every CLI whose sessions show up here. + + + + The same view, across every machine. + + + diff --git a/docs/de/agent-support.mdx b/docs/de/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/de/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/de/agenteye/cli-and-agents.mdx b/docs/de/agenteye/cli-and-agents.mdx deleted file mode 100644 index b8425520..00000000 --- a/docs/de/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "Ihr gesamtes Failproof AI Observability-Deployment, einen Befehl entfernt." ---- - - -Ihr gesamtes Failproof AI Observability-Deployment, einen Befehl entfernt. Prüfen Sie die Produktion, erstellen Sie einen API-Schlüssel oder bestätigen Sie einen Vorfall, ohne Ihr Terminal zu verlassen – und automatisieren Sie alles in CI oder lassen Sie einen Coding-Agenten es auf Englisch erledigen. - -```bash -pipx install agenteye -agenteye login --email you@example.com # a 6-digit code lands in your inbox -agenteye --json sessions --since 24h # every agent run from the last day, newest first -``` - -*Das `agenteye` CLI kommuniziert mit Ihrem Dashboard. Es ist ein anderes Werkzeug als der Collector, der Events an den Server sendet.* - -## Ihr gesamtes Deployment, einen Befehl entfernt - -Hören Sie auf, zwischen Tabs zu wechseln, um eine schnelle Frage zu beantworten. Das `agenteye` CLI liest Ihre Daten und verwaltet Ihre Organisation aus einer einzigen Binary heraus – eine Überprüfung, die früher das Durchklicken des Dashboards erforderte, ist jetzt eine einzige Zeile, die Sie erneut ausführen, als Alias anlegen oder in ein Runbook einfügen können. Sie erhalten vier Bereiche: - -- **Daten lesen:** `sessions`, `events`, `evals` und `errors`, gefiltert nach Zeit, Agent und Umgebung. -- **Organisation verwalten:** `keys`, `users`, `settings`, `alerts` und `incidents`. -- **Analysen ausführen:** gespeichertes SQL sowie ein Ad-hoc-`query`-Runner über Ihre Event-Daten. -- **Den Assistenten befragen:** `agent ask` erreicht denselben schreibgeschützten Analysten, mit dem Sie im Dashboard chatten. - -Installieren Sie es einmalig mit `pipx`, melden Sie sich mit einem per E-Mail zugesandten 6-stelligen Code an, und Sie sind startklar. Die Sitzung dauert etwa einen Tag; führen Sie `agenteye login` erneut aus, wenn sie abläuft. Nutzen Sie es für schnelle Produktionsprüfungen, das Bereitstellen eines Schlüssels oder die Triage eines aktiven Vorfalls – alles ohne Browser: - -```bash -agenteye errors --since 24h --aggregate # what is breaking, grouped by error type -agenteye incidents list --state firing # what is on fire right now -agenteye keys create ci --add events:add # a key that can only push events, secret shown once -``` - -Eine wichtige Konvention: Globale Optionen wie `--json` stehen vor dem Befehl. `agenteye --json sessions` ist korrekt; `agenteye sessions --json` ist es nicht. - -## Skripte und CI-Integration - -Jeder Befehl akzeptiert `--json`, und das ändert alles. Sauberes JSON geht nach stdout, während Statusmeldungen und Warnungen für Menschen nach stderr gehen – ein `--json`-Output lässt sich also direkt in `jq` pipen, ohne störende Zeilen herausfiltern zu müssen. Das macht das CLI gleichermaßen nützlich für Sie an der Eingabeaufforderung und für einen Coding-Agenten, der die Ausgabe verarbeitet: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -Es ist für den unbeaufsichtigten Betrieb ausgelegt. Bestätigungsabfragen werden automatisch übersprungen, wenn kein Terminal angebunden ist – nichts bleibt in einer Pipeline hängen –, und jeder Befehl gibt einen aussagekräftigen Exit-Code zurück: `0` Erfolg, `4` nicht eingeloggt, `5` fehlende Berechtigung (die Meldung nennt sie, z. B. `alerts:write`), `3` Dashboard nicht erreichbar. Ein Skript kann bei `4` eine erneute Authentifizierung einleiten oder bei `5` genau sagen, was bei einem Administrator anzufragen ist, anstatt ohne Rückmeldung zu scheitern. - -## Einen Coding-Agenten auf Englisch steuern lassen - -Noch besser: Sie sollten sich all diese Flags gar nicht merken müssen. Die **CLI-Skill** ist ein kleiner Agent-Skill-Ordner namens `agenteye-cli`, der einen Coding-Agenten wie Claude Code oder Codex beibringt, das CLI auf Basis von Anfragen in natürlicher Sprache zu bedienen. Fragen Sie „Ist heute etwas defekt?" und der Agent wählt den passenden Befehl, führt ihn in Ihrem Namen aus und antwortet in Prosa. - -Für Claude Code legen Sie den `agenteye-cli`-Ordner in `~/.claude/skills/` ab – er wird automatisch erkannt. Failproof AI Observability stellt den Ordner bereit; es ist nichts Zusätzliches zu installieren, da er nur das bereits installierte CLI steuert. Melden Sie sich zunächst selbst an: Die Skill kann den Login per E-Mail-Code nicht für Sie abschließen. - -Da der Agent das CLI unter Ihrer Identität ausführt, kann er alles tun, was Ihr Login erlaubt – Lesen und Schreiben gleichermaßen: Schlüssel erstellen, Einstellungen ändern, Vorfälle auflösen. Die „Sind Sie sicher?"-Abfrage des CLI wird für einen Agenten nicht ausgelöst, daher ist die Skill so gestaltet, dass sie den genauen Befehl nennt und auf Ihre Zustimmung wartet, bevor eine Änderung vorgenommen wird. Sie sind der Bestätigungsschritt. - -```text -you Why did session run-001 fail? - -agent Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -Lesevorgänge bleiben sofort, und jeder Schreibvorgang wartet auf Sie: - -```text -you Give CI a key that can only push events. - -agent This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you yes - -agent Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## Weiterführendes - -- [CLI-Referenz](/de/agenteye/cli): Alle Befehle, Flags und JSON-Strukturen. -- [CLI-Rezepte für Agenten](/de/agenteye/cli-recipes): Kopierfertige `jq`-Muster und Exit-Code-Behandlung. -- [CLI-Agent-Skill](/de/agenteye/cli-skill): Installation und Verwendung der `agenteye-cli`-Skill. -- [KI-Assistent](/de/agenteye/assistant): Der Dashboard-Analyst, mit dem `agent ask` kommuniziert. \ No newline at end of file diff --git a/docs/de/agenteye/cli-skill.mdx b/docs/de/agenteye/cli-skill.mdx deleted file mode 100644 index 3a39c7e2..00000000 --- a/docs/de/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Failproof AI Observability CLI Agent Skill" -description: "Fragen Sie Ihren Coding-Agenten, ob heute etwas nicht funktioniert, und lassen Sie ihn die Antwort aus Ihren Live-Failproof AI Observability-Daten beziehen – ohne Befehle auswendig lernen zu müssen." ---- - - -Fragen Sie Ihren Coding-Agenten *„Ist heute irgendetwas kaputt?"* und lassen Sie ihn die Antwort aus Ihren Live-Failproof AI Observability-Daten beziehen – ohne Befehle auswendig lernen zu müssen. Der **Failproof AI Observability CLI Skill** (`agenteye-cli`) ist ein *Agent Skill*: ein kleiner Ordner mit Anweisungen, den ein Coding-Agent wie Claude Code oder Codex bei Bedarf lädt. Er bringt dem Agenten bei, Ihre Observability-Deployment über die [`agenteye` CLI](/de/agenteye/cli) anhand von Anfragen in normalem Englisch zu bedienen – etwa *„Gib CI einen Schlüssel, der nur Events pushen kann"* oder *„Bestätige den ausgelösten Incident und weise ihn mir zu."* - -Es handelt sich **nicht** um einen Dienst oder eine separate Binärdatei; es gibt nichts zu deployen. Es setzt auf der bereits installierten CLI auf: Der Agent ruft `agenteye --json …` auf, analysiert das saubere JSON und antwortet Ihnen in Prosaform. Alles, was er tun kann, könnten Sie selbst durch Eingabe derselben Befehle tun. - ---- - -## Verhältnis zu den anderen Failproof AI Observability-Schnittstellen - -Failproof AI Observability bietet Ihnen vier Wege, um auf dieselben Daten und Steuerungsmöglichkeiten zuzugreifen. Sie ergänzen sich gegenseitig: - -| Schnittstelle | Was es ist | Wo es läuft | Verwenden Sie es, wenn | -|---|---|---|---| -| **[CLI](/de/agenteye/cli)** | Die Befehls-/Flag-Referenz für `agenteye` | Ihr Terminal | Sie einen bestimmten Befehl ausführen oder skripten möchten | -| **[CLI-Rezepte](/de/agenteye/cli-recipes)** | Copy-paste-`jq`/Pipeline-Muster | Ihr Terminal / Skripte | Sie die CLI in Automatisierungen einbinden | -| **CLI Skill** (dieses Dokument) | Eine natürlichsprachige Eingabetür zur CLI | Ihr Coding-Agent, auf Ihrer Workstation | Sie einfach fragen und den Agenten den Befehl wählen lassen möchten | -| **[Evaluator Skill](/de/agenteye/evaluator-skill)** | Ein verwandter Skill, der Ihren Scoring-Dienst entwirft und aufbaut | Ihr Coding-Agent, auf Ihrer Workstation | Sie Eval-Scores *erstellen* möchten, anstatt sie zu lesen | -| **[Python SDK Skill](/de/agenteye/python-sdk-skill)** | Ein verwandter Skill, der Ihren Agenten instrumentiert, damit er überhaupt Telemetrie aussendet | Ihr Coding-Agent, auf Ihrer Workstation | Ihr Agent die Events *erzeugen* soll, die dieser Skill liest | -| **[In-Dashboard-KI-Assistent](/de/agenteye/assistant)** | Ein im Dashboard eingebetteter Chat | Serverseitig (im Dashboard) | Sie Q&A über Ihre Daten direkt im Dashboard wünschen | - -Der Skill selbst hat keine eigenen Rechte; er übersetzt lediglich Ihre Worte in CLI-Aufrufe, die als Sie ausgeführt werden: - -```mermaid -flowchart TD - YOU["you: 'ack the firing incident'"] --> AGENT["coding agent (Claude Code / Codex)
loads the agenteye-cli skill"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|your authenticated CLI session| API["Observability dashboard API"] -``` - -### vs. dem In-Dashboard-KI-Assistenten: ein wichtiger Unterschied - -Dies sind zwei verschiedene Tools mit sehr unterschiedlichem Wirkungsradius: - -- Der **In-Dashboard-KI-Assistent** ([KI-Assistent](/de/agenteye/assistant)) ist ein im Dashboard eingebetteter Chat, der vom Agenten-Dienst unterstützt wird. Er ist **lesend plus genehmigungspflichtig beim Erstellen**: Er kann gespeicherte Abfragen und Dashboards entwerfen, aber jeder Schreibvorgang pausiert für Ihre ausdrückliche Klickgenehmigung, und er löscht nie. Er ist durch die Berechtigung `agent:use` geschützt und sieht immer nur Daten für die Organisation, die Sie gerade ansehen. -- Der **CLI Skill** läuft auf *Ihrer* Workstation innerhalb *Ihres* Coding-Agenten und steuert die `agenteye` CLI **als Sie**. Er kann die **gesamte CLI-Oberfläche nutzen, einschließlich Mutationen** (API-Schlüssel erstellen/rotieren/deaktivieren, Org-Einstellungen ändern, Incidents auflösen, gespeicherte Abfragen löschen) – begrenzt nur durch die Berechtigungen Ihres CLI-Logins. Gehen Sie damit genauso sorgfältig um, wie Sie diese Befehle manuell eingeben würden. - ---- - -## Voraussetzungen - -1. Die **`agenteye` CLI ist installiert** und im `PATH` (siehe [CLI](/de/agenteye/cli)-Referenz: `pipx install agenteye`). -2. Ihre **Dashboard-URL** ist gesetzt (`AGENTEYE_DASHBOARD_URL`, oder der Agent übergibt `--base-url`). -3. Eine **eingeloggte Sitzung**: Führen Sie `agenteye login` selbst zuerst aus. Der Skill **kann** den per E-Mail versendeten Einmalcode-Login nicht für Sie abschließen; er wird Sie auffordern, `agenteye login` auszuführen, wenn die Sitzung fehlt oder abgelaufen ist (CLI-Exit-Code `4`). - ---- - -## Wo Sie ihn bekommen - -Der Skill ist in Failproof AIs öffentlicher Skills-Sammlung veröffentlicht: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -Nichts daran ist gesperrt – das Repository ist öffentlich, und der Skill benötigt keine eigenen Anmeldeinformationen, da er nur die **öffentliche** `agenteye` CLI gegen *Ihr* Dashboard treibt und dabei die Sitzung verwendet, mit der *Sie* eingeloggt sind. Sie müssen niemanden darum bitten. - -Beachten Sie, dass er als eigener Ordner ausgeliefert wird und **nicht** im `pipx install agenteye`-Paket enthalten ist – suchen Sie dort also nicht danach. - -## Den Skill installieren - -Der schnellste Weg ist die [`skills`](https://skills.sh) CLI, die den Ordner holt und dort ablegt, wo Ihr Agent sucht: - -```bash -# Claude Code, nur dieses Projekt -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# jedes Projekt (installiert nach ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# stattdessen Codex -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -Verwalten Sie ihn dann wie jeden anderen Skill: - -```bash -npx skills list -a claude-code # was ist installiert -npx skills update agenteye-cli # neueste Version holen -npx skills remove agenteye-cli # entfernen -``` - -Möchten Sie lieber manuell installieren? Ein Agent Skill ist nur ein Ordner mit einer `SKILL.md` (plus optionalen Referenzen), daher funktioniert auch das Kopieren: - -- **Claude Code**: Legen Sie den Ordner `agenteye-cli/` in `~/.claude/skills/` (jedes Projekt) oder `/.claude/skills/` (nur dieses Repository). Claude Code erkennt ihn automatisch – überprüfen Sie es mit der `/skills`-Liste oder stellen Sie einfach eine Frage, die zu seiner Beschreibung passt. -- **Codex (OpenAI)**: Codex liest dieselbe `SKILL.md`. Das enthaltene `agents/openai.yaml` setzt `allow_implicit_invocation: true`, sodass Codex den Skill automatisch auswählt, wenn eine Aufgabe passt; andernfalls rufen Sie ihn explizit als `$agenteye-cli` auf. - ---- - -## Sicherheit: Mutationen zeigen KEINE Bestätigungsabfrage, wenn ein Agent die CLI ausführt - -> **Warnung:** Lesen Sie dies, bevor Sie einen Agenten Änderungen vornehmen lassen. - -Die `agenteye` CLI fragt normalerweise *„Sind Sie sicher?"* vor einer destruktiven Aktion. Sie **überspringt diese Bestätigung automatisch, wenn sie nicht an ein Terminal angehängt ist (was genau der Fall ist, wenn ein Coding-Agent sie ausführt), und `--json` überspringt sie ebenfalls.** Die Sicherheitsabfrage wird für den Agenten daher **nicht** ausgelöst. - -Der Skill ist so geschrieben, dass er dies ausgleicht: Er ist angewiesen, den genauen Befehl anzugeben, den er ausführen wird, und Ihre ausdrückliche **Zustimmung vor jeder Zustandsänderung** einzuholen. Halten Sie diese Disziplin aufrecht. Wenn Sie Failproof AI Observability über einen Agenten steuern, *sind Sie* der Bestätigungsschritt. Die zustandsändernden Befehle, auf die Sie achten sollten: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- die schreibenden `incidents`-Unterbefehle: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -Alles unter **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) ist schreibgeschützt und ändert nichts. - -Da der Agent **als Sie** agiert, kann er nur das tun, was Ihr Login erlaubt; Berechtigungen werden **pro Org** aufgelöst (siehe [API-Schlüssel](/de/agenteye/api-keys)). Ein Befehl, für den Sie keine Berechtigung haben, gibt Exit-Code `5` mit dem genauen Berechtigungsnamen zurück, sodass der Agent Ihnen genau sagen kann, was Sie einen Administrator fragen müssen, anstatt undurchsichtig zu scheitern. - ---- - -## Was Sie fragen können - -So sieht ein echter Ablauf aus – zuerst eine Leseanfrage, dann eine Änderung, die auf Ihre Zustimmung wartet: - -```text -you ▸ Is anything broken in the last day? - -agent ▸ Running: agenteye --json errors --since 24h --aggregate - 3 error types in the last 24h. The biggest is TimeoutError - (18 hits, all on the checkout-agent), then RateLimited (5). - Want the failing sessions for TimeoutError? - -you ▸ Give CI a key that can only push events. - -agent ▸ This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you ▸ yes - -agent ▸ Done. Key "ci" created with events:add only. - The secret is shown only once, so store it now. I can't reprint it. -``` - -Der Skill ordnet jede natürlichsprachige Absicht dem richtigen `agenteye`-Befehl zu, ermittelt dabei zuerst gültige Werte (`list `, `whoami`), rät nicht und gibt den genauen Befehl vor jeder Änderung an. Weitere Beispiele: - -- *„Ist irgendetwas kaputt / fehlgeschlagen in den letzten 24 Stunden?"* → `errors --since 24h --aggregate`, dann eine Aufschlüsselung. -- *„Warum ist Sitzung `run-001` fehlgeschlagen?"* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *„Wie entwickelt sich die Qualität diese Woche?"* → `evals --aggregate --since 7d`, dann Drilldown in schlecht bewertete Läufe. -- *„Gib CI einen Schlüssel, der nur Events pushen kann."* → `keys create ci --add events:add` (der Befehl wird angegeben, dann erstellt und das einmalige Secret erfasst). -- *„Wer hat Zugriff? Mache Dana schreibgeschützt."* → `users list` → `users update dana@… --permission-set read-only` (nach Ihrer Bestätigung). -- *„Bestätige den ausgelösten Incident und weise ihn mir zu."* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`. - -Die genauen Befehle, Flags und JSON-Strukturen hinter diesen Beispielen finden Sie in der [CLI](/de/agenteye/cli)-Referenz und den [CLI-Rezepten für Agenten](/de/agenteye/cli-recipes). - ---- - -## Nächste Schritte - -- **[CLI](/de/agenteye/cli)**: vollständige Befehls- und Flag-Referenz für `agenteye`. -- **[CLI-Rezepte für Agenten](/de/agenteye/cli-recipes)**: Copy-paste-`jq`-Muster und Exit-Code-Behandlung. -- **[Evaluator Agent Skill](/de/agenteye/evaluator-skill)**: der verwandte Skill zum Aufbau des Evaluators, dessen Scores `agenteye evals` liest. -- **[Python SDK Agent Skill](/de/agenteye/python-sdk-skill)**: der verwandte Skill zum Instrumentieren eines Agenten, damit er die Telemetrie aussendet, die `agenteye` liest. -- **[KI-Assistent](/de/agenteye/assistant)**: der In-Dashboard-Assistent (nicht mit diesem Terminal-Skill zu verwechseln). -- **[API-Schlüssel](/de/agenteye/api-keys)**: das Berechtigungsmodell pro Org, das den Wirkungsbereich des Skills begrenzt. \ No newline at end of file diff --git a/docs/de/agenteye/codex-capture.mdx b/docs/de/agenteye/codex-capture.mdx deleted file mode 100644 index fae232f5..00000000 --- a/docs/de/agenteye/codex-capture.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "Codex-Sitzungsaufzeichnung" -description: "Leite die lokalen OpenAI Codex-Sitzungen deines Teams als gewöhnliche Sessions und Events in AgentEye weiter – ohne Änderungen an ihrer Arbeitsweise." ---- - -Deine Entwickler nutzen OpenAI Codex bereits täglich. Die Codex-Sitzungsaufzeichnung bringt diese Coding-Sessions als gewöhnliche Sessions und Events in AgentEye, sodass du sie durchsuchen, wiedergeben und zusammen mit allem anderen, was du beobachtest, auswerten kannst. Sie ergänzt das [Python SDK](/de/agenteye/python-sdk): Das SDK instrumentiert Agenten, die du selbst schreibst, während dieses Feature die Codex-Arbeit deines Teams aufzeichnet – ohne dass sich an deren Arbeitsweise etwas ändert. - -Ein kleiner Hintergrundkollektor liest Codex' lokale Sitzungstranskripte, während sie geschrieben werden, und überträgt sie an AgentEye. Ein Kollektor pro Maschine erfasst alle lokalen Codex-Oberflächen gleichzeitig – es ist keine oberflächenspezifische Einrichtung erforderlich. - -Derselbe Kollektor erfasst auch andere Agenten – siehe [OpenClaw](/de/agenteye/openclaw-capture) und [Hermes](/de/agenteye/hermes-capture). Aktiviere jede Variante, die du verwendest; ein einzelner Kollektor kann mehrere gleichzeitig aufzeichnen. - ---- - -## Was aufgezeichnet wird - -Jede Codex-Oberfläche, die **lokal** ausgeführt wird, erzeugt dieselben Sitzungstranskripte auf der Festplatte, und der Kollektor liest alle davon: - -- das Codex **CLI** und `codex exec` -- die **VS Code / IDE-Erweiterung** -- die **Desktop-App**, wenn sie eine Sitzung lokal ausführt - -Jede Codex-Sitzung wird zu einer AgentEye-[Session](/de/agenteye/sessions); ihre Nutzer- und Assistentennachrichten, das Reasoning, Tool-Aufrufe, Tool-Ergebnisse und der Token-Verbrauch werden zu den entsprechenden [Events](/de/agenteye/event-stream). Die Oberfläche, von der die jeweilige Sitzung stammt (CLI, IDE oder Desktop), wird festgehalten, damit du sie unterscheiden kannst. - -> **Cloud-Sitzungen werden nicht aufgezeichnet.** Die Desktop-App führt Sitzungen zunehmend in der Codex-Cloud aus und speichert lokal nur deren Metadaten – es gibt kein lokales Transkript zum Lesen. Nur lokal ausgeführte Sitzungen werden aufgezeichnet. - ---- - -## Aktivierung - -Die Aufzeichnung ist standardmäßig deaktiviert. Installiere den Kollektor mit einem API-Schlüssel, der die Berechtigung `events:add` besitzt (siehe [API-Schlüssel](/de/agenteye/api-keys)), und aktiviere die Codex-Aufzeichnung: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -Damit wird der Kollektor installiert, als Hintergrunddienst registriert und die Aufzeichnung gestartet. Prüfe, ob er läuft: - -```bash -agenteye-collector health -``` - -Beim ersten Start werden deine vorhandenen Codex-Sitzungen einmalig nachgefüllt, danach werden neue Aktivitäten innerhalb von Sekunden übertragen. Die Dateien von Codex werden ausschließlich gelesen – niemals verändert, verschoben oder gelöscht – und jede Sitzung wird genau einmal übertragen, auch nach einem Neustart. - ---- - -## Wo die Daten erscheinen - -Aufgezeichnete Sitzungen erscheinen unter **Sessions** und ihre Events im **Events**-Stream – genauso wie bei jedem anderen beobachteten Agenten. Damit funktionieren [Session-Replay](/de/agenteye/sessions), [Suche](/de/agenteye/queries), [Auswertungen](/de/agenteye/evaluations) und [Alerts](/de/agenteye/alerts) für sie ganz normal. Filtere nach dem Codex-Agenten, um nur diese anzuzeigen. - ---- - -## Datenschutz - -Codex-Transkripte enthalten die vollständige Sitzung – einschließlich Befehlsausgaben, Dateiinhalten und allem, was Codex gelesen oder geschrieben hat – und können sensible Informationen enthalten. Aufgezeichnete Sitzungen werden unverändert übertragen. Aktiviere die Aufzeichnung daher nur auf Maschinen und für Teams, bei denen das Zentralisieren dieser Inhalte in AgentEye angemessen ist, und weise dem Kollektor ausschließlich einen Schlüssel mit dem Umfang `events:add` zu. Unter [Sicherheit](/de/agenteye/security) erfährst du, wie deine Daten isoliert aufbewahrt werden. \ No newline at end of file diff --git a/docs/de/agenteye/concepts.mdx b/docs/de/agenteye/concepts.mdx deleted file mode 100644 index 5f6c4000..00000000 --- a/docs/de/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "Konzepte" -description: "Das Vokabular hinter Failproof AI Observability — Events, Sessions, Evaluierungen, Audits, Findings und Incidents — an einem Ort definiert." ---- - - -Diese Seite definiert das Vokabular, das Failproof AI Observability verwendet. Wenn ein Begriff in einem anderen Leitfaden unbekannt ist, wird er hier erklärt. Sie müssen ihn nicht von Anfang bis Ende lesen: Überfliegen Sie ihn, oder kehren Sie zurück, wenn Sie ein Wort genauer nachschlagen möchten. - ---- - -## Das Datenmodell - -**Event** -Die kleinste Dateneinheit. Ein Event zeichnet einen einzelnen Schritt auf, den Ihr Agent ausgeführt hat: ein `tool_use`, ein `model_request`, ein `hook_completed`, ein `error` usw. Ihr Agent gibt Events über das [Python SDK](/de/agenteye/python-sdk) aus; sie erscheinen in Echtzeit auf der **Events**-Seite. - -**Session** -Ein einzelner Agent-Lauf, identifiziert durch eine `session_id`. Eine Session umfasst alle Events, die diese ID teilen, zusammengefasst in einer einzelnen Zeile auf der **Sessions**-Seite und als Ausführungsgraph auf ihrer Detailseite dargestellt. Eine Session beginnt üblicherweise mit `agent_start` und endet mit `agent_end`. - -**Agent** -Ein benannter Akteur innerhalb eines Laufs, identifiziert durch eine `agent_id`. Ein Lauf kann mehrere Agents umfassen: zum Beispiel einen Planer, der einen Zusammenfassungs-Sub-Agenten startet. Sub-Agents tragen eine `parent_id`, die es Failproof AI Observability ermöglicht, sie in eigenen Spuren im Ausführungsgraph darzustellen. - -**Environment** -Eine Bezeichnung für den Ort, an dem der Lauf stattgefunden hat: `production`, `staging`, `dev`. Sie legen sie einmalig bei der Konfiguration des SDK fest. Fast jede Dashboard-Seite kann nach Environment gefiltert werden. - -**Context-Window-Auslastung** -Der prozentuale Anteil des Context-Windows eines Modells, den eine Antwort verbraucht hat. Failproof AI Observability versieht `model_response`-Events bei erkannten Modellen mit diesem Wert, sodass das Wachstum von Prompts und bevorstehende Kompaktierungen direkt im Event-Stream sichtbar sind. - ---- - -## Qualität - -**Evaluation** -Eine Qualitätsbewertung für eine abgeschlossene Session, die von einem Scoring-Dienst erstellt wird, den Sie selbst betreiben. Evaluierungen sind optional: Bis Sie einen Evaluator anschließen, werden Sessions aufgezeichnet, aber nicht bewertet. Jede Evaluierung kann mehrere benannte Scores enthalten (zum Beispiel `helpfulness`, `factuality`, `tool_efficiency`), jeweils mit einer kurzen Begründungsnotiz. Siehe [Evaluation suite](/de/agenteye/evaluation-suite). - -**Score-Key** -Der Name einer Dimension, über die ein Evaluator berichtet, z. B. `helpfulness`. Alerts und Audits können einen bestimmten Score-Key im Zeitverlauf beobachten. - -**Evaluator** -Ihr Scoring-Dienst. Failproof AI Observability übermittelt das Transkript eines abgeschlossenen Laufs per POST an ihn und speichert die zurückgegebenen Scores. Ein Standard-Evaluator wird nicht mitgeliefert; die Bewertungslogik liegt bei Ihnen. - ---- - -## Fehler finden und beheben - -**Hook** -Eine Sicherheitsvorkehrung oder ein Nebeneffekt, den Ihr Agent-Framework um einen Schritt herum ausführt: eine Inhaltssicherheitsprüfung, PII-Schwärzung oder eine Budget-Überwachung. Hooks geben `hook_triggered`- / `hook_completed`-Events mit einem `outcome` (allow, deny, modify) aus und haben eine eigene Observe-Seite. - -**Alert-Regel** -Eine Regel, die ausgelöst wird, wenn eine Metrik einen von Ihnen festgelegten Schwellenwert überschreitet: Fehlerrate, p95-Latenz, Token-Kosten oder ein Evaluator-Score. Wenn eine Regel ausgelöst wird, öffnet sie einen Incident und benachrichtigt Ihre gewählten Kanäle (E-Mail, Slack, Webhook, im Dashboard). Siehe [Alerts](/de/agenteye/alerts). - -**Incident** -Ein offenes Problem, das entsteht, wenn eine Alert-Regel ausgelöst wird. Incidents haben einen Lebenszyklus (bestätigen, zuweisen, lösen) und eine Aktivitäts-Timeline, die jede Aktion aufzeichnet. Sie können auch manuell einen öffnen. - -**Audit** -Eine wiederkehrende Untersuchung (stündlich bis wöchentlich), die Ihre Logs *sitzungsübergreifend* nach Fehlermustern durchsucht, für die Sie noch keine Regel geschrieben haben: Fehler-Cluster, niedrige Scores, Latenz-Ausreißer, Tool-Call-Schleifen und Läufe, die nie abgeschlossen wurden. Während ein Alert eine Metrik überwacht, die Sie bereits kennen, zeigt Ihnen ein Audit, worauf Sie als Nächstes achten sollten. Siehe [Audits](/de/agenteye/audits). - -**Finding** -Ein priorisiertes, evidenzbasiertes Ergebnis eines Audit-Laufs. Ein Finding benennt ein Muster, verlinkt auf die genauen Sessions dahinter und trägt einen Triage-Lebenszyklus (bestätigen, lösen, stummschalten, verwerfen). Failproof AI Observability dedupliziert Findings laufübergreifend, sodass ein bekanntes Muster aktualisiert wird, anstatt sich anzuhäufen. - -**Der KI-Assistent** -Der im Dashboard integrierte Chat, der auf Englisch Fragen zu Ihren Agents beantwortet — basierend auf Ihren eigenen Daten. Er ist standardmäßig schreibgeschützt; alles, was er erstellt (eine gespeicherte Abfrage, ein Dashboard), erfordert eine Genehmigung, und er kann niemals löschen. Siehe [AI assistant](/de/agenteye/assistant). - ---- - -## Betrieb - -**Organisation (Tenant)** -Ein isolierter Arbeitsbereich. Eine Failproof AI Observability-Instanz kann viele Organisationen hosten, jede mit eigenen Benutzern, Schlüsseln und Daten. Jede Dashboard-URL ist unter Ihrem Org-Slug (`//…`) eingeschränkt. - -**Collector** -`agenteye-collector`, der schlanke Daemon, der auf jedem Agent-Rechner läuft, die Events bündelt, die das SDK auf die Festplatte schreibt, und sie an den Server übermittelt. - -**API-Key** -Ein bereichsbeschränktes Token, das einen Client gegenüber dem Server authentifiziert. Keys tragen granulare Berechtigungen (zum Beispiel `events:add` für den Collector, schreibgeschützte Bereiche für einen Dashboard-Key). Siehe [API keys](/de/agenteye/api-keys). - -**Server** -Der Ingest- und API-Dienst. Er nimmt Events entgegen, speichert den Betriebszustand in Ihren Datenbanken und stellt das Dashboard und die CLI bereit. - -**Dashboard** -Die Web-Oberfläche. Jede Seite ist auf eine Organisation beschränkt und liest über die API des Servers. - ---- - -## Nächste Schritte - -- [Overview](/de/agenteye/overview): Wie diese Teile zusammenpassen. -- [Observability](/de/agenteye/observability): Die Observe-Oberflächen (Events, Sessions, Models, Tools, Hooks, Errors). \ No newline at end of file diff --git a/docs/de/agenteye/evaluator-skill.mdx b/docs/de/agenteye/evaluator-skill.mdx deleted file mode 100644 index ccccd307..00000000 --- a/docs/de/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: "Failproof AI Observability Evaluator Agent Skill" -description: "Von »Ich glaube, unser Agent ist manchmal schlecht« zu einem produktiven Scoring-Service – während dein Coding-Agent sowohl die Konzeption als auch die Umsetzung übernimmt." ---- - - -Von *„Ich glaube, unser Agent ist manchmal schlecht"* zu einem produktiven Scoring-Service – während dein Coding-Agent sowohl die Konzeption als auch die Umsetzung übernimmt. Der **Failproof AI Observability Evaluator Skill** (`agenteye-evaluator`) ist ein *Agent Skill*: ein kleines Verzeichnis mit Anweisungen, das ein Coding-Agent wie Claude Code oder Codex bei Bedarf lädt. Er bringt dem Agenten bei, herauszufinden, welche Qualitätsdimensionen es für *deinen* Agenten zu verfolgen lohnt, und dann den [Evaluator-Service](/de/agenteye/evaluation-suite) zu schreiben, zu testen und zu deployen, der sie bewertet. - -Es handelt sich **nicht** um einen gehosteten Scorer, eine Registry zum Hochladen oder ein Plugin-System. Dein Evaluator bleibt dein eigener HTTP-Service auf deiner eigenen Infrastruktur, genau wie im [Evaluation suite](/de/agenteye/evaluation-suite)-Leitfaden beschrieben. Der Skill lehrt deinen Agenten nur, ihn gut zu bauen – alles, was er tut, könntest du selbst tun, indem du denselben Code schreibst. - ---- - -## Das Schwierige ist zu entscheiden, was bewertet werden soll - -Die SDK-Oberfläche ist klein – ein Decorator und zwei Modelle – und ein Agent kann das allein aus dem [Contract](/de/agenteye/evaluation-suite#http-contract) herleiten. Daran scheitern Evaluatoren nicht. Sie scheitern daran, dass sie das Falsche bewerten, und ein Evaluator, der das Falsche bewertet, ist schlimmer als keiner: Er produziert ein Dashboard, das alle lernen zu ignorieren. - -Deshalb liegt der Schwerpunkt des Skills auf dem Teil, bevor überhaupt Code entsteht. Der Agent interviewt dich (*„Beschreib einen Lauf, der gut war; jetzt einen, der schlecht war"*), zieht dann deine echten Sessions durch die [`agenteye` CLI](/de/agenteye/cli) und liest sie von Anfang bis Ende. Diese beiden Hälften widersprechen sich meistens, und genau das ist der Punkt: was du zu messen beabsichtigst versus was deine Transcripts tatsächlich hergeben. Eine Dimension überlebt nur, wenn sie aus den Events **berechenbar** und **diskriminierend** ist – wenn sie sowohl für deinen guten als auch für deinen schlechten Lauf 0,9 ergibt, lehrt sie nichts und wird gestrichen. - -Das Ergebnis ist ein Vorschlag von 2–4 Dimensionen mit der zugehörigen Begründung, dem du zustimmen musst, bevor eine Zeile Code geschrieben wird. - -```mermaid -flowchart TD - YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent (Claude Code / Codex)
loads the agenteye-evaluator skill"] - AGENT -->|"interview: what does good vs bad look like?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["your real sessions
what actually happens"] - DATA --> DIMS["2-4 dimensions, you sign off"] - DIMS --> SVC["your evaluator service
agenteye-evaluator SDK"] - SVC --> SCORES["scores land in the dashboard
and agenteye evals"] -``` - ---- - -## Beziehung zu den anderen Evaluation-Komponenten - -Vier Docs behandeln das Scoring und gehen in dieser Reihenfolge ineinander über: - -| Seite | Was es ist | Verwende es, wenn | -|---|---|---| -| **[Evaluations](/de/agenteye/evaluations)** | Das Feature: Scores im Sessions-Grid, Dashboards, Re-evaluate | Du wissen möchtest, was automatisches Scoring dir bringt | -| **[Evaluation suite](/de/agenteye/evaluation-suite)** | Der HTTP-Contract, das SDK, die Server-Umgebungsvariablen | Du den Evaluator selbst implementierst oder debuggst | -| **Evaluator Skill** (dieses Dokument) | Ein sprachbasierter Einstieg in das Designen *und* Bauen des Scorers | Du von „Ich will Evals" zu einem laufenden Service kommen möchtest | -| **[CLI skill](/de/agenteye/cli-skill)** | Ein sprachbasierter Einstieg in die `agenteye` CLI | Du die bereits vorhandenen Scores *lesen* möchtest | -| **[Python SDK skill](/de/agenteye/python-sdk-skill)** | Ein sprachbasierter Einstieg in die Instrumentierung deines Agenten | Dein Agent noch keine Sessions emittiert – es gibt noch nichts zu bewerten | - -### vs. CLI Skill: Bauen versus Lesen - -Die beiden Skills überschneiden sich bewusst nicht, und beide zu installieren ist der Normalfall – der Agent wählt je nach Anfrage zwischen ihnen: - -- **`agenteye-evaluator`** (dieses Dokument) baut das, was Scores *erzeugt*. Seine Aufgabe endet, wenn Scores zum ersten Mal eintreffen. -- **[`agenteye-cli`](/de/agenteye/cli-skill)** liest bereits vorhandene Scores (`agenteye evals`). *„Hat die Qualität diese Woche nachgelassen?"* ist seine Frage, nicht die dieses Skills. - ---- - -## Voraussetzungen - -1. **Die `agenteye` CLI installiert und eingeloggt** (`pipx install agenteye`, dann `agenteye login`). Der Skill nutzt sie an zwei Stellen: um die echten Sessions zu holen, gegen die er designed, und um am Ende zu bestätigen, dass deine Scores angekommen sind. Dein Login benötigt `events:read`, sowie `evaluations:read` für die abschließende Prüfung. Wie beim CLI Skill kann er das per E-Mail zugesandte Einmal-Code-Login **nicht** für dich abschließen. -2. **Einen Ort für den Evaluator.** Er wird in ein Image gebaut und als langlebiger Service betrieben, benötigt also ein echtes Repo, keine temporäre Datei. Evaluatoren leben oft in einem eigenen Repo, getrennt vom bewerteten Agenten – der Skill sucht nach einem vorhandenen und fragt, bevor er ein neues anlegt. -3. **Das `agenteye-evaluator` SDK Wheel** – lies den nächsten Abschnitt, bevor dein Agent `pip`-Befehle einzutippen beginnt. - ---- - -## Bezugsquelle - -Der Skill ist in Failproof AI's öffentlicher Skills-Sammlung veröffentlicht: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -Das Repository ist öffentlich und der Skill benötigt keine eigenen Zugangsdaten – er steuert nur die `agenteye` CLI mit dem Session, mit dem *du* eingeloggt bist, und schreibt Code in *dein* Repo. Beachte, dass er als eigenes Verzeichnis ausgeliefert wird und **nicht** im `pipx install agenteye`-Paket enthalten ist – such ihn dort also nicht. - -## Den Skill installieren - -Der schnellste Weg ist die [`skills`](https://skills.sh) CLI, die das Verzeichnis abruft und dort ablegt, wo dein Agent sucht: - -```bash -# Claude Code, nur dieses Projekt -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# jedes Projekt (installiert nach ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# stattdessen Codex -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -Anschließend verwaltest du ihn wie jeden anderen Skill: - -```bash -npx skills list -a claude-code # was installiert ist -npx skills update agenteye-evaluator # neueste Version holen -npx skills remove agenteye-evaluator # entfernen -``` - -Bevorzugst du manuelle Installation? Ein Agent Skill ist nur ein Verzeichnis mit einer `SKILL.md` (plus optionalen Referenzen), das Kopieren funktioniert also ebenfalls: - -- **Claude Code**: Lege das `agenteye-evaluator/`-Verzeichnis in `~/.claude/skills/` (jedes Projekt) oder `/.claude/skills/` (nur dieses Repo). Claude Code erkennt es automatisch – prüfe mit der `/skills`-Liste oder frage einfach nach Evals. -- **Codex (OpenAI)**: Codex liest dieselbe `SKILL.md`. Die mitgelieferte `agents/openai.yaml` setzt `allow_implicit_invocation: true`, sodass Codex den Skill automatisch auswählt, wenn eine Aufgabe passt; andernfalls rufst du ihn explizit als `$agenteye-evaluator` auf. - ---- - -## Das SDK ist nicht auf dem öffentlichen PyPI - -> **Warnung:** Lies dies, bevor du einen Agenten das SDK installieren lässt. - -Der Skill ist öffentlich; das SDK, das er verwendet, ist es nicht. `agenteye-evaluator` wird nur als privates Release-Artefakt ausgeliefert, und anders als `agenteye` ist der Name auf dem öffentlichen PyPI **nicht beansprucht** – ein blankes `pip install agenteye-evaluator` könnte also das Paket eines Fremden in den Service ziehen, der deine Produktions-Transcripts liest. Das ist ein Supply-Chain-Problem, kein Tippfehler. - -Der Skill weiß das und arbeitet stattdessen eine Installations-Leiter ab, die beim ersten zutreffenden Schritt stoppt: die Monorepo-Quelle, wenn du im AgentEye-Repo bist, sonst das private Release-Wheel von GitHub Releases (erfordert Zugriff), und wenn keines davon erreichbar ist, **stoppt er und teilt dir mit, deinen Failproof AI-Kontakt nach dem Wheel zu fragen**, anstatt zu improvisieren. - -Wenn dein Agent also ein blankes `pip install agenteye-evaluator` von Public PyPI vorschlägt, ist das das Zeichen, dass der Skill nie geladen wurde. Halte dort an und prüfe, ob er installiert ist. - ---- - -## Was du ihn fragen kannst - -Ein echter Durchlauf beginnt mit einer vagen Anfrage und endet mit einem abgezeichneten Design, nicht mit Code: - -```text -you ▸ I want evals for our support bot. I think it's sometimes bad. - -agent ▸ Before I write anything: describe a run that went well, - and one that went badly. - -you ▸ Good: it answers from the docs. Bad: it makes stuff up, - or gives up and says "contact support". - -agent ▸ Running: agenteye --json sessions --status error --since 7d - Reading 5 sessions, including two you'd call bad. - - Your words say factuality and deflection. The transcripts - agree on deflection (2 of 5 end with a support hand-off, - and it's plain in model_response.content), but factuality - needs a ground truth your events don't carry. - - So I'd score three things: - - deflection rules, cheap, separates your two runs - - tool_efficiency rules, catches the retry loop in run-004 - - groundedness LLM judge: is every claim in the docs it retrieved? - - "factuality" would need ground truth we don't have; groundedness - is the closest thing your transcripts can actually support. - OK with those three names? Renaming later splits the history. - -you ▸ yes - -agent ▸ No evaluator in this repo. Should I scaffold one here, or do - you have one elsewhere? -``` - -Anschließend schreibt er die regelbasierten Dimensionen zuerst (kostenlos, sofort, deterministisch), testet sie gegen eine echte erfasste Session – einschließlich der leeren und nie abgeschlossenen Sessions, die naive Evaluatoren zum Absturz bringen – und greift nur für die subjektive Dimension auf einen LLM-Judge zurück. Er kennt die [Grenzen des Dispatchers](/de/agenteye/evaluation-suite#configuring-the-server) – ein 30-Sekunden-Request-Timeout und 8 gleichzeitige Calls deployment-weit – wenn der Judge nicht zuverlässig hineinpasst, geht er daher asynchron mit `JobPending` vor, anstatt zuzulassen, dass dein Judge fünfmal abgebrochen und mit fünffachen Kosten neu versucht wird. - -Dann deployt er, setzt die beiden Server-Umgebungsvariablen und bestätigt mit `agenteye --json evals --session-id `, dass Scores tatsächlich angekommen sind. Das Ankommen der Scores ist der einzige Beweis. - ---- - -## Worauf du achten solltest - -- **Dimensionsnamen sind nahezu dauerhaft.** Score-Keys sind beliebige Strings, und die Plattform verfolgt Trends für alles, was du sendest – das bedeutet, nichts downstream korrigiert eine schlechte Wahl. Benennst du sie später um, teilt sich die Historie: Alte Sessions behalten den alten Key und der Trend bricht ab. Deshalb holt sich der Skill explizite Zustimmung, bevor er Code schreibt – nimm diese Aufforderung ernst. -- **Fixtures sind echte Produktions-Transcripts.** Das Design gegen echte Sessions bedeutet, sie auf die Festplatte zu holen, und sie können Kundendaten enthalten. Der Skill fragt, bevor er sie in Git committet; im Zweifelsfall halte `fixtures/` aus dem Repo heraus und lass jeden Entwickler seine eigenen holen. -- **Der Agent schreibt und deployt einen Service, der jeden Transcript liest.** Er handelt als du, gebunden durch die Berechtigungen deines CLI-Logins, aber überprüfe den Evaluator wie jeden anderen Code, der Produktionsdaten berührt. - ---- - -## Nächste Schritte - -- **[Evaluation suite](/de/agenteye/evaluation-suite)**: der HTTP-Contract, das SDK und die Server-Umgebungsvariablen, die der Skill konfiguriert. -- **[Evaluations](/de/agenteye/evaluations)**: wo die Scores erscheinen, sobald sie ankommen. -- **[CLI skill](/de/agenteye/cli-skill)**: der Schwester-Skill, zum Lesen von Ergebnissen statt zum Bauen des Scorers. -- **[CLI](/de/agenteye/cli)**: die Befehlsreferenz hinter den Session-Daten, gegen die der Skill designed. \ No newline at end of file diff --git a/docs/de/agenteye/hermes-capture.mdx b/docs/de/agenteye/hermes-capture.mdx deleted file mode 100644 index 33d785fa..00000000 --- a/docs/de/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Hermes-Sitzungsaufzeichnung" -description: "Bringen Sie die Hermes-Gateway-Sitzungen Ihres Teams — Slack, Telegram, CLI und geplante Ausführungen — als gewöhnliche Sitzungen und Ereignisse in AgentEye ein." ---- - -[Hermes](https://hermes-agent.nousresearch.com) beantwortet die Anfragen Ihres Teams von überall, wo es bereits arbeitet — Slack, Telegram, der CLI, geplante Ausführungen. Die Hermes-Sitzungsaufzeichnung bringt all das als gewöhnliche Sitzungen und Ereignisse in AgentEye ein, sodass der Assistent, mit dem Ihr Team täglich spricht, genauso beobachtbar ist wie die Agenten, die Sie selbst schreiben. - -Ein kleiner Hintergrund-Collector liest Hermes' lokalen Sitzungsspeicher, während dieser beschrieben wird, und überträgt die Sitzungen an AgentEye. Er funktioniert genauso wie die Aufzeichnung bei [Codex](/de/agenteye/codex-capture) und [OpenClaw](/de/agenteye/openclaw-capture), und ein einzelner Collector kann mehrere davon gleichzeitig aufzeichnen. - ---- - -## Was aufgezeichnet wird - -Jede Hermes-Sitzung auf dem Rechner wird aufgezeichnet, unabhängig davon, über welchen Kanal sie zustande kam. Jede einzelne wird zu einer AgentEye-[Sitzung](/de/agenteye/sessions); ihre Benutzer- und Assistentennachrichten, Tool-Aufrufe und Tool-Ergebnisse werden zu den entsprechenden [Ereignissen](/de/agenteye/event-stream). - -Der Kanal, über den eine Sitzung gestartet wurde — Slack, Telegram, CLI oder eine geplante Ausführung — wird in der Sitzung festgehalten, sodass Sie sie unterscheiden und nach einer bestimmten filtern können. Dazu kommen das Modell, auf dem die Sitzung lief, der Chat und die Person, von der sie gestartet wurde, sowie — wenn eine Sitzung eine weitere erzeugt hat — die Verknüpfung zurück zur übergeordneten Sitzung. - -Sitzungen erscheinen, sobald Hermes sie startet, unabhängig davon, ob bereits etwas gesagt wurde. Die Antwort eines Gesprächsschritts und seine Tool-Aufrufe bleiben in der Reihenfolge, in der sie tatsächlich stattfanden. Wenn eine Sitzung endet, erfahren Sie auch, warum sie endete, was sie gekostet hat und wie viele Tokens sie verbrauchte. - ---- - -## Aktivierung - -Die Aufzeichnung ist deaktiviert, bis Sie sie einschalten. Installieren Sie den Collector mit einem API-Schlüssel, der die Berechtigung `events:add` besitzt (siehe [API-Schlüssel](/de/agenteye/api-keys)), und aktivieren Sie die Hermes-Aufzeichnung: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -Dadurch wird der Collector installiert, als Hintergrunddienst registriert und die Aufzeichnung gestartet. Bestätigen Sie, dass er läuft: - -```bash -agenteye-collector health -``` - -Möchten Sie mehrere Agenten auf demselben Rechner aufzeichnen? Fügen Sie das Flag jedes Agenten demselben Befehl hinzu — zum Beispiel `--hermes-enabled --codex-enabled`. - -Beim ersten Start werden Ihre vorhandenen Hermes-Sitzungen einmalig nachgefüllt, und neue Aktivitäten werden dann innerhalb von Sekunden übertragen. Die eigenen Daten von Hermes werden dabei nur gelesen — niemals verändert oder gelöscht — und jede Nachricht wird genau einmal übertragen, auch nach Neustarts. - -`health` teilt Ihnen außerdem mit, ob alles, was der Collector aufgezeichnet hat, tatsächlich bei AgentEye angekommen ist. Wenn ein Batch nicht zugestellt werden konnte, wird er aufbewahrt und erneut versucht, anstatt verworfen zu werden. Die Prüfung meldet so lange einen ungesunden Zustand, wie noch etwas aussteht — „gesund" bedeutet also, dass Ihre Daten angekommen sind, nicht lediglich, dass der Prozess läuft. - ---- - -## Wo es erscheint - -Aufgezeichnete Sitzungen erscheinen unter **Sessions** und ihre Ereignisse im **Events**-Stream, genauso wie bei jedem anderen beobachteten Agenten — sodass [Sitzungswiedergabe](/de/agenteye/sessions), [Suche](/de/agenteye/queries), [Auswertungen](/de/agenteye/evaluations) und [Benachrichtigungen](/de/agenteye/alerts) alle darauf anwendbar sind. Filtern Sie nach dem Hermes-Agenten, um nur dessen Sitzungen anzuzeigen. - ---- - -## Datenschutz - -Hermes-Sitzungen enthalten das vollständige Gesprächsprotokoll — einschließlich Befehlsausgaben, Dateiinhalten und allem, was der Agent gelesen oder geschrieben hat — und können Geheimnisse enthalten. Aufgezeichnete Sitzungen werden unverändert übertragen. Aktivieren Sie die Aufzeichnung daher nur dort, wo die Zentralisierung dieser Inhalte in AgentEye angemessen ist, und vergeben Sie dem Collector einen Schlüssel, der ausschließlich auf `events:add` beschränkt ist. Unter [Sicherheit](/de/agenteye/security) erfahren Sie, wie Ihre Daten isoliert aufbewahrt werden. \ No newline at end of file diff --git a/docs/de/agenteye/observability.mdx b/docs/de/agenteye/observability.mdx deleted file mode 100644 index 60ecfd4c..00000000 --- a/docs/de/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Beobachten" -description: "Die Beobachtungsoberflächen zeigen Ihnen, was Ihre Agenten gerade tun, und ermöglichen den detaillierten Einblick in einzelne Ausführungen." ---- - - -Die Beobachtungsoberflächen zeigen Ihnen, was Ihre Agenten gerade tun, und ermöglichen den detaillierten Einblick in einzelne Ausführungen. Alle Daten hier sind live, auf Ihre Organisation beschränkt und nach Datumsbereich, Umgebung, Agent und Sitzung filterbar – so gelangen Sie in Sekunden von „irgendetwas stimmt nicht" zum exakten Ausführungslauf. - -![Der Live-Event-Stream, farbcodiert nach Typ und filterbar nach Umgebung, Agent und Sitzung](/agenteye/images/events-stream.png) - -Vier Oberflächen, jede mit einer eigenen Seite: - -- **[Event-Stream](/de/agenteye/event-stream)**: der live, schrittweise Verlauf jeder Ausführung über alle Agenten hinweg, neueste zuerst. Die Startseite Ihrer Organisation und erste Anlaufstelle bei der Fehlersuche. -- **[Sitzungen und Ausführungsgraph](/de/agenteye/sessions)**: diese Ereignisse zu einer Zeile pro Ausführung zusammengefasst, plus eine git-artige Darstellung des Ablaufs jeder Ausführung. -- **[Performance-Metriken](/de/agenteye/telemetry)**: Latenz-Heatmaps und p50/p95/p99-Kennwerte für Ihre Modelle, Tools und Hooks, sodass ein Ausreißer am oberen Ende sofort vom Median auffällt. -- **[Fehlerverfolgung](/de/agenteye/error-tracking)**: eine einzige Triage-Oberfläche für alles, was schiefgelaufen ist – mit einem Klick von einem ausgelösten Alert zum fehlerhaften Ausführungslauf. - -## Verwandte Themen - -- [Evaluierungen](/de/agenteye/evaluations): Bewerten Sie jeden Ausführungslauf hinsichtlich der Qualität. -- [Alerts](/de/agenteye/alerts): Wandeln Sie beliebige Schwellenwerte in Benachrichtigungsregeln um. -- [Audits](/de/agenteye/audits): Lassen Sie Failproof AI Observability Fehlermuster über Sitzungen hinweg für Sie finden. -- [CLI und Agenten](/de/agenteye/cli-and-agents): dieselbe Observability direkt aus Ihrem Terminal. \ No newline at end of file diff --git a/docs/de/agenteye/openclaw-capture.mdx b/docs/de/agenteye/openclaw-capture.mdx deleted file mode 100644 index ed337e98..00000000 --- a/docs/de/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "OpenClaw Session-Aufzeichnung" -description: "Leiten Sie die lokalen OpenClaw-Sitzungen Ihres Teams als gewöhnliche Sessions und Events in AgentEye weiter — ohne Änderungen an der Art, wie OpenClaw ausgeführt wird." ---- - -Wenn Ihr Team [OpenClaw](https://docs.openclaw.ai) nutzt, bringt die OpenClaw-Sitzungsaufzeichnung diese Sessions als gewöhnliche Sessions und Events in AgentEye ein. So können Sie sie durchsuchen, wiedergeben und gemeinsam mit allem anderen auswerten, was Sie beobachten. Sie ergänzt das [Python SDK](/de/agenteye/python-sdk): Das SDK instrumentiert Agenten, die Sie selbst schreiben, während diese Funktion die OpenClaw-Arbeit erfasst, die Ihr Team bereits durchführt — ohne Änderungen an deren Arbeitsweise. - -Ein kleiner Hintergrund-Collector liest OpenClaw's lokale Sitzungsprotokolle, während sie geschrieben werden, und übermittelt sie an AgentEye. Er funktioniert genauso wie der [Codex Capture](/de/agenteye/codex-capture), und ein einzelner Collector kann beide gleichzeitig erfassen. - ---- - -## Was aufgezeichnet wird - -Jeder Agent, der im OpenClaw-Setup eines Rechners konfiguriert ist, wird vom Collector dieses Rechners erfasst — es ist keine agentenspezifische Einrichtung erforderlich. - -Jede OpenClaw-Sitzung wird zu einer AgentEye-[Session](/de/agenteye/sessions); ihre Benutzer- und Assistentennachrichten, Tool-Aufrufe und Tool-Ergebnisse werden zu den entsprechenden [Events](/de/agenteye/event-stream). - ---- - -## Aktivierung - -Die Aufzeichnung ist deaktiviert, bis Sie sie einschalten. Installieren Sie den Collector mit einem API-Schlüssel, der die Berechtigung `events:add` besitzt (siehe [API-Schlüssel](/de/agenteye/api-keys)), und aktivieren Sie die OpenClaw-Aufzeichnung: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -Dadurch wird der Collector installiert, als Hintergrunddienst registriert und die Aufzeichnung gestartet. Bestätigen Sie, dass er läuft: - -```bash -agenteye-collector health -``` - -Möchten Sie mehr als einen Agenten auf demselben Rechner erfassen? Fügen Sie das Flag für jeden weiteren Agenten demselben Befehl hinzu — zum Beispiel `--openclaw-enabled --codex-enabled`. - -Beim ersten Start werden Ihre vorhandenen OpenClaw-Sitzungen einmalig nachträglich importiert, danach wird neue Aktivität innerhalb von Sekunden übertragen. Die eigenen Dateien von OpenClaw werden ausschließlich gelesen — niemals verändert, verschoben oder gelöscht — und jede Sitzung wird genau einmal übermittelt, auch nach Neustarts. - ---- - -## Wo die Daten erscheinen - -Aufgezeichnete Sitzungen erscheinen unter **Sessions** und ihre Events im **Events**-Stream, genau wie bei jedem anderen beobachteten Agenten — sodass [Session-Wiedergabe](/de/agenteye/sessions), [Suche](/de/agenteye/queries), [Auswertungen](/de/agenteye/evaluations) und [Benachrichtigungen](/de/agenteye/alerts) alle darauf anwendbar sind. Filtern Sie nach dem OpenClaw-Agenten, um nur dessen Daten anzuzeigen. - ---- - -## Datenschutz - -OpenClaw-Protokolle enthalten die vollständige Sitzung — einschließlich Befehlsausgaben, Dateiinhalte und alles, was der Agent gelesen oder geschrieben hat — und können vertrauliche Informationen enthalten. Aufgezeichnete Sitzungen werden unverändert übermittelt. Aktivieren Sie die Aufzeichnung daher nur auf Rechnern und für Teams, bei denen die Zentralisierung dieser Inhalte in AgentEye angemessen ist, und vergeben Sie dem Collector ausschließlich einen auf `events:add` beschränkten Schlüssel. Unter [Sicherheit](/de/agenteye/security) erfahren Sie, wie Ihre Daten isoliert aufbewahrt werden. \ No newline at end of file diff --git a/docs/de/agenteye/python-sdk-skill.mdx b/docs/de/agenteye/python-sdk-skill.mdx deleted file mode 100644 index 637dc1ac..00000000 --- a/docs/de/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,131 +0,0 @@ ---- -title: "Failproof AI Observability Python SDK Agent Skill" -description: "Von einem nicht instrumentierten Agenten zu sichtbaren Events – Ihr Coding-Agent findet die Instrumentierungspunkte, implementiert sie und beweist, dass sie korrekt funktionieren." ---- - -Sagen Sie Ihrem Coding-Agenten *„Füge Failproof AI Observability zu diesem Agenten hinzu"* und lassen Sie ihn Ihre Schleife lesen, die richtigen Instrumentierungspunkte ermitteln, den Code schreiben und die Events verifizieren – bevor er die Aufgabe als erledigt markiert. - -Der **Python SDK Skill** (`agenteye-python-sdk`) ist ein *Agent Skill*: ein Ordner mit Anweisungen, den ein Coding-Agent wie Claude Code oder Codex bei Bedarf lädt, wenn eine Aufgabe dazu passt. Er bringt dem Agenten bei, das [Python SDK](/de/agenteye/python-sdk) zu verwenden – er ist keine Bibliothek und ändert nichts an der Funktionsweise des SDK. - -## Instrumentierung ist leicht zu schreiben – und leicht still falsch zu machen - -Das SDK ist klein: dreizehn Event-Methoden, alle ausschließlich als Keyword-Argumente. Ein Coding-Agent kann die [Python SDK](/de/agenteye/python-sdk)-Referenz lesen und in einer Minute plausible Instrumentierung produzieren. - -Das Problem ist, dass dieses SDK keinen Fehler wirft, wenn etwas falsch ist – und falsche Instrumentierung sieht genauso aus wie richtige Instrumentierung, bis jemand ein Dashboard öffnet und es leer vorfindet. Die Fehler, die wirklich Zeit kosten, sind allesamt stille: - -| Der Fehler | Was Sie sehen | -|---|---| -| Kein `agent_start` | Alle Events landen. Null Sessions. | -| Environment nie gesetzt | Alles funktioniert, wird unter `dev` abgelegt. | -| `outcome="failure"` | Der Lauf zeigt grün – nur `failed`, `error`, `timeout`, `rejected` zählen. | -| Tippfehler im Feldnamen | Akzeptiert und als neues Feld gespeichert. | -| Events aus einem Thread-Pool emittiert | Werden still verworfen. | - -Keiner davon wirft einen Fehler. Keiner taucht in Tests auf. Jeder einzelne ist im Skill dokumentiert – als Vertrag zusammen mit dem Check, der ihn erkennt. - -## Was der Skill tut – der Reihe nach - -Der Skill durchläuft dieselben drei Schritte, die ein sorgfältiger Entwickler gehen würde: - -1. **Planen.** Er liest Ihre Agenten-Schleife und stellt die zwei Fragen, die nur Sie beantworten können: Was zählt als ein Lauf (Ihre `session_id`), und wer sind die unterscheidbaren Akteure (Ihre `agent_id`)? Das wird geklärt, bevor Code geschrieben wird – denn eine spätere Änderung spaltet Ihre Historie und bricht die Trends. -2. **Schreiben.** Er bindet die Identität einmal pro Lauf statt sie durch jede Aufrufstelle durchzufädeln, und wählt eine nebenläufigkeitssichere Form – ein Detail, das wichtig ist, weil die naheliegende Abkürzung zwei überlappende Läufe stillschweigend in einer einzigen Session vermischt. -3. **Verifizieren.** Er führt Ihren Agenten aus und liest die entstandenen Event-Dateien, prüft ob `agent_start` vorhanden ist, das Environment stimmt und ein Lauf genau eine Session erzeugt hat. - -Dieser dritte Schritt ist der, den die meisten überspringen. Das SDK schreibt Events in lokale Dateien, sodass eine vollständige Integration auf einem Laptop bewiesen werden kann – ohne Server, ohne API-Key, ohne Netzwerk. Genau deshalb besteht der Skill darauf, diesen Schritt durchzuführen. - -## Verhältnis zu den anderen Skills - -Drei Skills, eine klare Aufteilung: - -| Skill | Einsetzen wenn | Was er berührt | -|---|---|---| -| **Python SDK Skill** (diese Seite) | Sie möchten, dass Ihr Agent Telemetrie *emittiert* – „Observability hinzufügen", „Warum erscheint mein Agent nicht?" | Schreibt Code in Ihrem Agenten-Repo. Liest nichts. | -| **[Evaluator Skill](/de/agenteye/evaluator-skill)** | Sie möchten Läufe *bewerten* – „Was sollen wir überhaupt messen?" | Schreibt Code in Ihrem Repo; liest Telemetrie | -| **[CLI Skill](/de/agenteye/cli-skill)** | Sie möchten *nachlesen*, was passiert ist, oder Ihr Deployment betreiben | Steuert die CLI als Sie, inklusive Änderungen | - -Die Übergabe erfolgt in dieser Reihenfolge: Dieser Skill bringt Events zum Fließen, der Evaluator bewertet sie, die CLI liest sie aus. Es gibt nichts zu bewerten und nichts zu lesen, bis Ihr Agent Sessions emittiert – wenn Sie von vorne beginnen, fangen Sie hier an. - -## Voraussetzungen - -1. **Python 3.10+** und die Agenten-Codebasis, die Sie instrumentieren möchten. -2. **Das SDK.** Es wird an Kunden als privates Wheel ausgeliefert und nicht über einen öffentlichen Index – Ihr Onboarding erklärt, wie Sie es beziehen und installieren. Der Skill kennt den Installationspfad und fragt Sie, anstatt zu raten, falls er ihn nicht finden kann. -3. **Nichts weiter.** Kein Dashboard-Login, kein API-Key, kein Netzwerk. Der Skill verifiziert anhand der Event-Dateien, die das SDK schreibt, und kann seine Arbeit offline abschließen und beweisen. - -## Bezugsquelle - -Der Skill befindet sich in der öffentlichen [`FailproofAI/skills`](https://github.com/FailproofAI/skills)-Sammlung: - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -Fügen Sie `-g` hinzu, um ihn für alle Projekte statt nur das aktuelle zu installieren, und `--copy`, wenn Ihre Umgebung keine Symlinks unterstützt. Für Codex übergeben Sie `-a codex`. - -## Manuelle Installation - -Agent Skills sind Ordner, die eine `SKILL.md` plus Referenzen enthalten. Falls Sie den Installer nicht verwenden möchten: - -- **Claude Code**: Kopieren Sie den Ordner `agenteye-python-sdk/` nach `~/.claude/skills/` (alle Projekte) oder `/.claude/skills/` (nur dieses Repo). Claude Code erkennt ihn automatisch – prüfen Sie die `/skills`-Liste oder fragen Sie einfach etwas, das dazu passt. -- **Codex**: Codex liest dieselbe `SKILL.md`. Das mitgelieferte `agents/openai.yaml` setzt `allow_implicit_invocation: true`, sodass er automatisch gewählt wird, wenn eine Aufgabe passt; andernfalls rufen Sie ihn als `$agenteye-python-sdk` auf. - -Führen Sie Ihren Agenten **im Repository mit dem Code aus, den Sie instrumentieren möchten** – der Skill liest Ihre Agenten-Schleife, bevor er etwas vorschlägt. - -## Wie eine Session aussieht - -```text -you ▸ Add Failproof AI Observability to this agent. - -agent ▸ Read your loop — one tool dispatcher in tools.py, one Anthropic - wrapper in llm.py. Two questions before I write anything. - - What counts as one run? You handle requests concurrently, so I'd - use the request id you already have in ctx.request_id — then a - session here and a request in your logs are the same string. - - And who are the actors? I see a planner path and a researcher - path. Label them separately, or is that one agent to you? - -you ▸ Request id is right. They're one agent, call it "assistant". - -agent ▸ Done. Wrapped the dispatcher and the LLM client; agent_start and - agent_end bracket handle_request, including the error path. Used - a context-local for identity — your dispatcher runs tools in a - thread pool, and the simpler approach would have mixed concurrent - runs into one session without complaining. - - Verified against a real run: 11 events across 4 files — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, one session - per request, durations paired. - - One thing I did NOT do: your worker pool in batch.py submits - straight to the executor, so events from there would be dropped. - Want me to fix those too? -``` - -Das Muster, das auffällt: Er hat den Code gelesen, bevor er etwas vorschlug, nur die Fragen gestellt, die Sie beantworten können, eine bereits vorhandene ID wiederverwendet, die nebenläufigkeitssichere Form gewählt *weil* er einen Thread-Pool sah, und **durch Lesen der tatsächlichen Events verifiziert** statt einfach Erfolg zu erklären – und dann die eine Stelle markiert, von der er wusste, dass sie still versagen würde. - -## Was Sie ihn fragen können - -- *„Warum erscheint mein Agent nicht im Dashboard?"* → Geht die Leiter hinab: Werden Events geschrieben, ist `agent_start` vorhanden, stimmt das Environment, liest der Collector am richtigen Ort? -- *„Alles landet unter dev."* → Das Environment wurde nie gesetzt oder durch einen späteren Aufruf zurückgesetzt. -- *„Token-Tracking hinzufügen."* → Findet Ihren LLM-Wrapper und erfasst Modell, Stop-Grund und Nutzung. -- *„Auch die Sub-Agenten instrumentieren."* → Eine Session, eindeutige Agenten-Labels, verschachtelt unter ihrem Elternteil. -- *„Tests für die Instrumentierung schreiben."* → Zeigt das SDK auf ein temporäres Verzeichnis und macht Assertions auf die geschriebenen Events. - -## Worauf Sie achten sollten - -**Lassen Sie ihn verifizieren.** Der Schritt, der diesen Skill wertvoll macht, ist der letzte – Ihren Agenten ausführen und die Events zurücklesen. Ein Agent, der Instrumentierung schreibt und dann aufhört, hat die einfache Hälfte erledigt; die Hälfte, die still versagt, ist die andere. - -**Namen vereinbaren, bevor Code geschrieben wird.** `session_id` und `agent_id` sind die Achsen, nach denen jede Oberfläche gruppiert. Sie später umzubenennen spaltet die Historie: Alte Läufe behalten die alten Labels und Ihre Trends brechen. Der Skill wird fragen; die Antwort ist eine Minute Nachdenken wert. - -**Wenn Ihr Agent vorschlägt, das SDK von einem öffentlichen Index zu installieren, wurde der Skill nicht geladen.** Das SDK wird privat vertrieben. Dieser Vorschlag ist ein zuverlässiges Zeichen dafür, dass Ihr Coding-Agent rät statt dem Skill zu folgen – stoppen Sie ihn dort und prüfen Sie, ob der Skill installiert ist. - -Abgesehen davon ist der Wirkungsbereich überschaubar: Er schreibt Code in Ihrem Arbeitsverzeichnis und Event-Dateien dort, wo Sie es angeben. Er liest nichts aus Ihrem Deployment und ändert nichts daran. - -## Nächste Schritte - -- **[Python SDK](/de/agenteye/python-sdk)**: Die vollständige Event-Referenz – jeder Event-Typ und jedes Feld – hinter dem, was dieser Skill automatisiert. -- **[Sessions](/de/agenteye/sessions)**: Was Ihre Instrumentierung produziert, sobald Events ankommen. -- **[Evaluator Agent Skill](/de/agenteye/evaluator-skill)**: Der nächste Schritt, sobald Läufe ankommen – ihre Bewertung. -- **[CLI Agent Skill](/de/agenteye/cli-skill)**: Ihre Telemetrie zurücklesen. \ No newline at end of file diff --git a/docs/de/agenteye/security.mdx b/docs/de/agenteye/security.mdx deleted file mode 100644 index 9756abac..00000000 --- a/docs/de/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "Sicherheit" -description: "Failproof AI Observability ist darauf ausgelegt, nah an Ihren Produktions-Agents zu laufen – das bedeutet, es sieht Ihre Prompts, Tool-Eingaben und Ausgaben." ---- - - -Failproof AI Observability ist darauf ausgelegt, nah an Ihren Produktions-Agents zu laufen – das bedeutet, es sieht Ihre Prompts, Tool-Eingaben und Ausgaben. Diese Seite erklärt, wie die Daten isoliert, kontrolliert und in Ihren Händen bleiben. Wenn Sie Failproof AI Observability im Rahmen einer Sicherheitsprüfung evaluieren, beginnen Sie hier. - ---- - -## Ihre Daten bleiben in Ihrer Umgebung - -Failproof AI Observability wird selbst gehostet. Events, Prompts, Modellantworten und Analysen werden in Ihren eigenen Datenbanken, in Ihrer eigenen Umgebung gespeichert. Es werden keine Daten zur Speicherung an einen Drittanbieter-SaaS übermittelt – Ihre Daten verbleiben in Ihrem eigenen Cloud-Account. - ---- - -## Mandantenisolierung - -Eine Failproof AI Observability-Instanz kann viele Organisationen beherbergen, und jede ist auf Speicherebene isoliert – durchgesetzt von der Datenbank, nicht nur von der Benutzeroberfläche: - -- Die operativen Daten einer Organisation (Benutzer, Schlüssel, Dashboards, gespeicherte Abfragen) sind auf diese Organisation beschränkt, und organisationsübergreifende Lesezugriffe werden von der Datenbank selbst blockiert. -- Jedes aufgenommene Event wird mit der zugehörigen Organisation gestempelt, sodass die Events einer Organisation niemals von einer anderen gelesen werden können. - -Jede Dashboard-Route ist unter einem Org-Slug (`//…`) eingeschränkt. - ---- - -## Anmeldung - -Failproof AI Observability verwendet passwortlose, E-Mail-basierte Anmeldung. Es gibt kein Passwort, das abgephisht oder geleakt werden könnte. Ein Benutzer fordert einen Einmalcode (oder einen Magic Link zum einmaligen Klicken) an, der per E-Mail zugestellt wird und schnell abläuft. Die Anmeldung ist durch eine **Allowlist** gesichert: Nur E-Mail-Adressen (oder Domains), die Sie freigeben, können sich authentifizieren. - -![Der Anmeldebildschirm von Failproof AI Observability, der einen Einmalcode an Ihre E-Mail-Adresse sendet](/agenteye/images/login.png) - ---- - -## Eingeschränkter Zugriff mit API-Schlüsseln - -Jeder Client authentifiziert sich mit einem API-Schlüssel, der granulare, minimal privilegierte Berechtigungen trägt. Ein Collector benötigt lediglich `events:add`; ein Dashboard- oder Assistenten-Schlüssel kann schreibgeschützt sein; destruktive Aktionen (Löschen, Neugenerieren) sind separate Berechtigungen, die Sie gezielt vergeben. - -![Die API-Schlüssel-Seite: Berechtigungen jedes Schlüssels, farblich nach Lese-, Schreib- und destruktivem Umfang kodiert](/agenteye/images/api-keys.png) - -Behalten Sie den Admin-Bootstrap-Schlüssel für die Einrichtung, und vergeben Sie eingeschränkte Schlüssel für alles andere. Siehe [API-Schlüssel](/de/agenteye/api-keys). - ---- - -## Ein schreibgeschützter, genehmigungspflichtiger Assistent - -Der [KI-Assistent](/de/agenteye/assistant) im Dashboard beantwortet Fragen über Ihre Daten, ist aber bewusst eingeschränkt: - -- Er ist **standardmäßig schreibgeschützt**: Sein SQL wird durch einen Guard geleitet, der nur `SELECT`/`WITH`-Abfragen, einzelne Anweisungen und eine Zeilenbegrenzung erlaubt. -- Alles, was er erstellt (eine gespeicherte Abfrage, ein Dashboard), ist **genehmigungspflichtig**: Sie prüfen und genehmigen jeden Schreibvorgang, bevor er ausgeführt wird. -- Er **kann niemals löschen**. - -So kann ein Teammitglied fragen „Welche Agents haben diese Woche am häufigsten Fehler gemeldet?" und auf die Antwort reagieren – ohne dass der Assistent Ihre Daten eigenständig ändern oder entfernen kann. - ---- - -## Daten in Übertragung - -Der gesamte Datenverkehr läuft über HTTPS. Sie terminieren TLS mit Ihren eigenen Zertifikaten, sodass der Datenverkehr zwischen Collector und Server sowie zwischen Browser und Server verschlüsselt übertragen wird. - ---- - -## Nächste Schritte - -- [Übersicht](/de/agenteye/overview): Wie Failproof AI Observability zusammenarbeitet. -- [API-Schlüssel](/de/agenteye/api-keys): Zugriff für Collector, Dashboard und Assistent einschränken. -- [Observability](/de/agenteye/observability): Was Failproof AI Observability von Ihren Agents erfasst. \ No newline at end of file diff --git a/docs/de/cli/audit.mdx b/docs/de/audit.mdx similarity index 100% rename from docs/de/cli/audit.mdx rename to docs/de/audit.mdx diff --git a/docs/de/cli/backfill.mdx b/docs/de/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/de/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/de/cli/config.mdx b/docs/de/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/de/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/de/cli/flush.mdx b/docs/de/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/de/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/de/cli/harness.mdx b/docs/de/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/de/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/de/cli/migrate.mdx b/docs/de/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/de/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/de/cli/uninstall.mdx b/docs/de/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/de/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/de/cli/update.mdx b/docs/de/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/de/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/de/agenteye/api-keys.mdx b/docs/de/cloud/access.mdx similarity index 95% rename from docs/de/agenteye/api-keys.mdx rename to docs/de/cloud/access.mdx index b12d5f5a..07ec068e 100644 --- a/docs/de/agenteye/api-keys.mdx +++ b/docs/de/cloud/access.mdx @@ -1,10 +1,10 @@ --- title: "API Keys" -description: "API keys steuern, wer und was Ihren Failproof AI Observability-Server erreichen kann – ein Collector kann damit Events senden, ohne jemals Lese- oder Adminrechte zu erhalten." +description: "API keys steuern, wer und was Ihren FailproofAI Cloud-Server erreichen kann – ein Collector kann damit Events senden, ohne jemals Lese- oder Adminrechte zu erhalten." --- -API keys steuern, wer und was Ihren Failproof AI Observability-Server erreichen kann – ein Collector kann damit Events senden, ohne jemals Lese- oder Adminrechte zu erhalten. Jeder Key trägt eine oder mehrere Berechtigungen, und jede Berechtigung sichert bestimmte Server-Routen ab; Sie vergeben nur die Berechtigungen, die ein Job tatsächlich benötigt. Die meisten Deployments erstellen lediglich drei Arten von Keys. +API keys steuern, wer und was Ihren FailproofAI Cloud-Server erreichen kann – ein Collector kann damit Events senden, ohne jemals Lese- oder Adminrechte zu erhalten. Jeder Key trägt eine oder mehrere Berechtigungen, und jede Berechtigung sichert bestimmte Server-Routen ab; Sie vergeben nur die Berechtigungen, die ein Job tatsächlich benötigt. Die meisten Deployments erstellen lediglich drei Arten von Keys. ## Die 3 Keys, die die meisten Deployments benötigen @@ -82,7 +82,7 @@ Der Server erzwingt einen festen Berechtigungskatalog; jede Berechtigung sichert Diese Berechtigungen unterstützen die **Users**-Seite im Dashboard, auf der die vergebenen Scopes jedes Mitglieds als Chips angezeigt werden: -![Die Users-Seite: eine Karte pro Dashboard-Benutzer mit E-Mail-Adresse, vergebenen Berechtigungen sowie Bearbeiten- und Deaktivieren-Steuerelementen](/agenteye/images/users.png) +![Die Users-Seite: eine Karte pro Dashboard-Benutzer mit E-Mail-Adresse, vergebenen Berechtigungen sowie Bearbeiten- und Deaktivieren-Steuerelementen](/cloud/images/users.png) ### Betriebliche Einstellungen @@ -91,7 +91,7 @@ Diese Berechtigungen unterstützen die **Users**-Seite im Dashboard, auf der die | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | Dashboard-verwaltete Betriebseinstellungen und ihre Metadaten anzeigen; modellspezifische Context-Window-Overrides auflisten; und das effektive Window für ein Modell auflösen. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | Betriebseinstellungen bearbeiten sowie modellspezifische Context-Window-Overrides hinzufügen, ändern oder entfernen. Änderungen wirken sich auf neue Events aus, ohne den Server neu starten zu müssen. | -![Die Settings-Seite: dashboard-verwaltete Betriebseinstellungen wie erlaubte Anmeldemethoden und Session-/OTP-Lebensdauern, bearbeitbar ohne Neustart](/agenteye/images/settings.png) +![Die Settings-Seite: dashboard-verwaltete Betriebseinstellungen wie erlaubte Anmeldemethoden und Session-/OTP-Lebensdauern, bearbeitbar ohne Neustart](/cloud/images/settings.png) ### Alarme & Vorfälle @@ -247,7 +247,7 @@ Die **Keys**-Seite im Dashboard bietet eine Benutzeroberfläche für alle oben g Wenn Sie einen Key im Dashboard erstellen, geben Sie das Secret nicht selbst an; das Dashboard generiert ein starkes Secret für Sie und zeigt es **einmalig** bei der Erstellung an. Kopieren Sie es sofort und speichern Sie es sicher; es wird nie wieder angezeigt – genau wie beim Neu-Generieren. Sie können die Berechtigungen des Keys trotzdem direkt auswählen oder sie aus einem Berechtigungs-Set übernehmen (siehe unten). -![Die API-Keys-Seite: eine Karte pro Key mit Name, vergebenen Berechtigungen und Erstellungszeitpunkt sowie Aktionen zum Neu-Generieren und Deaktivieren; geschützte Keys wie `admin` sind gekennzeichnet](/agenteye/images/api-keys.png) +![Die API-Keys-Seite: eine Karte pro Key mit Name, vergebenen Berechtigungen und Erstellungszeitpunkt sowie Aktionen zum Neu-Generieren und Deaktivieren; geschützte Keys wie `admin` sind gekennzeichnet](/cloud/images/api-keys.png) --- @@ -276,5 +276,5 @@ Diese Hinweise sind nur relevant, wenn Sie eine bestehende Instanz aktualisieren ## Nächste Schritte -- [Python SDK](/de/agenteye/python-sdk): Wie Ihr Agent-Code sich beim Senden von Events authentifiziert. -- [Security](/de/agenteye/security): Wie Anmeldung, Zugriffskontrolle und organisationsweite Datenisolierung funktionieren. \ No newline at end of file +- [Python SDK](/de/cloud/sdk): Wie Ihr Agent-Code sich beim Senden von Events authentifiziert. +- [Security](/de/cloud/security): Wie Anmeldung, Zugriffskontrolle und organisationsweite Datenisolierung funktionieren. \ No newline at end of file diff --git a/docs/de/cloud/agent-skills.mdx b/docs/de/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/de/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/de/agenteye/alerts.mdx b/docs/de/cloud/alerts.mdx similarity index 76% rename from docs/de/agenteye/alerts.mdx rename to docs/de/cloud/alerts.mdx index 4a2b39b6..af5b7664 100644 --- a/docs/de/agenteye/alerts.mdx +++ b/docs/de/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "Erfahre sofort, wenn etwas deine Grenze überschreitet – auf dem --- -Erfahre sofort, wenn etwas deine Grenze überschreitet – auf dem Kanal, den dein Team bereits nutzt, statt es von einem Kunden zu hören. Lege eine Regel einmal fest, und Failproof AI Observability prüft sie nach einem Zeitplan und benachrichtigt dich per E-Mail, Slack, Webhook oder direkt im Dashboard. +Erfahre sofort, wenn etwas deine Grenze überschreitet – auf dem Kanal, den dein Team bereits nutzt, statt es von einem Kunden zu hören. Lege eine Regel einmal fest, und FailproofAI Cloud prüft sie nach einem Zeitplan und benachrichtigt dich per E-Mail, Slack, Webhook oder direkt im Dashboard. -![Die Alerts-Seite: ein Raster mit Alert-Regelkarten, jede mit ihrem Auslöser, dem Auswertungsfenster, den Kanälen und einem Info-, Warn- oder Kritisch-Schweregrad-Badge](/agenteye/images/alerts.png) +![Die Alerts-Seite: ein Raster mit Alert-Regelkarten, jede mit ihrem Auslöser, dem Auswertungsfenster, den Kanälen und einem Info-, Warn- oder Kritisch-Schweregrad-Badge](/cloud/images/alerts.png) *Alle Alert-Regeln auf einen Blick: was überwacht wird, wie oft, wohin benachrichtigt wird und wie dringend.* ## Erfahre von Problemen, bevor deine Nutzer es tun @@ -22,9 +22,9 @@ Kombiniere beliebige dieser Optionen für eine einzige Regel. Der Schweregrad (I ## Regeln per Formular erstellen, nicht per JSON -Du beschreibst, was „kaputt" bedeutet, in einem Formular, und Failproof AI Observability erstellt die zugrundeliegende Regel für dich. Die JSON-Spezifikation ist lediglich das, was dieses Formular intern erzeugt – du kannst sie lesen, um eine Regel zu verstehen, aber tippst sie selten manuell ein. +Du beschreibst, was „kaputt" bedeutet, in einem Formular, und FailproofAI Cloud erstellt die zugrundeliegende Regel für dich. Die JSON-Spezifikation ist lediglich das, was dieses Formular intern erzeugt – du kannst sie lesen, um eine Regel zu verstehen, aber tippst sie selten manuell ein. -![Das Formular für neue Alerts: Name und Beschreibung, ein Aktivierungsschalter und eine Auslöserauswahl mit Metrikschwellenwert, benutzerdefiniertem SQL, Auswertungsscore, zusammengesetzter Auswertung und ereignisbezogenen Bedingungen](/agenteye/images/alert-new.png) +![Das Formular für neue Alerts: Name und Beschreibung, ein Aktivierungsschalter und eine Auslöserauswahl mit Metrikschwellenwert, benutzerdefiniertem SQL, Auswertungsscore, zusammengesetzter Auswertung und ereignisbezogenen Bedingungen](/cloud/images/alert-new.png) *Wähle einen Auslöser und das Formular zeigt die richtigen Felder an; Speichern schreibt die Regel.* Der Standardablauf geht schnell: Name vergeben, einen **Auslöser** wählen (was überwacht werden soll), **Schwellenwert und Zeitfenster** festlegen (wie schlimm, über welchen Zeitraum), mindestens einen **Kanal** anhängen, dann **Speichern** und auf **Test** klicken, um eine synthetische Benachrichtigung auszulösen und zu bestätigen, dass jedes Ziel richtig verdrahtet ist. Intern entsteht dabei eine kleine Spezifikation wie: @@ -43,7 +43,7 @@ Du bist nicht auf eine einzige Art von Signal beschränkt. Wähle den Auslöser, | **Zusammengesetzte Auswertung** | mehrere Score-Prüfungen mit Beliebig-, Alle- oder Mindestens-N-Logik kombiniert werden, um eine Regression zu erkennen, die sich erst über mehrere Scores hinweg zeigt | | **Pro Ereignis** | ein einzelnes passendes Ereignis eintrifft: ein bestimmter Agent, ein bestimmter Fehlertyp oder ein Nachrichten-Substring | -Schaust du gerade auf der [Errors-Seite](/de/agenteye/error-tracking) auf einen Fehler? Jede Zeile dort hat eine **+ Alert**-Schaltfläche, die dieses Formular vorausgefüllt öffnet, um genau diesen Fehler beim nächsten Auftreten abzufangen – damit der Vorfall, den du gerade triagiert hast, beim nächsten Mal direkt eine Benachrichtigung auslöst. +Schaust du gerade auf der [Errors-Seite](/de/cloud/errors) auf einen Fehler? Jede Zeile dort hat eine **+ Alert**-Schaltfläche, die dieses Formular vorausgefüllt öffnet, um genau diesen Fehler beim nächsten Auftreten abzufangen – damit der Vorfall, den du gerade triagiert hast, beim nächsten Mal direkt eine Benachrichtigung auslöst. **Wo du es findest:** Alerts befinden sich unter `//alerts`. Zum Erstellen, Bearbeiten, Löschen und Testen von Regeln wird **`alerts:write`** benötigt; `alerts:read` reicht zum Anschauen. Die Empfängerauswahl listet die Mitglieder deiner Organisation namentlich auf, sodass du eine Person benachrichtigen kannst, ohne das Formular zu verlassen. @@ -53,11 +53,11 @@ Eine einzelne fehlerhafte Messung sollte dich nicht aufwecken. Der **M von N**-R ## Was passiert, wenn ein Alert ausgelöst wird -Ein Verstoß öffnet einen **Incident** und benachrichtigt deine Kanäle einmalig. Von dort aus bestätigt dein Team den Vorfall, weist einen Verantwortlichen zu, bespricht ihn und löst ihn auf – alles in einem übersichtlichen, zugeordneten Protokoll. Dieser Triage-Workflow hat seine eigene Seite: siehe [Incidents](/de/agenteye/incidents). +Ein Verstoß öffnet einen **Incident** und benachrichtigt deine Kanäle einmalig. Von dort aus bestätigt dein Team den Vorfall, weist einen Verantwortlichen zu, bespricht ihn und löst ihn auf – alles in einem übersichtlichen, zugeordneten Protokoll. Dieser Triage-Workflow hat seine eigene Seite: siehe [Incidents](/de/cloud/incidents). ## Verwandte Themen -- [Incidents](/de/agenteye/incidents): verfolge einen ausgelösten Alert von offen über bestätigt bis gelöst. -- [Error tracking](/de/agenteye/error-tracking): gruppiere Agent-Fehler und wandle einen mit einem Klick in einen Alert um. -- [Dashboards](/de/agenteye/dashboards): beobachte die gemeinsamen Boards, aus denen die überwachten Schwellenwerte stammen. -- [CLI and agents](/de/agenteye/cli-and-agents): erstelle Alerts und bestätige Incidents über dein Terminal oder integriere sie per Skript in CI. \ No newline at end of file +- [Incidents](/de/cloud/incidents): verfolge einen ausgelösten Alert von offen über bestätigt bis gelöst. +- [Error tracking](/de/cloud/errors): gruppiere Agent-Fehler und wandle einen mit einem Klick in einen Alert um. +- [Dashboards](/de/cloud/dashboards): beobachte die gemeinsamen Boards, aus denen die überwachten Schwellenwerte stammen. +- [CLI and agents](/de/cloud/cli): erstelle Alerts und bestätige Incidents über dein Terminal oder integriere sie per Skript in CI. \ No newline at end of file diff --git a/docs/de/agenteye/assistant.mdx b/docs/de/cloud/assistant.mdx similarity index 85% rename from docs/de/agenteye/assistant.mdx rename to docs/de/cloud/assistant.mdx index 961e49f7..ceead3a8 100644 --- a/docs/de/agenteye/assistant.mdx +++ b/docs/de/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "Stell deinen Agentendaten eine Frage auf Deutsch und erhalte eine --- -Stell deinen Agentendaten eine Frage in gewöhnlicher Sprache und erhalte eine Antwort, die direkt auf die Belege verlinkt. Kein SQL schreiben, kein Durchsuchen von Dashboards – der **Failproof AI Observability**-Assistent ist der schnellste Weg für jeden in deinem Team, Antworten zu euren Agenten zu bekommen. +Stell deinen Agentendaten eine Frage in gewöhnlicher Sprache und erhalte eine Antwort, die direkt auf die Belege verlinkt. Kein SQL schreiben, kein Durchsuchen von Dashboards – der **FailproofAI Cloud**-Assistent ist der schnellste Weg für jeden in deinem Team, Antworten zu euren Agenten zu bekommen. -![Der Failproof AI Observability-Assistent beantwortet eine Frage in natürlicher Sprache im Dashboard und zeigt dabei eine Live-Agenten-Aktivitätstabelle, eine Aufschlüsselung der Modellnutzung pro Agent und schriftliche Zusammenfassungen – die ausgeführten Abfragen werden inline angezeigt](/agenteye/images/assistant.png) +![Der FailproofAI Cloud-Assistent beantwortet eine Frage in natürlicher Sprache im Dashboard und zeigt dabei eine Live-Agenten-Aktivitätstabelle, eine Aufschlüsselung der Modellnutzung pro Agent und schriftliche Zusammenfassungen – die ausgeführten Abfragen werden inline angezeigt](/cloud/images/assistant.png) *Frag in natürlicher Sprache und erhalte eine Antwort, die aus deinen eigenen Daten aufgebaut ist. Hier wird aufgeschlüsselt, welche Agenten am stärksten ausgelastet sind und welche Modelle sie verwenden – die ausgeführten Abfragen werden angezeigt, damit du jede Zahl nachvollziehen kannst.* Es gibt nichts zu lernen. Öffne den Chat, tippe, was du wissen möchtest, und folge den Links, die zurückgegeben werden: @@ -36,7 +36,7 @@ Wenn eine Antwort es wert ist, behalten zu werden, bitte den Assistenten, sie zu Auf der **Queries**-Seite geht er noch einen Schritt weiter und wird zum SQL-Autor: Beschreibe die gewünschte Abfrage („zeige Fehlerrate nach Agent für die letzten 7 Tage") und er streamt SQL direkt in den Editor – mit einer Diff-Ansicht, damit du die Änderung **akzeptieren** oder **ablehnen** kannst, bevor sie übernommen wird. -![Die Observability-Queries-Seite und ihr SQL-Editor](/agenteye/images/query-lab.png) +![Die FailproofAI Cloud-Queries-Seite und ihr SQL-Editor](/cloud/images/query-lab.png) *Die Queries-Seite: In diesem Editor streamt der Assistent einen schreibgeschützten Entwurf, den du akzeptieren oder ablehnen kannst.* Das Erstellen von SQL per Frage hier verwendet die Berechtigung `queries:run` – dieselbe, die hinter dem **Ausführen**-Button des Editors steckt. Der Chat überall sonst benötigt `agent:use`. @@ -49,7 +49,7 @@ Du kannst den Assistenten für alle öffnen, ohne dir Gedanken darüber machen z - **Jeder Schreibvorgang wartet auf dich.** Gespeicherte Abfragen und Dashboards werden nur nach deinem ausdrücklichen Klick auf „Genehmigen" erstellt – und es gibt keine Einstellung, die diese Schranke deaktiviert. - **Er kann niemals etwas löschen.** Es ist kein Lösch-Tool verfügbar, und der Assistent hat keine Löschberechtigung. Löschvorgänge bleiben in deinen Händen, im Dashboard. - **Er bleibt in deiner Organisation.** Der Assistent sieht immer nur die Organisation, die du gerade ansiehst. -- **Deine Fragen gehören dir.** Eingaben und Antworten leben in deiner eigenen Observability-Datenbank; Produktanalysen zeichnen nur Nutzungsmetadaten auf, niemals deinen Fragentext. +- **Deine Fragen gehören dir.** Eingaben und Antworten leben in deiner eigenen FailproofAI Cloud-Datenbank; Produktanalysen zeichnen nur Nutzungsmetadaten auf, niemals deinen Fragentext. ## Wo du ihn findest @@ -57,7 +57,7 @@ Der Assistent befindet sich am rechten Rand jeder Seite unter deiner Organisatio ## Verwandte Themen -- [CLI and agents](/de/agenteye/cli-and-agents) -- [Queries](/de/agenteye/queries) -- [Dashboards](/de/agenteye/dashboards) -- [Evaluation suite](/de/agenteye/evaluation-suite) \ No newline at end of file +- [CLI and agents](/de/cloud/cli) +- [Queries](/de/cloud/queries) +- [Dashboards](/de/cloud/dashboards) +- [Evaluation suite](/de/cloud/evaluators) \ No newline at end of file diff --git a/docs/de/agenteye/audits.mdx b/docs/de/cloud/audits.mdx similarity index 80% rename from docs/de/agenteye/audits.mdx rename to docs/de/cloud/audits.mdx index 41b6f14a..7044609e 100644 --- a/docs/de/agenteye/audits.mdx +++ b/docs/de/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "Audits: Ihr automatischer Zuverlässigkeitsanalyst" -description: "Failproof AI Observability sucht nach den Fehlern, für die Sie nie eine Regel geschrieben haben, und liefert Ihnen eine priorisierte, evidenzbasierte Aufgabenliste mit genau dem, was behoben werden muss." +description: "FailproofAI Cloud sucht nach den Fehlern, für die Sie nie eine Regel geschrieben haben, und liefert Ihnen eine priorisierte, evidenzbasierte Aufgabenliste mit genau dem, was behoben werden muss." --- -Failproof AI Observability sucht nach den Fehlern, für die Sie nie eine Regel geschrieben haben, und liefert Ihnen eine priorisierte, evidenzbasierte Aufgabenliste mit genau dem, was behoben werden muss. Es ist so, als würde ein Analyst jede Nacht Ihre Logs durchforsten und Ihnen morgens die Kurzliste auf den Schreibtisch legen. +FailproofAI Cloud sucht nach den Fehlern, für die Sie nie eine Regel geschrieben haben, und liefert Ihnen eine priorisierte, evidenzbasierte Aufgabenliste mit genau dem, was behoben werden muss. Es ist so, als würde ein Analyst jede Nacht Ihre Logs durchforsten und Ihnen morgens die Kurzliste auf den Schreibtisch legen.
@@ -12,7 +12,7 @@ Failproof AI Observability sucht nach den Fehlern, für die Sie nie eine Regel g *Ein zweiminütiger Rundgang: vom geplanten Lauf bis zu einer umsetzbaren Lösung.* -![Die Audits-Seite: wiederkehrende Jobs, die Ihre Sessions auf Fehlermuster scannen, jeweils mit Zeitplan und Sensitivität](/agenteye/images/audits.png) +![Die Audits-Seite: wiederkehrende Jobs, die Ihre Sessions auf Fehlermuster scannen, jeweils mit Zeitplan und Sensitivität](/cloud/images/audits.png) *Jedes Audit ist ein wiederkehrender Job, der Ihre Sessions auswertet und priorisierte, evidenzbasierte Empfehlungen erstellt.* ## Hören Sie auf zu raten, was als Nächstes behoben werden soll @@ -48,7 +48,7 @@ Audits befinden sich im Dashboard unter **`//audits`** (Seitenleiste z ## Verwandtes -- [Alerts](/de/agenteye/alerts): Werden Sie benachrichtigt, sobald ein Schwellenwert, den Sie bereits kennen, überschritten wird. -- [Evaluations](/de/agenteye/evaluations): Bewerten Sie jeden Lauf, damit Qualitätsregressionen von selbst auffallen. -- [Error tracking](/de/agenteye/error-tracking): Gruppieren und verfolgen Sie die Fehler, die Ihre Agents ausgeben. -- [Incidents](/de/agenteye/incidents): Verfolgen Sie ein von einem Audit aufgedecktes Problem bis zu seiner Lösung. \ No newline at end of file +- [Alerts](/de/cloud/alerts): Werden Sie benachrichtigt, sobald ein Schwellenwert, den Sie bereits kennen, überschritten wird. +- [Evaluations](/de/cloud/evaluations): Bewerten Sie jeden Lauf, damit Qualitätsregressionen von selbst auffallen. +- [Error tracking](/de/cloud/errors): Gruppieren und verfolgen Sie die Fehler, die Ihre Agents ausgeben. +- [Incidents](/de/cloud/incidents): Verfolgen Sie ein von einem Audit aufgedecktes Problem bis zu seiner Lösung. \ No newline at end of file diff --git a/docs/de/cloud/capture.mdx b/docs/de/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/de/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/de/agenteye/cli-recipes.mdx b/docs/de/cloud/cli-recipes.mdx similarity index 90% rename from docs/de/agenteye/cli-recipes.mdx rename to docs/de/cloud/cli-recipes.mdx index e0645365..bc0b068f 100644 --- a/docs/de/agenteye/cli-recipes.mdx +++ b/docs/de/cloud/cli-recipes.mdx @@ -4,9 +4,9 @@ description: "Copy-paste-Abfragemuster und jq-Rezepte, die Sitzungs-, Ereignis- --- -Sitzungs-, Ereignis- und Auswertungsdaten direkt aus einem Skript oder Coding-Agenten abrufen (und Neuauswertungen auslösen), mit sauberem JSON auf stdout, das direkt in `jq` weitergeleitet werden kann. Diese Rezepte verwandeln die Daten von Failproof AI Observability in etwas, das ein Terminal-Nutzer oder ein KI-Coding-Agent (Claude Code, Cursor) abfragen und automatisieren kann – ohne durch das Dashboard zu klicken. +Sitzungs-, Ereignis- und Auswertungsdaten direkt aus einem Skript oder Coding-Agenten abrufen (und Neuauswertungen auslösen), mit sauberem JSON auf stdout, das direkt in `jq` weitergeleitet werden kann. Diese Rezepte verwandeln die Daten von FailproofAI Cloud in etwas, das ein Terminal-Nutzer oder ein KI-Coding-Agent (Claude Code, Cursor) abfragen und automatisieren kann – ohne durch das Dashboard zu klicken. -Die folgenden Muster sind copy-paste-bereit für die Failproof AI Observability CLI (`agenteye`). Installation, Authentifizierung und die vollständige Optionsliste finden Sie unter [CLI](/de/agenteye/cli); führen Sie `agenteye -h` oder `agenteye -h` für die integrierte Hilfe aus. +Die folgenden Muster sind copy-paste-bereit für die FailproofAI Cloud CLI (`agenteye`). Installation, Authentifizierung und die vollständige Optionsliste finden Sie unter [CLI](/de/cloud/cli); führen Sie `agenteye -h` oder `agenteye -h` für die integrierte Hilfe aus. ## Grundregeln @@ -173,7 +173,7 @@ Das `--fields` jedes Befehls akzeptiert genau die Feldnamen seines eigenen Eleme ## Nächste Schritte -- [CLI](/de/agenteye/cli): Installation, Authentifizierung und die vollständige Optionsreferenz für jeden Befehl. -- [CLI-Agent-Skill](/de/agenteye/cli-skill): Diese Rezepte als Skill verpacken, den Ihr Coding-Agent laden kann. -- [API-Schlüssel](/de/agenteye/api-keys): Schlüssel erstellen und eingrenzen, mit denen sich CLI, SDK und Collector authentifizieren. -- [Python SDK](/de/agenteye/python-sdk): Ereignisse in Failproof AI Observability senden, damit diese Rezepte Daten zum Abfragen haben. \ No newline at end of file +- [CLI](/de/cloud/cli): Installation, Authentifizierung und die vollständige Optionsreferenz für jeden Befehl. +- [CLI-Agent-Skill](/de/cloud/agent-skills): Diese Rezepte als Skill verpacken, den Ihr Coding-Agent laden kann. +- [API-Schlüssel](/de/cloud/access): Schlüssel erstellen und eingrenzen, mit denen sich CLI, SDK und Collector authentifizieren. +- [Python SDK](/de/cloud/sdk): Ereignisse in FailproofAI Cloud senden, damit diese Rezepte Daten zum Abfragen haben. \ No newline at end of file diff --git a/docs/de/agenteye/cli.mdx b/docs/de/cloud/cli.mdx similarity index 91% rename from docs/de/agenteye/cli.mdx rename to docs/de/cloud/cli.mdx index c7f03cf9..9b3348f6 100644 --- a/docs/de/agenteye/cli.mdx +++ b/docs/de/cloud/cli.mdx @@ -1,10 +1,10 @@ --- title: "CLI" -description: "Steuere die gesamte Failproof AI Observability vom Terminal oder einem Skript aus: kein Umweg über das Dashboard." +description: "Steuere die gesamte FailproofAI Cloud vom Terminal oder einem Skript aus: kein Umweg über das Dashboard." --- -Steuere die gesamte Failproof AI Observability vom Terminal oder einem Skript aus: kein Umweg über das Dashboard. Die `agenteye` CLI fragt deine Daten ab (Sessions, Event-Logs, Evaluierungen) und verwaltet deine Organisation (API-Keys, Nutzer, Einstellungen, Alerts, Incidents, gespeicherte Abfragen) – greife darauf zurück, wenn du eine Prüfung automatisieren, Observability in CI einbinden oder einen Coding-Agenten die Produktion inspizieren lassen möchtest. Jeder Befehl unterstützt ein `--json`-Flag, sodass er gleichermaßen für dich an der Eingabeaufforderung oder für einen Coding-Agenten (Claude Code, Cursor) funktioniert, der das Ergebnis parst. +Steuere die gesamte FailproofAI Cloud vom Terminal oder einem Skript aus: kein Umweg über das Dashboard. Die `agenteye` CLI fragt deine Daten ab (Sessions, Event-Logs, Evaluierungen) und verwaltet deine Organisation (API-Keys, Nutzer, Einstellungen, Alerts, Incidents, gespeicherte Abfragen) – greife darauf zurück, wenn du eine Prüfung automatisieren, FailproofAI Cloud in CI einbinden oder einen Coding-Agenten die Produktion inspizieren lassen möchtest. Jeder Befehl unterstützt ein `--json`-Flag, sodass er gleichermaßen für dich an der Eingabeaufforderung oder für einen Coding-Agenten (Claude Code, Cursor) funktioniert, der das Ergebnis parst. Mit einer einzigen Binary kannst du: @@ -69,7 +69,7 @@ agenteye --version agenteye --help ``` -> **Hinweis:** Das Failproof AI Observability Python SDK verwendet ebenfalls den Distributionsnamen `agenteye`. Die Installation der CLI mit `pipx` oder `uv tool` (statt `pip install` in ein gemeinsames Virtualenv) verhindert Konflikte zwischen beiden. Ein einfaches `pip install agenteye` ist nur dann problemlos, wenn das SDK nicht in derselben Umgebung installiert ist. +> **Hinweis:** Das FailproofAI Cloud Python SDK verwendet ebenfalls den Distributionsnamen `agenteye`. Die Installation der CLI mit `pipx` oder `uv tool` (statt `pip install` in ein gemeinsames Virtualenv) verhindert Konflikte zwischen beiden. Ein einfaches `pip install agenteye` ist nur dann problemlos, wenn das SDK nicht in derselben Umgebung installiert ist. --- @@ -91,7 +91,7 @@ agenteye logout # revoke the session and clear the stored token `whoami` schlägt bei einer fehlenden oder abgelaufenen Session nie fehl; stattdessen meldet es `logged_in: false`, sodass ein Skript oder Agent den Auth-Status sicher abfragen kann (es kann dennoch mit einem Nicht-Null-Wert enden, wenn keine Basis-URL gesetzt oder das Dashboard nicht erreichbar ist). -**Voraussetzungen:** Deine E-Mail-Adresse muss für die Anmeldung am Dashboard berechtigt sein (frage deinen Failproof AI Observability-Administrator), und das Dashboard muss über seine Basis-URL erreichbar sein (siehe [Konfiguration](#configuration)). Wenn du einen Code anforderst und keiner eintrifft, ist deine E-Mail-Adresse wahrscheinlich noch nicht für den Dashboard-Zugang freigeschalten. +**Voraussetzungen:** Deine E-Mail-Adresse muss für die Anmeldung am Dashboard berechtigt sein (frage deinen FailproofAI Cloud-Administrator), und das Dashboard muss über seine Basis-URL erreichbar sein (siehe [Konfiguration](#configuration)). Wenn du einen Code anforderst und keiner eintrifft, ist deine E-Mail-Adresse wahrscheinlich noch nicht für den Dashboard-Zugang freigeschalten. --- @@ -338,13 +338,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | Authentifiziert, aber dein Konto verfügt nicht über die erforderliche Berechtigung (die Meldung nennt sie) | | 6 | Die angeforderte Ressource wurde nicht gefunden (z. B. unbekannte Session- oder Incident-ID) | -Diese machen die CLI sicher skriptfähig: Ein Coding-Agent kann bei `4` darauf reagieren, dich zur erneuten Authentifizierung aufzufordern, oder bei `5` die fehlende Berechtigung anzeigen. Siehe [CLI-Rezepte für Agenten](/de/agenteye/cli-recipes) für Exit-Code-Behandlungsmuster und JSON-Ausgabeformen. +Diese machen die CLI sicher skriptfähig: Ein Coding-Agent kann bei `4` darauf reagieren, dich zur erneuten Authentifizierung aufzufordern, oder bei `5` die fehlende Berechtigung anzeigen. Siehe [CLI-Rezepte für Agenten](/de/cloud/cli-recipes) für Exit-Code-Behandlungsmuster und JSON-Ausgabeformen. --- ## Nächste Schritte -- **[CLI-Rezepte für Agenten](/de/agenteye/cli-recipes)**: Kopierfertige Abfragemuster, `jq`-Einzeiler, `--fields`-Projektionen, Exit-Code-Behandlung und JSON-Ausgabeformen – geschrieben für Coding-Agenten, die die CLI steuern. -- **[CLI-Agent-Skill](/de/agenteye/cli-skill)**: Paketiere diese CLI als installierbaren Claude Code / Codex-*Skill*, damit ein Coding-Agent Failproof AI Observability über einfache Textanfragen steuert. -- **[API-Keys](/de/agenteye/api-keys)**: Das Berechtigungsmodell hinter `keys create --add …`. -- **[KI-Assistent](/de/agenteye/assistant)**: Den Assistenten aktivieren, mit dem `agent ask` kommuniziert. \ No newline at end of file +- **[CLI-Rezepte für Agenten](/de/cloud/cli-recipes)**: Kopierfertige Abfragemuster, `jq`-Einzeiler, `--fields`-Projektionen, Exit-Code-Behandlung und JSON-Ausgabeformen – geschrieben für Coding-Agenten, die die CLI steuern. +- **[CLI-Agent-Skill](/de/cloud/agent-skills)**: Paketiere diese CLI als installierbaren Claude Code / Codex-*Skill*, damit ein Coding-Agent FailproofAI Cloud über einfache Textanfragen steuert. +- **[API-Keys](/de/cloud/access)**: Das Berechtigungsmodell hinter `keys create --add …`. +- **[KI-Assistent](/de/cloud/assistant)**: Den Assistenten aktivieren, mit dem `agent ask` kommuniziert. \ No newline at end of file diff --git a/docs/de/cloud/connect.mdx b/docs/de/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/de/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/de/agenteye/dashboards.mdx b/docs/de/cloud/dashboards.mdx similarity index 70% rename from docs/de/agenteye/dashboards.mdx rename to docs/de/cloud/dashboards.mdx index 90c4cd37..b26d0855 100644 --- a/docs/de/agenteye/dashboards.mdx +++ b/docs/de/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "Verwandeln Sie Ihre Live-Agentendaten in ein gemeinsames Bild, das Verwandeln Sie Ihre Live-Agentendaten in ein gemeinsames Bild, das Ihr gesamtes Team im Blick behält. Pinnen Sie die wichtigsten Abfragen als Diagramme, und alle sehen auf Anhieb dieselben Zahlen – ohne eine einzige Abfrage erneut ausführen zu müssen. -![Ein Dashboard aus gespeicherten Abfragen: eine Ereignisse-pro-Stunde-Linie, ein Fehler-nach-Typ-Balken, ein Latenz-Flächendiagramm und Tokens nach Modell](/agenteye/images/dashboard-fleet.png) +![Ein Dashboard aus gespeicherten Abfragen: eine Ereignisse-pro-Stunde-Linie, ein Fehler-nach-Typ-Balken, ein Latenz-Flächendiagramm und Tokens nach Modell](/cloud/images/dashboard-fleet.png) *Ein Board, vier gespeicherte Abfragen: Ereignisse pro Stunde, Fehler nach Typ, Latenz und Tokens nach Modell.* @@ -25,15 +25,15 @@ Ihre Boards finden Sie unter `//dashboards`. ## Gespeicherte Abfragen pinnen -Jede Kachel beginnt als gespeicherte Abfrage. Erstellen und speichern Sie die gewünschte Abfrage in der [Queries](/de/agenteye/queries)-Bibliothek (mit integrierten Voreinstellungen und eigenen Abfragen über Ihre Ereignisse und Auswertungen), und pinnen Sie sie dann als passendes Diagramm auf ein Dashboard: eine **Linie** für Trends über die Zeit, ein **Balken** für Kategorienvergleiche, eine **Fläche** für Volumina oder ein **Kreisdiagramm** für Anteile. +Jede Kachel beginnt als gespeicherte Abfrage. Erstellen und speichern Sie die gewünschte Abfrage in der [Queries](/de/cloud/queries)-Bibliothek (mit integrierten Voreinstellungen und eigenen Abfragen über Ihre Ereignisse und Auswertungen), und pinnen Sie sie dann als passendes Diagramm auf ein Dashboard: eine **Linie** für Trends über die Zeit, ein **Balken** für Kategorienvergleiche, eine **Fläche** für Volumina oder ein **Kreisdiagramm** für Anteile. Da eine Kachel lediglich Ihre gespeicherte Abfrage als Diagramm darstellt, müssen Sie nichts manuell synchronisieren. Aktualisieren Sie die Abfrage einmal, und jedes Dashboard, das sie verwendet, wird automatisch aktualisiert. ## Qualität im Blick behalten, nicht nur Volumen -Das Volumen zeigt Ihnen, dass die Agenten beschäftigt sind. Die Qualität zeigt Ihnen, ob sie ihre Aufgabe tatsächlich erfüllen. Richten Sie ein Dashboard auf Ihre [Auswertungs-Scores](/de/agenteye/evaluations) aus, und Sie erhalten ein Board, das verfolgt, wie gut die Ausführungen im Laufe der Zeit laufen – sodass ein Qualitätsrückgang als Einbruch im Diagramm erscheint und nicht als böse Überraschung eines Kunden. +Das Volumen zeigt Ihnen, dass die Agenten beschäftigt sind. Die Qualität zeigt Ihnen, ob sie ihre Aufgabe tatsächlich erfüllen. Richten Sie ein Dashboard auf Ihre [Auswertungs-Scores](/de/cloud/evaluations) aus, und Sie erhalten ein Board, das verfolgt, wie gut die Ausführungen im Laufe der Zeit laufen – sodass ein Qualitätsrückgang als Einbruch im Diagramm erscheint und nicht als böse Überraschung eines Kunden. -![Ein qualitätsorientiertes Dashboard aus gespeicherten Auswertungsabfragen](/agenteye/images/dashboard-quality.png) +![Ein qualitätsorientiertes Dashboard aus gespeicherten Auswertungsabfragen](/cloud/images/dashboard-quality.png) *Ein Qualitäts-Board hält Ihre Auswertungs-Scores stets im Vordergrund, direkt neben den operativen Kennzahlen.* @@ -41,6 +41,6 @@ Halten Sie ein Betriebs-Board und ein Qualitäts-Board nebeneinander, und Ihr Te ## Verwandtes -- [Queries](/de/agenteye/queries): Erstellen und speichern Sie die Abfragen, die zu Ihren Kacheln werden. -- [Evaluations](/de/agenteye/evaluations): Bewerten Sie Ihre Ausführungen, um die Qualität über die Zeit abzubilden. -- [Alerts](/de/agenteye/alerts): Wandeln Sie einen Schwellenwert für eine dieser Metriken in eine Benachrichtigung um. \ No newline at end of file +- [Queries](/de/cloud/queries): Erstellen und speichern Sie die Abfragen, die zu Ihren Kacheln werden. +- [Evaluations](/de/cloud/evaluations): Bewerten Sie Ihre Ausführungen, um die Qualität über die Zeit abzubilden. +- [Alerts](/de/cloud/alerts): Wandeln Sie einen Schwellenwert für eine dieser Metriken in eine Benachrichtigung um. \ No newline at end of file diff --git a/docs/de/agenteye/error-tracking.mdx b/docs/de/cloud/errors.mdx similarity index 65% rename from docs/de/agenteye/error-tracking.mdx rename to docs/de/cloud/errors.mdx index fd96ef44..b643974d 100644 --- a/docs/de/agenteye/error-tracking.mdx +++ b/docs/de/cloud/errors.mdx @@ -6,14 +6,14 @@ description: "Sehen Sie jeden Fehler Ihrer Agenten an einem Ort, gruppiert, dami Sehen Sie jeden Fehler Ihrer Agenten an einem Ort, gruppiert, damit ein Fehlerstoß als ein einziges Problem erscheint. Sie erhalten einen Klick-Pfad von „etwas ist rot" bis zum genauen Lauf, der abgebrochen ist, ohne einen Live-Feed durchscrollen zu müssen. -![Die Fehlerseite: ein Histogramm der Fehler über die Zeit über gruppierten roten Fehlerzeilen, jede mit einer Ein-Klick-Schaltfläche „+ alert"](/agenteye/images/errors.png) +![Die Fehlerseite: ein Histogramm der Fehler über die Zeit über gruppierten roten Fehlerzeilen, jede mit einer Ein-Klick-Schaltfläche „+ alert"](/cloud/images/errors.png) *Die Fehlerseite: ein Histogramm der Fehler über die Zeit, wobei wiederkehrende Fehler in einer Zeile pro Vorfall zusammengefasst werden.* ## Jeder Fehler, bereits für Sie gesammelt Wenn ein Agent abstürzt, sollten Sie keinen Live-Event-Stream durchscrollen müssen, um rote Zeilen zu finden, bevor sie verschwinden. Die **Fehlerseite** übernimmt das Sammeln für Sie. Sie bündelt alles, was das Dashboard rot markieren würde, auf einer einzigen Triage-Oberfläche – das Erste, was Sie sehen, ist, was fehlschlägt, nicht wo Sie danach suchen müssen. -Und sie erfasst mehr als die offensichtlichen Fehler. Neben expliziten `error`-Events macht Failproof AI Observability auch die stillen Fehler sichtbar: Jedes `tool_result`, `hook_completed` oder `agent_end`, dessen Payload einen Fehler enthält, wird hier angezeigt. Ein Tool, das einen Fehler zurückgegeben hat, oder ein Hook, der fehlerhaft beendet wurde, entgeht Ihnen nicht mehr, nur weil keine laute Exception ausgelöst wurde. +Und sie erfasst mehr als die offensichtlichen Fehler. Neben expliziten `error`-Events macht FailproofAI Cloud auch die stillen Fehler sichtbar: Jedes `tool_result`, `hook_completed` oder `agent_end`, dessen Payload einen Fehler enthält, wird hier angezeigt. Ein Tool, das einen Fehler zurückgegeben hat, oder ein Hook, der fehlerhaft beendet wurde, entgeht Ihnen nicht mehr, nur weil keine laute Exception ausgelöst wurde. Am oberen Rand zeigt ein Histogramm Fehler über die Zeit. Ein Blick zeigt Ihnen, ob es sich um ein stetiges Hintergrundrauschen oder um einen Anstieg handelt, der vor wenigen Minuten begann – damit wissen Sie sofort, ob Sie alles stehen und liegen lassen müssen. @@ -23,19 +23,19 @@ Wie jede Beobachtungsoberfläche ist die Fehlerseite auf Ihre Organisation begre Eine einzige defekte Abhängigkeit kann denselben Fehler hunderte Male pro Minute auslösen. Unbearbeitet ergibt das eine Wand aus nahezu identischen Zeilen, die das Wesentliche verbirgt. -Failproof AI Observability fasst wiederkehrende Fehler mit derselben Session und demselben Fehlertyp in einer einzigen Zeile zusammen. Ein Fehlerstoß erscheint als ein einziger Vorfall. Sie zählen Probleme, keine Log-Zeilen – und das Signal, das wichtig ist, bleibt oben, anstatt von seinem eigenen Volumen überwältigt zu werden. +FailproofAI Cloud fasst wiederkehrende Fehler mit derselben Session und demselben Fehlertyp in einer einzigen Zeile zusammen. Ein Fehlerstoß erscheint als ein einziger Vorfall. Sie zählen Probleme, keine Log-Zeilen – und das Signal, das wichtig ist, bleibt oben, anstatt von seinem eigenen Volumen überwältigt zu werden. ## Von „etwas ist rot" zum genauen Event Klicken Sie auf eine beliebige Zeile, um direkt in die Session dieses Laufs zu gelangen, positioniert auf dem genauen Event, das fehlgeschlagen ist. Kein Kopieren von Session-IDs, kein Scrollen, um den Moment des Fehlers zu finden: Sie landen genau dort, mit dem vollständigen Ausführungsgraph auf einen Blick, sodass Sie sehen können, was der Agent in den Momenten vor dem Absturz getan hat. -Wenn Sie `alerts:write`-Berechtigung haben, enthält jede Zeile auch eine **+ alert**-Schaltfläche. Klicken Sie darauf, öffnet Observability eine neue Alert-Regel, die bereits so ausgefüllt ist, dass sie denselben Fehler beim nächsten Mal erkennt. Der Vorfall, den Sie gerade triagiert haben, wird zu dem, der Sie beim nächsten Mal benachrichtigt – anstatt Sie zweimal zu überraschen. +Wenn Sie `alerts:write`-Berechtigung haben, enthält jede Zeile auch eine **+ alert**-Schaltfläche. Klicken Sie darauf, öffnet FailproofAI Cloud eine neue Alert-Regel, die bereits so ausgefüllt ist, dass sie denselben Fehler beim nächsten Mal erkennt. Der Vorfall, den Sie gerade triagiert haben, wird zu dem, der Sie beim nächsten Mal benachrichtigt – anstatt Sie zweimal zu überraschen. **Wo Sie es finden:** Die **Fehlerseite** befindet sich im Beobachtungsbereich des Dashboards unter `//errors`. ## Verwandte Themen -- [Alerts](/de/agenteye/alerts): Jeden Fehler in eine Benachrichtigungsregel umwandeln. -- [Incidents](/de/agenteye/incidents): Einen ausgelösten Alert von offen bis gelöst verfolgen. -- [Sessions](/de/agenteye/sessions): Den vollständigen Lauf hinter einem Fehler öffnen. -- [Audits](/de/agenteye/audits): Observability Fehlermuster in Ihren Läufen automatisch erkennen lassen. \ No newline at end of file +- [Alerts](/de/cloud/alerts): Jeden Fehler in eine Benachrichtigungsregel umwandeln. +- [Incidents](/de/cloud/incidents): Einen ausgelösten Alert von offen bis gelöst verfolgen. +- [Sessions](/de/cloud/sessions): Den vollständigen Lauf hinter einem Fehler öffnen. +- [Audits](/de/cloud/audits): FailproofAI Cloud Fehlermuster in Ihren Läufen automatisch erkennen lassen. \ No newline at end of file diff --git a/docs/de/agenteye/evaluations.mdx b/docs/de/cloud/evaluations.mdx similarity index 66% rename from docs/de/agenteye/evaluations.mdx rename to docs/de/cloud/evaluations.mdx index c1b3e732..ef108103 100644 --- a/docs/de/agenteye/evaluations.mdx +++ b/docs/de/cloud/evaluations.mdx @@ -3,15 +3,15 @@ title: "Evaluations" description: "Qualitätsprobleme finden Sie jetzt von selbst, anstatt erst durch eine Nutzerbeschwerde davon zu erfahren." --- -Qualitätsprobleme finden Sie jetzt von selbst, anstatt erst durch eine Nutzerbeschwerde davon zu erfahren. Verbinden Sie Ihren eigenen Scoring-Dienst einmalig, und Failproof AI Observability bewertet jeden abgeschlossenen Lauf automatisch – sodass ein Rückgang der Hilfsbereitschaft oder eine Häufung von Halluzinationen sichtbar wird, bevor ein Kunde es überhaupt merkt. +Qualitätsprobleme finden Sie jetzt von selbst, anstatt erst durch eine Nutzerbeschwerde davon zu erfahren. Verbinden Sie Ihren eigenen Scoring-Dienst einmalig, und FailproofAI Cloud bewertet jeden abgeschlossenen Lauf automatisch – sodass ein Rückgang der Hilfsbereitschaft oder eine Häufung von Halluzinationen sichtbar wird, bevor ein Kunde es überhaupt merkt. -![Das Sessions-Raster mit einer Score-Spalte: Jeder Lauf trägt eine Auswertungs-Statusanzeige sowie farbcodierte Badges für Hilfsbereitschaft, Faktentreue und Tool-Effizienz](/agenteye/images/sessions-list.png) +![Das Sessions-Raster mit einer Score-Spalte: Jeder Lauf trägt eine Auswertungs-Statusanzeige sowie farbcodierte Badges für Hilfsbereitschaft, Faktentreue und Tool-Effizienz](/cloud/images/sessions-list.png) *Jeder Lauf im Sessions-Raster trägt seine Bewertungen; rote, gelbe und grüne Badges machen schwache Läufe sofort erkennbar, ohne dass Sie ein einziges Transkript öffnen müssen.* ## Schluss mit manuellen Stichproben -Früher haben Sie eine Handvoll Läufe stichprobenartig geprüft und gehofft, der Rest sei in Ordnung. Jetzt wird jede abgeschlossene Session in dem Moment bewertet, in dem sie endet – anhand der Dimensionen, die Ihnen wichtig sind: Hilfsbereitschaft, Tool-Effizienz, Faktentreue, Sicherheit oder was auch immer Ihr Qualitätsmaßstab ist. Sie legen die Score-Schlüssel fest; Failproof AI Observability speichert, verfolgt und zeigt alles an, was Ihr Evaluator zurücksendet. Kein Lauf bleibt unbewertet, und Sie erfahren von einem Regressionsfall nicht mehr erst über ein Support-Ticket. +Früher haben Sie eine Handvoll Läufe stichprobenartig geprüft und gehofft, der Rest sei in Ordnung. Jetzt wird jede abgeschlossene Session in dem Moment bewertet, in dem sie endet – anhand der Dimensionen, die Ihnen wichtig sind: Hilfsbereitschaft, Tool-Effizienz, Faktentreue, Sicherheit oder was auch immer Ihr Qualitätsmaßstab ist. Sie legen die Score-Schlüssel fest; FailproofAI Cloud speichert, verfolgt und zeigt alles an, was Ihr Evaluator zurücksendet. Kein Lauf bleibt unbewertet, und Sie erfahren von einem Regressionsfall nicht mehr erst über ein Support-Ticket. Die Bewertungen erscheinen direkt im Sessions-Raster unter **`//sessions`** (Seitenleiste → *observe* → *sessions*), ein Badge-Cluster pro Zeile. Möchten Sie nur die Läufe sehen, die nicht die Erwartungen erfüllt haben? Filtern Sie das Raster nach Score-Bereich – etwa Hilfsbereitschaft unter 0,5 – und rufen Sie genau die Läufe auf, die es wert sind, gelesen zu werden. Zum Anzeigen von Bewertungen wird die Berechtigung `evaluations:read` benötigt. @@ -19,7 +19,7 @@ Die Bewertungen erscheinen direkt im Sessions-Raster unter **`//sessio Eine Zahl sagt Ihnen, dass ein Lauf schwach war; die Session-Seite erklärt Ihnen, warum. Öffnen Sie einen beliebigen Lauf, und die rechte Leiste beginnt mit der übergeordneten Zusammenfassung, gefolgt von einem Balken pro Dimension – jeweils mit der Begründung Ihres Evaluators darunter. So gelangen Sie in Sekunden von „factuality-Score 0,4" zu der genauen Aussage, die falsch war. -![Die rechte Leiste einer Session: oben die Auswertungszusammenfassung, darunter Score-Balken pro Dimension mit je einer Begründungszeile, neben der vollständigen Event-Timeline](/agenteye/images/session-detail.png) +![Die rechte Leiste einer Session: oben die Auswertungszusammenfassung, darunter Score-Balken pro Dimension mit je einer Begründungszeile, neben der vollständigen Event-Timeline](/cloud/images/session-detail.png) *Die Session-Detailansicht: Zusammenfassung, Score-Balken pro Dimension und die Begründung hinter jedem Score – direkt neben der Event-Timeline des Laufs.* @@ -29,7 +29,7 @@ Haben Sie einen präziseren Evaluator bereitgestellt oder schauen Sie sich einen Ein einzelner niedriger Score ist Rauschen; eine ganze Kohorte im Abwärtstrend ist ein Signal. Gespeicherte Dashboards wandeln Ihre Scores in einen Trend um, den Sie auf einen Blick verfolgen können: durchschnittliche Hilfsbereitschaft diese Woche im Vergleich zur letzten, pro Agent, pro Umgebung. -![Ein Qualitäts-Dashboard: durchschnittliche Score-Balken pro Evaluator-Dimension sowie ein zeitlicher Verlaufstrend](/agenteye/images/dashboard-quality.png) +![Ein Qualitäts-Dashboard: durchschnittliche Score-Balken pro Evaluator-Dimension sowie ein zeitlicher Verlaufstrend](/cloud/images/dashboard-quality.png) *Ein gespeichertes Qualitäts-Dashboard zeigt die Trends der von Ihnen hervorgehobenen Score-Schlüssel – sodass eine langsame Verschlechterung lange vor einem Vorfall offensichtlich wird.* @@ -37,14 +37,14 @@ Dashboards finden Sie unter **`//dashboards`** (Seitenleiste → *anal ## Einen Evaluator einmalig verbinden -Die Bewertung ist optional und bleibt vollständig deaktiviert, bis Sie Failproof AI Observability auf einen Scorer verweisen. Sie richten einen kleinen HTTP-Dienst ein (Observability liefert eine funktionierende Referenzimplementierung, die Sie kopieren können), setzen zwei Werte auf Ihrem Server, und von da an wird jeder Lauf automatisch bewertet. Die vollständige Anleitung, den Scoring-Vertrag und das SDK finden Sie im ausführlichen Leitfaden. +Die Bewertung ist optional und bleibt vollständig deaktiviert, bis Sie FailproofAI Cloud auf einen Scorer verweisen. Sie richten einen kleinen HTTP-Dienst ein (FailproofAI Cloud liefert eine funktionierende Referenzimplementierung, die Sie kopieren können), setzen zwei Werte auf Ihrem Server, und von da an wird jeder Lauf automatisch bewertet. Die vollständige Anleitung, den Scoring-Vertrag und das SDK finden Sie im ausführlichen Leitfaden. -Nicht sicher, welche Dimensionen es überhaupt wert sind, bewertet zu werden? Die [Evaluator Agent Skill](/de/agenteye/evaluator-skill) lässt Ihren Coding-Agenten das anhand Ihrer eigenen Sessions herausarbeiten und den Dienst anschließend erstellen und bereitstellen. +Nicht sicher, welche Dimensionen es überhaupt wert sind, bewertet zu werden? Die [Evaluator Agent Skill](/de/cloud/agent-skills) lässt Ihren Coding-Agenten das anhand Ihrer eigenen Sessions herausarbeiten und den Dienst anschließend erstellen und bereitstellen. ## Verwandte Themen -- [Evaluation Suite](/de/agenteye/evaluation-suite): Verbinden Sie Ihren Evaluator, den Scoring-Vertrag und das SDK. -- [Evaluator Agent Skill](/de/agenteye/evaluator-skill): Lassen Sie einen Coding-Agenten Ihre Score-Dimensionen auswählen und den Evaluator erstellen. -- [Sessions](/de/agenteye/sessions): Das laufbezogene Raster, in dem Scores erscheinen. -- [Dashboards](/de/agenteye/dashboards): Qualitätstrends speichern und organisationsweit teilen. -- [Audits](/de/agenteye/audits): Das andere automatische Qualitätsmerkmal von Observability, für sessionübergreifende Untersuchungen. \ No newline at end of file +- [Evaluation Suite](/de/cloud/evaluators): Verbinden Sie Ihren Evaluator, den Scoring-Vertrag und das SDK. +- [Evaluator Agent Skill](/de/cloud/agent-skills): Lassen Sie einen Coding-Agenten Ihre Score-Dimensionen auswählen und den Evaluator erstellen. +- [Sessions](/de/cloud/sessions): Das laufbezogene Raster, in dem Scores erscheinen. +- [Dashboards](/de/cloud/dashboards): Qualitätstrends speichern und organisationsweit teilen. +- [Audits](/de/cloud/audits): Das andere automatische Qualitätsmerkmal von FailproofAI Cloud, für sessionübergreifende Untersuchungen. \ No newline at end of file diff --git a/docs/de/agenteye/evaluation-suite.mdx b/docs/de/cloud/evaluators.mdx similarity index 83% rename from docs/de/agenteye/evaluation-suite.mdx rename to docs/de/cloud/evaluators.mdx index 917a5e29..76d1b127 100644 --- a/docs/de/agenteye/evaluation-suite.mdx +++ b/docs/de/cloud/evaluators.mdx @@ -1,20 +1,20 @@ --- title: "Evaluation Suite" -description: "Failproof AI Observability bewertet automatisch jeden abgeschlossenen Agenten-Lauf auf Qualität: Sie stellen einen kleinen Scoring-Dienst bereit, und Observability erledigt den Rest." +description: "FailproofAI Cloud bewertet automatisch jeden abgeschlossenen Agenten-Lauf auf Qualität: Sie stellen einen kleinen Scoring-Dienst bereit, und FailproofAI Cloud erledigt den Rest." --- -Failproof AI Observability kann jeden abgeschlossenen Agenten-Lauf automatisch auf Qualität bewerten: Sie stellen einen kleinen Scoring-Dienst bereit, und Observability erledigt den Rest. Nutzen Sie es, um die Dimensionen zu verfolgen, die Ihnen wichtig sind (Hilfsbereitschaft, Tool-Effizienz, Faktentreue, Sicherheit – Sie entscheiden), Regressionen frühzeitig zu erkennen und Agenten oder Umgebungen auf einen Blick zu vergleichen. Scoring ist optional: Die Pipeline tut nichts, bis Sie `EVALUATOR_ENDPOINT` auf dem Server setzen. +FailproofAI Cloud kann jeden abgeschlossenen Agenten-Lauf automatisch auf Qualität bewerten: Sie stellen einen kleinen Scoring-Dienst bereit, und FailproofAI Cloud erledigt den Rest. Nutzen Sie es, um die Dimensionen zu verfolgen, die Ihnen wichtig sind (Hilfsbereitschaft, Tool-Effizienz, Faktentreue, Sicherheit – Sie entscheiden), Regressionen frühzeitig zu erkennen und Agenten oder Umgebungen auf einen Blick zu vergleichen. Scoring ist optional: Die Pipeline tut nichts, bis Sie `EVALUATOR_ENDPOINT` auf dem Server setzen. -> **Hinweis:** Sie definieren die Score-Dimensionen. Ihr Evaluator kann beliebige numerische Schlüssel zurückgeben; Observability speichert, verfolgt und zeigt alles an, was Sie zurücksenden. +> **Hinweis:** Sie definieren die Score-Dimensionen. Ihr Evaluator kann beliebige numerische Schlüssel zurückgeben; FailproofAI Cloud speichert, verfolgt und zeigt alles an, was Sie zurücksenden. ## Auf einen Blick -1. **Schreiben Sie einen Scorer.** Starten Sie einen kleinen HTTP-Dienst, der ein Sitzungsprotokoll liest und Scores zurückgibt. Observability liefert ein funktionsfähiges Referenzbeispiel, das Sie kopieren können. Siehe [Evaluator mit dem SDK schreiben](#writing-an-evaluator-with-the-sdk). -2. **Richten Sie Observability darauf aus.** Setzen Sie `EVALUATOR_ENDPOINT` (und ein gemeinsames `EVALUATOR_TOKEN`) auf dem Serverprozess. +1. **Schreiben Sie einen Scorer.** Starten Sie einen kleinen HTTP-Dienst, der ein Sitzungsprotokoll liest und Scores zurückgibt. FailproofAI Cloud liefert ein funktionsfähiges Referenzbeispiel, das Sie kopieren können. Siehe [Evaluator mit dem SDK schreiben](#writing-an-evaluator-with-the-sdk). +2. **Richten Sie FailproofAI Cloud darauf aus.** Setzen Sie `EVALUATOR_ENDPOINT` (und ein gemeinsames `EVALUATOR_TOKEN`) auf dem Serverprozess. 3. **Beobachten Sie die eingehenden Scores.** Jede abgeschlossene Sitzung wird automatisch bewertet; die Ergebnisse erscheinen auf der Sitzungsdetailseite, im Sitzungsraster und in gespeicherten Dashboards. -![Eine Sitzungsdetailansicht mit der Bewertungszusammenfassung, Scores pro Dimension als Balken und Begründungstext in der rechten Spalte](/agenteye/images/session-detail.png) +![Eine Sitzungsdetailansicht mit der Bewertungszusammenfassung, Scores pro Dimension als Balken und Begründungstext in der rechten Spalte](/cloud/images/session-detail.png) *Sobald ein Evaluator konfiguriert ist, wird jeder abgeschlossene Lauf bewertet, und die Ergebnisse erscheinen in der rechten Spalte der Sitzung: oben die Zusammenfassung, dann Score-Balken pro Dimension mit Begründung.* @@ -24,7 +24,7 @@ Failproof AI Observability kann jeden abgeschlossenen Agenten-Lauf automatisch a ```mermaid flowchart LR - ING["ingest /events
agent_end"] --> SRV["Observability server"] + ING["ingest /events
agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -32,14 +32,14 @@ flowchart LR SRV --> RES["evaluations
terminal results"] ``` -Wenn das Observability SDK ein `agent_end`-Ereignis für eine Sitzung auslöst, plant der Server eine Bewertung. Er sendet dann per POST das vollständige Ereignisprotokoll an Ihren Evaluator-Dienst, der entweder: +Wenn das FailproofAI Cloud SDK ein `agent_end`-Ereignis für eine Sitzung auslöst, plant der Server eine Bewertung. Er sendet dann per POST das vollständige Ereignisprotokoll an Ihren Evaluator-Dienst, der entweder: - **Das Ergebnis direkt zurückgibt** mit `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`. Das Ergebnis wird an die Bewertungs-Timeline der Sitzung angehängt. `reasoning` und `summary` sind optional. -- **Verzögert** mit `{"status":"pending", "job_id":"abc-123"}`. Observability ruft dann `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` auf, bis Ihr Evaluator `{"status":"done", ...}` oder `{"status":"error", "error":"..."}` zurückgibt. +- **Verzögert** mit `{"status":"pending", "job_id":"abc-123"}`. FailproofAI Cloud ruft dann `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` auf, bis Ihr Evaluator `{"status":"done", ...}` oder `{"status":"error", "error":"..."}` zurückgibt. - Der Abfrageintervall ist pro Job konfigurierbar: Eine `pending`-Antwort kann `next_poll_secs` enthalten, um den Standardwert zu überschreiben; andernfalls verwendet Observability den Wert `default_poll_interval_secs` aus `GET /config`; ansonsten fällt der Server auf `EVALUATOR_POLLING_INTERVAL_SECS` zurück (Standard: 10 s). Alle Werte werden auf [1 s, 1 h] begrenzt. + Der Abfrageintervall ist pro Job konfigurierbar: Eine `pending`-Antwort kann `next_poll_secs` enthalten, um den Standardwert zu überschreiben; andernfalls verwendet FailproofAI Cloud den Wert `default_poll_interval_secs` aus `GET /config`; ansonsten fällt der Server auf `EVALUATOR_POLLING_INTERVAL_SECS` zurück (Standard: 10 s). Alle Werte werden auf [1 s, 1 h] begrenzt. -Sitzungen, die niemals `agent_end` auslösen (zum Beispiel ein abgestürzter Agentenprozess), können ebenfalls erfasst werden: Das `GET /config` des Evaluators kann `{"inactivity_timeout_secs": 1800}` zurückgeben, und Observability bewertet jede Sitzung, die so lange inaktiv war. Setzen Sie das Feld auf `null` oder lassen Sie es weg, um diesen Fallback zu deaktivieren. +Sitzungen, die niemals `agent_end` auslösen (zum Beispiel ein abgestürzter Agentenprozess), können ebenfalls erfasst werden: Das `GET /config` des Evaluators kann `{"inactivity_timeout_secs": 1800}` zurückgeben, und FailproofAI Cloud bewertet jede Sitzung, die so lange inaktiv war. Setzen Sie das Feld auf `null` oder lassen Sie es weg, um diesen Fallback zu deaktivieren. Die Pipeline ist vollständig inaktiv, wenn `EVALUATOR_ENDPOINT` nicht gesetzt ist. @@ -47,7 +47,7 @@ Eine Sitzung kann **mehrere abschließende Bewertungen im Laufe der Zeit** ansam Der Inaktivitäts-Fallback greift auch bei wiederaufgenommenen Sitzungen: Wenn nach einer vorherigen abschließenden Bewertung neue Ereignisse eintreffen und die Sitzung dann länger als `inactivity_timeout_secs` inaktiv bleibt, wird eine neue Bewertung in die Warteschlange gestellt. -Vorübergehende Fehler (5xx, 429, Timeouts, Netzwerkfehler) werden mit exponentiellem Backoff bis zu `EVALUATOR_MAX_ATTEMPTS` wiederholt; 4xx-Antworten sind endgültig. Observability kann sicher mit mehreren horizontal skalierten Serverinstanzen betrieben werden; die Arbeit wird so aufgeteilt, dass dieselbe Sitzung nie gleichzeitig zweimal verteilt wird. +Vorübergehende Fehler (5xx, 429, Timeouts, Netzwerkfehler) werden mit exponentiellem Backoff bis zu `EVALUATOR_MAX_ATTEMPTS` wiederholt; 4xx-Antworten sind endgültig. FailproofAI Cloud kann sicher mit mehreren horizontal skalierten Serverinstanzen betrieben werden; die Arbeit wird so aufgeteilt, dass dieselbe Sitzung nie gleichzeitig zweimal verteilt wird. --- @@ -55,7 +55,7 @@ Vorübergehende Fehler (5xx, 429, Timeouts, Netzwerkfehler) werden mit exponenti Alle authentifizierten Routen verwenden **Bearer-Token-Authentifizierung**. Derselbe Wert muss auf beiden Seiten konfiguriert sein: -- Observability-Server: Umgebungsvariable `EVALUATOR_TOKEN` +- FailproofAI Cloud-Server: Umgebungsvariable `EVALUATOR_TOKEN` - Evaluator-Dienst: auf dieselbe Weise konfiguriert (das `agenteye-evaluator` SDK liest `EVALUATOR_TOKEN` gemäß Konvention) Wenn `EVALUATOR_TOKEN` nicht gesetzt ist, sendet der Server keinen `Authorization`-Header; der Evaluator kann dann anonyme Anfragen akzeptieren, was für ein rein internes Netzwerk in Ordnung ist, im öffentlichen Internet jedoch nicht empfohlen wird. @@ -126,7 +126,7 @@ Der Server behandelt jeden anderen 2xx-Body als Protokollfehler und protokollier Sie müssen den HTTP-Vertrag nicht manuell implementieren. Das Python-Paket `agenteye-evaluator` bietet Ihnen einen typisierten FastAPI-Wrapper, der Authentifizierung, Routing und die Anfrage-/Antwortformate für Sie übernimmt. -Failproof AI Observability liefert auch einen **funktionsfähigen Referenz-Evaluator**, der `helpfulness`, `tool_efficiency` und `factuality` anhand der Struktur des Protokolls bewertet. Kopieren Sie ihn als Ausgangspunkt und tauschen Sie Ihre eigene Logik ein: ein LLM-Richter, eine Regelmaschine – was auch immer Ihrem Qualitätsstandard entspricht. +FailproofAI Cloud liefert auch einen **funktionsfähigen Referenz-Evaluator**, der `helpfulness`, `tool_efficiency` und `factuality` anhand der Struktur des Protokolls bewertet. Kopieren Sie ihn als Ausgangspunkt und tauschen Sie Ihre eigene Logik ein: ein LLM-Richter, eine Regelmaschine – was auch immer Ihrem Qualitätsstandard entspricht. Minimal funktionsfähiger Evaluator: @@ -149,7 +149,7 @@ def run(req: EvalRequest) -> EvalResponse: Die `app`-Instanz läuft unter jedem ASGI-Server, sodass `uvicorn module:app` sie startet. -Für Evaluatoren, die aufwändige Arbeit verzögern müssen, geben Sie stattdessen `JobPending` zurück und registrieren Sie einen `@app.job_lookup`-Handler; der Observability-Server fragt `GET /evaluate/{job_id}` ab, bis Sie einen endgültigen Status zurückgeben oder die Obergrenze `EVALUATOR_MAX_POLL_DURATION_SECS` (Standard: 1 h) erreicht wird. +Für Evaluatoren, die aufwändige Arbeit verzögern müssen, geben Sie stattdessen `JobPending` zurück und registrieren Sie einen `@app.job_lookup`-Handler; der FailproofAI Cloud-Server fragt `GET /evaluate/{job_id}` ab, bis Sie einen endgültigen Status zurückgeben oder die Obergrenze `EVALUATOR_MAX_POLL_DURATION_SECS` (Standard: 1 h) erreicht wird. Die vollständige API-Referenz, das asynchrone Muster und das Ereignisschema sind in der README des `agenteye-evaluator` SDK dokumentiert. @@ -157,7 +157,7 @@ Die vollständige API-Referenz, das asynchrone Muster und das Ereignisschema sin ## Ihren Evaluator betreiben -Der Evaluator ist **Ihr Dienst** – Failproof AI Observability liefert keinen Standard-Evaluator, daher erstellen und betreiben Sie ihn dort, wo Sie Ihre eigenen Dienste betreiben. Er läuft unter jedem ASGI-Server (zum Beispiel `uvicorn my_evaluator:app`); stellen Sie die Routen `/health`, `/config` und `/evaluate` gemäß dem [HTTP-Vertrag](#http-contract) bereit, und verweisen Sie den Server darauf (siehe [Server konfigurieren](#configuring-the-server)). +Der Evaluator ist **Ihr Dienst** – FailproofAI Cloud liefert keinen Standard-Evaluator, daher erstellen und betreiben Sie ihn dort, wo Sie Ihre eigenen Dienste betreiben. Er läuft unter jedem ASGI-Server (zum Beispiel `uvicorn my_evaluator:app`); stellen Sie die Routen `/health`, `/config` und `/evaluate` gemäß dem [HTTP-Vertrag](#http-contract) bereit, und verweisen Sie den Server darauf (siehe [Server konfigurieren](#configuring-the-server)). Sobald der Evaluator erreichbar ist, gibt `GET /health` `{"status":"ok"}` zurück. Nachdem ein Agent vollständig durchgelaufen ist, gibt `GET /evaluations` auf dem Server eine Zeile mit `status: "done"` und den von Ihrem Evaluator erzeugten Scores zurück. @@ -255,7 +255,7 @@ Der Bootstrap-Administrator (`ADMIN_KEY`, `ADMIN_EMAIL`) erhält diese automatis - **`/sessions`**: filterbares Sitzungsraster; die Score-Spalte zeigt den Bewertungsstatus und die Scores jeder Sitzung auf einen Blick. - **`/dashboards`**: gespeicherte Bewertungsqualitätsansichten (siehe [Dashboards](#dashboards) unten). -![Das Sitzungsraster mit Bewertungsstatuspillen pro Sitzung und farbcodierten Score-Abzeichen (helpfulness, factuality, tool_efficiency, safety, coherence)](/agenteye/images/sessions-list.png) +![Das Sitzungsraster mit Bewertungsstatuspillen pro Sitzung und farbcodierten Score-Abzeichen (helpfulness, factuality, tool_efficiency, safety, coherence)](/cloud/images/sessions-list.png) *Das Sitzungsraster zeigt den Bewertungsstatus und die Scores jedes Laufs auf einen Blick; rote/gelbe/grüne Abzeichen lassen niedrige Scores sofort auffallen.* @@ -272,7 +272,7 @@ Jedes Dashboard fixiert: Jede Karte zeigt die Anzahl übereinstimmender Sitzungen, eine done/error/timeout-Aufschlüsselung, den Durchschnitt jedes hervorgehobenen Scores und eine kleine Trend-Sparkline. Das Öffnen eines Dashboards zeigt die vollständigen Panels; **„In Sitzungen öffnen"** führt Sie zur Sitzungsseite, die genau auf dieses Segment vorge filtert ist. Metriken werden serverseitig über den gesamten übereinstimmenden Datensatz berechnet (über `GET /evaluations/aggregate`), sodass die Zahlen exakt und nicht gesampelt sind. -![Ein Bewertungsqualitäts-Dashboard mit durchschnittlichen Score-Balken pro Evaluatordimension, einer Tool-ok-vs-error-Aufschlüsselung, Top-Tools und einem Ereignisse-pro-Stunde-Trend](/agenteye/images/dashboard-quality.png) +![Ein Bewertungsqualitäts-Dashboard mit durchschnittlichen Score-Balken pro Evaluatordimension, einer Tool-ok-vs-error-Aufschlüsselung, Top-Tools und einem Ereignisse-pro-Stunde-Trend](/cloud/images/dashboard-quality.png) **Berechtigungen:** Anzeigen erfordert sowohl `dashboards:read` als auch `evaluations:read`; Erstellen und Bearbeiten erfordert `dashboards:write`; Löschen erfordert `dashboards:delete`. Der Bootstrap-Administrator erhält all diese automatisch. @@ -294,7 +294,7 @@ Jede Karte zeigt die Anzahl übereinstimmender Sitzungen, eine done/error/timeou ## Nächste Schritte -- [Evaluator-Agenten-Skill](/de/agenteye/evaluator-skill): Lassen Sie einen Coding-Agenten Ihre Dimensionen anhand echter Sitzungen entwerfen und diesen Dienst für Sie erstellen. -- [Python SDK](/de/agenteye/python-sdk): Die `agent_end`-Ereignisse auslösen, die das Scoring anstoßen. -- [API-Schlüssel](/de/agenteye/api-keys): Die Berechtigungen `evaluations:read` und `evaluations:trigger`. -- [Audits](/de/agenteye/audits): Die andere automatisierte Qualitätsfunktion von Observability für richtlinienbasierte Überprüfungen. \ No newline at end of file +- [Evaluator-Agenten-Skill](/de/cloud/agent-skills): Lassen Sie einen Coding-Agenten Ihre Dimensionen anhand echter Sitzungen entwerfen und diesen Dienst für Sie erstellen. +- [Python SDK](/de/cloud/sdk): Die `agent_end`-Ereignisse auslösen, die das Scoring anstoßen. +- [API-Schlüssel](/de/cloud/access): Die Berechtigungen `evaluations:read` und `evaluations:trigger`. +- [Audits](/de/cloud/audits): Die andere automatisierte Qualitätsfunktion von FailproofAI Cloud für richtlinienbasierte Überprüfungen. \ No newline at end of file diff --git a/docs/de/agenteye/event-stream.mdx b/docs/de/cloud/event-stream.mdx similarity index 73% rename from docs/de/agenteye/event-stream.mdx rename to docs/de/cloud/event-stream.mdx index 8bf38095..659323e8 100644 --- a/docs/de/agenteye/event-stream.mdx +++ b/docs/de/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "In dem Moment, in dem dein Agent etwas tut, siehst du es." In dem Moment, in dem dein Agent etwas tut, siehst du es. Der Event Stream ist dein Live-Puls auf jeden Agenten in der Produktion: kein Warten, kein Durchsuchen von Logs, kein Rätselraten, was gerade passiert ist. -![Der Live-Event-Stream: farblich kodierte Event-Zeilen, die in Echtzeit eingehen, filterbar nach Umgebung, Agent, Session, Event-Typ und Freitext](/agenteye/images/events-stream.png) +![Der Live-Event-Stream: farblich kodierte Event-Zeilen, die in Echtzeit eingehen, filterbar nach Umgebung, Agent, Session, Event-Typ und Freitext](/cloud/images/events-stream.png) *Jedes Event von jedem Agenten in deiner Organisation, neueste zuerst, aktualisiert sich in Echtzeit.* @@ -37,14 +37,14 @@ Die Freitextsuche führt dich direkt zu einer Nachricht, einem Tool-Namen oder e Der Event Stream ist die Startseite deiner Organisation. Melde dich an, und er ist die erste Ansicht, die du siehst, unter `//` – die Triage beginnt also in dem Moment, in dem du ankommst. -Im Hintergrund senden deine Agenten Events über das SDK, der Collector leitet sie an deinen Failproof AI Observability-Server weiter, und der Stream verfolgt sie, sobald sie in deiner kontrollierten Infrastruktur ankommen. Wenn du statt des rohen Trails die Gesamtübersicht möchtest, kollabieren die Events eines Laufs auf Sessions zu einer einzelnen Zeile – einen Klick entfernt. +Im Hintergrund senden deine Agenten Events über das SDK, der Collector leitet sie an deinen FailproofAI Cloud-Server weiter, und der Stream verfolgt sie, sobald sie in deiner kontrollierten Infrastruktur ankommen. Wenn du statt des rohen Trails die Gesamtübersicht möchtest, kollabieren die Events eines Laufs auf Sessions zu einer einzelnen Zeile – einen Klick entfernt. -Dies ist die rohe Quelle der Wahrheit, auf der jede andere Observability-Ansicht aufbaut. Wenn eine Zahl anderswo falsch aussieht, ist der Stream der Ort, an dem du bestätigst, was tatsächlich passiert ist. +Dies ist die rohe Quelle der Wahrheit, auf der jede andere FailproofAI Cloud-Ansicht aufbaut. Wenn eine Zahl anderswo falsch aussieht, ist der Stream der Ort, an dem du bestätigst, was tatsächlich passiert ist. ## Verwandte Themen -- [Sessions](/de/agenteye/sessions): dieselben Events zusammengefasst zu einer Zeile pro Lauf, mit einem Git-artigen Ausführungsgraphen. -- [Telemetry](/de/agenteye/telemetry): was deine Agenten senden und wie Events den Stream erreichen. -- [Error tracking](/de/agenteye/error-tracking): eine einzige Triage-Ansicht für alles, was schiefgelaufen ist. -- [Alerts](/de/agenteye/alerts): wandle jeden Schwellenwert in eine Benachrichtigungsregel um. -- [CLI and agents](/de/agenteye/cli-and-agents): derselbe Live-Trail aus deinem Terminal. \ No newline at end of file +- [Sessions](/de/cloud/sessions): dieselben Events zusammengefasst zu einer Zeile pro Lauf, mit einem Git-artigen Ausführungsgraphen. +- [Telemetry](/de/cloud/performance): was deine Agenten senden und wie Events den Stream erreichen. +- [Error tracking](/de/cloud/errors): eine einzige Triage-Ansicht für alles, was schiefgelaufen ist. +- [Alerts](/de/cloud/alerts): wandle jeden Schwellenwert in eine Benachrichtigungsregel um. +- [CLI and agents](/de/cloud/cli): derselbe Live-Trail aus deinem Terminal. \ No newline at end of file diff --git a/docs/de/cloud/fleet.mdx b/docs/de/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/de/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/de/agenteye/incidents.mdx b/docs/de/cloud/incidents.mdx similarity index 85% rename from docs/de/agenteye/incidents.mdx rename to docs/de/cloud/incidents.mdx index 37b831b5..eb77a09d 100644 --- a/docs/de/agenteye/incidents.mdx +++ b/docs/de/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "Wenn ein Alert ausgelöst wird, sieht jeder, dass der Incident off Wenn ein Alert ausgelöst wird, lautet die erste Frage immer: „Wer kümmert sich darum?" Incidents liefern die Antwort: Sobald eine Schwellenwertüberschreitung eintritt, sieht jeder, dass der Incident offen ist, wer ihn verantwortet und was bisher genau passiert ist – als saubere, zugeordnete Dokumentation, die sich direkt für eine Post-mortem-Analyse verwenden lässt. -![Der Incidents-Posteingang: alert-verknüpfte und manuell geöffnete Incident-Karten, nach Status gruppiert, jeweils mit Schweregrad-Badge und zugewiesener Person](/agenteye/images/incidents.png) +![Der Incidents-Posteingang: alert-verknüpfte und manuell geöffnete Incident-Karten, nach Status gruppiert, jeweils mit Schweregrad-Badge und zugewiesener Person](/cloud/images/incidents.png) *Der Posteingang gruppiert offene Incidents nach Status und filtert nach Schweregrad und zugewiesener Person, sodass sofort ersichtlich ist, was jetzt menschliches Eingreifen erfordert.* ## Auf einen Blick sehen, wer zuständig ist @@ -17,10 +17,10 @@ Kein „Schaut da gerade jemand drauf?" mehr im Chat. Eine Schwellenwertübersch Wenn der Incident abgeschlossen ist, ist das Protokoll bereits fertig. Beim Öffnen eines Incidents sind der Auslöser, eine Zusammenfassung der Überschreitung, zugewiesene Personen und Abonnenten, ein Kommentarbereich zur direkten Koordination sowie eine unveränderliche Aktivitäts-Timeline sichtbar. -![Eine Incident-Detailansicht: der übergeordnete Alert und die Überschreitungszusammenfassung, zugewiesene Personen und Abonnenten, eine zugeordnete Aktivitäts-Timeline und ein Kommentarbereich](/agenteye/images/incident-detail.png) +![Eine Incident-Detailansicht: der übergeordnete Alert und die Überschreitungszusammenfassung, zugewiesene Personen und Abonnenten, eine zugeordnete Aktivitäts-Timeline und ein Kommentarbereich](/cloud/images/incident-detail.png) *Alles, was passiert ist, in chronologischer Reihenfolge – jede Zeile mit dem Namen der verantwortlichen Person.* -Jede Aktion (geöffnet, bestätigt, gelöst usw.) wird in diese Timeline geschrieben und niemals nachträglich geändert. Jeder Eintrag ist zugeordnet: per E-Mail dem Operator, der die Aktion durchgeführt hat, oder **automated** für alles, was Failproof AI Observability selbstständig getan hat – beispielsweise das Öffnen des Incidents bei einer Schwellenwertüberschreitung. Nichts ist anonym und nichts geht verloren, sodass die Post-mortem-Analyse nahezu von selbst entsteht. +Jede Aktion (geöffnet, bestätigt, gelöst usw.) wird in diese Timeline geschrieben und niemals nachträglich geändert. Jeder Eintrag ist zugeordnet: per E-Mail dem Operator, der die Aktion durchgeführt hat, oder **automated** für alles, was FailproofAI Cloud selbstständig getan hat – beispielsweise das Öffnen des Incidents bei einer Schwellenwertüberschreitung. Nichts ist anonym und nichts geht verloren, sodass die Post-mortem-Analyse nahezu von selbst entsteht. ## Wie sich ein Incident entwickelt @@ -45,6 +45,6 @@ Incidents befinden sich unter `//incidents`. Für die Anzeige wird **` ## Verwandte Themen -- [Alerts](/de/agenteye/alerts): die Regeln, die Incidents öffnen, wenn ein Schwellenwert überschritten wird. -- [Error Tracking](/de/agenteye/error-tracking): alle Fehler an einem Ort einsehen und einen davon zu einem Alert heraufstufen. -- [Audits](/de/agenteye/audits): der geplante Analyst, der Fehler findet, die von keiner Regel überwacht wurden. \ No newline at end of file +- [Alerts](/de/cloud/alerts): die Regeln, die Incidents öffnen, wenn ein Schwellenwert überschritten wird. +- [Error Tracking](/de/cloud/errors): alle Fehler an einem Ort einsehen und einen davon zu einem Alert heraufstufen. +- [Audits](/de/cloud/audits): der geplante Analyst, der Fehler findet, die von keiner Regel überwacht wurden. \ No newline at end of file diff --git a/docs/de/cloud/managed-policies.mdx b/docs/de/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/de/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/de/agenteye/overview.mdx b/docs/de/cloud/overview.mdx similarity index 50% rename from docs/de/agenteye/overview.mdx rename to docs/de/cloud/overview.mdx index ee0650c8..3f589e33 100644 --- a/docs/de/agenteye/overview.mdx +++ b/docs/de/cloud/overview.mdx @@ -1,16 +1,16 @@ --- title: "Failproof AI: Agenten auf Fehler überwachen" -description: "Failproof AI Observability ist eine selbst gehostete Plattform zur Beobachtung, Bewertung und Verbesserung Ihrer KI-Agenten in der Produktion." +description: "FailproofAI Cloud ist eine selbst gehostete Plattform zur Beobachtung, Bewertung und Verbesserung Ihrer KI-Agenten in der Produktion." --- -Failproof AI Observability ist eine selbst gehostete Plattform zur Beobachtung, Bewertung und Verbesserung Ihrer KI-Agenten in der Produktion. Sie zeichnet alles auf, was Ihre Agenten tun (jeden Tool-Aufruf, jede Modellanfrage, jeden Hook und jeden Fehler), bewertet die Qualität jedes Durchlaufs und zeigt Ihnen die Fehler, nach denen Sie nicht aktiv gesucht haben – alles in einem Dashboard, das Sie in Ihrer eigenen Infrastruktur betreiben. +FailproofAI Cloud ist eine selbst gehostete Plattform zur Beobachtung, Bewertung und Verbesserung Ihrer KI-Agenten in der Produktion. Sie zeichnet alles auf, was Ihre Agenten tun (jeden Tool-Aufruf, jede Modellanfrage, jeden Hook und jeden Fehler), bewertet die Qualität jedes Durchlaufs und zeigt Ihnen die Fehler, nach denen Sie nicht aktiv gesucht haben – alles in einem Dashboard, das Sie in Ihrer eigenen Infrastruktur betreiben. -Wenn Sie KI-Agenten einsetzen und es leid sind zu rätseln, warum ein Durchlauf schiefgelaufen ist, sind Sie hier genau richtig. Diese Seite erklärt, was Failproof AI Observability Ihnen bietet und wie die einzelnen Teile zusammenpassen – noch bevor Sie irgendetwas installieren. +Wenn Sie KI-Agenten einsetzen und es leid sind zu rätseln, warum ein Durchlauf schiefgelaufen ist, sind Sie hier genau richtig. Diese Seite erklärt, was FailproofAI Cloud Ihnen bietet und wie die einzelnen Teile zusammenpassen – noch bevor Sie irgendetwas installieren. -> **Failproof AI Observability ist ein Enterprise-Produkt von Failproof AI.** Sie möchten es in Aktion sehen? Fordern Sie eine Demo an: E-Mail an [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +> **FailproofAI Cloud ist ein Enterprise-Produkt von Failproof AI.** Sie möchten es in Aktion sehen? Fordern Sie eine Demo an: E-Mail an [nikita@befailproof.ai](mailto:nikita@befailproof.ai). -![Eine Failproof AI Observability-Sitzung als git-ähnlicher Ausführungsgraph neben der Ereigniszeitachse, mit einer Aufschlüsselung von Tools, Modellen und Hooks in der rechten Seitenleiste](/agenteye/images/session-detail.png) +![Eine FailproofAI Cloud-Sitzung als git-ähnlicher Ausführungsgraph neben der Ereigniszeitachse, mit einer Aufschlüsselung von Tools, Modellen und Hooks in der rechten Seitenleiste](/cloud/images/session-detail.png) *Jeder Agentendurchlauf wird als git-ähnlicher Ausführungsgraph (links) neben seiner Ereigniszeitachse dargestellt. Parallele Unteragenten erhalten jeweils ihre eigene Spur; die rechte Seitenleiste schlüsselt die Tools, Modelle, Hooks und den Token-Verbrauch des Durchlaufs auf.* @@ -30,52 +30,52 @@ Zwei kurze Videos zeigen die zwei Dinge, die Teams zuerst nutzen: einen Durchlau
-*Failproof Audit: Lassen Sie Failproof AI Observability Ihre Logs sitzungsübergreifend durchsuchen und erfahren Sie, was behoben werden muss.* +*Failproof Audit: Lassen Sie FailproofAI Cloud Ihre Logs sitzungsübergreifend durchsuchen und erfahren Sie, was behoben werden muss.* --- ## Warum Teams es nutzen - **Sehen Sie, was Ihr Agent wirklich getan hat.** Jeder Durchlauf wird zu einem lesbaren, git-ähnlichen Ausführungsgraphen: welche Tools parallel liefen, welche Unteragenten abgezweigt wurden, wo es ins Stocken geriet und was es gekostet hat. -- **Qualitätsrückgänge automatisch erkennen.** Verbinden Sie einen kleinen Scoring-Dienst, und Failproof AI Observability bewertet jeden abgeschlossenen Durchlauf – sodass ein Rückgang der Hilfsbereitschaft oder ein Anstieg von Halluzinationen von selbst sichtbar wird. +- **Qualitätsrückgänge automatisch erkennen.** Verbinden Sie einen kleinen Scoring-Dienst, und FailproofAI Cloud bewertet jeden abgeschlossenen Durchlauf – sodass ein Rückgang der Hilfsbereitschaft oder ein Anstieg von Halluzinationen von selbst sichtbar wird. - **Fehler finden, für die Sie keine Regel geschrieben haben.** Regelmäßige Audits durchsuchen Ihre Logs sitzungsübergreifend nach Fehlerclustern, Latenz-Ausreißern, niedrigen Bewertungen und hängenden Durchläufen und liefern Ihnen priorisierte, evidenzbasierte Erkenntnisse. - **Benachrichtigt werden, wenn es darauf ankommt.** Schwellenwertregeln reagieren auf Fehlerrate, Latenz, Kosten oder Evaluator-Scores und eröffnen Incidents, die Sie bestätigen, zuweisen und lösen können. - **Fragen in natürlicher Sprache stellen.** Ein KI-Assistent im Dashboard beantwortet Fragen wie „Wie entwickelt sich die Qualität in der Produktion diese Woche?" – auf Basis Ihrer eigenen Daten. Jede Änderung, die er vornimmt, ist genehmigungspflichtig. -- **Ihre Daten behalten.** Failproof AI Observability ist selbst gehostet: Ereignisse, Prompts und Analysen bleiben in der von Ihnen kontrollierten Infrastruktur. +- **Ihre Daten behalten.** FailproofAI Cloud ist selbst gehostet: Ereignisse, Prompts und Analysen bleiben in der von Ihnen kontrollierten Infrastruktur. --- ## Was Sie erhalten -Failproof AI Observability ist um drei Ideen herum organisiert (**Beobachten**, **Analysieren** und **Verwalten**), die in der linken Seitenleiste des Dashboards gespiegelt werden. +FailproofAI Cloud ist um drei Ideen herum organisiert (**Beobachten**, **Analysieren** und **Verwalten**), die in der linken Seitenleiste des Dashboards gespiegelt werden. **Beobachten** (die unverfälschte Wahrheit dessen, was passiert ist): -- **[Ereignis-Stream](/de/agenteye/event-stream)**: die Live-Aufzeichnung jedes einzelnen Schritts jedes Durchlaufs (Tool-Aufrufe, Modellaufrufe, Hooks, Fehler). -- **[Sitzungen](/de/agenteye/sessions)**: diese Ereignisse zusammengefasst zu einer Zeile pro Durchlauf, jeweils bereit zur Bewertung, mit einem git-ähnlichen Ausführungsgraphen. -- **[Performance-Metriken](/de/agenteye/telemetry)**: Latenz-Heatmaps pro Oberfläche und p50/p95/p99-Werte für Modelle, Tools und Hooks, damit ein Ausreißer im langen Ende sofort auffällt. -- **[Fehlerverfolgung](/de/agenteye/error-tracking)**: eine einzige Triage-Oberfläche für alles, was schiefgelaufen ist, einen Klick von einem ausgelösten Alert entfernt. +- **[Ereignis-Stream](/de/cloud/event-stream)**: die Live-Aufzeichnung jedes einzelnen Schritts jedes Durchlaufs (Tool-Aufrufe, Modellaufrufe, Hooks, Fehler). +- **[Sitzungen](/de/cloud/sessions)**: diese Ereignisse zusammengefasst zu einer Zeile pro Durchlauf, jeweils bereit zur Bewertung, mit einem git-ähnlichen Ausführungsgraphen. +- **[Performance-Metriken](/de/cloud/performance)**: Latenz-Heatmaps pro Oberfläche und p50/p95/p99-Werte für Modelle, Tools und Hooks, damit ein Ausreißer im langen Ende sofort auffällt. +- **[Fehlerverfolgung](/de/cloud/errors)**: eine einzige Triage-Oberfläche für alles, was schiefgelaufen ist, einen Klick von einem ausgelösten Alert entfernt. -![Die Tools-Beobachtungsseite: eine Latenz-Heatmap, ein Perzentil-Band und ein Tool-Verteilungsbalken über 24 Zeitabschnitte](/agenteye/images/tools.png) +![Die Tools-Beobachtungsseite: eine Latenz-Heatmap, ein Perzentil-Band und ein Tool-Verteilungsbalken über 24 Zeitabschnitte](/cloud/images/tools.png) *Jede Beobachtungsoberfläche kombiniert eine Sparkline und p50/p95/p99-Werte mit einer Latenz-Heatmap und einem Perzentil-Band. Hier gezeigt: Tools.* **Analysieren** (Aktivitäten in Erkenntnisse verwandeln): -- **[Abfragen](/de/agenteye/queries)** und **[Dashboards](/de/agenteye/dashboards)**: gespeichertes SQL über Ihre Ereignisse und Evaluierungen, als geteilte, organisationsweite Dashboards visualisiert. -- **[Evaluierungen](/de/agenteye/evaluations)**: Qualitätsbewertungen, die von Ihrem eigenen Evaluator-Dienst erstellt werden, mit Begründung pro Bewertung. -- **[Audits](/de/agenteye/audits)**: wiederkehrende Untersuchungen, die Fehlermuster sitzungsübergreifend aufdecken. -- **[Alerts](/de/agenteye/alerts)** und **[Incidents](/de/agenteye/incidents)**: Schwellenwertregeln, die Sie benachrichtigen, sowie ein Incident-Workflow zur Triage. +- **[Abfragen](/de/cloud/queries)** und **[Dashboards](/de/cloud/dashboards)**: gespeichertes SQL über Ihre Ereignisse und Evaluierungen, als geteilte, organisationsweite Dashboards visualisiert. +- **[Evaluierungen](/de/cloud/evaluations)**: Qualitätsbewertungen, die von Ihrem eigenen Evaluator-Dienst erstellt werden, mit Begründung pro Bewertung. +- **[Audits](/de/cloud/audits)**: wiederkehrende Untersuchungen, die Fehlermuster sitzungsübergreifend aufdecken. +- **[Alerts](/de/cloud/alerts)** und **[Incidents](/de/cloud/incidents)**: Schwellenwertregeln, die Sie benachrichtigen, sowie ein Incident-Workflow zur Triage. **Schnittstellen** (auf Ihre Daten auf Ihre Weise zugreifen): -- **[CLI](/de/agenteye/cli-and-agents)**: Steuern Sie Ihre gesamte Deployment vom Terminal oder einem Skript aus, und lassen Sie einen Coding-Agenten dies für Sie in natürlicher Sprache erledigen. -- **[KI-Assistent](/de/agenteye/assistant)**: Stellen Sie Fragen zu Ihren Agenten in natürlicher Sprache, direkt im Dashboard. -- **REST API**: Alles, was Dashboard und CLI tun, wird durch eine REST API unterstützt, die Sie direkt mit einem bereichsbegrenzten [API-Schlüssel](/de/agenteye/api-keys) aufrufen können – Ereignisse erfassen, Sitzungen und Evaluierungen abfragen sowie Dashboards, Alerts, Audits, Benutzer und Schlüssel verwalten, sodass Sie Failproof AI Observability in Ihr eigenes Tooling integrieren können. +- **[CLI](/de/cloud/cli)**: Steuern Sie Ihre gesamte Deployment vom Terminal oder einem Skript aus, und lassen Sie einen Coding-Agenten dies für Sie in natürlicher Sprache erledigen. +- **[KI-Assistent](/de/cloud/assistant)**: Stellen Sie Fragen zu Ihren Agenten in natürlicher Sprache, direkt im Dashboard. +- **REST API**: Alles, was Dashboard und CLI tun, wird durch eine REST API unterstützt, die Sie direkt mit einem bereichsbegrenzten [API-Schlüssel](/de/cloud/access) aufrufen können – Ereignisse erfassen, Sitzungen und Evaluierungen abfragen sowie Dashboards, Alerts, Audits, Benutzer und Schlüssel verwalten, sodass Sie FailproofAI Cloud in Ihr eigenes Tooling integrieren können. **Verwaltung** (für Ihr Team betreiben): -- **[API-Schlüssel](/de/agenteye/api-keys)**: bereichsbegrenzte Token für den Collector, das Dashboard und den Assistenten. +- **[API-Schlüssel](/de/cloud/access)**: bereichsbegrenzte Token für den Collector, das Dashboard und den Assistenten. - **Benutzer**: passwortlose, E-Mail-basierte Anmeldung mit einer Zulassungsliste. - **Einstellungen**: organisationsweite Konfiguration, einschließlich Modell-Kontextfenster-Überschreibungen. @@ -91,18 +91,18 @@ Daten fließen in eine Richtung, von Ihrem Agenten-Code zum Dashboard: Ihr Agent - **Dashboard**: wo Sie alles erkunden. - **Optionale Dienste**: ein Scoring-Dienst (Evaluierungen) und ein KI-Assistenten-Dienst (der In-Dashboard-Chat). -Für das in der gesamten Dokumentation verwendete Vokabular (*Ereignis, Sitzung, Evaluierung, Audit, Befund, Incident*) siehe [Konzepte](/de/agenteye/concepts). +Für das in der gesamten Dokumentation verwendete Vokabular (*Ereignis, Sitzung, Evaluierung, Audit, Befund, Incident*) siehe [Konzepte](/de/concepts). --- -## Failproof AI Observability erhalten +## FailproofAI Cloud erhalten -Failproof AI Observability ist ein Enterprise-Produkt von Failproof AI und funktioniert zusammen mit Failproof AI Enforcement – dem Richtlinien- und Guardrail-Produkt – unter der Failproof AI-Marke. Es läuft vollständig in Ihrer eigenen Umgebung. Wenn Sie noch keinen Zugang zu den Paketen haben, fordern Sie eine Demo an, und wir richten alles für Sie ein: E-Mail an [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +FailproofAI Cloud ist ein Enterprise-Produkt von Failproof AI und funktioniert zusammen mit FailproofAI guardrails – dem Richtlinien- und Guardrail-Produkt – unter der Failproof AI-Marke. Es läuft vollständig in Ihrer eigenen Umgebung. Wenn Sie noch keinen Zugang zu den Paketen haben, fordern Sie eine Demo an, und wir richten alles für Sie ein: E-Mail an [nikita@befailproof.ai](mailto:nikita@befailproof.ai). --- ## Nächste Schritte -- [Konzepte](/de/agenteye/concepts): das Failproof AI Observability-Vokabular an einem Ort. -- [Observability](/de/agenteye/observability): Verfolgen Sie, was Ihre Agenten tun, Durchlauf für Durchlauf. -- [Sicherheit](/de/agenteye/security): Wie Failproof AI Observability Ihre Daten isoliert und unter Ihrer Kontrolle hält. \ No newline at end of file +- [Konzepte](/de/concepts): das FailproofAI Cloud-Vokabular an einem Ort. +- [FailproofAI Cloud](/de/cloud/overview): Verfolgen Sie, was Ihre Agenten tun, Durchlauf für Durchlauf. +- [Sicherheit](/de/cloud/security): Wie FailproofAI Cloud Ihre Daten isoliert und unter Ihrer Kontrolle hält. \ No newline at end of file diff --git a/docs/de/agenteye/telemetry.mdx b/docs/de/cloud/performance.mdx similarity index 82% rename from docs/de/agenteye/telemetry.mdx rename to docs/de/cloud/performance.mdx index 0a731789..603f9e5d 100644 --- a/docs/de/agenteye/telemetry.mdx +++ b/docs/de/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "Erkenne sofort, wenn deine Modelle, Tools oder Hooks langsamer wer Erkenne sofort, wenn deine Modelle, Tools oder Hooks langsamer werden oder Kosten verursachen, und fange Tail-Latency-Spitzen ab, bevor deine Nutzer sie überhaupt bemerken. Drei dedizierte Seiten verwandeln rohe Laufzeiten in p50, p95 und p99, die du auf einen Blick ablesen kannst. -![Die Models-Seite mit einer Latency-Heatmap, einem Percentile-Band und modellspezifischen Token-, Kosten- und Kontextfenster-Werten](/agenteye/images/models.png) +![Die Models-Seite mit einer Latency-Heatmap, einem Percentile-Band und modellspezifischen Token-, Kosten- und Kontextfenster-Werten](/cloud/images/models.png) *Die Models-Seite: eine Latency-Heatmap, ein Percentile-Band sowie modellspezifische Token-Zahlen, geschätzte Kosten und die Kontextfenster-Auslastung.* ## Lass Durchschnittswerte nicht mehr deine schlechtesten Läufe verbergen @@ -24,13 +24,13 @@ Ein gemeinsames Hover-Fadenkreuz verknüpft Heatmap und Band zeitlich miteinande Die Models-Seite (oben abgebildet) beantwortet die zwei Fragen, die eine Rechnung immer aufwirft: Welches Modell, und wie viel? Zusätzlich zur gemeinsamen Latenzansicht zeigt sie **modellspezifischen Token-Verbrauch**, **geschätzte Kosten** und die **Kontextfenster-Auslastung** – damit unkontrolliertes Prompt-Wachstum und eine bevorstehende Kompaktierung sichtbar werden, bevor sie dich überraschen. -Failproof AI Observability erkennt gängige Modell-IDs automatisch. Falls ein Fenster falsch aussieht oder du ein eigenes privates Modell betreibst, korrigiere es oder füge eines unter **Settings** bei **model context windows** hinzu – die Auslastungsanzeigen passen sich entsprechend an. +FailproofAI Cloud erkennt gängige Modell-IDs automatisch. Falls ein Fenster falsch aussieht oder du ein eigenes privates Modell betreibst, korrigiere es oder füge eines unter **Settings** bei **model context windows** hinzu – die Auslastungsanzeigen passen sich entsprechend an. ## Tools: unterscheide langsam von defekt Ein Tool-Aufruf kann langsam sein oder stillschweigend fehlschlagen – und du möchtest das in Sekunden wissen, nicht erst nach stundenlangem Log-Wühlen. -![Die Tools-Seite mit der gemeinsamen Latency-Heatmap und dem Percentile-Band neben einer Erfolgs- und Fehleraufschlüsselung sowie einem Tool-Verteilungsbalken](/agenteye/images/tools.png) +![Die Tools-Seite mit der gemeinsamen Latency-Heatmap und dem Percentile-Band neben einer Erfolgs- und Fehleraufschlüsselung sowie einem Tool-Verteilungsbalken](/cloud/images/tools.png) *Die Tools-Seite: dieselbe Heatmap und dasselbe Percentile-Band, ergänzt um eine Erfolgs- und Fehleraufschlüsselung sowie einen Tool-Verteilungsbalken.* Neben der gemeinsamen Latenzansicht fügt die Tools-Seite eine **Erfolgs- und Fehleraufschlüsselung** sowie einen **Tool-Verteilungsbalken** hinzu, sodass du auf einen Blick siehst, welche Tools du am häufigsten verwendest und welche dein Fehlerbudget auffressen. @@ -39,14 +39,14 @@ Neben der gemeinsamen Latenzansicht fügt die Tools-Seite eine **Erfolgs- und Fe Wenn ein Lifecycle-Hook einen Lauf verlangsamt, kannst du mit der Aussage „Hooks sind langsam" nichts anfangen. Die Hooks-Seite führt dich direkt zu dem einen, der das Problem verursacht. -![Die Hooks-Seite mit nach Hook-Name und Trigger-Event aufgeschlüsselter Latenz über der gemeinsamen Heatmap und dem Percentile-Band](/agenteye/images/hooks.png) +![Die Hooks-Seite mit nach Hook-Name und Trigger-Event aufgeschlüsselter Latenz über der gemeinsamen Heatmap und dem Percentile-Band](/cloud/images/hooks.png) *Die Hooks-Seite: Latenz aufgeschlüsselt nach Hook-Name und Trigger-Event.* Über derselben Latency-Heatmap und demselben Percentile-Band schlüsselt die Hooks-Seite die Aktivität nach **Hook-Name** und **Trigger-Event** auf, sodass du direkt bei dem einen Hook und dem einen Event landest, der Aufmerksamkeit erfordert. ## Verwandte Seiten -- [Event-Stream](/de/agenteye/event-stream): der Live-Feed aller Events, farblich kodiert. -- [Sessions](/de/agenteye/sessions): Events zu einer Zeile pro Lauf zusammenfassen und den Ausführungsgraphen öffnen. -- [Fehlerverfolgung](/de/agenteye/error-tracking): eine zentrale Triage-Oberfläche für alles, was das Dashboard rot einfärbt. -- [Dashboards](/de/agenteye/dashboards): Übersichtsansichten über deine gesamte Flotte. \ No newline at end of file +- [Event-Stream](/de/cloud/event-stream): der Live-Feed aller Events, farblich kodiert. +- [Sessions](/de/cloud/sessions): Events zu einer Zeile pro Lauf zusammenfassen und den Ausführungsgraphen öffnen. +- [Fehlerverfolgung](/de/cloud/errors): eine zentrale Triage-Oberfläche für alles, was das Dashboard rot einfärbt. +- [Dashboards](/de/cloud/dashboards): Übersichtsansichten über deine gesamte Flotte. \ No newline at end of file diff --git a/docs/de/agenteye/queries.mdx b/docs/de/cloud/queries.mdx similarity index 76% rename from docs/de/agenteye/queries.mdx rename to docs/de/cloud/queries.mdx index 6ea854c5..1f7b5bf9 100644 --- a/docs/de/agenteye/queries.mdx +++ b/docs/de/cloud/queries.mdx @@ -4,9 +4,9 @@ description: "Stellen Sie Ihren Agentendaten beliebige Fragen und erhalten Sie i --- -Stellen Sie Ihren Agentendaten beliebige Fragen und erhalten Sie in Sekunden eine Antwort. Failproof AI Observability bietet Ihnen eine Bibliothek gespeicherter, sofort ausführbarer Abfragen über Ihre Events und Auswertungen – damit starten Sie mit einem funktionierenden Beispiel statt vor einem leeren SQL-Editor. +Stellen Sie Ihren Agentendaten beliebige Fragen und erhalten Sie in Sekunden eine Antwort. FailproofAI Cloud bietet Ihnen eine Bibliothek gespeicherter, sofort ausführbarer Abfragen über Ihre Events und Auswertungen – damit starten Sie mit einem funktionierenden Beispiel statt vor einem leeren SQL-Editor. -![Die Bibliothek gespeicherter Abfragen: ein Raster wiederverwendbarer Abfragen, sowohl eingebaute Vorlagen als auch eigene](/agenteye/images/queries.png) +![Die Bibliothek gespeicherter Abfragen: ein Raster wiederverwendbarer Abfragen, sowohl eingebaute Vorlagen als auch eigene](/cloud/images/queries.png) *Ihre Bibliothek gespeicherter Abfragen unter `//queries`: eingebaute Vorlagen neben den Abfragen, die Ihr Team gespeichert hat.* @@ -22,7 +22,7 @@ Sie finden die Bibliothek unter `//queries`. Öffnen Sie eine beliebige Abfrage, und sie wird im SQL-Composer angezeigt, wo Sie sie anpassen und die Antwort sofort sehen können – kein Export, kein Umweg, kein Warten auf jemand anderen. -![Der SQL-Abfrage-Composer mit einer gespeicherten Abfrage, einer Schema-Seitenleiste und einem Live-Ergebnisraster](/agenteye/images/query-lab.png) +![Der SQL-Abfrage-Composer mit einer gespeicherten Abfrage, einer Schema-Seitenleiste und einem Live-Ergebnisraster](/cloud/images/query-lab.png) *Der SQL-Composer: Ihre Abfrage auf der linken Seite, eine Schema-Seitenleiste damit Sie nie einen Spaltennamen erraten müssen, und ein Live-Ergebnisraster darunter.* @@ -43,14 +43,14 @@ agenteye query list # die gleichen gespeicherten Abfragen, aus I agenteye query run errs --arg prod # eine ausführen und die Zeilen ausgeben (--json zum Weiterleiten hinzufügen) ``` - Siehe [CLI und Agenten](/de/agenteye/cli-and-agents) für den vollständigen Befehlssatz. + Siehe [CLI und Agenten](/de/cloud/cli) für den vollständigen Befehlssatz. -- **Über den KI-Assistenten.** Sie sind unsicher, wie Sie das SQL formulieren sollen? Fragen Sie den [KI-Assistenten](/de/agenteye/assistant) im Dashboard auf normalem Deutsch, und er wird die Abfrage entwerfen und für Sie in Ihrer Bibliothek speichern. +- **Über den KI-Assistenten.** Sie sind unsicher, wie Sie das SQL formulieren sollen? Fragen Sie den [KI-Assistenten](/de/cloud/assistant) im Dashboard auf normalem Deutsch, und er wird die Abfrage entwerfen und für Sie in Ihrer Bibliothek speichern. Das Ausführen einer gespeicherten Abfrage ist durch die Berechtigung `queries:run` geschützt, die getrennt von den Berechtigungen zum Erstellen oder Löschen von Abfragen verwaltet wird. So können Sie Lesezugriff erteilen, ohne allen zu erlauben, die Bibliothek umzuschreiben. ## Verwandte Themen -- [Dashboards](/de/agenteye/dashboards): Abfrageergebnisse in geteilte, organisationsweite Diagramme einbinden. -- [KI-Assistent](/de/agenteye/assistant): Fragen auf normalem Deutsch stellen und eine fertige Abfrage erhalten. -- [CLI und Agenten](/de/agenteye/cli-and-agents): Dieselben Abfragen über das Terminal ausführen und speichern. \ No newline at end of file +- [Dashboards](/de/cloud/dashboards): Abfrageergebnisse in geteilte, organisationsweite Diagramme einbinden. +- [KI-Assistent](/de/cloud/assistant): Fragen auf normalem Deutsch stellen und eine fertige Abfrage erhalten. +- [CLI und Agenten](/de/cloud/cli): Dieselben Abfragen über das Terminal ausführen und speichern. \ No newline at end of file diff --git a/docs/de/agenteye/python-sdk.mdx b/docs/de/cloud/sdk.mdx similarity index 92% rename from docs/de/agenteye/python-sdk.mdx rename to docs/de/cloud/sdk.mdx index fbbcf938..ac877468 100644 --- a/docs/de/agenteye/python-sdk.mdx +++ b/docs/de/cloud/sdk.mdx @@ -4,11 +4,11 @@ description: "Beobachte genau, was deine KI-Agenten in der Produktion getan habe --- -Beobachte genau, was deine KI-Agenten in der Produktion getan haben: jeden Agentenlauf, Tool-Aufruf, Modellanfrage, Hook und menschlichen Eingriff. Das Failproof AI Observability Python SDK zeichnet diesen Verlauf direkt aus deinem Agenten-Code auf, damit du debuggen, auditieren und nachvollziehen kannst, was passiert ist. Verwende es immer dann, wenn Failproof AI Observability deine Agenten beobachten soll. +Beobachte genau, was deine KI-Agenten in der Produktion getan haben: jeden Agentenlauf, Tool-Aufruf, Modellanfrage, Hook und menschlichen Eingriff. Das FailproofAI Cloud Python SDK zeichnet diesen Verlauf direkt aus deinem Agenten-Code auf, damit du debuggen, auditieren und nachvollziehen kannst, was passiert ist. Verwende es immer dann, wenn FailproofAI Cloud deine Agenten beobachten soll. Intern schreibt das SDK strukturierte Events in lokale JSONL-Dateien, und der Collector-Daemon liest diese und überträgt sie automatisch an die Plattform. Du musst diese Dateien nicht selbst verwalten. -> **Tipp:** Neu bei Failproof AI Observability? Diese Seite ist die vollständige SDK-Event-Referenz. +> **Tipp:** Neu bei FailproofAI Cloud? Diese Seite ist die vollständige SDK-Event-Referenz.
@@ -26,7 +26,7 @@ Sobald es installiert ist, überprüfe die Installation: python -c "import agenteye; print(agenteye.__version__)" ``` -Möchtest du die gesamte Integration von einem Coding-Agent erledigen lassen? Der [Python SDK Agent Skill](/de/agenteye/python-sdk-skill) kennt den Installationspfad, plant die Instrumentierungspunkte, schreibt sie und überprüft, ob die Events ankommen. +Möchtest du die gesamte Integration von einem Coding-Agent erledigen lassen? Der [Python SDK Agent Skill](/de/cloud/agent-skills) kennt den Installationspfad, plant die Instrumentierungspunkte, schreibt sie und überprüft, ob die Events ankommen. --- @@ -60,7 +60,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### Einen echten Aufruf instrumentieren -In der Praxis umhüllst du deinen bestehenden Agenten-Code. Klammere einen Modellaufruf mit `model_request` davor und `model_response` danach ein, sodass die beiden Events die echte Anfrage umspannen und Failproof AI Observability sie zuordnen kann: +In der Praxis umhüllst du deinen bestehenden Agenten-Code. Klammere einen Modellaufruf mit `model_request` davor und `model_response` danach ein, sodass die beiden Events die echte Anfrage umspannen und FailproofAI Cloud sie zuordnen kann: ```python import anthropic @@ -99,7 +99,7 @@ Umhülle Tool-Aufrufe auf dieselbe Weise mit `tool_use` und `tool_result`, wobei So sehen diese Events aus, sobald sie das Dashboard erreichen — farblich nach Typ kodiert und filterbar nach Umgebung, Agent und Session: -![Der Live-Events-Stream, farblich nach Event-Typ kodiert und filterbar nach Umgebung, Agent und Session](/agenteye/images/events-stream.png) +![Der Live-Events-Stream, farblich nach Event-Typ kodiert und filterbar nach Umgebung, Agent und Session](/cloud/images/events-stream.png) --- @@ -156,9 +156,9 @@ Das macht die Bereinigung zu deiner Wahl und Verantwortung. Wenn ein Prompt oder ## Event-Referenz -Die meisten Events kommen in Start-/End-Paaren, die eine Korrelations-ID teilen: `tool_use` und `tool_result` teilen eine `tool_call_id`, `hook_triggered` und `hook_completed` teilen eine `hook_id`, und `human_wait` und `human_input` teilen eine `input_id`. Sende das Start-Event, führe die Arbeit aus und sende dann das End-Event mit derselben ID. Failproof AI Observability ordnet das Paar zu und berechnet `duration_ms` für dich, sodass du `duration_ms` nie selbst übergibst. +Die meisten Events kommen in Start-/End-Paaren, die eine Korrelations-ID teilen: `tool_use` und `tool_result` teilen eine `tool_call_id`, `hook_triggered` und `hook_completed` teilen eine `hook_id`, und `human_wait` und `human_input` teilen eine `input_id`. Sende das Start-Event, führe die Arbeit aus und sende dann das End-Event mit derselben ID. FailproofAI Cloud ordnet das Paar zu und berechnet `duration_ms` für dich, sodass du `duration_ms` nie selbst übergibst. -![Der git-artige Ausführungsgraph einer Session neben ihrer Event-Zeitleiste, aus den gepaarten Events rekonstruiert, mit dem Tool/Modell/Hook-Aufschlüsselungspanel](/agenteye/images/session-detail.png) +![Der git-artige Ausführungsgraph einer Session neben ihrer Event-Zeitleiste, aus den gepaarten Events rekonstruiert, mit dem Tool/Modell/Hook-Aufschlüsselungspanel](/cloud/images/session-detail.png) Alle Event-Methoden erfordern diese zwei Felder: @@ -432,5 +432,5 @@ Jede Datei wird atomar geschrieben: Das SDK schreibt zunächst in eine temporär ## Nächste Schritte -- [Event-Stream](/de/agenteye/event-stream): Beobachte, wie diese Events live ankommen, farblich kodiert und filterbar nach Umgebung, Agent und Session. -- [Sessions](/de/agenteye/sessions): Sieh, wie die gepaarten Events jeden Agentenlauf als Ausführungsgraph und Zeitleiste rekonstruieren. \ No newline at end of file +- [Event-Stream](/de/cloud/event-stream): Beobachte, wie diese Events live ankommen, farblich kodiert und filterbar nach Umgebung, Agent und Session. +- [Sessions](/de/cloud/sessions): Sieh, wie die gepaarten Events jeden Agentenlauf als Ausführungsgraph und Zeitleiste rekonstruieren. \ No newline at end of file diff --git a/docs/de/cloud/security.mdx b/docs/de/cloud/security.mdx new file mode 100644 index 00000000..3b0a8b4d --- /dev/null +++ b/docs/de/cloud/security.mdx @@ -0,0 +1,68 @@ +--- +title: "Sicherheit" +description: "FailproofAI Cloud ist darauf ausgelegt, nah an Ihren Produktions-Agents zu laufen – das bedeutet, es sieht Ihre Prompts, Tool-Eingaben und Ausgaben." +--- + + +FailproofAI Cloud ist darauf ausgelegt, nah an Ihren Produktions-Agents zu laufen – das bedeutet, es sieht Ihre Prompts, Tool-Eingaben und Ausgaben. Diese Seite erklärt, wie die Daten isoliert, kontrolliert und in Ihren Händen bleiben. Wenn Sie FailproofAI Cloud im Rahmen einer Sicherheitsprüfung evaluieren, beginnen Sie hier. + +--- + +## Ihre Daten bleiben in Ihrer Umgebung + +FailproofAI Cloud wird selbst gehostet. Events, Prompts, Modellantworten und Analysen werden in Ihren eigenen Datenbanken, in Ihrer eigenen Umgebung gespeichert. Es werden keine Daten zur Speicherung an einen Drittanbieter-SaaS übermittelt – Ihre Daten verbleiben in Ihrem eigenen Cloud-Account. + +--- + +## Mandantenisolierung + +Eine FailproofAI Cloud-Instanz kann viele Organisationen beherbergen, und jede ist auf Speicherebene isoliert – durchgesetzt von der Datenbank, nicht nur von der Benutzeroberfläche: + +- Die operativen Daten einer Organisation (Benutzer, Schlüssel, Dashboards, gespeicherte Abfragen) sind auf diese Organisation beschränkt, und organisationsübergreifende Lesezugriffe werden von der Datenbank selbst blockiert. +- Jedes aufgenommene Event wird mit der zugehörigen Organisation gestempelt, sodass die Events einer Organisation niemals von einer anderen gelesen werden können. + +Jede Dashboard-Route ist unter einem Org-Slug (`//…`) eingeschränkt. + +--- + +## Anmeldung + +FailproofAI Cloud verwendet passwortlose, E-Mail-basierte Anmeldung. Es gibt kein Passwort, das abgephisht oder geleakt werden könnte. Ein Benutzer fordert einen Einmalcode (oder einen Magic Link zum einmaligen Klicken) an, der per E-Mail zugestellt wird und schnell abläuft. Die Anmeldung ist durch eine **Allowlist** gesichert: Nur E-Mail-Adressen (oder Domains), die Sie freigeben, können sich authentifizieren. + +![Der Anmeldebildschirm von FailproofAI Cloud, der einen Einmalcode an Ihre E-Mail-Adresse sendet](/cloud/images/login.png) + +--- + +## Eingeschränkter Zugriff mit API-Schlüsseln + +Jeder Client authentifiziert sich mit einem API-Schlüssel, der granulare, minimal privilegierte Berechtigungen trägt. Ein Collector benötigt lediglich `events:add`; ein Dashboard- oder Assistenten-Schlüssel kann schreibgeschützt sein; destruktive Aktionen (Löschen, Neugenerieren) sind separate Berechtigungen, die Sie gezielt vergeben. + +![Die API-Schlüssel-Seite: Berechtigungen jedes Schlüssels, farblich nach Lese-, Schreib- und destruktivem Umfang kodiert](/cloud/images/api-keys.png) + +Behalten Sie den Admin-Bootstrap-Schlüssel für die Einrichtung, und vergeben Sie eingeschränkte Schlüssel für alles andere. Siehe [API-Schlüssel](/de/cloud/access). + +--- + +## Ein schreibgeschützter, genehmigungspflichtiger Assistent + +Der [KI-Assistent](/de/cloud/assistant) im Dashboard beantwortet Fragen über Ihre Daten, ist aber bewusst eingeschränkt: + +- Er ist **standardmäßig schreibgeschützt**: Sein SQL wird durch einen Guard geleitet, der nur `SELECT`/`WITH`-Abfragen, einzelne Anweisungen und eine Zeilenbegrenzung erlaubt. +- Alles, was er erstellt (eine gespeicherte Abfrage, ein Dashboard), ist **genehmigungspflichtig**: Sie prüfen und genehmigen jeden Schreibvorgang, bevor er ausgeführt wird. +- Er **kann niemals löschen**. + +So kann ein Teammitglied fragen „Welche Agents haben diese Woche am häufigsten Fehler gemeldet?" und auf die Antwort reagieren – ohne dass der Assistent Ihre Daten eigenständig ändern oder entfernen kann. + +--- + +## Daten in Übertragung + +Der gesamte Datenverkehr läuft über HTTPS. Sie terminieren TLS mit Ihren eigenen Zertifikaten, sodass der Datenverkehr zwischen Collector und Server sowie zwischen Browser und Server verschlüsselt übertragen wird. + +--- + +## Nächste Schritte + +- [Übersicht](/de/cloud/overview): Wie FailproofAI Cloud zusammenarbeitet. +- [API-Schlüssel](/de/cloud/access): Zugriff für Collector, Dashboard und Assistent einschränken. +- [FailproofAI Cloud](/de/cloud/overview): Was FailproofAI Cloud von Ihren Agents erfasst. \ No newline at end of file diff --git a/docs/de/agenteye/sessions.mdx b/docs/de/cloud/sessions.mdx similarity index 75% rename from docs/de/agenteye/sessions.mdx rename to docs/de/cloud/sessions.mdx index 547a8800..e9fea066 100644 --- a/docs/de/agenteye/sessions.mdx +++ b/docs/de/cloud/sessions.mdx @@ -4,9 +4,9 @@ description: "Alle Ereignisse eines Runs in einer übersichtlichen Zeile zusamme --- -Schluss mit dem Rätseln, warum ein Run fehlgeschlagen ist. Failproof AI Observability fasst alle Ereignisse eines Runs in einer lesbaren Zeile zusammen und zeichnet den gesamten Run als Git-ähnliches Diagramm, das du in Sekunden erfassen kannst – so siehst du genau, was dein Agent Schritt für Schritt getan hat. +Schluss mit dem Rätseln, warum ein Run fehlgeschlagen ist. FailproofAI Cloud fasst alle Ereignisse eines Runs in einer lesbaren Zeile zusammen und zeichnet den gesamten Run als Git-ähnliches Diagramm, das du in Sekunden erfassen kannst – so siehst du genau, was dein Agent Schritt für Schritt getan hat. -![Die Sessions-Liste: eine Zeile pro Run, über Umgebungen und Agents hinweg, mit Status-Pills und Bewertungsbadges](/agenteye/images/sessions-list.png) +![Die Sessions-Liste: eine Zeile pro Run, über Umgebungen und Agents hinweg, mit Status-Pills und Bewertungsbadges](/cloud/images/sessions-list.png) *Eine Zeile pro Run: der Status-Pill zeigt auf einen Blick, wie der Run geendet hat, und ein Score-Badge erscheint, sobald ein Evaluator verbunden ist.* @@ -30,7 +30,7 @@ Sobald du einen Evaluator verbindest, wird jeder abgeschlossene Run automatisch ## Den gesamten Run als Diagramm lesen -![Der Git-ähnliche Ausführungsgraph einer Session neben ihrer Ereigniszeitleiste, mit dem Panel für Tool-, Modell- und Hook-Aufschlüsselung](/agenteye/images/session-detail.png) +![Der Git-ähnliche Ausführungsgraph einer Session neben ihrer Ereigniszeitleiste, mit dem Panel für Tool-, Modell- und Hook-Aufschlüsselung](/cloud/images/session-detail.png) *Der Ausführungsgraph (links) liegt neben der Ereigniszeitleiste; die rechte Leiste schlüsselt Tools, Modelle, Hooks und Token-Verbrauch des Runs auf.* @@ -38,7 +38,7 @@ Klicke auf eine beliebige Session, um ihren Ausführungsgraph zu öffnen: eine G Die rechte Leiste liefert dir die Run-spezifische Aufschlüsselung: welche Tools und Modelle liefen, welche Hooks gefeuert haben und was der Run an Tokens gekostet hat. Das ist die Antwort auf „Warum hat dieser Run so viel gekostet?" oder „Welches Tool ist das langsame?" – direkt neben dem Graphen, der dazu geführt hat. -Einzelne Ereignisse sind adressierbar, sodass du jemandem einen Link zu einem bestimmten Moment schicken kannst, anstatt „die Session, ungefähr zwei Drittel runter". Kopiere den Link aus einem beliebigen Ereignis, oder folge einem Link aus einem [Audit](/de/agenteye/audits)-Fund oder einem Fehler – die Session öffnet sich dann mit dem ausgewählten und angezeigten Ereignis. Das gilt auch für sehr lange Runs: Die Zeitleiste lädt aus Rücksicht auf deinen Browser ein begrenztes Fenster, und ein Link, der über dieses Fenster hinausweist, findet sein Ereignis trotzdem, anstatt dich am Anfang abzusetzen. Wenn das Ereignis aus deinem Aufbewahrungsfenster herausgefallen ist, teilt dir die Seite das mit, anstatt stillschweigend nichts auszuwählen. +Einzelne Ereignisse sind adressierbar, sodass du jemandem einen Link zu einem bestimmten Moment schicken kannst, anstatt „die Session, ungefähr zwei Drittel runter". Kopiere den Link aus einem beliebigen Ereignis, oder folge einem Link aus einem [Audit](/de/cloud/audits)-Fund oder einem Fehler – die Session öffnet sich dann mit dem ausgewählten und angezeigten Ereignis. Das gilt auch für sehr lange Runs: Die Zeitleiste lädt aus Rücksicht auf deinen Browser ein begrenztes Fenster, und ein Link, der über dieses Fenster hinausweist, findet sein Ereignis trotzdem, anstatt dich am Anfang abzusetzen. Wenn das Ereignis aus deinem Aufbewahrungsfenster herausgefallen ist, teilt dir die Seite das mit, anstatt stillschweigend nichts auszuwählen. --- @@ -46,12 +46,12 @@ Einzelne Ereignisse sind adressierbar, sodass du jemandem einen Link zu einem be Jede Dashboard-Seite ist auf deine Org beschränkt (`//…`). Sessions findest du unter **Observe** in der linken Seitenleiste, neben Events, mit den Filtern für Datumsbereich, Umgebung, Agent und Session am oberen Rand der Liste. Jede Zeile ist einen Klick von ihrem vollständigen Ausführungsgraph entfernt. -Um die Score-Badges und die Score-Bereich-Filterung zu aktivieren, verbinde einen Evaluator: siehe [Evaluations](/de/agenteye/evaluations). +Um die Score-Badges und die Score-Bereich-Filterung zu aktivieren, verbinde einen Evaluator: siehe [Evaluations](/de/cloud/evaluations). --- ## Verwandte Themen -- [Event stream](/de/agenteye/event-stream): der rohe, schrittweise Verlauf, aus dem jede Session zusammengesetzt wird. -- [Evaluations](/de/agenteye/evaluations): verbinde einen Evaluator, damit jeder Run einen Score-Badge erhält, nach dem du filtern kannst. -- [Telemetry](/de/agenteye/telemetry): wie Runs von deinem Agent in diese Sessions gelangen. \ No newline at end of file +- [Event stream](/de/cloud/event-stream): der rohe, schrittweise Verlauf, aus dem jede Session zusammengesetzt wird. +- [Evaluations](/de/cloud/evaluations): verbinde einen Evaluator, damit jeder Run einen Score-Badge erhält, nach dem du filtern kannst. +- [Telemetry](/de/cloud/performance): wie Runs von deinem Agent in diese Sessions gelangen. \ No newline at end of file diff --git a/docs/de/concepts.mdx b/docs/de/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/de/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/de/daemon.mdx b/docs/de/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/de/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/de/dashboard.mdx b/docs/de/dashboard.mdx index 75c76c7d..8d663e10 100644 --- a/docs/de/dashboard.mdx +++ b/docs/de/dashboard.mdx @@ -69,7 +69,7 @@ Ein charaktergetriebener Bericht darüber, wie sich Ihr Agent tatsächlich in ve 4. **So verbessern Sie sich** — ruhige Zeilenliste, eine pro empfohlener Richtlinie: Richtlinienname in Weiß, einzeilige Beschreibung, Installationsbefehl + Kopierschaltfläche auf der rechten Seite. Die Abschnittsüberschrift lautet `enable all N → projected · ` (die Punktzahl, die Sie mit allen angewendeten Korrekturen erreichen würden), und die Schaltfläche `[install all]` kopiert den kombinierten `failproofai policy add a b c …`-Befehl für jede empfohlene Richtlinie. 5. **Komm besser zurück** — zwei nebeneinander liegende Karten. Links: Erinnerung setzen (`3d` / `7d` / `14d` / `30d` Kadenz-Auswahl; wird nach Authentifizierung über `/api/auth/reminder` gespeichert). Rechts: failproof-Vorteile freischalten — `invite a friend` öffnet ein Modal, das eine komma-/leerzeichen-/zeilenumbruchgetrennte Liste von Freundes-E-Mails akzeptiert (max. 10 pro Sendung), POSTet diese an `/api/audit/invite`, das sie an den API-Server unter `POST /v0/invite` weiterleitet. Der API-Server sendet eine E-Mail pro Empfänger von `invite@failproof.ai` mit dem Absender im Cc und gesetztem `Reply-To`, sodass der Empfänger sieht, wer ihn eingeladen hat, und der Absender eine Kopie in seinem Posteingang erhält. Anonyme Benutzer werden zuerst durch den `AuthDialog` geleitet, damit die E-Mail des Absenders bekannt ist, bevor Einladungen verschickt werden. Ansprüche/Vorteilserfüllung folgt. -Angetrieben von der `failproofai audit`-Laufzeit — siehe [Audit CLI](/de/cli/audit) für die zugrunde liegende Scan-Engine, unterstützte Flags und sitzungsspezifische Cache-Invarianten. Das Dashboard speichert das neueste Ergebnis unter `~/.failproofai/audit-dashboard.json` (Modus `0600`, einzelner Slot, neue Läufe überschreiben), sodass erneute Besuche sofort laden; **sowohl der transkriptspezifische als auch der gesamtergebnisbezogene Cache werden beim Lesen abgelehnt, sobald sie älter als 7 Tage sind**, damit das Dashboard kein einwöchiges Ergebnis stillschweigend ausliefert — nach Ablauf der TTL fällt `/audit` in seinen Leerzustand und fordert einen neuen Lauf an. Ein Klick auf `[ re-audit now ]` nahe am Ende des Berichts sendet einen POST an `/api/audit/run` mit `noCache: true` — ein Re-Audit umgeht den transkriptspezifischen Cache und scannt jedes Transkript von Grund auf neu, anstatt stillschweigend das zwischengespeicherte Ergebnis zurückzugeben — und das Dashboard fragt `/api/audit/status` mit 1 Hz ab, bis der Lauf abgeschlossen ist; ein pinker Fortschrittsbalken wird während des Laufs mit einem Zeitmesser oben im Viewport fixiert, und das frische Ergebnis wird bei Erfolg an Ort und Stelle ausgetauscht (kein vollständiges Neuladen der Seite; ein fehlgeschlagener Re-Audit lässt den vorherigen Bericht intakt). Bei einem Fehler wird der Balken rot mit einer auf den `RerunError.kind` abgestimmten Meldung (`timeout` / `network` / `post_failed`). Leerzustand (kein Cache oder abgelaufen) und Null-Sitzungen-Zustand (Cache vorhanden, aber der Scan hat keine Transkripte gefunden) werden separat angezeigt. +Angetrieben von der `failproofai audit`-Laufzeit — siehe [Audit CLI](/de/audit) für die zugrunde liegende Scan-Engine, unterstützte Flags und sitzungsspezifische Cache-Invarianten. Das Dashboard speichert das neueste Ergebnis unter `~/.failproofai/audit-dashboard.json` (Modus `0600`, einzelner Slot, neue Läufe überschreiben), sodass erneute Besuche sofort laden; **sowohl der transkriptspezifische als auch der gesamtergebnisbezogene Cache werden beim Lesen abgelehnt, sobald sie älter als 7 Tage sind**, damit das Dashboard kein einwöchiges Ergebnis stillschweigend ausliefert — nach Ablauf der TTL fällt `/audit` in seinen Leerzustand und fordert einen neuen Lauf an. Ein Klick auf `[ re-audit now ]` nahe am Ende des Berichts sendet einen POST an `/api/audit/run` mit `noCache: true` — ein Re-Audit umgeht den transkriptspezifischen Cache und scannt jedes Transkript von Grund auf neu, anstatt stillschweigend das zwischengespeicherte Ergebnis zurückzugeben — und das Dashboard fragt `/api/audit/status` mit 1 Hz ab, bis der Lauf abgeschlossen ist; ein pinker Fortschrittsbalken wird während des Laufs mit einem Zeitmesser oben im Viewport fixiert, und das frische Ergebnis wird bei Erfolg an Ort und Stelle ausgetauscht (kein vollständiges Neuladen der Seite; ein fehlgeschlagener Re-Audit lässt den vorherigen Bericht intakt). Bei einem Fehler wird der Balken rot mit einer auf den `RerunError.kind` abgestimmten Meldung (`timeout` / `network` / `post_failed`). Leerzustand (kein Cache oder abgelaufen) und Null-Sitzungen-Zustand (Cache vorhanden, aber der Scan hat keine Transkripte gefunden) werden separat angezeigt. ### Richtlinien diff --git a/docs/de/architecture.mdx b/docs/de/how-it-works.mdx similarity index 100% rename from docs/de/architecture.mdx rename to docs/de/how-it-works.mdx diff --git a/docs/de/introduction.mdx b/docs/de/introduction.mdx index 1ff567ef..43da77a1 100644 --- a/docs/de/introduction.mdx +++ b/docs/de/introduction.mdx @@ -54,4 +54,4 @@ failproofai policies --install # enable policies (or skip — `failproofai` wi failproofai # launch the dashboard ``` -Die vollständige Anleitung finden Sie im [Erste-Schritte-Leitfaden](/de/getting-started). \ No newline at end of file +Die vollständige Anleitung finden Sie im [Erste-Schritte-Leitfaden](/de/quickstart). \ No newline at end of file diff --git a/docs/de/policies.mdx b/docs/de/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/de/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/de/getting-started.mdx b/docs/de/quickstart.mdx similarity index 100% rename from docs/de/getting-started.mdx rename to docs/de/quickstart.mdx diff --git a/docs/de/reference/files.mdx b/docs/de/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/de/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/docs.json b/docs/docs.json index 744a6e59..ac0d26f1 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -32,110 +32,106 @@ "language": "en", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "agenteye/overview", - "agenteye/concepts", - "agenteye/observability" + "introduction", + "quickstart", + "how-it-works", + "concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "agenteye/event-stream", - "agenteye/sessions", - "agenteye/telemetry", - "agenteye/error-tracking", - "agenteye/evaluations", - "agenteye/queries", - "agenteye/dashboards", - "agenteye/audits", - "agenteye/alerts", - "agenteye/incidents", - "agenteye/assistant", - "agenteye/cli-and-agents" + "policies", + "built-in-policies", + "custom-policies", + "configuration", + "agent-support", + "daemon", + "testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "agenteye/python-sdk", - "agenteye/python-sdk-skill", - "agenteye/codex-capture", - "agenteye/openclaw-capture", - "agenteye/hermes-capture", - "agenteye/cli", - "agenteye/cli-skill", - "agenteye/cli-recipes", - "agenteye/evaluation-suite", - "agenteye/evaluator-skill" + "dashboard", + "audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "agenteye/api-keys", - "agenteye/security" + "cloud/overview", + "cloud/connect", + "cloud/fleet", + "cloud/managed-policies", + "cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "Getting Started", + "group": "Observe your agents", "pages": [ - "introduction", - "getting-started" + "cloud/event-stream", + "cloud/sessions", + "cloud/performance", + "cloud/errors" ] }, { - "group": "Core Concepts", + "group": "Analyze and act", "pages": [ - "built-in-policies", - "custom-policies", - "configuration" + "cloud/evaluations", + "cloud/evaluators", + "cloud/queries", + "cloud/dashboards", + "cloud/audits", + "cloud/alerts", + "cloud/incidents", + "cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "cli/dashboard", + "cloud/sdk", + "cloud/cli", + "cloud/cli-recipes", + "cloud/agent-skills", + "cloud/access", + "cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "cli/config", "cli/install-policies", "cli/remove-policies", "cli/list-policies", + "cli/harness", + "cli/backfill", + "cli/flush", "cli/hook", - "cli/audit", + "cli/dashboard", "cli/update", "cli/migrate", + "cli/uninstall", "cli/version", "cli/environment-variables" ] }, { - "group": "Tools", - "pages": [ - "dashboard" - ] - }, - { - "group": "Advanced", + "group": "Reference", "pages": [ - "architecture", - "testing", + "reference/files", + "examples", "package-aliases", "for-agents" ] - }, - { - "group": "Examples", - "pages": [ - "examples" - ] } ] } @@ -145,108 +141,106 @@ "language": "zh", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "zh/agenteye/overview", - "zh/agenteye/concepts", - "zh/agenteye/observability" + "zh/introduction", + "zh/quickstart", + "zh/how-it-works", + "zh/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "zh/agenteye/event-stream", - "zh/agenteye/sessions", - "zh/agenteye/telemetry", - "zh/agenteye/error-tracking", - "zh/agenteye/evaluations", - "zh/agenteye/queries", - "zh/agenteye/dashboards", - "zh/agenteye/audits", - "zh/agenteye/alerts", - "zh/agenteye/incidents", - "zh/agenteye/assistant", - "zh/agenteye/cli-and-agents" + "zh/policies", + "zh/built-in-policies", + "zh/custom-policies", + "zh/configuration", + "zh/agent-support", + "zh/daemon", + "zh/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "zh/agenteye/python-sdk", - "zh/agenteye/python-sdk-skill", - "zh/agenteye/codex-capture", - "zh/agenteye/openclaw-capture", - "zh/agenteye/hermes-capture", - "zh/agenteye/cli", - "zh/agenteye/cli-skill", - "zh/agenteye/cli-recipes", - "zh/agenteye/evaluation-suite", - "zh/agenteye/evaluator-skill" + "zh/dashboard", + "zh/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "zh/agenteye/api-keys", - "zh/agenteye/security" + "zh/cloud/overview", + "zh/cloud/connect", + "zh/cloud/fleet", + "zh/cloud/managed-policies", + "zh/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "快速开始", + "group": "Observe your agents", "pages": [ - "zh/introduction", - "zh/getting-started" + "zh/cloud/event-stream", + "zh/cloud/sessions", + "zh/cloud/performance", + "zh/cloud/errors" ] }, { - "group": "核心概念", + "group": "Analyze and act", "pages": [ - "zh/built-in-policies", - "zh/custom-policies", - "zh/configuration" + "zh/cloud/evaluations", + "zh/cloud/evaluators", + "zh/cloud/queries", + "zh/cloud/dashboards", + "zh/cloud/audits", + "zh/cloud/alerts", + "zh/cloud/incidents", + "zh/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "zh/cli/dashboard", + "zh/cloud/sdk", + "zh/cloud/cli", + "zh/cloud/cli-recipes", + "zh/cloud/agent-skills", + "zh/cloud/access", + "zh/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "zh/cli/config", "zh/cli/install-policies", "zh/cli/remove-policies", "zh/cli/list-policies", + "zh/cli/harness", + "zh/cli/backfill", + "zh/cli/flush", "zh/cli/hook", - "zh/cli/audit", + "zh/cli/dashboard", + "zh/cli/update", + "zh/cli/migrate", + "zh/cli/uninstall", "zh/cli/version", "zh/cli/environment-variables" ] }, { - "group": "工具", - "pages": [ - "zh/dashboard" - ] - }, - { - "group": "进阶", + "group": "Reference", "pages": [ - "zh/architecture", - "zh/testing", + "zh/reference/files", + "zh/examples", "zh/package-aliases", "zh/for-agents" ] - }, - { - "group": "示例", - "pages": [ - "zh/examples" - ] } ] } @@ -256,108 +250,106 @@ "language": "ja", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "ja/agenteye/overview", - "ja/agenteye/concepts", - "ja/agenteye/observability" + "ja/introduction", + "ja/quickstart", + "ja/how-it-works", + "ja/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "ja/agenteye/event-stream", - "ja/agenteye/sessions", - "ja/agenteye/telemetry", - "ja/agenteye/error-tracking", - "ja/agenteye/evaluations", - "ja/agenteye/queries", - "ja/agenteye/dashboards", - "ja/agenteye/audits", - "ja/agenteye/alerts", - "ja/agenteye/incidents", - "ja/agenteye/assistant", - "ja/agenteye/cli-and-agents" + "ja/policies", + "ja/built-in-policies", + "ja/custom-policies", + "ja/configuration", + "ja/agent-support", + "ja/daemon", + "ja/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "ja/agenteye/python-sdk", - "ja/agenteye/python-sdk-skill", - "ja/agenteye/codex-capture", - "ja/agenteye/openclaw-capture", - "ja/agenteye/hermes-capture", - "ja/agenteye/cli", - "ja/agenteye/cli-skill", - "ja/agenteye/cli-recipes", - "ja/agenteye/evaluation-suite", - "ja/agenteye/evaluator-skill" + "ja/dashboard", + "ja/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "ja/agenteye/api-keys", - "ja/agenteye/security" + "ja/cloud/overview", + "ja/cloud/connect", + "ja/cloud/fleet", + "ja/cloud/managed-policies", + "ja/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "はじめに", + "group": "Observe your agents", "pages": [ - "ja/introduction", - "ja/getting-started" + "ja/cloud/event-stream", + "ja/cloud/sessions", + "ja/cloud/performance", + "ja/cloud/errors" ] }, { - "group": "基本概念", + "group": "Analyze and act", "pages": [ - "ja/built-in-policies", - "ja/custom-policies", - "ja/configuration" + "ja/cloud/evaluations", + "ja/cloud/evaluators", + "ja/cloud/queries", + "ja/cloud/dashboards", + "ja/cloud/audits", + "ja/cloud/alerts", + "ja/cloud/incidents", + "ja/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "ja/cli/dashboard", + "ja/cloud/sdk", + "ja/cloud/cli", + "ja/cloud/cli-recipes", + "ja/cloud/agent-skills", + "ja/cloud/access", + "ja/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "ja/cli/config", "ja/cli/install-policies", "ja/cli/remove-policies", "ja/cli/list-policies", + "ja/cli/harness", + "ja/cli/backfill", + "ja/cli/flush", "ja/cli/hook", - "ja/cli/audit", + "ja/cli/dashboard", + "ja/cli/update", + "ja/cli/migrate", + "ja/cli/uninstall", "ja/cli/version", "ja/cli/environment-variables" ] }, { - "group": "ツール", + "group": "Reference", "pages": [ - "ja/dashboard" - ] - }, - { - "group": "上級", - "pages": [ - "ja/architecture", - "ja/testing", + "ja/reference/files", + "ja/examples", "ja/package-aliases", "ja/for-agents" ] - }, - { - "group": "例", - "pages": [ - "ja/examples" - ] } ] } @@ -367,108 +359,106 @@ "language": "ko", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "ko/agenteye/overview", - "ko/agenteye/concepts", - "ko/agenteye/observability" + "ko/introduction", + "ko/quickstart", + "ko/how-it-works", + "ko/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "ko/agenteye/event-stream", - "ko/agenteye/sessions", - "ko/agenteye/telemetry", - "ko/agenteye/error-tracking", - "ko/agenteye/evaluations", - "ko/agenteye/queries", - "ko/agenteye/dashboards", - "ko/agenteye/audits", - "ko/agenteye/alerts", - "ko/agenteye/incidents", - "ko/agenteye/assistant", - "ko/agenteye/cli-and-agents" + "ko/policies", + "ko/built-in-policies", + "ko/custom-policies", + "ko/configuration", + "ko/agent-support", + "ko/daemon", + "ko/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "ko/agenteye/python-sdk", - "ko/agenteye/python-sdk-skill", - "ko/agenteye/codex-capture", - "ko/agenteye/openclaw-capture", - "ko/agenteye/hermes-capture", - "ko/agenteye/cli", - "ko/agenteye/cli-skill", - "ko/agenteye/cli-recipes", - "ko/agenteye/evaluation-suite", - "ko/agenteye/evaluator-skill" + "ko/dashboard", + "ko/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "ko/agenteye/api-keys", - "ko/agenteye/security" + "ko/cloud/overview", + "ko/cloud/connect", + "ko/cloud/fleet", + "ko/cloud/managed-policies", + "ko/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "시작하기", + "group": "Observe your agents", "pages": [ - "ko/introduction", - "ko/getting-started" + "ko/cloud/event-stream", + "ko/cloud/sessions", + "ko/cloud/performance", + "ko/cloud/errors" ] }, { - "group": "핵심 개념", + "group": "Analyze and act", "pages": [ - "ko/built-in-policies", - "ko/custom-policies", - "ko/configuration" + "ko/cloud/evaluations", + "ko/cloud/evaluators", + "ko/cloud/queries", + "ko/cloud/dashboards", + "ko/cloud/audits", + "ko/cloud/alerts", + "ko/cloud/incidents", + "ko/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "ko/cli/dashboard", + "ko/cloud/sdk", + "ko/cloud/cli", + "ko/cloud/cli-recipes", + "ko/cloud/agent-skills", + "ko/cloud/access", + "ko/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "ko/cli/config", "ko/cli/install-policies", "ko/cli/remove-policies", "ko/cli/list-policies", + "ko/cli/harness", + "ko/cli/backfill", + "ko/cli/flush", "ko/cli/hook", - "ko/cli/audit", + "ko/cli/dashboard", + "ko/cli/update", + "ko/cli/migrate", + "ko/cli/uninstall", "ko/cli/version", "ko/cli/environment-variables" ] }, { - "group": "도구", + "group": "Reference", "pages": [ - "ko/dashboard" - ] - }, - { - "group": "고급", - "pages": [ - "ko/architecture", - "ko/testing", + "ko/reference/files", + "ko/examples", "ko/package-aliases", "ko/for-agents" ] - }, - { - "group": "예제", - "pages": [ - "ko/examples" - ] } ] } @@ -478,108 +468,106 @@ "language": "es", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "es/agenteye/overview", - "es/agenteye/concepts", - "es/agenteye/observability" + "es/introduction", + "es/quickstart", + "es/how-it-works", + "es/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "es/agenteye/event-stream", - "es/agenteye/sessions", - "es/agenteye/telemetry", - "es/agenteye/error-tracking", - "es/agenteye/evaluations", - "es/agenteye/queries", - "es/agenteye/dashboards", - "es/agenteye/audits", - "es/agenteye/alerts", - "es/agenteye/incidents", - "es/agenteye/assistant", - "es/agenteye/cli-and-agents" + "es/policies", + "es/built-in-policies", + "es/custom-policies", + "es/configuration", + "es/agent-support", + "es/daemon", + "es/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "es/agenteye/python-sdk", - "es/agenteye/python-sdk-skill", - "es/agenteye/codex-capture", - "es/agenteye/openclaw-capture", - "es/agenteye/hermes-capture", - "es/agenteye/cli", - "es/agenteye/cli-skill", - "es/agenteye/cli-recipes", - "es/agenteye/evaluation-suite", - "es/agenteye/evaluator-skill" + "es/dashboard", + "es/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "es/agenteye/api-keys", - "es/agenteye/security" + "es/cloud/overview", + "es/cloud/connect", + "es/cloud/fleet", + "es/cloud/managed-policies", + "es/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "Primeros pasos", + "group": "Observe your agents", "pages": [ - "es/introduction", - "es/getting-started" + "es/cloud/event-stream", + "es/cloud/sessions", + "es/cloud/performance", + "es/cloud/errors" ] }, { - "group": "Conceptos principales", + "group": "Analyze and act", "pages": [ - "es/built-in-policies", - "es/custom-policies", - "es/configuration" + "es/cloud/evaluations", + "es/cloud/evaluators", + "es/cloud/queries", + "es/cloud/dashboards", + "es/cloud/audits", + "es/cloud/alerts", + "es/cloud/incidents", + "es/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "es/cli/dashboard", + "es/cloud/sdk", + "es/cloud/cli", + "es/cloud/cli-recipes", + "es/cloud/agent-skills", + "es/cloud/access", + "es/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "es/cli/config", "es/cli/install-policies", "es/cli/remove-policies", "es/cli/list-policies", + "es/cli/harness", + "es/cli/backfill", + "es/cli/flush", "es/cli/hook", - "es/cli/audit", + "es/cli/dashboard", + "es/cli/update", + "es/cli/migrate", + "es/cli/uninstall", "es/cli/version", "es/cli/environment-variables" ] }, { - "group": "Herramientas", - "pages": [ - "es/dashboard" - ] - }, - { - "group": "Avanzado", + "group": "Reference", "pages": [ - "es/architecture", - "es/testing", + "es/reference/files", + "es/examples", "es/package-aliases", "es/for-agents" ] - }, - { - "group": "Ejemplos", - "pages": [ - "es/examples" - ] } ] } @@ -589,108 +577,106 @@ "language": "pt-BR", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "pt-br/agenteye/overview", - "pt-br/agenteye/concepts", - "pt-br/agenteye/observability" + "pt-br/introduction", + "pt-br/quickstart", + "pt-br/how-it-works", + "pt-br/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "pt-br/agenteye/event-stream", - "pt-br/agenteye/sessions", - "pt-br/agenteye/telemetry", - "pt-br/agenteye/error-tracking", - "pt-br/agenteye/evaluations", - "pt-br/agenteye/queries", - "pt-br/agenteye/dashboards", - "pt-br/agenteye/audits", - "pt-br/agenteye/alerts", - "pt-br/agenteye/incidents", - "pt-br/agenteye/assistant", - "pt-br/agenteye/cli-and-agents" + "pt-br/policies", + "pt-br/built-in-policies", + "pt-br/custom-policies", + "pt-br/configuration", + "pt-br/agent-support", + "pt-br/daemon", + "pt-br/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "pt-br/agenteye/python-sdk", - "pt-br/agenteye/python-sdk-skill", - "pt-br/agenteye/codex-capture", - "pt-br/agenteye/openclaw-capture", - "pt-br/agenteye/hermes-capture", - "pt-br/agenteye/cli", - "pt-br/agenteye/cli-skill", - "pt-br/agenteye/cli-recipes", - "pt-br/agenteye/evaluation-suite", - "pt-br/agenteye/evaluator-skill" + "pt-br/dashboard", + "pt-br/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "pt-br/agenteye/api-keys", - "pt-br/agenteye/security" + "pt-br/cloud/overview", + "pt-br/cloud/connect", + "pt-br/cloud/fleet", + "pt-br/cloud/managed-policies", + "pt-br/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "Começando", + "group": "Observe your agents", "pages": [ - "pt-br/introduction", - "pt-br/getting-started" + "pt-br/cloud/event-stream", + "pt-br/cloud/sessions", + "pt-br/cloud/performance", + "pt-br/cloud/errors" ] }, { - "group": "Conceitos principais", + "group": "Analyze and act", "pages": [ - "pt-br/built-in-policies", - "pt-br/custom-policies", - "pt-br/configuration" + "pt-br/cloud/evaluations", + "pt-br/cloud/evaluators", + "pt-br/cloud/queries", + "pt-br/cloud/dashboards", + "pt-br/cloud/audits", + "pt-br/cloud/alerts", + "pt-br/cloud/incidents", + "pt-br/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "pt-br/cli/dashboard", + "pt-br/cloud/sdk", + "pt-br/cloud/cli", + "pt-br/cloud/cli-recipes", + "pt-br/cloud/agent-skills", + "pt-br/cloud/access", + "pt-br/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "pt-br/cli/config", "pt-br/cli/install-policies", "pt-br/cli/remove-policies", "pt-br/cli/list-policies", + "pt-br/cli/harness", + "pt-br/cli/backfill", + "pt-br/cli/flush", "pt-br/cli/hook", - "pt-br/cli/audit", + "pt-br/cli/dashboard", + "pt-br/cli/update", + "pt-br/cli/migrate", + "pt-br/cli/uninstall", "pt-br/cli/version", "pt-br/cli/environment-variables" ] }, { - "group": "Ferramentas", - "pages": [ - "pt-br/dashboard" - ] - }, - { - "group": "Avançado", + "group": "Reference", "pages": [ - "pt-br/architecture", - "pt-br/testing", + "pt-br/reference/files", + "pt-br/examples", "pt-br/package-aliases", "pt-br/for-agents" ] - }, - { - "group": "Exemplos", - "pages": [ - "pt-br/examples" - ] } ] } @@ -700,108 +686,106 @@ "language": "de", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "de/agenteye/overview", - "de/agenteye/concepts", - "de/agenteye/observability" + "de/introduction", + "de/quickstart", + "de/how-it-works", + "de/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "de/agenteye/event-stream", - "de/agenteye/sessions", - "de/agenteye/telemetry", - "de/agenteye/error-tracking", - "de/agenteye/evaluations", - "de/agenteye/queries", - "de/agenteye/dashboards", - "de/agenteye/audits", - "de/agenteye/alerts", - "de/agenteye/incidents", - "de/agenteye/assistant", - "de/agenteye/cli-and-agents" + "de/policies", + "de/built-in-policies", + "de/custom-policies", + "de/configuration", + "de/agent-support", + "de/daemon", + "de/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "de/agenteye/python-sdk", - "de/agenteye/python-sdk-skill", - "de/agenteye/codex-capture", - "de/agenteye/openclaw-capture", - "de/agenteye/hermes-capture", - "de/agenteye/cli", - "de/agenteye/cli-skill", - "de/agenteye/cli-recipes", - "de/agenteye/evaluation-suite", - "de/agenteye/evaluator-skill" + "de/dashboard", + "de/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "de/agenteye/api-keys", - "de/agenteye/security" + "de/cloud/overview", + "de/cloud/connect", + "de/cloud/fleet", + "de/cloud/managed-policies", + "de/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "Erste Schritte", + "group": "Observe your agents", "pages": [ - "de/introduction", - "de/getting-started" + "de/cloud/event-stream", + "de/cloud/sessions", + "de/cloud/performance", + "de/cloud/errors" ] }, { - "group": "Kernkonzepte", + "group": "Analyze and act", "pages": [ - "de/built-in-policies", - "de/custom-policies", - "de/configuration" + "de/cloud/evaluations", + "de/cloud/evaluators", + "de/cloud/queries", + "de/cloud/dashboards", + "de/cloud/audits", + "de/cloud/alerts", + "de/cloud/incidents", + "de/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "de/cli/dashboard", + "de/cloud/sdk", + "de/cloud/cli", + "de/cloud/cli-recipes", + "de/cloud/agent-skills", + "de/cloud/access", + "de/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "de/cli/config", "de/cli/install-policies", "de/cli/remove-policies", "de/cli/list-policies", + "de/cli/harness", + "de/cli/backfill", + "de/cli/flush", "de/cli/hook", - "de/cli/audit", + "de/cli/dashboard", + "de/cli/update", + "de/cli/migrate", + "de/cli/uninstall", "de/cli/version", "de/cli/environment-variables" ] }, { - "group": "Werkzeuge", + "group": "Reference", "pages": [ - "de/dashboard" - ] - }, - { - "group": "Fortgeschritten", - "pages": [ - "de/architecture", - "de/testing", + "de/reference/files", + "de/examples", "de/package-aliases", "de/for-agents" ] - }, - { - "group": "Beispiele", - "pages": [ - "de/examples" - ] } ] } @@ -811,108 +795,106 @@ "language": "fr", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "fr/agenteye/overview", - "fr/agenteye/concepts", - "fr/agenteye/observability" + "fr/introduction", + "fr/quickstart", + "fr/how-it-works", + "fr/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "fr/agenteye/event-stream", - "fr/agenteye/sessions", - "fr/agenteye/telemetry", - "fr/agenteye/error-tracking", - "fr/agenteye/evaluations", - "fr/agenteye/queries", - "fr/agenteye/dashboards", - "fr/agenteye/audits", - "fr/agenteye/alerts", - "fr/agenteye/incidents", - "fr/agenteye/assistant", - "fr/agenteye/cli-and-agents" + "fr/policies", + "fr/built-in-policies", + "fr/custom-policies", + "fr/configuration", + "fr/agent-support", + "fr/daemon", + "fr/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "fr/agenteye/python-sdk", - "fr/agenteye/python-sdk-skill", - "fr/agenteye/codex-capture", - "fr/agenteye/openclaw-capture", - "fr/agenteye/hermes-capture", - "fr/agenteye/cli", - "fr/agenteye/cli-skill", - "fr/agenteye/cli-recipes", - "fr/agenteye/evaluation-suite", - "fr/agenteye/evaluator-skill" + "fr/dashboard", + "fr/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "fr/agenteye/api-keys", - "fr/agenteye/security" + "fr/cloud/overview", + "fr/cloud/connect", + "fr/cloud/fleet", + "fr/cloud/managed-policies", + "fr/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "Démarrage", + "group": "Observe your agents", "pages": [ - "fr/introduction", - "fr/getting-started" + "fr/cloud/event-stream", + "fr/cloud/sessions", + "fr/cloud/performance", + "fr/cloud/errors" ] }, { - "group": "Concepts clés", + "group": "Analyze and act", "pages": [ - "fr/built-in-policies", - "fr/custom-policies", - "fr/configuration" + "fr/cloud/evaluations", + "fr/cloud/evaluators", + "fr/cloud/queries", + "fr/cloud/dashboards", + "fr/cloud/audits", + "fr/cloud/alerts", + "fr/cloud/incidents", + "fr/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "fr/cli/dashboard", + "fr/cloud/sdk", + "fr/cloud/cli", + "fr/cloud/cli-recipes", + "fr/cloud/agent-skills", + "fr/cloud/access", + "fr/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "fr/cli/config", "fr/cli/install-policies", "fr/cli/remove-policies", "fr/cli/list-policies", + "fr/cli/harness", + "fr/cli/backfill", + "fr/cli/flush", "fr/cli/hook", - "fr/cli/audit", + "fr/cli/dashboard", + "fr/cli/update", + "fr/cli/migrate", + "fr/cli/uninstall", "fr/cli/version", "fr/cli/environment-variables" ] }, { - "group": "Outils", + "group": "Reference", "pages": [ - "fr/dashboard" - ] - }, - { - "group": "Avancé", - "pages": [ - "fr/architecture", - "fr/testing", + "fr/reference/files", + "fr/examples", "fr/package-aliases", "fr/for-agents" ] - }, - { - "group": "Exemples", - "pages": [ - "fr/examples" - ] } ] } @@ -922,108 +904,106 @@ "language": "ru", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", + "pages": [ + "ru/introduction", + "ru/quickstart", + "ru/how-it-works", + "ru/concepts" + ] + }, + { + "group": "Guardrails", "pages": [ - "ru/agenteye/overview", - "ru/agenteye/concepts", - "ru/agenteye/observability" + "ru/policies", + "ru/built-in-policies", + "ru/custom-policies", + "ru/configuration", + "ru/agent-support", + "ru/daemon", + "ru/testing" ] }, { - "group": "Features", + "group": "See what happened", "pages": [ - "ru/agenteye/event-stream", - "ru/agenteye/sessions", - "ru/agenteye/telemetry", - "ru/agenteye/error-tracking", - "ru/agenteye/evaluations", - "ru/agenteye/queries", - "ru/agenteye/dashboards", - "ru/agenteye/audits", - "ru/agenteye/alerts", - "ru/agenteye/incidents", - "ru/agenteye/assistant", - "ru/agenteye/cli-and-agents" + "ru/dashboard", + "ru/audit" ] }, { - "group": "SDK and CLI", + "group": "FailproofAI Cloud", "pages": [ - "ru/agenteye/python-sdk", - "ru/agenteye/python-sdk-skill", - "ru/agenteye/codex-capture", - "ru/agenteye/openclaw-capture", - "ru/agenteye/hermes-capture", - "ru/agenteye/cli", - "ru/agenteye/cli-skill", - "ru/agenteye/cli-recipes", - "ru/agenteye/evaluation-suite", - "ru/agenteye/evaluator-skill" + "ru/cloud/overview", + "ru/cloud/connect", + "ru/cloud/fleet", + "ru/cloud/managed-policies", + "ru/cloud/capture" ] }, { - "group": "Administration", + "group": "Observe your agents", "pages": [ - "ru/agenteye/api-keys", - "ru/agenteye/security" + "ru/cloud/event-stream", + "ru/cloud/sessions", + "ru/cloud/performance", + "ru/cloud/errors" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "Начало работы", + "group": "Analyze and act", "pages": [ - "ru/introduction", - "ru/getting-started" + "ru/cloud/evaluations", + "ru/cloud/evaluators", + "ru/cloud/queries", + "ru/cloud/dashboards", + "ru/cloud/audits", + "ru/cloud/alerts", + "ru/cloud/incidents", + "ru/cloud/assistant" ] }, { - "group": "Основные концепции", + "group": "Build on it", "pages": [ - "ru/built-in-policies", - "ru/custom-policies", - "ru/configuration" + "ru/cloud/sdk", + "ru/cloud/cli", + "ru/cloud/cli-recipes", + "ru/cloud/agent-skills", + "ru/cloud/access", + "ru/cloud/security" ] }, { - "group": "CLI", + "group": "CLI reference", "pages": [ - "ru/cli/dashboard", + "ru/cli/config", "ru/cli/install-policies", "ru/cli/remove-policies", "ru/cli/list-policies", + "ru/cli/harness", + "ru/cli/backfill", + "ru/cli/flush", "ru/cli/hook", - "ru/cli/audit", + "ru/cli/dashboard", + "ru/cli/update", + "ru/cli/migrate", + "ru/cli/uninstall", "ru/cli/version", "ru/cli/environment-variables" ] }, { - "group": "Инструменты", - "pages": [ - "ru/dashboard" - ] - }, - { - "group": "Продвинутый", + "group": "Reference", "pages": [ - "ru/architecture", - "ru/testing", + "ru/reference/files", + "ru/examples", "ru/package-aliases", "ru/for-agents" ] - }, - { - "group": "Примеры", - "pages": [ - "ru/examples" - ] } ] } @@ -1033,108 +1013,106 @@ "language": "hi", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "hi/agenteye/overview", - "hi/agenteye/concepts", - "hi/agenteye/observability" + "hi/introduction", + "hi/quickstart", + "hi/how-it-works", + "hi/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "hi/agenteye/event-stream", - "hi/agenteye/sessions", - "hi/agenteye/telemetry", - "hi/agenteye/error-tracking", - "hi/agenteye/evaluations", - "hi/agenteye/queries", - "hi/agenteye/dashboards", - "hi/agenteye/audits", - "hi/agenteye/alerts", - "hi/agenteye/incidents", - "hi/agenteye/assistant", - "hi/agenteye/cli-and-agents" + "hi/policies", + "hi/built-in-policies", + "hi/custom-policies", + "hi/configuration", + "hi/agent-support", + "hi/daemon", + "hi/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "hi/agenteye/python-sdk", - "hi/agenteye/python-sdk-skill", - "hi/agenteye/codex-capture", - "hi/agenteye/openclaw-capture", - "hi/agenteye/hermes-capture", - "hi/agenteye/cli", - "hi/agenteye/cli-skill", - "hi/agenteye/cli-recipes", - "hi/agenteye/evaluation-suite", - "hi/agenteye/evaluator-skill" + "hi/dashboard", + "hi/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "hi/agenteye/api-keys", - "hi/agenteye/security" + "hi/cloud/overview", + "hi/cloud/connect", + "hi/cloud/fleet", + "hi/cloud/managed-policies", + "hi/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "शुरू करें", + "group": "Observe your agents", "pages": [ - "hi/introduction", - "hi/getting-started" + "hi/cloud/event-stream", + "hi/cloud/sessions", + "hi/cloud/performance", + "hi/cloud/errors" ] }, { - "group": "मूल अवधारणाएँ", + "group": "Analyze and act", "pages": [ - "hi/built-in-policies", - "hi/custom-policies", - "hi/configuration" + "hi/cloud/evaluations", + "hi/cloud/evaluators", + "hi/cloud/queries", + "hi/cloud/dashboards", + "hi/cloud/audits", + "hi/cloud/alerts", + "hi/cloud/incidents", + "hi/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "hi/cli/dashboard", + "hi/cloud/sdk", + "hi/cloud/cli", + "hi/cloud/cli-recipes", + "hi/cloud/agent-skills", + "hi/cloud/access", + "hi/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "hi/cli/config", "hi/cli/install-policies", "hi/cli/remove-policies", "hi/cli/list-policies", + "hi/cli/harness", + "hi/cli/backfill", + "hi/cli/flush", "hi/cli/hook", - "hi/cli/audit", + "hi/cli/dashboard", + "hi/cli/update", + "hi/cli/migrate", + "hi/cli/uninstall", "hi/cli/version", "hi/cli/environment-variables" ] }, { - "group": "उपकरण", - "pages": [ - "hi/dashboard" - ] - }, - { - "group": "उन्नत", + "group": "Reference", "pages": [ - "hi/architecture", - "hi/testing", + "hi/reference/files", + "hi/examples", "hi/package-aliases", "hi/for-agents" ] - }, - { - "group": "उदाहरण", - "pages": [ - "hi/examples" - ] } ] } @@ -1144,108 +1122,106 @@ "language": "tr", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "tr/agenteye/overview", - "tr/agenteye/concepts", - "tr/agenteye/observability" + "tr/introduction", + "tr/quickstart", + "tr/how-it-works", + "tr/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "tr/agenteye/event-stream", - "tr/agenteye/sessions", - "tr/agenteye/telemetry", - "tr/agenteye/error-tracking", - "tr/agenteye/evaluations", - "tr/agenteye/queries", - "tr/agenteye/dashboards", - "tr/agenteye/audits", - "tr/agenteye/alerts", - "tr/agenteye/incidents", - "tr/agenteye/assistant", - "tr/agenteye/cli-and-agents" + "tr/policies", + "tr/built-in-policies", + "tr/custom-policies", + "tr/configuration", + "tr/agent-support", + "tr/daemon", + "tr/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "tr/agenteye/python-sdk", - "tr/agenteye/python-sdk-skill", - "tr/agenteye/codex-capture", - "tr/agenteye/openclaw-capture", - "tr/agenteye/hermes-capture", - "tr/agenteye/cli", - "tr/agenteye/cli-skill", - "tr/agenteye/cli-recipes", - "tr/agenteye/evaluation-suite", - "tr/agenteye/evaluator-skill" + "tr/dashboard", + "tr/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "tr/agenteye/api-keys", - "tr/agenteye/security" + "tr/cloud/overview", + "tr/cloud/connect", + "tr/cloud/fleet", + "tr/cloud/managed-policies", + "tr/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "Başlangıç", + "group": "Observe your agents", "pages": [ - "tr/introduction", - "tr/getting-started" + "tr/cloud/event-stream", + "tr/cloud/sessions", + "tr/cloud/performance", + "tr/cloud/errors" ] }, { - "group": "Temel Kavramlar", + "group": "Analyze and act", "pages": [ - "tr/built-in-policies", - "tr/custom-policies", - "tr/configuration" + "tr/cloud/evaluations", + "tr/cloud/evaluators", + "tr/cloud/queries", + "tr/cloud/dashboards", + "tr/cloud/audits", + "tr/cloud/alerts", + "tr/cloud/incidents", + "tr/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "tr/cli/dashboard", + "tr/cloud/sdk", + "tr/cloud/cli", + "tr/cloud/cli-recipes", + "tr/cloud/agent-skills", + "tr/cloud/access", + "tr/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "tr/cli/config", "tr/cli/install-policies", "tr/cli/remove-policies", "tr/cli/list-policies", + "tr/cli/harness", + "tr/cli/backfill", + "tr/cli/flush", "tr/cli/hook", - "tr/cli/audit", + "tr/cli/dashboard", + "tr/cli/update", + "tr/cli/migrate", + "tr/cli/uninstall", "tr/cli/version", "tr/cli/environment-variables" ] }, { - "group": "Araçlar", + "group": "Reference", "pages": [ - "tr/dashboard" - ] - }, - { - "group": "Gelişmiş", - "pages": [ - "tr/architecture", - "tr/testing", + "tr/reference/files", + "tr/examples", "tr/package-aliases", "tr/for-agents" ] - }, - { - "group": "Örnekler", - "pages": [ - "tr/examples" - ] } ] } @@ -1255,108 +1231,106 @@ "language": "vi", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "vi/agenteye/overview", - "vi/agenteye/concepts", - "vi/agenteye/observability" + "vi/introduction", + "vi/quickstart", + "vi/how-it-works", + "vi/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "vi/agenteye/event-stream", - "vi/agenteye/sessions", - "vi/agenteye/telemetry", - "vi/agenteye/error-tracking", - "vi/agenteye/evaluations", - "vi/agenteye/queries", - "vi/agenteye/dashboards", - "vi/agenteye/audits", - "vi/agenteye/alerts", - "vi/agenteye/incidents", - "vi/agenteye/assistant", - "vi/agenteye/cli-and-agents" + "vi/policies", + "vi/built-in-policies", + "vi/custom-policies", + "vi/configuration", + "vi/agent-support", + "vi/daemon", + "vi/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "vi/agenteye/python-sdk", - "vi/agenteye/python-sdk-skill", - "vi/agenteye/codex-capture", - "vi/agenteye/openclaw-capture", - "vi/agenteye/hermes-capture", - "vi/agenteye/cli", - "vi/agenteye/cli-skill", - "vi/agenteye/cli-recipes", - "vi/agenteye/evaluation-suite", - "vi/agenteye/evaluator-skill" + "vi/dashboard", + "vi/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "vi/agenteye/api-keys", - "vi/agenteye/security" + "vi/cloud/overview", + "vi/cloud/connect", + "vi/cloud/fleet", + "vi/cloud/managed-policies", + "vi/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "Bắt đầu", + "group": "Observe your agents", "pages": [ - "vi/introduction", - "vi/getting-started" + "vi/cloud/event-stream", + "vi/cloud/sessions", + "vi/cloud/performance", + "vi/cloud/errors" ] }, { - "group": "Khái niệm cốt lõi", + "group": "Analyze and act", "pages": [ - "vi/built-in-policies", - "vi/custom-policies", - "vi/configuration" + "vi/cloud/evaluations", + "vi/cloud/evaluators", + "vi/cloud/queries", + "vi/cloud/dashboards", + "vi/cloud/audits", + "vi/cloud/alerts", + "vi/cloud/incidents", + "vi/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "vi/cli/dashboard", + "vi/cloud/sdk", + "vi/cloud/cli", + "vi/cloud/cli-recipes", + "vi/cloud/agent-skills", + "vi/cloud/access", + "vi/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "vi/cli/config", "vi/cli/install-policies", "vi/cli/remove-policies", "vi/cli/list-policies", + "vi/cli/harness", + "vi/cli/backfill", + "vi/cli/flush", "vi/cli/hook", - "vi/cli/audit", + "vi/cli/dashboard", + "vi/cli/update", + "vi/cli/migrate", + "vi/cli/uninstall", "vi/cli/version", "vi/cli/environment-variables" ] }, { - "group": "Công cụ", - "pages": [ - "vi/dashboard" - ] - }, - { - "group": "Nâng cao", + "group": "Reference", "pages": [ - "vi/architecture", - "vi/testing", + "vi/reference/files", + "vi/examples", "vi/package-aliases", "vi/for-agents" ] - }, - { - "group": "Ví dụ", - "pages": [ - "vi/examples" - ] } ] } @@ -1366,108 +1340,106 @@ "language": "it", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", + "pages": [ + "it/introduction", + "it/quickstart", + "it/how-it-works", + "it/concepts" + ] + }, + { + "group": "Guardrails", "pages": [ - "it/agenteye/overview", - "it/agenteye/concepts", - "it/agenteye/observability" + "it/policies", + "it/built-in-policies", + "it/custom-policies", + "it/configuration", + "it/agent-support", + "it/daemon", + "it/testing" ] }, { - "group": "Features", + "group": "See what happened", "pages": [ - "it/agenteye/event-stream", - "it/agenteye/sessions", - "it/agenteye/telemetry", - "it/agenteye/error-tracking", - "it/agenteye/evaluations", - "it/agenteye/queries", - "it/agenteye/dashboards", - "it/agenteye/audits", - "it/agenteye/alerts", - "it/agenteye/incidents", - "it/agenteye/assistant", - "it/agenteye/cli-and-agents" + "it/dashboard", + "it/audit" ] }, { - "group": "SDK and CLI", + "group": "FailproofAI Cloud", "pages": [ - "it/agenteye/python-sdk", - "it/agenteye/python-sdk-skill", - "it/agenteye/codex-capture", - "it/agenteye/openclaw-capture", - "it/agenteye/hermes-capture", - "it/agenteye/cli", - "it/agenteye/cli-skill", - "it/agenteye/cli-recipes", - "it/agenteye/evaluation-suite", - "it/agenteye/evaluator-skill" + "it/cloud/overview", + "it/cloud/connect", + "it/cloud/fleet", + "it/cloud/managed-policies", + "it/cloud/capture" ] }, { - "group": "Administration", + "group": "Observe your agents", "pages": [ - "it/agenteye/api-keys", - "it/agenteye/security" + "it/cloud/event-stream", + "it/cloud/sessions", + "it/cloud/performance", + "it/cloud/errors" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "Per iniziare", + "group": "Analyze and act", "pages": [ - "it/introduction", - "it/getting-started" + "it/cloud/evaluations", + "it/cloud/evaluators", + "it/cloud/queries", + "it/cloud/dashboards", + "it/cloud/audits", + "it/cloud/alerts", + "it/cloud/incidents", + "it/cloud/assistant" ] }, { - "group": "Concetti chiave", + "group": "Build on it", "pages": [ - "it/built-in-policies", - "it/custom-policies", - "it/configuration" + "it/cloud/sdk", + "it/cloud/cli", + "it/cloud/cli-recipes", + "it/cloud/agent-skills", + "it/cloud/access", + "it/cloud/security" ] }, { - "group": "CLI", + "group": "CLI reference", "pages": [ - "it/cli/dashboard", + "it/cli/config", "it/cli/install-policies", "it/cli/remove-policies", "it/cli/list-policies", + "it/cli/harness", + "it/cli/backfill", + "it/cli/flush", "it/cli/hook", - "it/cli/audit", + "it/cli/dashboard", + "it/cli/update", + "it/cli/migrate", + "it/cli/uninstall", "it/cli/version", "it/cli/environment-variables" ] }, { - "group": "Strumenti", + "group": "Reference", "pages": [ - "it/dashboard" - ] - }, - { - "group": "Avanzato", - "pages": [ - "it/architecture", - "it/testing", + "it/reference/files", + "it/examples", "it/package-aliases", "it/for-agents" ] - }, - { - "group": "Esempi", - "pages": [ - "it/examples" - ] } ] } @@ -1477,108 +1449,106 @@ "language": "ar", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "ar/agenteye/overview", - "ar/agenteye/concepts", - "ar/agenteye/observability" + "ar/introduction", + "ar/quickstart", + "ar/how-it-works", + "ar/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "ar/agenteye/event-stream", - "ar/agenteye/sessions", - "ar/agenteye/telemetry", - "ar/agenteye/error-tracking", - "ar/agenteye/evaluations", - "ar/agenteye/queries", - "ar/agenteye/dashboards", - "ar/agenteye/audits", - "ar/agenteye/alerts", - "ar/agenteye/incidents", - "ar/agenteye/assistant", - "ar/agenteye/cli-and-agents" + "ar/policies", + "ar/built-in-policies", + "ar/custom-policies", + "ar/configuration", + "ar/agent-support", + "ar/daemon", + "ar/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "ar/agenteye/python-sdk", - "ar/agenteye/python-sdk-skill", - "ar/agenteye/codex-capture", - "ar/agenteye/openclaw-capture", - "ar/agenteye/hermes-capture", - "ar/agenteye/cli", - "ar/agenteye/cli-skill", - "ar/agenteye/cli-recipes", - "ar/agenteye/evaluation-suite", - "ar/agenteye/evaluator-skill" + "ar/dashboard", + "ar/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "ar/agenteye/api-keys", - "ar/agenteye/security" + "ar/cloud/overview", + "ar/cloud/connect", + "ar/cloud/fleet", + "ar/cloud/managed-policies", + "ar/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "البداية", + "group": "Observe your agents", "pages": [ - "ar/introduction", - "ar/getting-started" + "ar/cloud/event-stream", + "ar/cloud/sessions", + "ar/cloud/performance", + "ar/cloud/errors" ] }, { - "group": "المفاهيم الأساسية", + "group": "Analyze and act", "pages": [ - "ar/built-in-policies", - "ar/custom-policies", - "ar/configuration" + "ar/cloud/evaluations", + "ar/cloud/evaluators", + "ar/cloud/queries", + "ar/cloud/dashboards", + "ar/cloud/audits", + "ar/cloud/alerts", + "ar/cloud/incidents", + "ar/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "ar/cli/dashboard", + "ar/cloud/sdk", + "ar/cloud/cli", + "ar/cloud/cli-recipes", + "ar/cloud/agent-skills", + "ar/cloud/access", + "ar/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "ar/cli/config", "ar/cli/install-policies", "ar/cli/remove-policies", "ar/cli/list-policies", + "ar/cli/harness", + "ar/cli/backfill", + "ar/cli/flush", "ar/cli/hook", - "ar/cli/audit", + "ar/cli/dashboard", + "ar/cli/update", + "ar/cli/migrate", + "ar/cli/uninstall", "ar/cli/version", "ar/cli/environment-variables" ] }, { - "group": "الأدوات", - "pages": [ - "ar/dashboard" - ] - }, - { - "group": "متقدم", + "group": "Reference", "pages": [ - "ar/architecture", - "ar/testing", + "ar/reference/files", + "ar/examples", "ar/package-aliases", "ar/for-agents" ] - }, - { - "group": "أمثلة", - "pages": [ - "ar/examples" - ] } ] } @@ -1588,133 +1558,112 @@ "language": "he", "tabs": [ { - "tab": "Observability", + "tab": "Documentation", "groups": [ { - "group": "Overview", + "group": "Start here", "pages": [ - "he/agenteye/overview", - "he/agenteye/concepts", - "he/agenteye/observability" + "he/introduction", + "he/quickstart", + "he/how-it-works", + "he/concepts" ] }, { - "group": "Features", + "group": "Guardrails", "pages": [ - "he/agenteye/event-stream", - "he/agenteye/sessions", - "he/agenteye/telemetry", - "he/agenteye/error-tracking", - "he/agenteye/evaluations", - "he/agenteye/queries", - "he/agenteye/dashboards", - "he/agenteye/audits", - "he/agenteye/alerts", - "he/agenteye/incidents", - "he/agenteye/assistant", - "he/agenteye/cli-and-agents" + "he/policies", + "he/built-in-policies", + "he/custom-policies", + "he/configuration", + "he/agent-support", + "he/daemon", + "he/testing" ] }, { - "group": "SDK and CLI", + "group": "See what happened", "pages": [ - "he/agenteye/python-sdk", - "he/agenteye/python-sdk-skill", - "he/agenteye/codex-capture", - "he/agenteye/openclaw-capture", - "he/agenteye/hermes-capture", - "he/agenteye/cli", - "he/agenteye/cli-skill", - "he/agenteye/cli-recipes", - "he/agenteye/evaluation-suite", - "he/agenteye/evaluator-skill" + "he/dashboard", + "he/audit" ] }, { - "group": "Administration", + "group": "FailproofAI Cloud", "pages": [ - "he/agenteye/api-keys", - "he/agenteye/security" + "he/cloud/overview", + "he/cloud/connect", + "he/cloud/fleet", + "he/cloud/managed-policies", + "he/cloud/capture" ] - } - ] - }, - { - "tab": "Enforcement", - "groups": [ + }, { - "group": "תחילת עבודה", + "group": "Observe your agents", "pages": [ - "he/introduction", - "he/getting-started" + "he/cloud/event-stream", + "he/cloud/sessions", + "he/cloud/performance", + "he/cloud/errors" ] }, { - "group": "מושגי יסוד", + "group": "Analyze and act", "pages": [ - "he/built-in-policies", - "he/custom-policies", - "he/configuration" + "he/cloud/evaluations", + "he/cloud/evaluators", + "he/cloud/queries", + "he/cloud/dashboards", + "he/cloud/audits", + "he/cloud/alerts", + "he/cloud/incidents", + "he/cloud/assistant" ] }, { - "group": "CLI", + "group": "Build on it", "pages": [ - "he/cli/dashboard", + "he/cloud/sdk", + "he/cloud/cli", + "he/cloud/cli-recipes", + "he/cloud/agent-skills", + "he/cloud/access", + "he/cloud/security" + ] + }, + { + "group": "CLI reference", + "pages": [ + "he/cli/config", "he/cli/install-policies", "he/cli/remove-policies", "he/cli/list-policies", + "he/cli/harness", + "he/cli/backfill", + "he/cli/flush", "he/cli/hook", - "he/cli/audit", + "he/cli/dashboard", + "he/cli/update", + "he/cli/migrate", + "he/cli/uninstall", "he/cli/version", "he/cli/environment-variables" ] }, { - "group": "כלים", + "group": "Reference", "pages": [ - "he/dashboard" - ] - }, - { - "group": "מתקדם", - "pages": [ - "he/architecture", - "he/testing", + "he/reference/files", + "he/examples", "he/package-aliases", "he/for-agents" ] - }, - { - "group": "דוגמאות", - "pages": [ - "he/examples" - ] } ] } ] } - ], - "global": { - "anchors": [ - { - "anchor": "GitHub", - "href": "https://github.com/failproofai/failproofai", - "icon": "github" - }, - { - "anchor": "npm", - "href": "https://www.npmjs.com/package/failproofai", - "icon": "npm" - }, - { - "anchor": "Discord", - "href": "https://discord.befailproof.ai/", - "icon": "discord" - } - ] - } + ] }, "navbar": { "links": [], @@ -1770,56 +1719,232 @@ }, "redirects": [ { - "source": "/agenteye/collector-installation", - "destination": "/agenteye/overview" + "source": "/agenteye/alerts", + "destination": "/cloud/alerts" + }, + { + "source": "/agenteye/api-keys", + "destination": "/cloud/access" + }, + { + "source": "/agenteye/assistant", + "destination": "/cloud/assistant" + }, + { + "source": "/agenteye/audits", + "destination": "/cloud/audits" + }, + { + "source": "/agenteye/cli", + "destination": "/cloud/cli" + }, + { + "source": "/agenteye/cli-and-agents", + "destination": "/cloud/cli" + }, + { + "source": "/agenteye/cli-recipes", + "destination": "/cloud/cli-recipes" + }, + { + "source": "/agenteye/cli-skill", + "destination": "/cloud/agent-skills" + }, + { + "source": "/agenteye/codex-capture", + "destination": "/cloud/capture" + }, + { + "source": "/agenteye/concepts", + "destination": "/concepts" + }, + { + "source": "/agenteye/dashboards", + "destination": "/cloud/dashboards" + }, + { + "source": "/agenteye/error-tracking", + "destination": "/cloud/errors" + }, + { + "source": "/agenteye/evaluation-suite", + "destination": "/cloud/evaluators" + }, + { + "source": "/agenteye/evaluations", + "destination": "/cloud/evaluations" + }, + { + "source": "/agenteye/evaluator-skill", + "destination": "/cloud/agent-skills" + }, + { + "source": "/agenteye/event-stream", + "destination": "/cloud/event-stream" + }, + { + "source": "/agenteye/hermes-capture", + "destination": "/cloud/capture" + }, + { + "source": "/agenteye/incidents", + "destination": "/cloud/incidents" + }, + { + "source": "/agenteye/observability", + "destination": "/cloud/overview" + }, + { + "source": "/agenteye/openclaw-capture", + "destination": "/cloud/capture" + }, + { + "source": "/agenteye/overview", + "destination": "/cloud/overview" + }, + { + "source": "/agenteye/python-sdk", + "destination": "/cloud/sdk" + }, + { + "source": "/agenteye/python-sdk-skill", + "destination": "/cloud/agent-skills" + }, + { + "source": "/agenteye/queries", + "destination": "/cloud/queries" + }, + { + "source": "/agenteye/security", + "destination": "/cloud/security" + }, + { + "source": "/agenteye/sessions", + "destination": "/cloud/sessions" + }, + { + "source": "/agenteye/telemetry", + "destination": "/cloud/performance" + }, + { + "source": "/architecture", + "destination": "/how-it-works" + }, + { + "source": "/cli/audit", + "destination": "/audit" + }, + { + "source": "/cloud/api-keys", + "destination": "/cloud/access" + }, + { + "source": "/cloud/cli-and-agents", + "destination": "/cloud/cli" + }, + { + "source": "/cloud/cli-skill", + "destination": "/cloud/agent-skills" + }, + { + "source": "/cloud/codex-capture", + "destination": "/cloud/capture" + }, + { + "source": "/cloud/collector-installation", + "destination": "/cloud/overview" + }, + { + "source": "/cloud/collector-migration", + "destination": "/cloud/overview" + }, + { + "source": "/cloud/concepts", + "destination": "/concepts" + }, + { + "source": "/cloud/deployment", + "destination": "/cloud/overview" + }, + { + "source": "/cloud/deployment-options", + "destination": "/cloud/overview" + }, + { + "source": "/cloud/error-tracking", + "destination": "/cloud/errors" + }, + { + "source": "/cloud/evaluation-suite", + "destination": "/cloud/evaluators" + }, + { + "source": "/cloud/evaluator-skill", + "destination": "/cloud/agent-skills" + }, + { + "source": "/cloud/faq", + "destination": "/cloud/overview" + }, + { + "source": "/cloud/getting-started", + "destination": "/cloud/overview" + }, + { + "source": "/cloud/github-token", + "destination": "/cloud/access" + }, + { + "source": "/cloud/health-monitoring", + "destination": "/cloud/overview" }, { - "source": "/agenteye/collector-migration", - "destination": "/agenteye/overview" + "source": "/cloud/hermes-capture", + "destination": "/cloud/capture" }, { - "source": "/agenteye/deployment", - "destination": "/agenteye/overview" + "source": "/cloud/kubernetes-deployment", + "destination": "/cloud/overview" }, { - "source": "/agenteye/deployment-options", - "destination": "/agenteye/overview" + "source": "/cloud/managed-deployment", + "destination": "/cloud/overview" }, { - "source": "/agenteye/faq", - "destination": "/agenteye/overview" + "source": "/cloud/observability", + "destination": "/cloud/overview" }, { - "source": "/agenteye/getting-started", - "destination": "/agenteye/overview" + "source": "/cloud/openclaw-capture", + "destination": "/cloud/capture" }, { - "source": "/agenteye/github-token", - "destination": "/agenteye/api-keys" + "source": "/cloud/python-sdk", + "destination": "/cloud/sdk" }, { - "source": "/agenteye/health-monitoring", - "destination": "/agenteye/observability" + "source": "/cloud/python-sdk-skill", + "destination": "/cloud/agent-skills" }, { - "source": "/agenteye/kubernetes-deployment", - "destination": "/agenteye/overview" + "source": "/cloud/single-pod-deployment", + "destination": "/cloud/overview" }, { - "source": "/agenteye/managed-deployment", - "destination": "/agenteye/overview" + "source": "/cloud/telemetry", + "destination": "/cloud/performance" }, { - "source": "/agenteye/single-pod-deployment", - "destination": "/agenteye/overview" + "source": "/cloud/tenant-management", + "destination": "/cloud/access" }, { - "source": "/agenteye/tenant-management", - "destination": "/agenteye/api-keys" + "source": "/cloud/troubleshooting", + "destination": "/cloud/overview" }, { - "source": "/agenteye/troubleshooting", - "destination": "/agenteye/overview" + "source": "/getting-started", + "destination": "/quickstart" } ], "integrations": { diff --git a/docs/es/agent-support.mdx b/docs/es/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/es/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/es/agenteye/cli-and-agents.mdx b/docs/es/agenteye/cli-and-agents.mdx deleted file mode 100644 index 8c9aba70..00000000 --- a/docs/es/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "Todo tu despliegue de Failproof AI Observability, a un comando de distancia." ---- - - -Todo tu despliegue de Failproof AI Observability, a un comando de distancia. Revisa producción, genera una clave de API o reconoce un incidente sin salir de tu terminal, luego automatiza cualquiera de estas acciones en CI, o deja que un agente de código lo haga por ti en lenguaje natural. - -```bash -pipx install agenteye -agenteye login --email you@example.com # a 6-digit code lands in your inbox -agenteye --json sessions --since 24h # every agent run from the last day, newest first -``` - -*El CLI `agenteye` se comunica con tu dashboard. Es una herramienta distinta al colector, que envía eventos al servidor.* - -## Todo tu despliegue, a un comando de distancia - -Deja de cambiar de pestaña para responder una pregunta rápida. El CLI `agenteye` lee tus datos y administra tu organización desde un único binario, de modo que una verificación que antes requería navegar por el dashboard se convierte en una línea que puedes volver a ejecutar, crear un alias o pegar en un runbook. Dispones de cuatro áreas: - -- **Lee tus datos:** `sessions`, `events`, `evals` y `errors`, filtrados por tiempo, agente y entorno. -- **Administra tu organización:** `keys`, `users`, `settings`, `alerts` e `incidents`. -- **Ejecuta análisis:** SQL guardado y un ejecutor `query` ad hoc sobre tus datos de eventos. -- **Consulta al asistente:** `agent ask` accede al mismo analista de solo lectura con el que conversas en el dashboard. - -Instálalo una vez con `pipx`, inicia sesión con un código de 6 dígitos enviado por correo, y ya estás listo. La sesión dura aproximadamente un día; vuelve a ejecutar `agenteye login` cuando expire. Úsalo para revisar producción a fondo, aprovisionar una clave o clasificar un incidente activo, todo sin abrir un navegador: - -```bash -agenteye errors --since 24h --aggregate # what is breaking, grouped by error type -agenteye incidents list --state firing # what is on fire right now -agenteye keys create ci --add events:add # a key that can only push events, secret shown once -``` - -Un hábito importante: las opciones globales como `--json` van antes del comando. `agenteye --json sessions` es correcto; `agenteye sessions --json`, no. - -## Automatízalo, intégralo en CI - -Cada comando acepta `--json`, y eso lo cambia todo. El JSON limpio va a stdout mientras los mensajes de estado y advertencias van a stderr, por lo que una captura con `--json` se puede pasar directamente a `jq` sin necesidad de limpiar líneas adicionales. Esto hace que el CLI sea igual de útil tanto para ti en la terminal como para un agente de código que procesa la salida: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -Está diseñado para ejecutarse de forma desatendida. Las confirmaciones se omiten automáticamente cuando no hay terminal conectada, así que nada queda bloqueado en un pipeline, y cada comando devuelve un código de salida significativo: `0` éxito, `4` no autenticado, `5` permiso faltante (el mensaje lo indica, por ejemplo `alerts:write`), `3` dashboard inaccesible. Un script puede bifurcarse en un `4` para reautenticarse o en un `5` para saber exactamente qué solicitar a un administrador, en lugar de fallar sin información. - -## Deja que un agente de código lo maneje en lenguaje natural - -Mejor aún, no deberías tener que recordar ninguna de estas opciones. El **CLI skill** es una pequeña carpeta de Agent Skill llamada `agenteye-cli` que enseña a un agente de código como Claude Code o Codex a manejar el CLI mediante solicitudes en lenguaje natural. Pregunta "¿hay algo roto hoy?" y el agente selecciona el comando, lo ejecuta como tú y responde en prosa. - -Para Claude Code, coloca la carpeta `agenteye-cli` en `~/.claude/skills/` y se descubre automáticamente. Failproof AI Observability proporciona la carpeta; no hay nada extra que instalar, ya que solo controla el CLI que ya tienes instalado. Inicia sesión tú mismo primero: el skill no puede completar el inicio de sesión con código enviado por correo en tu lugar. - -Como el agente ejecuta el CLI como tú, puede hacer todo lo que tu sesión permite, tanto lecturas como escrituras: crear claves, cambiar configuraciones, resolver incidentes. La confirmación interactiva del CLI no se activa para un agente, por lo que el skill está diseñado para indicar el comando exacto y esperar tu aprobación antes de cualquier cambio. Tú eres el paso de confirmación. - -```text -you Why did session run-001 fail? - -agent Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -Las lecturas son inmediatas, y cada escritura se pausa esperando tu confirmación: - -```text -you Give CI a key that can only push events. - -agent This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you yes - -agent Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## Relacionado - -- [Referencia del CLI](/es/agenteye/cli): cada comando, opción y estructura JSON. -- [Recetas del CLI para agentes](/es/agenteye/cli-recipes): patrones `jq` listos para usar y manejo de códigos de salida. -- [CLI agent skill](/es/agenteye/cli-skill): instala y ejecuta el skill `agenteye-cli`. -- [Asistente de IA](/es/agenteye/assistant): el analista integrado en el dashboard con el que `agent ask` se comunica. \ No newline at end of file diff --git a/docs/es/agenteye/cli-skill.mdx b/docs/es/agenteye/cli-skill.mdx deleted file mode 100644 index e0920f32..00000000 --- a/docs/es/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Habilidad de CLI para Observabilidad de Failproof AI" -description: "Pregúntale a tu agente de codificación «¿hay algo roto hoy?» y deja que responda con tus datos en vivo de Observabilidad de Failproof AI, sin comandos que memorizar." ---- - - -Pregúntale a tu agente de codificación *«¿hay algo roto hoy?»* y deja que responda con tus datos en vivo de Observabilidad de Failproof AI, sin comandos que memorizar. La **habilidad de CLI de Observabilidad de Failproof AI** (`agenteye-cli`) es una *Agent Skill*: una pequeña carpeta de instrucciones que un agente de codificación como Claude Code o Codex carga bajo demanda. Le enseña al agente a operar tu despliegue de Observabilidad a través de la [`agenteye` CLI](/es/agenteye/cli) mediante solicitudes en lenguaje natural como *«dale a CI una clave que solo pueda enviar eventos»* o *«acepta el incidente activo y asígnamelo»*. - -**No** es un servicio ni un binario independiente; no hay nada que desplegar. Se apoya en la CLI que ya tienes instalada: el agente invoca `agenteye --json …`, analiza el JSON limpio resultante y te responde en prosa. Todo lo que puede hacer, tú también podrías hacerlo escribiendo los mismos comandos. - ---- - -## Relación con las demás interfaces de Observabilidad de Failproof AI - -Failproof AI Observability te ofrece cuatro formas de acceder a los mismos datos y controles. Se complementan entre sí: - -| Interfaz | Qué es | Dónde se ejecuta | Úsala cuando | -|---|---|---|---| -| **[CLI](/es/agenteye/cli)** | La referencia de comandos y opciones de `agenteye` | Tu terminal | Quieres ejecutar o automatizar un comando específico | -| **[Recetas de CLI](/es/agenteye/cli-recipes)** | Patrones de `jq`/pipeline listos para copiar y pegar | Tu terminal / scripts | Estás integrando la CLI en automatizaciones | -| **Habilidad de CLI** (este doc) | Una puerta de entrada en lenguaje natural a la CLI | Tu agente de codificación, en tu estación de trabajo | Quieres *simplemente preguntar* y dejar que el agente elija el comando | -| **[Habilidad de evaluador](/es/agenteye/evaluator-skill)** | Una habilidad hermana que diseña y construye tu servicio de puntuación | Tu agente de codificación, en tu estación de trabajo | Quieres *producir* puntuaciones de evaluación en lugar de leerlas | -| **[Habilidad del SDK de Python](/es/agenteye/python-sdk-skill)** | Una habilidad hermana que instrumenta tu agente para que emita telemetría | Tu agente de codificación, en tu estación de trabajo | Quieres que tu agente *produzca* los eventos que esta habilidad lee | -| **[Asistente de IA en el dashboard](/es/agenteye/assistant)** | Un chat integrado en el dashboard | Del lado del servidor (en el dashboard) | Quieres hacer preguntas sobre tus datos dentro del dashboard | - -La habilidad en sí no tiene privilegios propios; simplemente convierte tus palabras en llamadas a la CLI que se ejecutan como tú: - -```mermaid -flowchart TD - YOU["tú: 'acepta el incidente activo'"] --> AGENT["agente de codificación (Claude Code / Codex)
carga la habilidad agenteye-cli"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|tu sesión autenticada de CLI| API["API del dashboard de Observabilidad"] -``` - -### vs. el asistente de IA en el dashboard: una distinción importante - -Son dos herramientas distintas con radios de acción muy diferentes: - -- El **asistente de IA en el dashboard** ([asistente de IA](/es/agenteye/assistant)) es un chat integrado en el dashboard, respaldado por el servicio de agente. Es **de solo lectura más autoría con aprobación**: puede redactar consultas guardadas y dashboards, pero cada escritura se pausa para esperar tu aprobación explícita con un clic, y nunca elimina nada. Requiere el permiso `agent:use` y solo accede a los datos de la organización que estás viendo. -- La **habilidad de CLI** se ejecuta en *tu* estación de trabajo dentro de *tu* agente de codificación y maneja la `agenteye` CLI **como tú**. Puede realizar la **superficie completa de la CLI, incluidas las mutaciones** (crear/rotar/deshabilitar claves API, cambiar configuraciones de la organización, resolver incidentes, eliminar consultas guardadas), limitada únicamente por los permisos de tu sesión de CLI. Trátala con exactamente el mismo cuidado con el que tratarías ejecutar esos comandos tú mismo. - ---- - -## Requisitos previos - -1. La **CLI `agenteye` instalada** y disponible en el `PATH` (consulta la referencia de [CLI](/es/agenteye/cli): `pipx install agenteye`). -2. Tu **URL del dashboard** configurada (`AGENTEYE_DASHBOARD_URL`, o el agente pasa `--base-url`). -3. Una **sesión activa**: ejecuta `agenteye login` tú mismo primero. La habilidad **no puede** completar el proceso de inicio de sesión con código de un solo uso enviado por correo; te indicará que ejecutes `agenteye login` si la sesión falta o ha expirado (código de salida `4` de la CLI). - ---- - -## Dónde obtenerla - -La habilidad está publicada en la colección pública de habilidades de Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -No hay ninguna restricción de acceso: el repositorio es público y la habilidad no necesita credenciales propias, ya que solo maneja la `agenteye` CLI **pública** contra *tu* dashboard, usando la sesión con la que *tú* iniciaste sesión. No necesitas pedírsela a nadie. - -Ten en cuenta que se distribuye como su propia carpeta y **no** está incluida en el paquete `pipx install agenteye`, así que no la busques allí. - -## Instalación de la habilidad - -La forma más rápida es usando la CLI [`skills`](https://skills.sh), que descarga la carpeta y la coloca donde tu agente la busca: - -```bash -# Claude Code, solo este proyecto -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# todos los proyectos (instala en ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex en su lugar -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -Luego adminístrala como cualquier otra habilidad: - -```bash -npx skills list -a claude-code # qué está instalado -npx skills update agenteye-cli # obtener la última versión -npx skills remove agenteye-cli # eliminarla -``` - -¿Prefieres instalarla manualmente? Una Agent Skill es simplemente una carpeta que contiene un `SKILL.md` (más referencias opcionales), así que copiarla también funciona: - -- **Claude Code**: coloca la carpeta `agenteye-cli/` en `~/.claude/skills/` (todos los proyectos) o `/.claude/skills/` (solo ese repositorio). Claude Code la descubre automáticamente — verifica con la lista `/skills`, o simplemente haz una pregunta que coincida con su descripción. -- **Codex (OpenAI)**: Codex lee el mismo `SKILL.md`. El archivo `agents/openai.yaml` incluido configura `allow_implicit_invocation: true`, por lo que Codex selecciona automáticamente la habilidad cuando una tarea coincide; de lo contrario, invócala explícitamente como `$agenteye-cli`. - ---- - -## Seguridad: las mutaciones NO solicitan confirmación cuando un agente ejecuta la CLI - -> **Advertencia:** Lee esto antes de permitir que un agente realice cambios. - -La `agenteye` CLI normalmente pregunta *«¿estás seguro?»* antes de una acción destructiva. **Omite automáticamente esa confirmación cuando no está conectada a un terminal (que es exactamente cómo la ejecuta un agente de codificación), y `--json` también la omite.** Por lo tanto, el aviso de seguridad **no** se activará para el agente. - -La habilidad está diseñada para compensar esto: está instruida para indicar el comando exacto que ejecutará y obtener tu **aprobación explícita antes de cualquier cambio de estado**. Mantén esa disciplina. Cuando manejas Failproof AI Observability a través de un agente, *tú* eres el paso de confirmación. Los comandos que cambian estado a los que debes prestar atención: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- los subcomandos de escritura de `incidents`: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -Todo lo que está bajo **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) es de solo lectura y no cambia nada. - -Dado que el agente actúa **como tú**, solo puede hacer lo que tu sesión tiene permiso para hacer; los permisos se resuelven **por organización** (consulta [API keys](/es/agenteye/api-keys)). Un comando para el que no tienes permiso devuelve el código de salida `5` con el nombre exacto del permiso requerido, por lo que el agente puede decirte exactamente qué pedirle a un administrador en lugar de fallar de forma opaca. - ---- - -## Qué puedes pedirle - -Aquí tienes un ejemplo de una interacción real completa: primero una lectura, luego un cambio que espera tu aprobación: - -```text -tú ▸ ¿Hay algo roto en el último día? - -agente▸ Ejecutando: agenteye --json errors --since 24h --aggregate - 3 tipos de error en las últimas 24h. El más frecuente es TimeoutError - (18 ocurrencias, todas en checkout-agent), luego RateLimited (5). - ¿Quieres ver las sesiones fallidas de TimeoutError? - -tú ▸ Dale a CI una clave que solo pueda enviar eventos. - -agente▸ Esto crea una clave API. Ejecutaré: - agenteye keys create ci --add events:add - ¿Procedo? - -tú ▸ sí - -agente▸ Listo. Clave "ci" creada con solo events:add. - El secreto se muestra una sola vez, así que guárdalo ahora. No puedo volver a mostrarlo. -``` - -La habilidad mapea cada intención en lenguaje natural al comando `agenteye` correcto, descubriendo primero los valores válidos (`list `, `whoami`) para no adivinar, e indicando el comando exacto antes de cualquier cambio. Más ejemplos: - -- *«¿Hay algo roto o fallando en las últimas 24 horas?»* → `errors --since 24h --aggregate`, luego un desglose. -- *«¿Por qué falló la sesión `run-001`?»* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *«¿Cómo está evolucionando la calidad esta semana?»* → `evals --aggregate --since 7d`, luego profundizar en las ejecuciones con puntuación baja. -- *«Dale a CI una clave que solo pueda enviar eventos.»* → `keys create ci --add events:add` (indica el comando, luego lo crea y captura el secreto de un solo uso). -- *«¿Quién tiene acceso? Dale a Dana permisos de solo lectura.»* → `users list` → `users update dana@… --permission-set read-only` (después de confirmar contigo). -- *«Acepta el incidente activo y asígnamelo.»* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`. - -Para los comandos exactos, opciones y formatos JSON detrás de estos, consulta la referencia de [CLI](/es/agenteye/cli) y las [recetas de CLI para agentes](/es/agenteye/cli-recipes). - ---- - -## Próximos pasos - -- **[CLI](/es/agenteye/cli)**: referencia completa de comandos y opciones de `agenteye`. -- **[Recetas de CLI para agentes](/es/agenteye/cli-recipes)**: patrones de `jq` listos para copiar y pegar, y manejo de códigos de salida. -- **[Habilidad del agente evaluador](/es/agenteye/evaluator-skill)**: la habilidad hermana, para construir el evaluador cuyas puntuaciones lee `agenteye evals`. -- **[Habilidad del agente SDK de Python](/es/agenteye/python-sdk-skill)**: la habilidad hermana, para instrumentar un agente y que emita la telemetría que lee `agenteye`. -- **[Asistente de IA](/es/agenteye/assistant)**: el asistente integrado en el dashboard (no confundir con esta habilidad de terminal). -- **[API keys](/es/agenteye/api-keys)**: el modelo de permisos por organización que limita lo que la habilidad puede hacer. \ No newline at end of file diff --git a/docs/es/agenteye/codex-capture.mdx b/docs/es/agenteye/codex-capture.mdx deleted file mode 100644 index ec3ae5e8..00000000 --- a/docs/es/agenteye/codex-capture.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "Captura de sesiones de Codex" -description: "Lleva las sesiones locales de OpenAI Codex de tu equipo a AgentEye como sesiones y eventos ordinarios, sin modificar la forma en que los ejecutan." ---- - -Tus ingenieros ya usan OpenAI Codex a diario. La captura de sesiones de Codex trae esas sesiones de trabajo a AgentEye como sesiones y eventos ordinarios, para que puedas buscarlas, reproducirlas y evaluarlas junto con todo lo demás que observas. Complementa el [SDK de Python](/es/agenteye/python-sdk): el SDK instrumenta los agentes que tú escribes, mientras que esto captura el trabajo en Codex que tu equipo ya realiza, sin cambiar nada en su flujo habitual. - -Un pequeño recolector en segundo plano lee las transcripciones de sesiones locales de Codex a medida que se van escribiendo y las envía a AgentEye. Un único recolector por máquina captura todas las superficies locales de Codex a la vez — no es necesario configurar nada por cada superficie. - -El mismo recolector también captura otros agentes — consulta [OpenClaw](/es/agenteye/openclaw-capture) y [Hermes](/es/agenteye/hermes-capture). Activa los que uses; un solo recolector puede capturar varios a la vez. - ---- - -## Qué captura - -Todas las superficies de Codex que se ejecutan **localmente** producen las mismas transcripciones de sesión en disco, y el recolector las recoge todas: - -- la **CLI** de Codex y `codex exec` -- la **extensión de VS Code / IDE** -- la **aplicación de escritorio**, cuando ejecuta una sesión de forma local - -Cada sesión de Codex se convierte en una [sesión](/es/agenteye/sessions) de AgentEye; sus mensajes de usuario y asistente, razonamiento, llamadas a herramientas, resultados de herramientas y uso de tokens se convierten en los [eventos](/es/agenteye/event-stream) correspondientes. La superficie de la que proviene cada sesión (CLI, IDE o escritorio) queda registrada para que puedas distinguirlas. - -> **Las sesiones en la nube no se capturan.** La aplicación de escritorio ejecuta cada vez más sesiones en la nube de Codex y solo guarda sus metadatos en la máquina local — no hay transcripción local que leer. Solo se capturan las sesiones ejecutadas localmente. - ---- - -## Cómo activarlo - -La captura está desactivada hasta que la habilites. Instala el recolector con una clave de API que tenga el permiso `events:add` (consulta [Claves de API](/es/agenteye/api-keys)) y activa la captura de Codex: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -Esto instala el recolector, lo registra como servicio en segundo plano y comienza la captura. Confirma que está en ejecución: - -```bash -agenteye-collector health -``` - -En el primer arranque, las sesiones de Codex existentes se importan de una sola vez y la actividad nueva fluye en cuestión de segundos. Los archivos propios de Codex solo se leen — nunca se modifican, mueven ni eliminan — y cada sesión se envía exactamente una vez, incluso tras reinicios. - ---- - -## Dónde aparece - -Las sesiones capturadas aparecen en **Sessions**, y sus eventos en el flujo de **Events**, igual que cualquier otro agente que observes — así que la [reproducción de sesiones](/es/agenteye/sessions), la [búsqueda](/es/agenteye/queries), las [evaluaciones](/es/agenteye/evaluations) y las [alertas](/es/agenteye/alerts) funcionan con ellas. Filtra por el agente de Codex para verlas por separado. - ---- - -## Privacidad - -Las transcripciones de Codex contienen la sesión completa — incluyendo la salida de comandos, el contenido de archivos y todo lo que Codex leyó o escribió — y pueden contener secretos. Las sesiones capturadas se envían tal cual, así que activa la captura únicamente en las máquinas y para los equipos en los que centralizar ese contenido en AgentEye sea apropiado, y proporciona al recolector una clave con alcance exclusivo a `events:add`. Consulta [Seguridad](/es/agenteye/security) para saber cómo se mantienen aislados tus datos. \ No newline at end of file diff --git a/docs/es/agenteye/concepts.mdx b/docs/es/agenteye/concepts.mdx deleted file mode 100644 index 5a7b57e2..00000000 --- a/docs/es/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "Conceptos" -description: "El vocabulario de Failproof AI Observability — eventos, sesiones, evaluaciones, auditorías, hallazgos e incidentes — definido en un solo lugar." ---- - - -Esta página define el vocabulario que utiliza Failproof AI Observability. Si algún término en otra guía te resulta desconocido, aquí encontrarás su definición. No es necesario leerla de principio a fin: puedes hojearla o volver cuando encuentres una palabra que quieras aclarar. - ---- - -## El modelo de datos - -**Evento** -La unidad mínima de datos. Un evento registra un único paso que realizó tu agente: un `tool_use`, un `model_request`, un `hook_completed`, un `error`, entre otros. Tu agente emite eventos a través del [Python SDK](/es/agenteye/python-sdk); aparecen en tiempo real en la página de **Events**. - -**Sesión** -Una ejecución del agente, identificada por un `session_id`. Una sesión agrupa todos los eventos que comparten ese identificador, se muestra como una fila en la página de **Sessions** y se representa como un grafo de ejecución en su página de detalle. Por lo general, una sesión comienza con `agent_start` y termina con `agent_end`. - -**Agente** -Un actor con nombre dentro de una ejecución, identificado por un `agent_id`. Una ejecución puede involucrar varios agentes: por ejemplo, un planificador que genera un sub-agente de resumen. Los sub-agentes llevan un `parent_id`, que es lo que permite a Failproof AI Observability representarlos en sus propios carriles dentro del grafo de ejecución. - -**Entorno** -Una etiqueta que indica dónde ocurrió la ejecución: `production`, `staging`, `dev`. Se configura una sola vez al configurar el SDK. Casi todas las páginas del panel permiten filtrar por entorno. - -**Llenado de la ventana de contexto** -El porcentaje de la ventana de contexto de un modelo que consumió una respuesta. Failproof AI Observability lo registra en los eventos `model_response` para los modelos que reconoce, de modo que el crecimiento del prompt y la compactación inminente sean visibles directamente en el flujo de eventos. - ---- - -## Calidad - -**Evaluación** -Una puntuación de calidad para una sesión finalizada, generada por un servicio de puntuación que tú ejecutas. Las evaluaciones son opcionales: hasta que conectes un evaluador, las sesiones se registran pero no se puntúan. Cada evaluación puede incluir varias puntuaciones con nombre (por ejemplo, `helpfulness`, `factuality`, `tool_efficiency`), cada una con una breve nota de razonamiento. Consulta [Evaluation suite](/es/agenteye/evaluation-suite). - -**Clave de puntuación** -El nombre de una dimensión que reporta un evaluador, como `helpfulness`. Las alertas y auditorías pueden monitorear una clave de puntuación específica a lo largo del tiempo. - -**Evaluador** -Tu servicio de puntuación. Failproof AI Observability envía mediante POST la transcripción de una ejecución finalizada y almacena las puntuaciones que devuelve. No incluye un evaluador predeterminado; la lógica de puntuación es tuya. - ---- - -## Detección y corrección de fallos - -**Hook** -Una barrera de protección o efecto secundario que el framework de tu agente ejecuta alrededor de un paso: una verificación de seguridad de contenido, la eliminación de PII, un control de presupuesto. Los hooks emiten eventos `hook_triggered` / `hook_completed` con un `outcome` (allow, deny, modify), y tienen su propia página de observabilidad. - -**Regla de alerta** -Una regla que se activa cuando una métrica supera un umbral que tú defines: tasa de errores, latencia p95, costo en tokens o una puntuación del evaluador. Cuando se activa una regla, abre un incidente y notifica a los canales que hayas configurado (correo electrónico, Slack, webhook, panel de control). Consulta [Alerts](/es/agenteye/alerts). - -**Incidente** -Un problema abierto que se crea cuando se activa una regla de alerta. Los incidentes tienen un ciclo de vida (reconocer, asignar, resolver) y una línea de tiempo de actividad que registra cada acción. También puedes abrir uno manualmente. - -**Auditoría** -Una investigación recurrente (de cada hora a semanal) que analiza tus registros *entre* sesiones en busca de patrones de fallo para los que aún no has escrito una regla: clústeres de errores, puntuaciones bajas, valores atípicos de latencia, bucles de llamadas a herramientas y ejecuciones que nunca terminaron. Mientras que una alerta monitorea una métrica que ya conoces, una auditoría te indica qué deberías revisar a continuación. Consulta [Audits](/es/agenteye/audits). - -**Hallazgo** -Un resultado priorizado y respaldado por evidencia de una ejecución de auditoría. Un hallazgo identifica un patrón, enlaza con las sesiones exactas que lo respaldan y tiene un ciclo de vida de triaje (reconocer, resolver, silenciar, descartar). Failproof AI Observability deduplica los hallazgos entre ejecuciones, de modo que un patrón conocido se actualiza en lugar de acumularse. - -**El asistente de IA** -El chat integrado en el panel que responde preguntas sobre tus agentes en lenguaje natural, utilizando tus propios datos. Es de solo lectura por defecto; todo lo que crea (una consulta guardada, un panel de control) requiere aprobación, y nunca puede eliminar datos. Consulta [AI assistant](/es/agenteye/assistant). - ---- - -## Ejecución - -**Organización (tenant)** -Un espacio de trabajo aislado. Una instancia de Failproof AI Observability puede albergar muchas organizaciones, cada una con sus propios usuarios, claves y datos. Cada URL del panel está delimitada por el slug de tu organización (`//…`). - -**Recolector** -`agenteye-collector`, el daemon ligero que se ejecuta en cada máquina de agente, agrupa los eventos que el SDK escribe en disco y los envía al servidor. - -**Clave de API** -Un token con permisos acotados que autentica a un cliente frente al servidor. Las claves tienen permisos granulares (por ejemplo, `events:add` para el recolector, permisos de solo lectura para una clave de panel). Consulta [API keys](/es/agenteye/api-keys). - -**Servidor** -El servicio de ingesta y API. Recibe eventos, almacena el estado operativo en tus bases de datos y sirve el panel de control y la CLI. - -**Panel de control** -La interfaz web. Cada página está delimitada a una organización y accede a los datos a través de la API del servidor. - ---- - -## Próximos pasos - -- [Overview](/es/agenteye/overview): cómo encajan todas estas piezas. -- [Observability](/es/agenteye/observability): las superficies de observabilidad (Events, Sessions, Models, Tools, Hooks, Errors). \ No newline at end of file diff --git a/docs/es/agenteye/evaluator-skill.mdx b/docs/es/agenteye/evaluator-skill.mdx deleted file mode 100644 index 8f646072..00000000 --- a/docs/es/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: "Habilidad del Agente Evaluador de Observabilidad de Failproof AI" -description: "Pasa de «creo que nuestro agente a veces falla» a un servicio de puntuación desplegado, con tu agente de programación tomando las decisiones y construyendo la solución." ---- - - -Pasa de *«creo que nuestro agente a veces falla»* a un servicio de puntuación desplegado, con tu agente de programación tomando las decisiones y construyendo la solución. La **habilidad evaluadora de Observabilidad de Failproof AI** (`agenteye-evaluator`) es una *Agent Skill*: una pequeña carpeta de instrucciones que un agente de programación como Claude Code o Codex carga bajo demanda. Le enseña al agente a determinar qué dimensiones de calidad vale la pena rastrear para *tu* agente y luego escribir, probar y desplegar el [servicio evaluador](/es/agenteye/evaluation-suite) que las puntúa. - -**No** es un puntuador alojado, un registro al que subir archivos ni un sistema de plugins. Tu evaluador permanece como tu propio servicio HTTP en tu propia infraestructura, exactamente como se describe en la guía de la [Suite de evaluación](/es/agenteye/evaluation-suite). La habilidad solo enseña a tu agente a construirlo bien, de modo que todo lo que hace, podrías hacerlo tú mismo escribiendo el mismo código. - ---- - -## La parte difícil es decidir qué puntuar - -La superficie del SDK es pequeña — un decorador y dos modelos — y un agente puede escribirla a partir del [contrato](/es/agenteye/evaluation-suite#http-contract) por sí solo. Ahí no es donde fallan los evaluadores. Fallan porque puntúan la cosa equivocada, y un evaluador que puntúa la cosa equivocada es peor que ninguno: produce un dashboard que todos aprenden a ignorar. - -Por eso la mayor parte de la habilidad es la etapa previa a que exista cualquier código. Hace que el agente te entreviste (*«describe una ejecución que salió bien; ahora una que salió mal»*), luego recorre tus sesiones reales a través de la [CLI `agenteye`](/es/agenteye/cli) y las lee de principio a fin. Esas dos mitades suelen no coincidir, y la brecha es precisamente el punto: lo que pretendes medir frente a lo que tus transcripciones pueden respaldar realmente. Una dimensión solo sobrevive si es **computable** a partir de los eventos y **discriminante** — si puntúa 0,9 tanto en tu ejecución buena como en la mala, no enseña nada y se elimina. - -Lo que se devuelve es una propuesta de 2 a 4 dimensiones con el razonamiento adjunto, para que la apruebes antes de que se escriba una sola línea. - -```mermaid -flowchart TD - YOU["tú: 'Quiero evaluaciones para mi bot de soporte'"] --> AGENT["agente de programación (Claude Code / Codex)
carga la habilidad agenteye-evaluator"] - AGENT -->|"entrevista: ¿cómo se ve bueno vs malo?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["tus sesiones reales
lo que realmente ocurre"] - DATA --> DIMS["2-4 dimensiones, tú las apruebas"] - DIMS --> SVC["tu servicio evaluador
SDK agenteye-evaluator"] - SVC --> SCORES["las puntuaciones aparecen en el dashboard
y en agenteye evals"] -``` - ---- - -## Su relación con las demás piezas de evaluación - -Cuatro documentos cubren la puntuación, y se encadenan entre sí en orden: - -| Página | Qué es | Úsala cuando | -|---|---|---| -| **[Evaluaciones](/es/agenteye/evaluations)** | La funcionalidad: puntuaciones en la cuadrícula de sesiones, dashboards, re-evaluación | Quieres saber qué te aporta la puntuación automática | -| **[Suite de evaluación](/es/agenteye/evaluation-suite)** | El contrato HTTP, el SDK, las variables de entorno del servidor | Estás implementando o depurando el evaluador tú mismo | -| **Habilidad evaluadora** (este doc) | Una puerta de entrada en lenguaje natural para diseñar *y* construir el puntuador | Quieres pasar de «quiero evaluaciones» a un servicio en ejecución | -| **[Habilidad CLI](/es/agenteye/cli-skill)** | Una puerta de entrada en lenguaje natural para la CLI `agenteye` | Quieres *leer* las puntuaciones que ya tienes | -| **[Habilidad Python SDK](/es/agenteye/python-sdk-skill)** | Una puerta de entrada en lenguaje natural para instrumentar tu agente | Tu agente aún no emite sesiones — no hay nada que puntuar | - -### vs. la habilidad CLI: construir versus leer - -Las dos habilidades están deliberadamente diseñadas para no solaparse, e instalar ambas es la configuración normal — el agente elige entre ellas según lo que le pidas: - -- **`agenteye-evaluator`** (este doc) construye la cosa que *produce* puntuaciones. Su trabajo termina cuando las puntuaciones aparecen por primera vez. -- **[`agenteye-cli`](/es/agenteye/cli-skill)** lee las puntuaciones que ya existen (`agenteye evals`). *«¿Bajó la calidad esta semana?»* es su pregunta, no la de esta habilidad. - ---- - -## Requisitos previos - -1. **La CLI `agenteye` instalada e iniciada sesión** (`pipx install agenteye`, luego `agenteye login`). La habilidad la utiliza en dos momentos: para obtener las sesiones reales con las que diseña, y para confirmar que tus puntuaciones llegaron al final. Tu sesión necesita `events:read`, más `evaluations:read` para esa verificación final. Al igual que con la habilidad CLI, **no puede** completar el inicio de sesión con código de un solo uso enviado por correo electrónico en tu lugar. -2. **Un lugar donde alojar el evaluador.** Se construye como una imagen y se ejecuta como un servicio de larga duración, por lo que necesita un repositorio real, no un archivo temporal. Los evaluadores suelen vivir en su propio repositorio, separado del agente que se está puntuando — la habilidad busca uno existente y pregunta antes de crear un andamiaje nuevo. -3. **El wheel del SDK `agenteye-evaluator`** — lee la siguiente sección antes de dejar que tu agente empiece a escribir comandos `pip`. - ---- - -## Dónde conseguirla - -La habilidad está publicada en la colección pública de habilidades de Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -El repositorio es público y la habilidad no necesita credenciales propias — solo maneja la CLI `agenteye` con la sesión *tuya* en la que iniciaste sesión, y escribe código en *tu* repositorio. Ten en cuenta que se distribuye como su propia carpeta y **no** está dentro del paquete `pipx install agenteye`, así que no la busques ahí. - -## Instalación de la habilidad - -La forma más rápida es la CLI [`skills`](https://skills.sh), que descarga la carpeta y la coloca donde tu agente la busca: - -```bash -# Claude Code, solo este proyecto -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# todos los proyectos (instala en ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codex en su lugar -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -Luego adminístrala como cualquier otra habilidad: - -```bash -npx skills list -a claude-code # qué está instalado -npx skills update agenteye-evaluator # obtener la última versión -npx skills remove agenteye-evaluator # eliminarla -``` - -¿Prefieres instalar manualmente? Una Agent Skill es simplemente una carpeta que contiene un `SKILL.md` (más referencias opcionales), así que copiarla también funciona: - -- **Claude Code**: coloca la carpeta `agenteye-evaluator/` en `~/.claude/skills/` (todos los proyectos) o en `/.claude/skills/` (solo ese repositorio). Claude Code la descubre automáticamente — verifica con la lista `/skills`, o simplemente pide evaluaciones. -- **Codex (OpenAI)**: Codex lee el mismo `SKILL.md`. El archivo incluido `agents/openai.yaml` establece `allow_implicit_invocation: true`, por lo que Codex selecciona la habilidad automáticamente cuando una tarea coincide; de lo contrario, invócala explícitamente como `$agenteye-evaluator`. - ---- - -## El SDK no está en PyPI público - -> **Advertencia:** Lee esto antes de dejar que un agente instale el SDK. - -La habilidad es pública; el SDK que maneja no lo es. `agenteye-evaluator` se distribuye únicamente como un artefacto de lanzamiento privado, y a diferencia de `agenteye`, el nombre está **sin reclamar en PyPI público** — así que un `pip install agenteye-evaluator` sin más podría descargar el paquete de un desconocido en el servicio que lee tus transcripciones de producción. Eso es un problema de cadena de suministro, no un error tipográfico. - -La habilidad lo sabe y recorre una escalera de instalación en su lugar, deteniéndose en el primer peldaño que aplica: el código fuente del monorepo si estás dentro del repositorio de AgentEye, de lo contrario el wheel de lanzamiento privado desde GitHub Releases (requiere acceso), y si ninguno es accesible **se detiene y te dice que pidas el wheel a tu contacto de Failproof AI** en lugar de improvisar. - -Por tanto, si tu agente propone un `pip install agenteye-evaluator` simple desde PyPI público, esa es la señal de que la habilidad nunca se cargó. Detente ahí y comprueba que está instalada. - ---- - -## Qué puedes pedirle - -Un ciclo completo real empieza con una petición vaga y termina con un diseño aprobado, no con código: - -```text -tú ▸ Quiero evaluaciones para nuestro bot de soporte. Creo que a veces falla. - -agente▸ Antes de escribir nada: describe una ejecución que salió bien, - y una que salió mal. - -tú ▸ Bien: responde a partir de la documentación. Mal: se inventa cosas, - o se rinde y dice "contacta con soporte". - -agente▸ Ejecutando: agenteye --json sessions --status error --since 7d - Leyendo 5 sesiones, incluyendo dos que considerarías malas. - - Tus palabras dicen veracidad y evasión. Las transcripciones - coinciden en evasión (2 de 5 terminan con una derivación a soporte, - y es evidente en model_response.content), pero la veracidad - necesita una verdad de base que tus eventos no contienen. - - Así que puntuaría tres cosas: - - deflection reglas, barato, separa tus dos ejecuciones - - tool_efficiency reglas, detecta el bucle de reintentos en run-004 - - groundedness juez LLM: ¿cada afirmación está en los docs que recuperó? - - "factuality" necesitaría una verdad de base que no tenemos; groundedness - es lo más cercano que tus transcripciones pueden respaldar realmente. - ¿De acuerdo con esos tres nombres? Renombrar después divide el historial. - -tú ▸ sí - -agente▸ No hay evaluador en este repositorio. ¿Debo crear el andamiaje aquí, o - tienes uno en otro lugar? -``` - -A partir de ahí escribe primero las dimensiones basadas en reglas (gratuitas, instantáneas, deterministas), las prueba contra una sesión capturada real incluyendo las vacías y las que nunca se completaron y que hacen fallar a los evaluadores ingenuos, y solo recurre a un juez LLM para la dimensión subjetiva. Conoce los [límites del dispatcher](/es/agenteye/evaluation-suite#configuring-the-server) — un tiempo de espera de solicitud de 30s y 8 llamadas concurrentes en todo el despliegue — así que si el juez no cabe de forma fiable, va asíncrono con `JobPending` en lugar de dejar que tu juez sea cancelado y reintentado cinco veces a cinco veces el coste. - -Luego lo despliega, configura las dos variables de entorno del servidor y confirma con `agenteye --json evals --session-id ` que las puntuaciones realmente llegaron. Que lleguen las puntuaciones es la única prueba. - ---- - -## Qué tener en cuenta - -- **Los nombres de las dimensiones son casi permanentes.** Las claves de puntuación son cadenas arbitrarias y la plataforma traza tendencias de lo que envíes, lo que significa que nada en el downstream corrige una mala elección. Renombrar después divide el historial: las sesiones antiguas conservan la clave antigua y la tendencia se rompe. Por eso la habilidad obtiene una aprobación explícita antes de escribir código — tómate ese aviso en serio. -- **Los fixtures son transcripciones reales de producción.** Diseñar contra sesiones reales implica descargarlas al disco, y pueden contener datos de clientes. La habilidad pregunta antes de agregarlos a git; en caso de duda, mantén `fixtures/` fuera del repositorio y pide a cada desarrollador que descargue las suyas propias. -- **El agente escribe y despliega un servicio que lee cada transcripción.** Actúa como tú, acotado por los permisos de tu sesión de CLI, pero revisa el evaluador como cualquier otro código que toque datos de producción. - ---- - -## Próximos pasos - -- **[Suite de evaluación](/es/agenteye/evaluation-suite)**: el contrato HTTP, el SDK y las variables de entorno del servidor que configura la habilidad. -- **[Evaluaciones](/es/agenteye/evaluations)**: dónde aparecen las puntuaciones una vez que llegan. -- **[Habilidad CLI](/es/agenteye/cli-skill)**: la habilidad hermana, para leer resultados en lugar de construir el puntuador. -- **[CLI](/es/agenteye/cli)**: la referencia de comandos detrás de los datos de sesión con los que la habilidad diseña. \ No newline at end of file diff --git a/docs/es/agenteye/hermes-capture.mdx b/docs/es/agenteye/hermes-capture.mdx deleted file mode 100644 index 8975feab..00000000 --- a/docs/es/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Captura de sesiones de Hermes" -description: "Incorpora las sesiones del gateway Hermes de tu equipo — Slack, Telegram, CLI y ejecuciones programadas — en AgentEye como sesiones y eventos ordinarios." ---- - -[Hermes](https://hermes-agent.nousresearch.com) responde a tu equipo desde donde ya trabajan — Slack, Telegram, la CLI, ejecuciones programadas. La captura de sesiones de Hermes lleva todo eso a AgentEye como sesiones y eventos ordinarios, de modo que el asistente con el que tu equipo habla cada día sea tan observable como los agentes que tú mismo escribes. - -Un pequeño recolector en segundo plano lee el almacén de sesiones local de Hermes a medida que se va escribiendo y envía las sesiones a AgentEye. Funciona igual que la captura de [Codex](/es/agenteye/codex-capture) y [OpenClaw](/es/agenteye/openclaw-capture), y un único recolector puede capturar varios al mismo tiempo. - ---- - -## Qué captura - -Se captura cada sesión de Hermes en la máquina, independientemente del canal por el que llegó. Cada una se convierte en una [sesión](/es/agenteye/sessions) de AgentEye; sus mensajes de usuario y asistente, llamadas a herramientas y resultados de herramientas se convierten en los [eventos](/es/agenteye/event-stream) correspondientes. - -El canal desde el que se inició una sesión — Slack, Telegram, CLI o una ejecución programada — queda registrado en la sesión, de modo que puedes distinguirlas y filtrar por una a la vez. Junto a esto se almacenan el modelo sobre el que se ejecutó la sesión, el chat y la persona desde la que se inició, y, cuando una sesión dio lugar a otra, el enlace de vuelta a su sesión padre. - -Las sesiones aparecen en cuanto Hermes las inicia, independientemente de si ya se ha dicho algo, y la respuesta de un turno y sus llamadas a herramientas se mantienen en el orden en que realmente ocurrieron. Cuando una sesión finaliza, también obtienes el motivo del cierre, su coste y cuántos tokens utilizó. - ---- - -## Cómo activarlo - -La captura está desactivada hasta que la habilites. Instala el recolector con una clave de API que tenga el permiso `events:add` (consulta [API keys](/es/agenteye/api-keys)) y activa la captura de Hermes: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -Esto instala el recolector, lo registra como servicio en segundo plano y comienza la captura. Confirma que está en ejecución: - -```bash -agenteye-collector health -``` - -¿Capturas más de un agente en la misma máquina? Añade el indicador de cada uno al mismo comando — por ejemplo `--hermes-enabled --codex-enabled`. - -En la primera ejecución, tus sesiones de Hermes existentes se importan retroactivamente una vez y la nueva actividad se transmite en segundos. Los datos propios de Hermes solo se leen — nunca se modifican ni eliminan — y cada mensaje se envía una sola vez, incluso tras reinicios. - -`health` también te indica si todo lo que capturó el recolector llegó realmente a AgentEye. Si un lote no pudo entregarse, se conserva y se reintenta en lugar de descartarse, y la comprobación reporta estado no saludable mientras haya algo pendiente — así que "saludable" significa que tus datos han llegado, no simplemente que el proceso está activo. - ---- - -## Dónde aparece - -Las sesiones capturadas aparecen en **Sessions**, y sus eventos en el flujo **Events**, igual que cualquier otro agente que observes — de modo que la [reproducción de sesiones](/es/agenteye/sessions), la [búsqueda](/es/agenteye/queries), las [evaluaciones](/es/agenteye/evaluations) y las [alertas](/es/agenteye/alerts) funcionan sobre ellas. Filtra por el agente Hermes para verlas por separado. - ---- - -## Privacidad - -Las sesiones de Hermes contienen la transcripción completa — incluyendo la salida de comandos, el contenido de archivos y todo lo que el agente leyó o escribió — y pueden contener secretos. Las sesiones capturadas se envían tal cual, así que activa la captura solo donde sea apropiado centralizar ese contenido en AgentEye, y proporciona al recolector una clave con alcance limitado a `events:add`. Consulta [Security](/es/agenteye/security) para saber cómo se mantienen aislados tus datos. \ No newline at end of file diff --git a/docs/es/agenteye/observability.mdx b/docs/es/agenteye/observability.mdx deleted file mode 100644 index 9e9b7552..00000000 --- a/docs/es/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Observar" -description: "Las superficies de observación son donde vigilas lo que hacen tus agentes en tiempo real y profundizas en cualquier ejecución individual." ---- - - -Las superficies de observación son donde vigilas lo que hacen tus agentes en tiempo real y profundizas en cualquier ejecución individual. Todo aquí es en vivo, limitado a tu organización y filtrable por rango de fechas, entorno, agente y sesión, para que pases de "algo no cuadra" a la ejecución exacta en segundos. - -![El flujo de eventos en vivo, con código de colores por tipo y filtrable por entorno, agente y sesión](/agenteye/images/events-stream.png) - -Cuatro superficies, cada una con su propia página: - -- **[Flujo de eventos](/es/agenteye/event-stream)**: el rastro en vivo, paso a paso, de cada ejecución de todos los agentes, del más reciente al más antiguo. El inicio de tu organización y el primer punto de triaje. -- **[Sesiones y grafo de ejecución](/es/agenteye/sessions)**: esos eventos agrupados en una fila por ejecución, más una imagen estilo git de cómo se desarrolló cada ejecución. -- **[Métricas de rendimiento](/es/agenteye/telemetry)**: mapas de calor de latencia y métricas p50/p95/p99 para tus modelos, herramientas y hooks, para que un pico en la cola destaque frente a la mediana. -- **[Seguimiento de errores](/es/agenteye/error-tracking)**: una superficie de triaje unificada para todo lo que salió mal, a un clic de una alerta activa a la ejecución que falló. - -## Relacionado - -- [Evaluaciones](/es/agenteye/evaluations): puntúa cada ejecución en términos de calidad. -- [Alertas](/es/agenteye/alerts): convierte cualquier umbral en una regla de notificación. -- [Auditorías](/es/agenteye/audits): deja que Failproof AI Observability encuentre patrones de fallo en las sesiones por ti. -- [CLI y agentes](/es/agenteye/cli-and-agents): la misma observabilidad desde tu terminal. \ No newline at end of file diff --git a/docs/es/agenteye/openclaw-capture.mdx b/docs/es/agenteye/openclaw-capture.mdx deleted file mode 100644 index 9ac113a4..00000000 --- a/docs/es/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "Captura de sesiones de OpenClaw" -description: "Transmite las sesiones locales de OpenClaw de tu equipo a AgentEye como sesiones y eventos ordinarios, sin cambiar la forma en que OpenClaw se ejecuta." ---- - -Si tu equipo usa [OpenClaw](https://docs.openclaw.ai), la captura de sesiones de OpenClaw incorpora esas sesiones en AgentEye como sesiones y eventos ordinarios, para que puedas buscarlas, reproducirlas y evaluarlas junto al resto de lo que observas. Complementa el [SDK de Python](/es/agenteye/python-sdk): el SDK instrumenta los agentes que tú escribes, mientras que esto captura el trabajo de OpenClaw que tu equipo ya realiza, sin ningún cambio en cómo lo ejecuta. - -Un pequeño recolector en segundo plano lee los transcritos de sesión locales de OpenClaw conforme se van escribiendo y los envía a AgentEye. Funciona de la misma manera que la [captura de Codex](/es/agenteye/codex-capture), y un único recolector puede capturar ambos al mismo tiempo. - ---- - -## Qué captura - -Cada agente configurado en la instalación de OpenClaw de una máquina es capturado por el recolector de esa máquina; no se requiere configuración por agente. - -Cada sesión de OpenClaw se convierte en una [sesión](/es/agenteye/sessions) de AgentEye; sus mensajes de usuario y asistente, llamadas a herramientas y resultados de herramientas se convierten en los [eventos](/es/agenteye/event-stream) correspondientes. - ---- - -## Cómo activarlo - -La captura está desactivada hasta que la habilites. Instala el recolector con una clave de API que tenga el permiso `events:add` (consulta [Claves de API](/es/agenteye/api-keys)) y activa la captura de OpenClaw: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -Esto instala el recolector, lo registra como un servicio en segundo plano y comienza a capturar. Para confirmar que está en ejecución: - -```bash -agenteye-collector health -``` - -¿Capturas más de un agente en la misma máquina? Añade el indicador de cada uno al mismo comando; por ejemplo, `--openclaw-enabled --codex-enabled`. - -En la primera ejecución, tus sesiones de OpenClaw existentes se importan de forma retroactiva una sola vez, y la nueva actividad se transmite en cuestión de segundos. Los archivos propios de OpenClaw solo se leen; nunca se modifican, mueven ni eliminan, y cada sesión se envía exactamente una vez, incluso tras reinicios. - ---- - -## Dónde aparece - -Las sesiones capturadas aparecen en **Sessions**, y sus eventos en el flujo de **Events**, igual que cualquier otro agente que observes; por lo tanto, la [reproducción de sesiones](/es/agenteye/sessions), la [búsqueda](/es/agenteye/queries), las [evaluaciones](/es/agenteye/evaluations) y las [alertas](/es/agenteye/alerts) funcionan con ellas. Filtra por el agente de OpenClaw para verlas de forma independiente. - ---- - -## Privacidad - -Los transcritos de OpenClaw contienen la sesión completa, incluida la salida de comandos, el contenido de archivos y todo lo que el agente leyó o escribió, y pueden contener secretos. Las sesiones capturadas se envían tal cual, así que activa la captura únicamente en máquinas y para equipos donde centralizar ese contenido en AgentEye sea apropiado, y proporciona al recolector una clave con alcance limitado a `events:add`. Consulta [Seguridad](/es/agenteye/security) para conocer cómo se mantienen tus datos aislados. \ No newline at end of file diff --git a/docs/es/agenteye/python-sdk-skill.mdx b/docs/es/agenteye/python-sdk-skill.mdx deleted file mode 100644 index 4254c8b6..00000000 --- a/docs/es/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,135 +0,0 @@ ---- -title: "Skill del Agente Python SDK de Observabilidad de Failproof AI" -description: "Pasa de un agente sin instrumentación a eventos que puedes ver, con tu agente de código encontrando los puntos de instrumentación, escribiéndolos y verificando que funcionan." ---- - -Dile a tu agente de código *"agrega Observabilidad de Failproof AI a este agente"* y deja que lea tu bucle, determine dónde corresponde la instrumentación, la escriba y verifique los eventos antes de dar el trabajo por terminado. - -El **skill de Python SDK** (`agenteye-python-sdk`) es un *Agent Skill*: una carpeta de instrucciones que un agente de código como Claude Code o Codex carga a demanda cuando una tarea coincide con él. Le enseña al agente a usar el [Python SDK](/es/agenteye/python-sdk) — no es una librería y no cambia nada sobre cómo funciona el SDK. - -## La instrumentación es fácil de escribir y fácil de hacer mal sin notarlo - -El SDK es pequeño: trece métodos de eventos, todos con argumentos nombrados. Un agente de código puede leer la referencia del [Python SDK](/es/agenteye/python-sdk) y producir instrumentación plausible en un minuto. - -El problema es que este SDK no lanza errores cuando algo está mal, y la instrumentación incorrecta se ve exactamente igual a la correcta hasta que alguien abre un dashboard y lo encuentra vacío. Los errores que cuestan tiempo real son todos silencios: - -| El error | Lo que ves | -|---|---| -| Sin `agent_start` | Todos los eventos llegan. Cero sesiones. | -| Entorno nunca configurado | Todo funciona, archivado bajo `dev`. | -| `outcome="failure"` | La ejecución muestra verde — solo `failed`, `error`, `timeout`, `rejected` cuentan. | -| Un nombre de campo mal escrito | Aceptado y almacenado como un nuevo campo. | -| Eventos emitidos desde un thread pool | Descartados silenciosamente. | - -Ninguno lanza errores. Ninguno aparece en las pruebas. Todos están en el skill, declarados como un contrato con la verificación que los detecta. - -## Lo que hace, en orden - -El skill ejecuta los mismos tres pasos que haría un ingeniero cuidadoso: - -1. **Planificar.** Lee tu bucle de agente y hace las dos preguntas que solo tú puedes responder: qué cuenta como una ejecución (tu `session_id`), y quiénes son los actores distinguibles (tu `agent_id`). Las acuerda antes de escribir código, porque cambiarlas después divide tu historial y rompe las tendencias. -2. **Escribir.** Vincula la identidad una vez por ejecución en lugar de pasarla por cada punto de llamada, y elige una forma segura para concurrencia — un detalle importante, porque el atajo obvio mezcla silenciosamente dos ejecuciones superpuestas en una sola sesión. -3. **Verificar.** Ejecuta tu agente y lee los archivos de eventos resultantes, comprobando que `agent_start` está presente, que el entorno es correcto y que una ejecución produjo una sesión. - -Ese tercer paso es el que la gente omite. El SDK escribe eventos en archivos locales, por lo que una integración completa puede probarse en una laptop sin servidor, sin clave de API y sin red — que es exactamente por qué el skill insiste en hacerlo. - -## Cómo se relaciona con los otros skills - -Tres skills, una división clara: - -| Skill | Úsalo cuando | Qué modifica | -|---|---|---| -| **Skill de Python SDK** (esta página) | Quieres que tu agente *emita* telemetría — "agrega observabilidad", "¿por qué no aparece mi agente?" | Escribe código en el repositorio de tu agente. No lee nada. | -| **[Skill Evaluator](/es/agenteye/evaluator-skill)** | Quieres *puntuar* ejecuciones — "¿qué deberíamos medir?" | Escribe código en tu repositorio; lee telemetría | -| **[Skill CLI](/es/agenteye/cli-skill)** | Quieres *leer* lo que ocurrió, u operar tu despliegue | Maneja la CLI en tu nombre, incluyendo cambios | - -Se encadenan en ese orden: este skill hace que los eventos fluyan, el evaluador los puntúa, la CLI los lee. No hay nada que evaluar ni nada que leer hasta que tu agente emita sesiones, así que si estás empezando desde cero, comienza aquí. - -## Requisitos previos - -1. **Python 3.10+** y el código base del agente que quieres instrumentar. -2. **El SDK.** Se distribuye a los clientes como un wheel privado en lugar de desde un índice público — tu proceso de incorporación explica cómo obtenerlo e instalarlo. El skill conoce la ruta de instalación y te preguntará en lugar de adivinar si no puede encontrarla. -3. **Nada más.** Sin inicio de sesión en el dashboard, sin clave de API, sin red. El skill verifica contra los archivos de eventos que escribe el SDK, por lo que puede terminar y demostrar su trabajo sin conexión. - -## Dónde obtenerlo - -El skill vive en la colección pública [`FailproofAI/skills`](https://github.com/FailproofAI/skills): - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -Agrega `-g` para instalarlo en todos los proyectos en lugar de solo en el actual, y `--copy` si tu entorno no sigue enlaces simbólicos. Para Codex, pasa `-a codex`. - -## Instalarlo manualmente - -Los Agent Skills son carpetas que contienen un `SKILL.md` más referencias. Si prefieres no usar el instalador: - -- **Claude Code**: copia la carpeta `agenteye-python-sdk/` en `~/.claude/skills/` (todos los proyectos) o `/.claude/skills/` (solo ese repositorio). Claude Code la descubre automáticamente — revisa la lista `/skills`, o simplemente pregunta algo que coincida con ella. -- **Codex**: Codex lee el mismo `SKILL.md`. El archivo `agents/openai.yaml` incluido establece `allow_implicit_invocation: true`, por lo que se selecciona automáticamente cuando una tarea coincide; de lo contrario, invócalo como `$agenteye-python-sdk`. - -Ejecuta tu agente **en el repositorio que contiene el código que quieres instrumentar** — el skill lee tu bucle de agente antes de proponer cualquier cosa. - -## Cómo luce una sesión - -```text -tú ▸ Agrega Observabilidad de Failproof AI a este agente. - -agente▸ Leo tu bucle — un despachador de herramientas en tools.py, un - wrapper de Anthropic en llm.py. Dos preguntas antes de escribir - cualquier cosa. - - ¿Qué cuenta como una ejecución? Manejas solicitudes de forma - concurrente, así que usaría el id de solicitud que ya tienes en - ctx.request_id — así una sesión aquí y una solicitud en tus - logs son el mismo string. - - ¿Y quiénes son los actores? Veo un camino de planificador y un - camino de investigador. ¿Los etiqueto por separado, o para ti - es un solo agente? - -tú ▸ El id de solicitud está bien. Son un solo agente, llámalo - "assistant". - -agente▸ Listo. Envolví el despachador y el cliente LLM; agent_start y - agent_end delimitan handle_request, incluido el camino de error. - Usé un context-local para la identidad — tu despachador ejecuta - herramientas en un thread pool, y el enfoque más simple habría - mezclado ejecuciones concurrentes en una sola sesión sin avisar. - - Verificado contra una ejecución real: 11 eventos en 4 archivos — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, una sesión - por solicitud, duraciones emparejadas. - - Una cosa que NO hice: tu pool de workers en batch.py envía - directamente al executor, así que los eventos de ahí se - descartarían. ¿Quieres que los corrija también? -``` - -El patrón a notar: leyó el código antes de proponer, hizo solo las preguntas que tú puedes responder, reutilizó un id que ya tenías, eligió la forma segura para concurrencia *porque* vio un thread pool, y **verificó leyendo los eventos reales** en lugar de declarar éxito — luego señaló el único lugar que sabía que fallaría silenciosamente. - -## Qué puedes pedirle - -- *"¿Por qué no aparece mi agente en el dashboard?"* → recorre la escalera: si los eventos se están escribiendo, si `agent_start` está ahí, si el entorno es correcto, si el colector está leyendo el mismo lugar. -- *"Todo está llegando bajo dev."* → el entorno nunca se configuró, o fue restablecido por una llamada posterior. -- *"Agrega seguimiento de tokens."* → encuentra tu wrapper LLM y registra modelo, razón de parada y uso. -- *"Instrumenta los sub-agentes también."* → una sesión, etiquetas de agente distintas, anidadas bajo su padre. -- *"Escribe pruebas para la instrumentación."* → apunta el SDK a un directorio temporal y hace aserciones sobre los eventos que escribió. - -## Qué tener en cuenta - -**Deja que verifique.** El paso que hace que valga la pena usar este skill es el último — ejecutar tu agente y leer los eventos de vuelta. Un agente que escribe instrumentación y se detiene ha hecho la mitad fácil, y la mitad que falla silenciosamente es la otra. - -**Acuerda los nombres antes del código.** `session_id` y `agent_id` son los ejes por los que agrupa cada vista. Renombrarlos después divide el historial: las ejecuciones antiguas conservan las etiquetas anteriores y tus tendencias se rompen. El skill preguntará; la respuesta vale un minuto de reflexión. - -**Si tu agente propone instalar el SDK desde un índice público, el skill no se cargó.** El SDK se distribuye de forma privada. Esa propuesta es una señal clara de que tu agente de código está adivinando en lugar de seguir el skill — detenlo ahí y verifica que el skill esté instalado. - -Más allá de eso, su radio de acción es pequeño: escribe código en tu directorio de trabajo y archivos de eventos donde tú le indiques. No lee nada de tu despliegue ni cambia nada en él. - -## Próximos pasos - -- **[Python SDK](/es/agenteye/python-sdk)**: la referencia completa de eventos — cada tipo de evento y campo — detrás de lo que automatiza este skill. -- **[Sessions](/es/agenteye/sessions)**: lo que produce tu instrumentación una vez que los eventos llegan. -- **[Evaluator Agent Skill](/es/agenteye/evaluator-skill)**: el siguiente paso una vez que las ejecuciones están llegando — puntuarlas. -- **[CLI Agent Skill](/es/agenteye/cli-skill)**: leer tu telemetría de vuelta. \ No newline at end of file diff --git a/docs/es/agenteye/security.mdx b/docs/es/agenteye/security.mdx deleted file mode 100644 index 7bcada09..00000000 --- a/docs/es/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "Seguridad" -description: "Failproof AI Observability está diseñado para situarse cerca de tus agentes en producción, lo que significa que tiene acceso a tus prompts, entradas de herramientas y salidas." ---- - - -Failproof AI Observability está diseñado para situarse cerca de tus agentes en producción, lo que significa que tiene acceso a tus prompts, entradas de herramientas y salidas. Esta página explica cómo mantiene esos datos aislados, bajo control y en tus manos. Si estás evaluando Failproof AI Observability para una revisión de seguridad, comienza aquí. - ---- - -## Tus datos permanecen en tu entorno - -Failproof AI Observability es autoalojado. Los eventos, prompts, respuestas del modelo y las analíticas se almacenan en tus propias bases de datos, en tu propio entorno. Nada se envía a un SaaS de terceros para su almacenamiento, y tus datos permanecen en tu propia cuenta en la nube. - ---- - -## Aislamiento de inquilinos - -Una sola instancia de Failproof AI Observability puede alojar muchas organizaciones, y cada una está aislada a nivel de la capa de almacenamiento — aplicado por la base de datos, no solo por la interfaz de usuario: - -- Los datos operativos de una organización (usuarios, claves, paneles, consultas guardadas) están delimitados a esa organización, y las lecturas entre organizaciones están bloqueadas por la propia base de datos. -- Cada evento ingestado lleva el sello de la organización propietaria, por lo que los eventos de una organización nunca pueden ser leídos por otra. - -Cada ruta del panel está delimitada bajo un slug de organización (`//…`). - ---- - -## Inicio de sesión - -Failproof AI Observability utiliza inicio de sesión sin contraseña, basado en correo electrónico. No hay contraseña que pueda ser objeto de phishing o filtrarse. Un usuario solicita un código de un solo uso (o un enlace mágico de un clic), que se envía por correo electrónico y expira rápidamente. El inicio de sesión está controlado por una **lista de permitidos**: solo las direcciones de correo electrónico (o dominios) que tú autorices pueden autenticarse. - -![La pantalla de inicio de sesión de Failproof AI Observability, que envía un código de uso único a tu correo electrónico](/agenteye/images/login.png) - ---- - -## Acceso delimitado con claves de API - -Cada cliente se autentica con una clave de API que lleva permisos granulares de mínimo privilegio. Un recopilador solo necesita `events:add`; una clave de panel o asistente puede ser de solo lectura; las acciones destructivas (eliminar, regenerar) son permisos separados que tú decides incluir. - -![La página de claves de API: los permisos de cada clave, codificados por color según el alcance de lectura, escritura y destructivo](/agenteye/images/api-keys.png) - -Conserva la clave de arranque de administrador para la configuración, y emite claves con permisos reducidos para todo lo demás. Consulta [Claves de API](/es/agenteye/api-keys). - ---- - -## Un asistente de solo lectura con aprobación previa - -El [asistente de IA](/es/agenteye/assistant) del panel responde preguntas sobre tus datos, pero está restringido por diseño: - -- Es **de solo lectura por defecto**: su SQL se ejecuta a través de un guardián que solo permite consultas `SELECT`/`WITH`, de una sola instrucción, con un límite de filas. -- Todo lo que crea (una consulta guardada, un panel) requiere **aprobación previa**: tú revisas y apruebas cada escritura antes de que ocurra. -- **Nunca puede eliminar**. - -Así, un compañero de equipo puede preguntar "¿qué agentes tuvieron más errores esta semana?" y actuar sobre la respuesta, sin que el asistente pueda modificar o eliminar tus datos por su cuenta. - ---- - -## En tránsito - -Todo el tráfico circula a través de HTTPS. Tú terminas el TLS con tus propios certificados, por lo que el tráfico entre el recopilador y el servidor, y entre el navegador y el servidor, está cifrado en tránsito. - ---- - -## Próximos pasos - -- [Descripción general](/es/agenteye/overview): cómo encaja Failproof AI Observability en conjunto. -- [Claves de API](/es/agenteye/api-keys): delimita el acceso para el recopilador, el panel y el asistente. -- [Observabilidad](/es/agenteye/observability): qué captura Failproof AI Observability de tus agentes. \ No newline at end of file diff --git a/docs/es/cli/audit.mdx b/docs/es/audit.mdx similarity index 100% rename from docs/es/cli/audit.mdx rename to docs/es/audit.mdx diff --git a/docs/es/cli/backfill.mdx b/docs/es/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/es/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/es/cli/config.mdx b/docs/es/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/es/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/es/cli/flush.mdx b/docs/es/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/es/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/es/cli/harness.mdx b/docs/es/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/es/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/es/cli/migrate.mdx b/docs/es/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/es/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/es/cli/uninstall.mdx b/docs/es/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/es/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/es/cli/update.mdx b/docs/es/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/es/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/es/agenteye/api-keys.mdx b/docs/es/cloud/access.mdx similarity index 98% rename from docs/es/agenteye/api-keys.mdx rename to docs/es/cloud/access.mdx index ce8b5c45..19c319d2 100644 --- a/docs/es/agenteye/api-keys.mdx +++ b/docs/es/cloud/access.mdx @@ -82,7 +82,7 @@ El servidor aplica un catálogo fijo de permisos; cada uno protege rutas HTTP es Estos permisos respaldan la página **Users** del dashboard, donde los ámbitos otorgados a cada miembro se muestran como chips: -![La página Users: una tarjeta por usuario del dashboard con su email, permisos otorgados y controles de edición/deshabilitación](/agenteye/images/users.png) +![La página Users: una tarjeta por usuario del dashboard con su email, permisos otorgados y controles de edición/deshabilitación](/cloud/images/users.png) ### Configuración operacional @@ -91,7 +91,7 @@ Estos permisos respaldan la página **Users** del dashboard, donde los ámbitos | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | Ver la configuración operacional gestionada por el dashboard y sus metadatos; listar las anulaciones de ventana de contexto por modelo; y resolver la ventana efectiva para un modelo. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | Editar la configuración operacional y agregar, cambiar o eliminar anulaciones de ventana de contexto por modelo. Los cambios afectan a los nuevos eventos sin necesidad de reiniciar el servidor. | -![La página Settings: configuración operacional gestionada por el dashboard, como los inicios de sesión permitidos y los tiempos de vida de sesión/OTP, editable sin reiniciar](/agenteye/images/settings.png) +![La página Settings: configuración operacional gestionada por el dashboard, como los inicios de sesión permitidos y los tiempos de vida de sesión/OTP, editable sin reiniciar](/cloud/images/settings.png) ### Alertas e incidentes @@ -247,7 +247,7 @@ La página **Keys** del dashboard proporciona una interfaz de usuario para todas Cuando creas una clave desde el dashboard no proporcionas el secreto; el dashboard genera un secreto robusto por ti y lo muestra **una sola vez** al crearlo. Cópialo de inmediato y guárdalo de forma segura; nunca se vuelve a mostrar, exactamente igual que con una regeneración. Puedes seguir seleccionando los permisos de la clave directamente, o inicializarlos desde un conjunto de permisos (ver más abajo). -![La página API Keys: una tarjeta por clave con su nombre, permisos otorgados y fecha de creación, con acciones de regenerar y deshabilitar; las claves protegidas como `admin` están marcadas](/agenteye/images/api-keys.png) +![La página API Keys: una tarjeta por clave con su nombre, permisos otorgados y fecha de creación, con acciones de regenerar y deshabilitar; las claves protegidas como `admin` están marcadas](/cloud/images/api-keys.png) --- @@ -276,5 +276,5 @@ Solo necesitas estas notas si estás actualizando una instancia existente; los n ## Próximos pasos -- [SDK de Python](/es/agenteye/python-sdk): cómo se autentica el código de tu agente al enviar eventos. -- [Seguridad](/es/agenteye/security): cómo funcionan el inicio de sesión, el control de acceso y el aislamiento de datos por organización. \ No newline at end of file +- [SDK de Python](/es/cloud/sdk): cómo se autentica el código de tu agente al enviar eventos. +- [Seguridad](/es/cloud/security): cómo funcionan el inicio de sesión, el control de acceso y el aislamiento de datos por organización. \ No newline at end of file diff --git a/docs/es/cloud/agent-skills.mdx b/docs/es/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/es/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/es/agenteye/alerts.mdx b/docs/es/cloud/alerts.mdx similarity index 76% rename from docs/es/agenteye/alerts.mdx rename to docs/es/cloud/alerts.mdx index 82beeb55..b6cab28d 100644 --- a/docs/es/agenteye/alerts.mdx +++ b/docs/es/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "Entérate en el momento en que algo cruza tu línea, en el canal q --- -Entérate en el momento en que algo cruza tu línea, en el canal que tu equipo ya monitorea, en lugar de que te lo diga un cliente. Define una regla una vez y Failproof AI Observability la evalúa según un calendario, notificándote por email, Slack, webhook o directamente en el dashboard. +Entérate en el momento en que algo cruza tu línea, en el canal que tu equipo ya monitorea, en lugar de que te lo diga un cliente. Define una regla una vez y FailproofAI Cloud la evalúa según un calendario, notificándote por email, Slack, webhook o directamente en el dashboard. -![La página de Alertas: una cuadrícula de tarjetas de reglas de alerta, cada una con su disparador, ventana de evaluación, canales y una insignia de severidad informativa, de advertencia o crítica](/agenteye/images/alerts.png) +![La página de Alertas: una cuadrícula de tarjetas de reglas de alerta, cada una con su disparador, ventana de evaluación, canales y una insignia de severidad informativa, de advertencia o crítica](/cloud/images/alerts.png) *Todas las reglas de alerta de un vistazo: qué monitorea, con qué frecuencia, dónde notifica y qué tan urgente es.* ## Entérate de los problemas antes que tus usuarios @@ -22,9 +22,9 @@ Combina cualquier cantidad de canales en una sola regla, y su severidad (informa ## Crea la regla en un formulario, no en JSON -Describes lo que significa "roto" en un formulario y Failproof AI Observability escribe la regla subyacente por ti. La especificación JSON es simplemente lo que produce ese formulario internamente, así que puedes leerla para entender una regla, pero rara vez necesitarás escribirla. +Describes lo que significa "roto" en un formulario y FailproofAI Cloud escribe la regla subyacente por ti. La especificación JSON es simplemente lo que produce ese formulario internamente, así que puedes leerla para entender una regla, pero rara vez necesitarás escribirla. -![El formulario de nueva alerta: nombre y descripción, un interruptor de activación y un selector de disparador que ofrece umbral de métrica, SQL personalizado, puntuación de evaluación, evaluación compuesta y condiciones por evento](/agenteye/images/alert-new.png) +![El formulario de nueva alerta: nombre y descripción, un interruptor de activación y un selector de disparador que ofrece umbral de métrica, SQL personalizado, puntuación de evaluación, evaluación compuesta y condiciones por evento](/cloud/images/alert-new.png) *Elige un disparador y el formulario muestra los campos correctos; Guardar escribe la regla.* El flujo principal es rápido: nómbrala, elige un **disparador** (qué monitorear), define el **umbral y la ventana** (qué tan grave, durante cuánto tiempo), adjunta al menos un **canal**, luego **Guarda** y haz clic en **Probar** para enviar una notificación sintética y confirmar que todos los destinos están correctamente configurados. Internamente, eso produce una pequeña especificación como esta: @@ -43,7 +43,7 @@ No estás limitado a un solo tipo de señal. Elige el disparador que se adapte a | **Evaluación compuesta** | varias verificaciones de puntuación se combinan con lógica any, all o al-menos-N para detectar una regresión que solo se manifiesta entre múltiples puntuaciones | | **Por evento** | llega un único evento coincidente: un agente específico, un tipo de error específico o una subcadena de mensaje | -¿Ya estás mirando un fallo en la [página de Errores](/es/agenteye/error-tracking)? Cada fila tiene un botón **+ alerta** que abre este mismo formulario prellenado para detectar exactamente ese fallo en el futuro, de modo que el incidente que acabas de atender sea el que te notifique la próxima vez. +¿Ya estás mirando un fallo en la [página de Errores](/es/cloud/errors)? Cada fila tiene un botón **+ alerta** que abre este mismo formulario prellenado para detectar exactamente ese fallo en el futuro, de modo que el incidente que acabas de atender sea el que te notifique la próxima vez. **Dónde encontrarlo:** Las Alertas están en `//alerts`. Para crear, editar, eliminar y probar reglas se necesita **`alerts:write`**; con `alerts:read` es suficiente para consultar. El selector de destinatarios lista a los miembros de tu organización por nombre, así que puedes notificar a una persona sin salir del formulario. @@ -53,11 +53,11 @@ Una medición errónea no debería despertarte. El filtro de ruido **M de N** co ## Qué ocurre cuando se activa una alerta -Una infracción abre un **incidente** y notifica a tus canales una sola vez. A partir de ahí, tu equipo lo reconoce, asigna un responsable, lo discute y lo resuelve, todo con un registro limpio y atribuido. Ese flujo de trabajo de triaje tiene su propio espacio: consulta [Incidentes](/es/agenteye/incidents). +Una infracción abre un **incidente** y notifica a tus canales una sola vez. A partir de ahí, tu equipo lo reconoce, asigna un responsable, lo discute y lo resuelve, todo con un registro limpio y atribuido. Ese flujo de trabajo de triaje tiene su propio espacio: consulta [Incidentes](/es/cloud/incidents). ## Relacionado -- [Incidentes](/es/agenteye/incidents): sigue una alerta activa desde abierta hasta reconocida y resuelta. -- [Seguimiento de errores](/es/agenteye/error-tracking): agrupa fallos de agentes y conviértelos en una alerta con un clic. -- [Dashboards](/es/agenteye/dashboards): monitorea los paneles compartidos de los que provienen los umbrales que alertas. -- [CLI y agentes](/es/agenteye/cli-and-agents): crea alertas y confirma incidentes desde tu terminal, o incorpóralos a scripts de CI. \ No newline at end of file +- [Incidentes](/es/cloud/incidents): sigue una alerta activa desde abierta hasta reconocida y resuelta. +- [Seguimiento de errores](/es/cloud/errors): agrupa fallos de agentes y conviértelos en una alerta con un clic. +- [Dashboards](/es/cloud/dashboards): monitorea los paneles compartidos de los que provienen los umbrales que alertas. +- [CLI y agentes](/es/cloud/cli): crea alertas y confirma incidentes desde tu terminal, o incorpóralos a scripts de CI. \ No newline at end of file diff --git a/docs/es/agenteye/assistant.mdx b/docs/es/cloud/assistant.mdx similarity index 84% rename from docs/es/agenteye/assistant.mdx rename to docs/es/cloud/assistant.mdx index c66f3058..a3b37d10 100644 --- a/docs/es/agenteye/assistant.mdx +++ b/docs/es/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "Haz una pregunta en lenguaje natural sobre los datos de tu agente --- -Haz una pregunta en lenguaje natural sobre los datos de tu agente y obtén una respuesta vinculada directamente a la evidencia. Sin SQL que escribir, sin dashboards que explorar — el asistente de **Failproof AI Observability** es la forma más rápida para que cualquier miembro de tu equipo obtenga respuestas sobre sus agentes. +Haz una pregunta en lenguaje natural sobre los datos de tu agente y obtén una respuesta vinculada directamente a la evidencia. Sin SQL que escribir, sin dashboards que explorar — el asistente de **FailproofAI Cloud** es la forma más rápida para que cualquier miembro de tu equipo obtenga respuestas sobre sus agentes. -![El asistente de Failproof AI Observability respondiendo una pregunta en lenguaje natural dentro del dashboard, mostrando una tabla de actividad de agentes en vivo, un desglose de uso de modelos por agente y conclusiones escritas, con las consultas ejecutadas mostradas inline](/agenteye/images/assistant.png) +![El asistente de FailproofAI Cloud respondiendo una pregunta en lenguaje natural dentro del dashboard, mostrando una tabla de actividad de agentes en vivo, un desglose de uso de modelos por agente y conclusiones escritas, con las consultas ejecutadas mostradas inline](/cloud/images/assistant.png) *Pregunta en lenguaje natural y obtén una respuesta construida a partir de tus propios datos. Aquí desglosa qué agentes están más activos y qué modelos usan, y muestra las consultas que ejecutó para que puedas verificar cada número.* No hay nada que aprender. Abre el chat, escribe lo que quieres saber y sigue los enlaces que te devuelve: @@ -36,7 +36,7 @@ Cuando una respuesta vale la pena conservar, pídele al asistente que la guarde. En la página de **Queries** va un paso más allá y se convierte en autor de SQL: describe la consulta que quieres ("muestra la tasa de errores por agente durante los últimos 7 días") y transmite SQL directamente al editor, abriendo una vista de diferencias para que puedas **Aceptar** o **Rechazar** el cambio antes de que se aplique. -![La página Queries de Observability y su editor SQL](/agenteye/images/query-lab.png) +![La página Queries de FailproofAI Cloud y su editor SQL](/cloud/images/query-lab.png) *La página Queries: este editor es donde el asistente transmite un borrador de consulta de solo lectura para que lo aceptes o rechaces.* Crear SQL mediante preguntas aquí usa el permiso `queries:run`, el mismo que hay detrás del botón **Run** del editor. El chat en cualquier otro lugar necesita `agent:use`. @@ -49,7 +49,7 @@ Puedes abrir el asistente a todos sin preocuparte por lo que podría tocar: - **Cada escritura espera tu confirmación.** Las consultas guardadas y los dashboards solo se crean tras tu clic explícito en Aprobar, y no existe ninguna configuración que desactive esa barrera. - **Nunca puede eliminar nada.** No hay ninguna herramienta de eliminación expuesta y el asistente no tiene permiso de eliminación. Las eliminaciones permanecen en tus manos, en el dashboard. - **Se mantiene dentro de tu organización.** El asistente solo ve la organización que estás visualizando en ese momento. -- **Tus preguntas son tuyas.** Los prompts y las respuestas viven en tu propia base de datos de Observability; los análisis del producto registran solo metadatos de uso, nunca el texto de tus prompts. +- **Tus preguntas son tuyas.** Los prompts y las respuestas viven en tu propia base de datos de FailproofAI Cloud; los análisis del producto registran solo metadatos de uso, nunca el texto de tus prompts. ## Dónde encontrarlo @@ -57,7 +57,7 @@ El asistente aparece en el borde derecho de cada página bajo tu organización ( ## Relacionado -- [CLI y agentes](/es/agenteye/cli-and-agents) -- [Queries](/es/agenteye/queries) -- [Dashboards](/es/agenteye/dashboards) -- [Suite de evaluación](/es/agenteye/evaluation-suite) \ No newline at end of file +- [CLI y agentes](/es/cloud/cli) +- [Queries](/es/cloud/queries) +- [Dashboards](/es/cloud/dashboards) +- [Suite de evaluación](/es/cloud/evaluators) \ No newline at end of file diff --git a/docs/es/agenteye/audits.mdx b/docs/es/cloud/audits.mdx similarity index 82% rename from docs/es/agenteye/audits.mdx rename to docs/es/cloud/audits.mdx index c7072e4f..719845d4 100644 --- a/docs/es/agenteye/audits.mdx +++ b/docs/es/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "Auditorías: tu analista de fiabilidad automático" -description: "Failproof AI Observability busca los fallos para los que nunca escribiste una regla y te entrega una lista priorizada de exactamente qué corregir, respaldada por evidencias." +description: "FailproofAI Cloud busca los fallos para los que nunca escribiste una regla y te entrega una lista priorizada de exactamente qué corregir, respaldada por evidencias." --- -Failproof AI Observability busca los fallos para los que nunca escribiste una regla y te entrega una lista priorizada de exactamente qué corregir, respaldada por evidencias. Es como tener un analista que revisa tus logs cada noche y te deja la lista corta sobre el escritorio antes de que empiece el día. +FailproofAI Cloud busca los fallos para los que nunca escribiste una regla y te entrega una lista priorizada de exactamente qué corregir, respaldada por evidencias. Es como tener un analista que revisa tus logs cada noche y te deja la lista corta sobre el escritorio antes de que empiece el día.
@@ -12,7 +12,7 @@ Failproof AI Observability busca los fallos para los que nunca escribiste una re *Un recorrido de dos minutos: desde una ejecución programada hasta una corrección sobre la que puedes actuar.* -![La página de Auditorías: trabajos recurrentes que analizan tus sesiones en busca de patrones de fallo, cada uno con una programación y sensibilidad](/agenteye/images/audits.png) +![La página de Auditorías: trabajos recurrentes que analizan tus sesiones en busca de patrones de fallo, cada uno con una programación y sensibilidad](/cloud/images/audits.png) *Cada auditoría es un trabajo recurrente que examina tus sesiones y elabora recomendaciones priorizadas respaldadas por evidencias.* ## Deja de adivinar qué corregir a continuación @@ -48,7 +48,7 @@ Las auditorías se encuentran en el panel de control en **`//audits`** ## Relacionado -- [Alertas](/es/agenteye/alerts): recibe una notificación en el momento en que se supera un umbral que ya conoces. -- [Evaluaciones](/es/agenteye/evaluations): puntúa cada ejecución para que las regresiones de calidad se detecten por sí solas. -- [Seguimiento de errores](/es/agenteye/error-tracking): agrupa y sigue los errores que lanzan tus agentes. -- [Incidentes](/es/agenteye/incidents): rastrea un problema que detecta una auditoría hasta su corrección. \ No newline at end of file +- [Alertas](/es/cloud/alerts): recibe una notificación en el momento en que se supera un umbral que ya conoces. +- [Evaluaciones](/es/cloud/evaluations): puntúa cada ejecución para que las regresiones de calidad se detecten por sí solas. +- [Seguimiento de errores](/es/cloud/errors): agrupa y sigue los errores que lanzan tus agentes. +- [Incidentes](/es/cloud/incidents): rastrea un problema que detecta una auditoría hasta su corrección. \ No newline at end of file diff --git a/docs/es/cloud/capture.mdx b/docs/es/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/es/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/es/agenteye/cli-recipes.mdx b/docs/es/cloud/cli-recipes.mdx similarity index 91% rename from docs/es/agenteye/cli-recipes.mdx rename to docs/es/cloud/cli-recipes.mdx index 4f1d8e35..79470942 100644 --- a/docs/es/agenteye/cli-recipes.mdx +++ b/docs/es/cloud/cli-recipes.mdx @@ -3,9 +3,9 @@ title: "Recetas de CLI para agentes" description: "Patrones de consulta y recetas de jq listos para copiar y pegar que convierten datos de sesiones, eventos y evaluaciones en algo que un script o agente de código puede automatizar." --- -Extrae datos de sesiones, eventos y evaluaciones (y dispara reevaluaciones) directamente desde un script o agente de código, con JSON limpio en stdout que se puede redirigir a `jq`. Estas recetas convierten los datos de Failproof AI Observability en algo que un usuario de terminal o un agente de código de IA (Claude Code, Cursor) puede consultar y automatizar, sin necesidad de navegar por el panel. +Extrae datos de sesiones, eventos y evaluaciones (y dispara reevaluaciones) directamente desde un script o agente de código, con JSON limpio en stdout que se puede redirigir a `jq`. Estas recetas convierten los datos de FailproofAI Cloud en algo que un usuario de terminal o un agente de código de IA (Claude Code, Cursor) puede consultar y automatizar, sin necesidad de navegar por el panel. -Los patrones que se muestran a continuación están listos para copiar y pegar en la CLI de Failproof AI Observability (`agenteye`). Para la instalación, autenticación y la lista completa de opciones, consulta [CLI](/es/agenteye/cli); ejecuta `agenteye -h` o `agenteye -h` para ver la ayuda integrada. +Los patrones que se muestran a continuación están listos para copiar y pegar en la CLI de FailproofAI Cloud (`agenteye`). Para la instalación, autenticación y la lista completa de opciones, consulta [CLI](/es/cloud/cli); ejecuta `agenteye -h` o `agenteye -h` para ver la ayuda integrada. ## Reglas de oro @@ -172,7 +172,7 @@ El argumento `--fields` de cada comando acepta exactamente los nombres de campo ## Próximos pasos -- [CLI](/es/agenteye/cli): instalación, autenticación y la referencia completa de opciones para cada comando. -- [Skill de agente CLI](/es/agenteye/cli-skill): empaqueta estas recetas como una skill que tu agente de código pueda cargar. -- [Claves API](/es/agenteye/api-keys): crea y limita el alcance de las claves con las que se autentican la CLI, el SDK y el collector. -- [Python SDK](/es/agenteye/python-sdk): envía eventos a Failproof AI Observability para que haya datos que estas recetas puedan consultar. \ No newline at end of file +- [CLI](/es/cloud/cli): instalación, autenticación y la referencia completa de opciones para cada comando. +- [Skill de agente CLI](/es/cloud/agent-skills): empaqueta estas recetas como una skill que tu agente de código pueda cargar. +- [Claves API](/es/cloud/access): crea y limita el alcance de las claves con las que se autentican la CLI, el SDK y el collector. +- [Python SDK](/es/cloud/sdk): envía eventos a FailproofAI Cloud para que haya datos que estas recetas puedan consultar. \ No newline at end of file diff --git a/docs/es/agenteye/cli.mdx b/docs/es/cloud/cli.mdx similarity index 97% rename from docs/es/agenteye/cli.mdx rename to docs/es/cloud/cli.mdx index b424c7d7..3fca7b1c 100644 --- a/docs/es/agenteye/cli.mdx +++ b/docs/es/cloud/cli.mdx @@ -338,13 +338,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | Autenticado, pero tu cuenta no tiene el permiso requerido (el mensaje lo nombra) | | 6 | El recurso solicitado no se encontró (p. ej., sesión o ID de incidente desconocido) | -Esto hace que el CLI sea seguro para usar en scripts: un agente de código puede ramificar en un `4` para pedirte que te vuelvas a autenticar, o en un `5` para mostrar el permiso faltante. Consulta [recetas de CLI para agentes](/es/agenteye/cli-recipes) para patrones de manejo de códigos de salida y formas de salida JSON. +Esto hace que el CLI sea seguro para usar en scripts: un agente de código puede ramificar en un `4` para pedirte que te vuelvas a autenticar, o en un `5` para mostrar el permiso faltante. Consulta [recetas de CLI para agentes](/es/cloud/cli-recipes) para patrones de manejo de códigos de salida y formas de salida JSON. --- ## Próximos pasos -- **[Recetas de CLI para agentes](/es/agenteye/cli-recipes)**: patrones de consulta listos para copiar, one-liners de `jq`, proyecciones con `--fields`, manejo de códigos de salida y formas de salida JSON, escritos para agentes de código que controlan el CLI. -- **[Habilidad de CLI para agentes](/es/agenteye/cli-skill)**: empaqueta este CLI como una *skill* instalable de Claude Code / Codex para que un agente de código controle la Observabilidad de Failproof AI desde solicitudes en lenguaje natural. -- **[Claves de API](/es/agenteye/api-keys)**: el modelo de permisos detrás de `keys create --add …`. -- **[Asistente de IA](/es/agenteye/assistant)**: cómo habilitar el asistente con el que habla `agent ask`. \ No newline at end of file +- **[Recetas de CLI para agentes](/es/cloud/cli-recipes)**: patrones de consulta listos para copiar, one-liners de `jq`, proyecciones con `--fields`, manejo de códigos de salida y formas de salida JSON, escritos para agentes de código que controlan el CLI. +- **[Habilidad de CLI para agentes](/es/cloud/agent-skills)**: empaqueta este CLI como una *skill* instalable de Claude Code / Codex para que un agente de código controle la Observabilidad de Failproof AI desde solicitudes en lenguaje natural. +- **[Claves de API](/es/cloud/access)**: el modelo de permisos detrás de `keys create --add …`. +- **[Asistente de IA](/es/cloud/assistant)**: cómo habilitar el asistente con el que habla `agent ask`. \ No newline at end of file diff --git a/docs/es/cloud/connect.mdx b/docs/es/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/es/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/es/agenteye/dashboards.mdx b/docs/es/cloud/dashboards.mdx similarity index 69% rename from docs/es/agenteye/dashboards.mdx rename to docs/es/cloud/dashboards.mdx index 5521f920..09ac34df 100644 --- a/docs/es/agenteye/dashboards.mdx +++ b/docs/es/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "Convierte los datos en vivo de tus agentes en una vista compartida Convierte los datos en vivo de tus agentes en una vista compartida que todo tu equipo puede consultar. Fija las consultas más importantes como gráficos, y todos verán los mismos números de un vistazo, sin necesidad de volver a ejecutar ni una sola consulta. -![Un dashboard construido a partir de consultas guardadas: una línea de eventos por hora, una barra de errores por tipo, un gráfico de área de latencia y tokens por modelo](/agenteye/images/dashboard-fleet.png) +![Un dashboard construido a partir de consultas guardadas: una línea de eventos por hora, una barra de errores por tipo, un gráfico de área de latencia y tokens por modelo](/cloud/images/dashboard-fleet.png) *Un tablero, cuatro consultas guardadas: eventos por hora, errores por tipo, latencia y tokens por modelo.* @@ -25,15 +25,15 @@ Encontrarás tus tableros en `//dashboards`. ## Fija las consultas que ya tienes guardadas -Cada mosaico comienza como una consulta guardada. Crea y guarda la consulta que necesitas en la biblioteca de [Consultas](/es/agenteye/queries) (presets integrados más los tuyos propios, sobre tus eventos y evaluaciones), y luego fíjala en un dashboard como el gráfico que mejor se adapte a los datos: una **línea** para tendencias en el tiempo, una **barra** para comparar categorías, un **área** para volumen, o un **pastel** para mostrar proporciones. +Cada mosaico comienza como una consulta guardada. Crea y guarda la consulta que necesitas en la biblioteca de [Consultas](/es/cloud/queries) (presets integrados más los tuyos propios, sobre tus eventos y evaluaciones), y luego fíjala en un dashboard como el gráfico que mejor se adapte a los datos: una **línea** para tendencias en el tiempo, una **barra** para comparar categorías, un **área** para volumen, o un **pastel** para mostrar proporciones. Como un mosaico no es más que tu consulta guardada representada como gráfico, no hay nada que mantener sincronizado manualmente. Actualiza la consulta una vez y todos los dashboards que la usan se actualizan también. ## Monitorea la calidad, no solo el volumen -El volumen te dice que los agentes están ocupados. La calidad te dice que realmente están haciendo bien su trabajo. Apunta un dashboard a tus [puntuaciones de evaluación](/es/agenteye/evaluations) y obtendrás un tablero que rastrea el rendimiento de las ejecuciones a lo largo del tiempo, de modo que una regresión de calidad aparece como una caída en el gráfico en lugar de como una sorpresa de un cliente. +El volumen te dice que los agentes están ocupados. La calidad te dice que realmente están haciendo bien su trabajo. Apunta un dashboard a tus [puntuaciones de evaluación](/es/cloud/evaluations) y obtendrás un tablero que rastrea el rendimiento de las ejecuciones a lo largo del tiempo, de modo que una regresión de calidad aparece como una caída en el gráfico en lugar de como una sorpresa de un cliente. -![Un dashboard enfocado en calidad, construido a partir de consultas de evaluación guardadas](/agenteye/images/dashboard-quality.png) +![Un dashboard enfocado en calidad, construido a partir de consultas de evaluación guardadas](/cloud/images/dashboard-quality.png) *Un tablero de calidad mantiene tus puntuaciones de evaluación en primer plano, justo junto a los números operativos.* @@ -41,6 +41,6 @@ Mantén un tablero de operaciones y un tablero de calidad lado a lado, y tu equi ## Relacionados -- [Consultas](/es/agenteye/queries): crea y guarda las consultas que se convertirán en tus mosaicos. -- [Evaluaciones](/es/agenteye/evaluations): puntúa tus ejecuciones para poder graficar la calidad a lo largo del tiempo. -- [Alertas](/es/agenteye/alerts): convierte un umbral en cualquiera de estas métricas en una notificación. \ No newline at end of file +- [Consultas](/es/cloud/queries): crea y guarda las consultas que se convertirán en tus mosaicos. +- [Evaluaciones](/es/cloud/evaluations): puntúa tus ejecuciones para poder graficar la calidad a lo largo del tiempo. +- [Alertas](/es/cloud/alerts): convierte un umbral en cualquiera de estas métricas en una notificación. \ No newline at end of file diff --git a/docs/es/agenteye/error-tracking.mdx b/docs/es/cloud/errors.mdx similarity index 66% rename from docs/es/agenteye/error-tracking.mdx rename to docs/es/cloud/errors.mdx index 536c3e81..18ada564 100644 --- a/docs/es/agenteye/error-tracking.mdx +++ b/docs/es/cloud/errors.mdx @@ -6,14 +6,14 @@ description: "Ve todos los fallos que producen tus agentes en un solo lugar, agr Ve todos los fallos que producen tus agentes en un solo lugar, agrupados para que una ráfaga ruidosa se lea como un único problema. Tienes un camino de un solo clic desde "algo está en rojo" hasta la ejecución exacta que falló, sin tener que desplazarte por un feed en vivo para encontrarlo. -![La página de Errores: un histograma de fallos a lo largo del tiempo encima de filas de errores en rojo agrupados, cada una con un botón "+ alert" de un solo clic](/agenteye/images/errors.png) +![La página de Errores: un histograma de fallos a lo largo del tiempo encima de filas de errores en rojo agrupados, cada una con un botón "+ alert" de un solo clic](/cloud/images/errors.png) *La página de Errores: un histograma de fallos a lo largo del tiempo, con los fallos repetidos colapsados en una sola fila por incidente.* ## Todos los fallos, ya recopilados por ti Cuando un agente falla, no deberías tener que desplazarte por un stream de eventos en vivo esperando capturar las filas en rojo antes de que desaparezcan. La página **Errors** se encarga de la recopilación por ti. Reúne todo lo que el panel pintaría de rojo en una única superficie de triaje, para que lo primero que veas sea qué está fallando, no dónde tienes que ir a buscarlo. -Y detecta más que los fallos obvios. Además de los eventos explícitos de tipo `error`, Failproof AI Observability también muestra los fallos silenciosos: cualquier `tool_result`, `hook_completed` o `agent_end` cuyo payload contenga un fallo aparece aquí. Una herramienta que devolvió un error, o un hook que terminó mal, ya no pasa desapercibido simplemente porque nada lanzó una excepción sonora. +Y detecta más que los fallos obvios. Además de los eventos explícitos de tipo `error`, FailproofAI Cloud también muestra los fallos silenciosos: cualquier `tool_result`, `hook_completed` o `agent_end` cuyo payload contenga un fallo aparece aquí. Una herramienta que devolvió un error, o un hook que terminó mal, ya no pasa desapercibido simplemente porque nada lanzó una excepción sonora. En la parte superior, un histograma muestra los errores a lo largo del tiempo. Un vistazo te dice si se trata de un goteo de fondo constante o de un pico que empezó hace unos minutos, para que sepas de inmediato si debes dejar lo que estás haciendo. @@ -23,19 +23,19 @@ Como cualquier superficie de observabilidad, la página de Errores está delimit Una sola dependencia rota puede disparar el mismo error cientos de veces por minuto. Tal cual, eso es una pared de líneas casi idénticas que entierra lo único que realmente necesitas ver. -Failproof AI Observability colapsa los fallos repetidos que comparten la misma sesión y tipo de error en una sola fila. Una ráfaga se lee como un único incidente. Acabas contando problemas, no líneas de log, y la señal que importa se mantiene en primer plano en lugar de ahogarse en su propio volumen. +FailproofAI Cloud colapsa los fallos repetidos que comparten la misma sesión y tipo de error en una sola fila. Una ráfaga se lee como un único incidente. Acabas contando problemas, no líneas de log, y la señal que importa se mantiene en primer plano en lugar de ahogarse en su propio volumen. ## De "algo está en rojo" al evento exacto Haz clic en cualquier fila para ir directamente al interior de la sesión de esa ejecución, posicionado en el evento exacto que falló. Sin copiar IDs de sesión, sin desplazarte buscando el momento en que algo salió mal: llegas justo ahí, con el grafo de ejecución completo a un vistazo para que puedas ver qué hizo el agente en los momentos previos al fallo. -Si tienes `alerts:write`, cada fila también incluye un botón **+ alert**. Haz clic en él y Observability abre una nueva regla de alerta ya configurada para detectar ese mismo fallo de nuevo. El incidente que acabas de triar se convierte en el que te avisará la próxima vez, en lugar de sorprenderte dos veces. +Si tienes `alerts:write`, cada fila también incluye un botón **+ alert**. Haz clic en él y FailproofAI Cloud abre una nueva regla de alerta ya configurada para detectar ese mismo fallo de nuevo. El incidente que acabas de triar se convierte en el que te avisará la próxima vez, en lugar de sorprenderte dos veces. **Dónde encontrarlo:** la página **Errors** se encuentra en la sección de observabilidad del panel, en `//errors`. ## Relacionado -- [Alerts](/es/agenteye/alerts): convierte cualquier fallo en una regla de notificación. -- [Incidents](/es/agenteye/incidents): sigue una alerta activa desde que se abre hasta que se resuelve. -- [Sessions](/es/agenteye/sessions): abre la ejecución completa detrás de cualquier error. -- [Audits](/es/agenteye/audits): deja que Observability encuentre patrones de fallos en tus ejecuciones por ti. \ No newline at end of file +- [Alerts](/es/cloud/alerts): convierte cualquier fallo en una regla de notificación. +- [Incidents](/es/cloud/incidents): sigue una alerta activa desde que se abre hasta que se resuelve. +- [Sessions](/es/cloud/sessions): abre la ejecución completa detrás de cualquier error. +- [Audits](/es/cloud/audits): deja que FailproofAI Cloud encuentre patrones de fallos en tus ejecuciones por ti. \ No newline at end of file diff --git a/docs/es/agenteye/evaluations.mdx b/docs/es/cloud/evaluations.mdx similarity index 68% rename from docs/es/agenteye/evaluations.mdx rename to docs/es/cloud/evaluations.mdx index af799316..4bfd42e2 100644 --- a/docs/es/agenteye/evaluations.mdx +++ b/docs/es/cloud/evaluations.mdx @@ -3,15 +3,15 @@ title: "Evaluaciones" description: "Los problemas de calidad te encuentran a ti, en lugar de que te enteres por una queja de un usuario." --- -Los problemas de calidad te encuentran a ti, en lugar de que te enteres por una queja de un usuario. Conecta tu propio servicio de puntuación una sola vez y Failproof AI Observability califica automáticamente cada ejecución completada, de modo que una caída en la utilidad o un aumento en las alucinaciones aparece por sí solo, antes de que el cliente lo sienta. +Los problemas de calidad te encuentran a ti, en lugar de que te enteres por una queja de un usuario. Conecta tu propio servicio de puntuación una sola vez y FailproofAI Cloud califica automáticamente cada ejecución completada, de modo que una caída en la utilidad o un aumento en las alucinaciones aparece por sí solo, antes de que el cliente lo sienta. -![La cuadrícula de sesiones con una columna de puntuación: cada ejecución lleva una etiqueta de estado de evaluación y distintivos codificados por color de utilidad, factualidad y eficiencia de herramientas](/agenteye/images/sessions-list.png) +![La cuadrícula de sesiones con una columna de puntuación: cada ejecución lleva una etiqueta de estado de evaluación y distintivos codificados por color de utilidad, factualidad y eficiencia de herramientas](/cloud/images/sessions-list.png) *Cada ejecución en la cuadrícula de sesiones lleva sus puntuaciones; los distintivos rojos, ámbar y verdes hacen que las ejecuciones débiles resalten sin necesidad de abrir ni una sola transcripción.* ## Deja de revisar ejecuciones manualmente -Antes tenías que verificar un puñado de ejecuciones y esperar que el resto estuviera bien. Ahora cada sesión completada se puntúa en el momento en que termina, en las dimensiones que te importan: utilidad, eficiencia de herramientas, factualidad, seguridad, lo que sea que defina tu estándar de calidad. Tú defines las claves de puntuación; Failproof AI Observability almacena, sigue las tendencias y muestra lo que tu evaluador devuelva. Ninguna ejecución queda sin puntuar, y dejas de enterarte de una regresión a través de un ticket de soporte. +Antes tenías que verificar un puñado de ejecuciones y esperar que el resto estuviera bien. Ahora cada sesión completada se puntúa en el momento en que termina, en las dimensiones que te importan: utilidad, eficiencia de herramientas, factualidad, seguridad, lo que sea que defina tu estándar de calidad. Tú defines las claves de puntuación; FailproofAI Cloud almacena, sigue las tendencias y muestra lo que tu evaluador devuelva. Ninguna ejecución queda sin puntuar, y dejas de enterarte de una regresión a través de un ticket de soporte. Las puntuaciones aparecen en la cuadrícula de sesiones en **`//sessions`** (barra lateral → *observe* → *sessions*), con un grupo de distintivos por fila. ¿Quieres solo las ejecuciones que no alcanzaron el nivel? Filtra la cuadrícula por rango de puntuación, por ejemplo utilidad por debajo de 0,5, y obtén exactamente las ejecuciones que vale la pena revisar. Ver las puntuaciones requiere el permiso `evaluations:read`. @@ -19,7 +19,7 @@ Las puntuaciones aparecen en la cuadrícula de sesiones en **`//sessio Un número te dice que una ejecución fue débil; la página de sesión te dice por qué. Abre cualquier ejecución y el panel lateral derecho muestra primero el resumen general, seguido de una barra por dimensión con el razonamiento propio de tu evaluador debajo de cada una, de modo que pasas de "esto obtuvo 0,4 en factualidad" a la afirmación exacta que falló en cuestión de segundos. -![El panel lateral derecho de una sesión: el resumen de evaluación arriba, luego barras de puntuación por dimensión con una línea de razonamiento en cada una, junto a la línea de tiempo completa de eventos](/agenteye/images/session-detail.png) +![El panel lateral derecho de una sesión: el resumen de evaluación arriba, luego barras de puntuación por dimensión con una línea de razonamiento en cada una, junto a la línea de tiempo completa de eventos](/cloud/images/session-detail.png) *La vista de detalle de sesión: resumen, barras de puntuación por dimensión y el razonamiento detrás de cada puntuación, justo al lado de la línea de tiempo de eventos de la ejecución.* @@ -29,7 +29,7 @@ Un número te dice que una ejecución fue débil; la página de sesión te dice Una ejecución con puntuación baja es ruido; un grupo entero descendiendo es una señal. Los dashboards guardados convierten tus puntuaciones en una tendencia que puedes monitorear de un vistazo: utilidad promedio esta semana frente a la anterior, por agente, por entorno. -![Un dashboard de calidad: barras de puntuación promedio por dimensión del evaluador junto a una tendencia a lo largo del tiempo](/agenteye/images/dashboard-quality.png) +![Un dashboard de calidad: barras de puntuación promedio por dimensión del evaluador junto a una tendencia a lo largo del tiempo](/cloud/images/dashboard-quality.png) *Un dashboard de calidad guardado sigue la tendencia de las claves de puntuación que destacas, de modo que una deriva lenta es obvia mucho antes de convertirse en un incidente.* @@ -37,14 +37,14 @@ Los dashboards se encuentran en **`//dashboards`** (barra lateral → ## Conecta un evaluador una sola vez -La puntuación es opcional y permanece completamente desactivada hasta que apuntes Failproof AI Observability a un puntuador. Configuras un pequeño servicio HTTP (Observability incluye una referencia funcional que puedes copiar), estableces dos valores en tu servidor, y a partir de entonces todas las ejecuciones se puntúan automáticamente. La guía completa, el contrato de puntuación y el SDK están disponibles en la guía detallada. +La puntuación es opcional y permanece completamente desactivada hasta que apuntes FailproofAI Cloud a un puntuador. Configuras un pequeño servicio HTTP (FailproofAI Cloud incluye una referencia funcional que puedes copiar), estableces dos valores en tu servidor, y a partir de entonces todas las ejecuciones se puntúan automáticamente. La guía completa, el contrato de puntuación y el SDK están disponibles en la guía detallada. -¿No sabes qué dimensiones vale la pena puntuar en primer lugar? La [habilidad de agente evaluador](/es/agenteye/evaluator-skill) hace que tu agente de codificación lo determine en función de tus propias sesiones, y luego construye y despliega el servicio. +¿No sabes qué dimensiones vale la pena puntuar en primer lugar? La [habilidad de agente evaluador](/es/cloud/agent-skills) hace que tu agente de codificación lo determine en función de tus propias sesiones, y luego construye y despliega el servicio. ## Relacionado -- [Suite de evaluación](/es/agenteye/evaluation-suite): conecta tu evaluador, el contrato de puntuación y el SDK. -- [Habilidad de agente evaluador](/es/agenteye/evaluator-skill): deja que un agente de codificación elija tus dimensiones de puntuación y construya el evaluador. -- [Sesiones](/es/agenteye/sessions): la cuadrícula ejecución por ejecución donde aparecen las puntuaciones. -- [Dashboards](/es/agenteye/dashboards): guarda y comparte tendencias de calidad en toda tu organización. -- [Auditorías](/es/agenteye/audits): la otra función de calidad automática de Observability, para investigaciones entre sesiones. \ No newline at end of file +- [Suite de evaluación](/es/cloud/evaluators): conecta tu evaluador, el contrato de puntuación y el SDK. +- [Habilidad de agente evaluador](/es/cloud/agent-skills): deja que un agente de codificación elija tus dimensiones de puntuación y construya el evaluador. +- [Sesiones](/es/cloud/sessions): la cuadrícula ejecución por ejecución donde aparecen las puntuaciones. +- [Dashboards](/es/cloud/dashboards): guarda y comparte tendencias de calidad en toda tu organización. +- [Auditorías](/es/cloud/audits): la otra función de calidad automática de FailproofAI Cloud, para investigaciones entre sesiones. \ No newline at end of file diff --git a/docs/es/agenteye/evaluation-suite.mdx b/docs/es/cloud/evaluators.mdx similarity index 83% rename from docs/es/agenteye/evaluation-suite.mdx rename to docs/es/cloud/evaluators.mdx index 202347ab..19bd823e 100644 --- a/docs/es/agenteye/evaluation-suite.mdx +++ b/docs/es/cloud/evaluators.mdx @@ -1,20 +1,20 @@ --- title: "Suite de Evaluación" -description: "Failproof AI Observability puede puntuar automáticamente cada ejecución de agente finalizada: tú proporcionas un pequeño servicio de puntuación y Observability se encarga del resto." +description: "FailproofAI Cloud puede puntuar automáticamente cada ejecución de agente finalizada: tú proporcionas un pequeño servicio de puntuación y FailproofAI Cloud se encarga del resto." --- -Failproof AI Observability puede puntuar automáticamente cada ejecución de agente finalizada para medir su calidad: tú proporcionas un pequeño servicio de puntuación y Observability se encarga del resto. Úsalo para rastrear las dimensiones que te importan (utilidad, eficiencia de herramientas, factualidad, seguridad; tú decides), detectar regresiones a tiempo y comparar agentes o entornos de un vistazo. La puntuación es opcional: el pipeline no hace nada hasta que configures `EVALUATOR_ENDPOINT` en el servidor. +FailproofAI Cloud puede puntuar automáticamente cada ejecución de agente finalizada para medir su calidad: tú proporcionas un pequeño servicio de puntuación y FailproofAI Cloud se encarga del resto. Úsalo para rastrear las dimensiones que te importan (utilidad, eficiencia de herramientas, factualidad, seguridad; tú decides), detectar regresiones a tiempo y comparar agentes o entornos de un vistazo. La puntuación es opcional: el pipeline no hace nada hasta que configures `EVALUATOR_ENDPOINT` en el servidor. -> **Nota:** Tú defines las dimensiones de puntuación. Tu evaluador puede devolver las claves numéricas que quiera; Observability almacena, analiza tendencias y muestra todo lo que le envíes. +> **Nota:** Tú defines las dimensiones de puntuación. Tu evaluador puede devolver las claves numéricas que quiera; FailproofAI Cloud almacena, analiza tendencias y muestra todo lo que le envíes. ## Resumen rápido -1. **Escribe un evaluador.** Levanta un pequeño servicio HTTP que lea la transcripción de una sesión y devuelva puntuaciones. Observability incluye una referencia funcional que puedes copiar. Consulta [Escribir un evaluador con el SDK](#writing-an-evaluator-with-the-sdk). -2. **Apunta Observability hacia él.** Configura `EVALUATOR_ENDPOINT` (y un `EVALUATOR_TOKEN` compartido) en el proceso del servidor. +1. **Escribe un evaluador.** Levanta un pequeño servicio HTTP que lea la transcripción de una sesión y devuelva puntuaciones. FailproofAI Cloud incluye una referencia funcional que puedes copiar. Consulta [Escribir un evaluador con el SDK](#writing-an-evaluator-with-the-sdk). +2. **Apunta FailproofAI Cloud hacia él.** Configura `EVALUATOR_ENDPOINT` (y un `EVALUATOR_TOKEN` compartido) en el proceso del servidor. 3. **Observa cómo llegan las puntuaciones.** Cada sesión completada se puntúa automáticamente; los resultados aparecen en la página de detalle de sesión, la cuadrícula de sesiones y los dashboards guardados. -![Vista de detalle de sesión con el resumen de evaluación, barras de puntuación por dimensión y texto de razonamiento en el panel lateral derecho](/agenteye/images/session-detail.png) +![Vista de detalle de sesión con el resumen de evaluación, barras de puntuación por dimensión y texto de razonamiento en el panel lateral derecho](/cloud/images/session-detail.png) *Una vez configurado un evaluador, cada ejecución completada recibe una puntuación y los resultados aparecen en el panel lateral derecho de la sesión: el resumen en la parte superior, seguido de barras de puntuación por dimensión con su razonamiento.* @@ -24,7 +24,7 @@ Failproof AI Observability puede puntuar automáticamente cada ejecución de age ```mermaid flowchart LR - ING["ingest /events
agent_end"] --> SRV["Observability server"] + ING["ingest /events
agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -32,14 +32,14 @@ flowchart LR SRV --> RES["evaluations
terminal results"] ``` -Cuando el SDK de Observability emite un evento `agent_end` para una sesión, el servidor programa una evaluación. Luego envía mediante POST la transcripción completa de eventos a tu servicio evaluador, que puede: +Cuando el SDK de FailproofAI Cloud emite un evento `agent_end` para una sesión, el servidor programa una evaluación. Luego envía mediante POST la transcripción completa de eventos a tu servicio evaluador, que puede: - **Devolver el resultado de forma inmediata** con `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`. El resultado se añade a la línea temporal de evaluaciones de la sesión. `reasoning` y `summary` son opcionales. -- **Diferir la respuesta** con `{"status":"pending", "job_id":"abc-123"}`. Observability entonces llama a `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` hasta que tu evaluador devuelva `{"status":"done", ...}` o `{"status":"error", "error":"..."}`. +- **Diferir la respuesta** con `{"status":"pending", "job_id":"abc-123"}`. FailproofAI Cloud entonces llama a `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` hasta que tu evaluador devuelva `{"status":"done", ...}` o `{"status":"error", "error":"..."}`. - La cadencia de sondeo es por trabajo: una respuesta `pending` puede incluir `next_poll_secs` para sobreescribirla; de lo contrario, Observability usa el valor `default_poll_interval_secs` de `GET /config`; si tampoco está definido, el servidor recurre a `EVALUATOR_POLLING_INTERVAL_SECS` (10s por defecto). Todos los valores se limitan al rango [1s, 1h]. + La cadencia de sondeo es por trabajo: una respuesta `pending` puede incluir `next_poll_secs` para sobreescribirla; de lo contrario, FailproofAI Cloud usa el valor `default_poll_interval_secs` de `GET /config`; si tampoco está definido, el servidor recurre a `EVALUATOR_POLLING_INTERVAL_SECS` (10s por defecto). Todos los valores se limitan al rango [1s, 1h]. -Las sesiones que nunca emiten `agent_end` (por ejemplo, un proceso de agente que se ha bloqueado) también pueden procesarse: el `GET /config` del evaluador puede devolver `{"inactivity_timeout_secs": 1800}`, y Observability evaluará cualquier sesión que haya estado inactiva durante ese tiempo. Establece el campo en `null` u omítelo para desactivar este comportamiento alternativo. +Las sesiones que nunca emiten `agent_end` (por ejemplo, un proceso de agente que se ha bloqueado) también pueden procesarse: el `GET /config` del evaluador puede devolver `{"inactivity_timeout_secs": 1800}`, y FailproofAI Cloud evaluará cualquier sesión que haya estado inactiva durante ese tiempo. Establece el campo en `null` u omítelo para desactivar este comportamiento alternativo. El pipeline es completamente inactivo cuando `EVALUATOR_ENDPOINT` no está configurado. @@ -47,7 +47,7 @@ Una sesión puede acumular **múltiples evaluaciones terminales a lo largo del t La recuperación por inactividad también se activa en sesiones reanudadas: si llegan nuevos eventos después de una evaluación terminal anterior y la sesión vuelve a quedar inactiva pasando el umbral de `inactivity_timeout_secs`, se pone en cola una nueva evaluación. -Los fallos transitorios (5xx, 429, timeouts, errores de red) se reintentan con retroceso exponencial hasta `EVALUATOR_MAX_ATTEMPTS`; las respuestas 4xx son terminales. Observability es seguro de ejecutar con múltiples instancias de servidor escaladas horizontalmente; el trabajo se distribuye de forma que la misma sesión nunca se despacha dos veces de forma concurrente. +Los fallos transitorios (5xx, 429, timeouts, errores de red) se reintentan con retroceso exponencial hasta `EVALUATOR_MAX_ATTEMPTS`; las respuestas 4xx son terminales. FailproofAI Cloud es seguro de ejecutar con múltiples instancias de servidor escaladas horizontalmente; el trabajo se distribuye de forma que la misma sesión nunca se despacha dos veces de forma concurrente. --- @@ -55,7 +55,7 @@ Los fallos transitorios (5xx, 429, timeouts, errores de red) se reintentan con r Todas las rutas autenticadas usan **autenticación mediante token bearer**. El mismo valor debe configurarse en ambos lados: -- Servidor de Observability: variable de entorno `EVALUATOR_TOKEN` +- Servidor de FailproofAI Cloud: variable de entorno `EVALUATOR_TOKEN` - Servicio evaluador: configurado de la misma forma (el SDK `agenteye-evaluator` lee `EVALUATOR_TOKEN` por convención) Si `EVALUATOR_TOKEN` no está configurado, el servidor no envía cabecera `Authorization`; el evaluador puede entonces aceptar solicitudes anónimas, lo cual es aceptable en una red exclusivamente interna pero no recomendado en internet público. @@ -126,7 +126,7 @@ El servidor trata cualquier otro cuerpo 2xx como un error de protocolo y registr No tienes que implementar el contrato HTTP a mano. El paquete Python `agenteye-evaluator` te proporciona un wrapper tipado de FastAPI que gestiona la autenticación, el enrutamiento y los formatos de solicitud/respuesta por ti. -Failproof AI Observability también incluye un **evaluador de referencia funcional** que puntúa `helpfulness`, `tool_efficiency` y `factuality` a partir de la estructura de la transcripción. Cópialo como punto de partida y sustituye la lógica por la tuya: un juez LLM, un motor de reglas, lo que mejor se adapte a tu criterio de calidad. +FailproofAI Cloud también incluye un **evaluador de referencia funcional** que puntúa `helpfulness`, `tool_efficiency` y `factuality` a partir de la estructura de la transcripción. Cópialo como punto de partida y sustituye la lógica por la tuya: un juez LLM, un motor de reglas, lo que mejor se adapte a tu criterio de calidad. Evaluador mínimo viable: @@ -149,7 +149,7 @@ def run(req: EvalRequest) -> EvalResponse: La instancia `app` se ejecuta bajo cualquier servidor ASGI, por lo que `uvicorn module:app` la pone en marcha. -Para evaluadores que necesitan diferir trabajo costoso, devuelve `JobPending` en su lugar y registra un handler `@app.job_lookup`; el servidor de Observability sondea `GET /evaluate/{job_id}` hasta que devuelves un estado terminal o se agota el límite de `EVALUATOR_MAX_POLL_DURATION_SECS` (1 h por defecto). +Para evaluadores que necesitan diferir trabajo costoso, devuelve `JobPending` en su lugar y registra un handler `@app.job_lookup`; el servidor de FailproofAI Cloud sondea `GET /evaluate/{job_id}` hasta que devuelves un estado terminal o se agota el límite de `EVALUATOR_MAX_POLL_DURATION_SECS` (1 h por defecto). La referencia completa de la API, el patrón asíncrono y el esquema de eventos están documentados en el README del SDK `agenteye-evaluator`. @@ -157,7 +157,7 @@ La referencia completa de la API, el patrón asíncrono y el esquema de eventos ## Ejecutar tu evaluador -El evaluador es **tu servicio** — Failproof AI Observability no incluye un evaluador por defecto, así que lo construyes y ejecutas donde ejecutas tus propios servicios. Se ejecuta bajo cualquier servidor ASGI (por ejemplo `uvicorn my_evaluator:app`); sirve las rutas `/health`, `/config` y `/evaluate` del [contrato HTTP](#http-contract) y luego apunta el servidor hacia él (consulta [Configurar el servidor](#configuring-the-server)). +El evaluador es **tu servicio** — FailproofAI Cloud no incluye un evaluador por defecto, así que lo construyes y ejecutas donde ejecutas tus propios servicios. Se ejecuta bajo cualquier servidor ASGI (por ejemplo `uvicorn my_evaluator:app`); sirve las rutas `/health`, `/config` y `/evaluate` del [contrato HTTP](#http-contract) y luego apunta el servidor hacia él (consulta [Configurar el servidor](#configuring-the-server)). Una vez que el evaluador sea accesible, `GET /health` devuelve `{"status":"ok"}`. Después de que un agente se ejecute de principio a fin, `GET /evaluations` en el servidor devuelve una fila con `status: "done"` y las puntuaciones que produjo tu evaluador. @@ -255,7 +255,7 @@ El administrador bootstrap (`ADMIN_KEY`, `ADMIN_EMAIL`) recibe estos permisos au - **`/sessions`**: cuadrícula de sesiones filtrable; la columna de puntuación muestra el estado de evaluación y las puntuaciones de cada sesión de un vistazo. - **`/dashboards`**: vistas guardadas de salud de evaluación (consulta [Dashboards](#dashboards) más abajo). -![La cuadrícula de sesiones con indicadores de estado de evaluación por sesión e insignias de puntuación con código de colores (helpfulness, factuality, tool_efficiency, safety, coherence)](/agenteye/images/sessions-list.png) +![La cuadrícula de sesiones con indicadores de estado de evaluación por sesión e insignias de puntuación con código de colores (helpfulness, factuality, tool_efficiency, safety, coherence)](/cloud/images/sessions-list.png) *La cuadrícula de sesiones muestra el estado de evaluación y las puntuaciones de cada ejecución de un vistazo; las insignias en rojo/ámbar/verde hacen que las puntuaciones bajas destaquen.* @@ -272,7 +272,7 @@ Cada dashboard fija: Cada tarjeta muestra el número de sesiones coincidentes, un desglose done/error/timeout, el promedio de cada puntuación destacada y una pequeña línea de tendencia. Abrir un dashboard muestra los paneles a tamaño completo; **"open in sessions"** te lleva a la página de sesiones prefiltrada exactamente a ese subconjunto. Las métricas se calculan en el servidor sobre todo el conjunto coincidente (mediante `GET /evaluations/aggregate`), por lo que los números son exactos y no muestreados. -![Un dashboard de salud de evaluación con barras de puntuación media por dimensión del evaluador, un desglose ok-vs-error de herramientas, las principales herramientas y una tendencia de eventos por hora](/agenteye/images/dashboard-quality.png) +![Un dashboard de salud de evaluación con barras de puntuación media por dimensión del evaluador, un desglose ok-vs-error de herramientas, las principales herramientas y una tendencia de eventos por hora](/cloud/images/dashboard-quality.png) **Permisos:** para ver se necesita tanto `dashboards:read` como `evaluations:read`; para crear y editar se necesita `dashboards:write`; para eliminar se necesita `dashboards:delete`. El administrador bootstrap recibe todos estos permisos automáticamente. @@ -294,7 +294,7 @@ Cada tarjeta muestra el número de sesiones coincidentes, un desglose done/error ## Próximos pasos -- [Habilidad de agente evaluador](/es/agenteye/evaluator-skill): haz que un agente de programación diseñe tus dimensiones a partir de sesiones reales y construya este servicio por ti. -- [Python SDK](/es/agenteye/python-sdk): emite los eventos `agent_end` que desencadenan la puntuación. -- [Claves de API](/es/agenteye/api-keys): los permisos `evaluations:read` y `evaluations:trigger`. -- [Auditorías](/es/agenteye/audits): la otra función de calidad automatizada de Observability, para revisión basada en políticas. \ No newline at end of file +- [Habilidad de agente evaluador](/es/cloud/agent-skills): haz que un agente de programación diseñe tus dimensiones a partir de sesiones reales y construya este servicio por ti. +- [Python SDK](/es/cloud/sdk): emite los eventos `agent_end` que desencadenan la puntuación. +- [Claves de API](/es/cloud/access): los permisos `evaluations:read` y `evaluations:trigger`. +- [Auditorías](/es/cloud/audits): la otra función de calidad automatizada de FailproofAI Cloud, para revisión basada en políticas. \ No newline at end of file diff --git a/docs/es/agenteye/event-stream.mdx b/docs/es/cloud/event-stream.mdx similarity index 85% rename from docs/es/agenteye/event-stream.mdx rename to docs/es/cloud/event-stream.mdx index 68e102b2..a74f63d2 100644 --- a/docs/es/agenteye/event-stream.mdx +++ b/docs/es/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "En el momento en que tu agente hace algo, tú lo ves." En el momento en que tu agente hace algo, tú lo ves. El Stream de Eventos es tu pulso en tiempo real sobre cada agente en producción: sin esperas, sin buscar entre logs, sin adivinar qué acaba de pasar. -![El Stream de Eventos en vivo: filas de eventos con código de colores actualizándose en tiempo real, filtrables por entorno, agente, sesión, tipo de evento y texto libre](/agenteye/images/events-stream.png) +![El Stream de Eventos en vivo: filas de eventos con código de colores actualizándose en tiempo real, filtrables por entorno, agente, sesión, tipo de evento y texto libre](/cloud/images/events-stream.png) *Cada evento de cada agente en tu organización, del más reciente al más antiguo, actualizándose en tiempo real.* @@ -43,8 +43,8 @@ Esta es la fuente de verdad en bruto sobre la que se construye cada otra superfi ## Relacionado -- [Sesiones](/es/agenteye/sessions): los mismos eventos agrupados en una fila por ejecución, con un gráfico de ejecución al estilo de git. -- [Telemetría](/es/agenteye/telemetry): qué envían tus agentes y cómo llegan los eventos al stream. -- [Seguimiento de errores](/es/agenteye/error-tracking): una sola superficie de triaje para todo lo que salió mal. -- [Alertas](/es/agenteye/alerts): convierte cualquier umbral en una regla de notificación. -- [CLI y agentes](/es/agenteye/cli-and-agents): el mismo historial en tiempo real desde tu terminal. \ No newline at end of file +- [Sesiones](/es/cloud/sessions): los mismos eventos agrupados en una fila por ejecución, con un gráfico de ejecución al estilo de git. +- [Telemetría](/es/cloud/performance): qué envían tus agentes y cómo llegan los eventos al stream. +- [Seguimiento de errores](/es/cloud/errors): una sola superficie de triaje para todo lo que salió mal. +- [Alertas](/es/cloud/alerts): convierte cualquier umbral en una regla de notificación. +- [CLI y agentes](/es/cloud/cli): el mismo historial en tiempo real desde tu terminal. \ No newline at end of file diff --git a/docs/es/cloud/fleet.mdx b/docs/es/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/es/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/es/agenteye/incidents.mdx b/docs/es/cloud/incidents.mdx similarity index 84% rename from docs/es/agenteye/incidents.mdx rename to docs/es/cloud/incidents.mdx index 53dcd83a..50525721 100644 --- a/docs/es/agenteye/incidents.mdx +++ b/docs/es/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "Cuando se dispara una alerta, todos pueden ver que el incidente es Cuando se dispara una alerta, la primera pregunta siempre es "¿quién lo está atendiendo?". Los incidentes responden esa pregunta: en el momento en que algo supera un umbral, todos pueden ver que el incidente está abierto, quién lo gestiona y exactamente qué ha ocurrido hasta ahora, con un registro limpio y atribuido que puedes entregar directamente a una revisión post-mortem. -![La bandeja de incidentes: tarjetas de incidentes vinculadas a alertas y abiertas manualmente, agrupadas por estado, cada una con un indicador de severidad y un responsable asignado](/agenteye/images/incidents.png) +![La bandeja de incidentes: tarjetas de incidentes vinculadas a alertas y abiertas manualmente, agrupadas por estado, cada una con un indicador de severidad y un responsable asignado](/cloud/images/incidents.png) *La bandeja agrupa los incidentes abiertos por estado y filtra por severidad y responsable, para que veas de inmediato qué requiere atención humana.* ## Saber quién lo tiene, de un vistazo @@ -17,10 +17,10 @@ No más "¿alguien está mirando esto?" en un hilo de chat. Un incumplimiento ab Cuando el incidente termina, ya tienes el informe escrito. Abre cualquier incidente y verás la evidencia del incumplimiento, sus responsables y suscriptores, un hilo de comentarios para coordinar en el momento, y una línea de tiempo de actividad de solo escritura. -![Vista detallada de un incidente: la alerta padre y el resumen del incumplimiento, responsables y suscriptores, una línea de tiempo de actividad atribuida y un hilo de comentarios](/agenteye/images/incident-detail.png) +![Vista detallada de un incidente: la alerta padre y el resumen del incumplimiento, responsables y suscriptores, una línea de tiempo de actividad atribuida y un hilo de comentarios](/cloud/images/incident-detail.png) *Todo lo que ocurrió, en orden, cada línea firmada por quien lo hizo.* -Cada acción (abierto, aceptado, resuelto, etc.) queda registrada en esa línea de tiempo y nunca se edita ni elimina. Cada entrada está atribuida: al operador que la realizó, por correo electrónico, o a **automated** para cualquier cosa que Failproof AI Observability hizo de forma autónoma, como abrir el incidente al detectar el incumplimiento. Nada es anónimo y nada se pierde, por lo que el post-mortem prácticamente se escribe solo. +Cada acción (abierto, aceptado, resuelto, etc.) queda registrada en esa línea de tiempo y nunca se edita ni elimina. Cada entrada está atribuida: al operador que la realizó, por correo electrónico, o a **automated** para cualquier cosa que FailproofAI Cloud hizo de forma autónoma, como abrir el incidente al detectar el incumplimiento. Nada es anónimo y nada se pierde, por lo que el post-mortem prácticamente se escribe solo. ## Cómo progresa un incidente @@ -45,6 +45,6 @@ Los incidentes se encuentran en `//incidents`. Para ver los incidentes ## Relacionado -- [Alertas](/es/agenteye/alerts): las reglas que abren estos incidentes cuando se supera un umbral. -- [Seguimiento de errores](/es/agenteye/error-tracking): ve todos los fallos en un solo lugar y promueve uno a alerta. -- [Auditorías](/es/agenteye/audits): el analista programado que encuentra los fallos que ninguna regla estaba supervisando. \ No newline at end of file +- [Alertas](/es/cloud/alerts): las reglas que abren estos incidentes cuando se supera un umbral. +- [Seguimiento de errores](/es/cloud/errors): ve todos los fallos en un solo lugar y promueve uno a alerta. +- [Auditorías](/es/cloud/audits): el analista programado que encuentra los fallos que ninguna regla estaba supervisando. \ No newline at end of file diff --git a/docs/es/cloud/managed-policies.mdx b/docs/es/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/es/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/es/agenteye/overview.mdx b/docs/es/cloud/overview.mdx similarity index 51% rename from docs/es/agenteye/overview.mdx rename to docs/es/cloud/overview.mdx index 2429a23c..6d5ef2e0 100644 --- a/docs/es/agenteye/overview.mdx +++ b/docs/es/cloud/overview.mdx @@ -1,16 +1,16 @@ --- title: "Failproof AI: Observa Agentes en Busca de Fallos" -description: "Failproof AI Observability es una plataforma autoalojada para observar, evaluar y mejorar tus agentes de IA en producción." +description: "FailproofAI Cloud es una plataforma autoalojada para observar, evaluar y mejorar tus agentes de IA en producción." --- -Failproof AI Observability es una plataforma autoalojada para observar, evaluar y mejorar tus agentes de IA en producción. Registra todo lo que hacen tus agentes (cada llamada a herramientas, petición al modelo, hook y error), puntúa la calidad de cada ejecución y pone de manifiesto los fallos que no sabías que debías buscar, todo ello en un panel de control que ejecutas dentro de tu propia infraestructura. +FailproofAI Cloud es una plataforma autoalojada para observar, evaluar y mejorar tus agentes de IA en producción. Registra todo lo que hacen tus agentes (cada llamada a herramientas, petición al modelo, hook y error), puntúa la calidad de cada ejecución y pone de manifiesto los fallos que no sabías que debías buscar, todo ello en un panel de control que ejecutas dentro de tu propia infraestructura. -Si despliegas agentes de IA y estás cansado de adivinar por qué falló una ejecución, esta es la página por la que empezar. Explica qué te ofrece Failproof AI Observability y cómo encajan las piezas, antes de que instales nada. +Si despliegas agentes de IA y estás cansado de adivinar por qué falló una ejecución, esta es la página por la que empezar. Explica qué te ofrece FailproofAI Cloud y cómo encajan las piezas, antes de que instales nada. -> **Failproof AI Observability es un producto empresarial de Failproof AI.** ¿Quieres verlo en acción? Solicita una demo: escribe a [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +> **FailproofAI Cloud es un producto empresarial de Failproof AI.** ¿Quieres verlo en acción? Solicita una demo: escribe a [nikita@befailproof.ai](mailto:nikita@befailproof.ai). -![Una sesión de Failproof AI Observability dibujada como un grafo de ejecución estilo git junto a su cronología de eventos, con un desglose por ejecución de herramientas, modelos y hooks en el panel derecho](/agenteye/images/session-detail.png) +![Una sesión de FailproofAI Cloud dibujada como un grafo de ejecución estilo git junto a su cronología de eventos, con un desglose por ejecución de herramientas, modelos y hooks en el panel derecho](/cloud/images/session-detail.png) *Cada ejecución del agente se representa como un grafo de ejecución estilo git (izquierda) junto a su cronología de eventos. Los subagentes paralelos tienen su propio carril; el panel derecho desglosa las herramientas, modelos, hooks y gasto en tokens de la ejecución.* @@ -30,52 +30,52 @@ Dos vídeos cortos muestran las dos cosas a las que los equipos recurren primero
-*Failproof Audit: deja que Failproof AI Observability analice tus registros entre sesiones y te indique qué debes corregir.* +*Failproof Audit: deja que FailproofAI Cloud analice tus registros entre sesiones y te indique qué debes corregir.* --- ## Por qué los equipos lo usan - **Ve lo que tu agente hizo realmente.** Cada ejecución se convierte en un grafo de ejecución legible estilo git: qué herramientas se ejecutaron en paralelo, qué subagentes se ramificaron, dónde se atascó y cuánto consumió. -- **Detecta regresiones de calidad automáticamente.** Conecta un pequeño servicio de puntuación y Failproof AI Observability puntuará cada ejecución completada, de modo que una caída en utilidad o un pico en alucinaciones aparecerá por sí solo. +- **Detecta regresiones de calidad automáticamente.** Conecta un pequeño servicio de puntuación y FailproofAI Cloud puntuará cada ejecución completada, de modo que una caída en utilidad o un pico en alucinaciones aparecerá por sí solo. - **Encuentra fallos para los que no escribiste ninguna regla.** Las auditorías recurrentes analizan tus registros entre sesiones en busca de clústeres de errores, valores atípicos de latencia, puntuaciones bajas y ejecuciones bloqueadas, y te entregan hallazgos clasificados y respaldados por evidencias. - **Recibe alertas cuando importa.** Las reglas de umbral se activan por tasa de error, latencia, coste o puntuaciones del evaluador, y abren incidentes que puedes reconocer, asignar y resolver. - **Haz preguntas en lenguaje natural.** Un asistente de IA integrado en el panel responde preguntas como «¿cómo evoluciona la calidad en producción esta semana?» sobre tus propios datos. Cualquier cambio que realice requiere aprobación. -- **Mantén el control de tus datos.** Failproof AI Observability es autoalojada: los eventos, los prompts y los análisis permanecen en la infraestructura que tú controlas. +- **Mantén el control de tus datos.** FailproofAI Cloud es autoalojada: los eventos, los prompts y los análisis permanecen en la infraestructura que tú controlas. --- ## Qué obtienes -Failproof AI Observability se organiza en torno a tres conceptos (**observar**, **analizar** y **administrar**), reflejados en la barra lateral izquierda del panel de control. +FailproofAI Cloud se organiza en torno a tres conceptos (**observar**, **analizar** y **administrar**), reflejados en la barra lateral izquierda del panel de control. **Observar** (la verdad bruta de lo que ocurrió): -- **[Flujo de eventos](/es/agenteye/event-stream)**: el rastro en tiempo real, paso a paso, de cada ejecución (llamadas a herramientas, llamadas al modelo, hooks, errores). -- **[Sesiones](/es/agenteye/sessions)**: esos eventos agrupados en una fila por ejecución, cada una lista para ser puntuada, con un grafo de ejecución estilo git. -- **[Métricas de rendimiento](/es/agenteye/telemetry)**: mapas de calor de latencia por superficie y valores p50/p95/p99 para modelos, herramientas y hooks, para que un pico en la cola destaque sobre la mediana. -- **[Seguimiento de errores](/es/agenteye/error-tracking)**: una única superficie de triaje para todo lo que salió mal, a un clic de una alerta activa. +- **[Flujo de eventos](/es/cloud/event-stream)**: el rastro en tiempo real, paso a paso, de cada ejecución (llamadas a herramientas, llamadas al modelo, hooks, errores). +- **[Sesiones](/es/cloud/sessions)**: esos eventos agrupados en una fila por ejecución, cada una lista para ser puntuada, con un grafo de ejecución estilo git. +- **[Métricas de rendimiento](/es/cloud/performance)**: mapas de calor de latencia por superficie y valores p50/p95/p99 para modelos, herramientas y hooks, para que un pico en la cola destaque sobre la mediana. +- **[Seguimiento de errores](/es/cloud/errors)**: una única superficie de triaje para todo lo que salió mal, a un clic de una alerta activa. -![La página de observación de Tools: un mapa de calor de latencia, una banda de percentiles y una barra de distribución de herramientas en 24 intervalos de tiempo](/agenteye/images/tools.png) +![La página de observación de Tools: un mapa de calor de latencia, una banda de percentiles y una barra de distribución de herramientas en 24 intervalos de tiempo](/cloud/images/tools.png) *Cada superficie de observación combina un minigráfico y valores p50/p95/p99 con un mapa de calor de latencia y una banda de percentiles. Mostrado aquí: Tools.* **Analizar** (convertir la actividad en respuestas): -- **[Consultas](/es/agenteye/queries)** y **[paneles](/es/agenteye/dashboards)**: SQL guardado sobre tus eventos y evaluaciones, representado en paneles compartidos con ámbito de organización. -- **[Evaluaciones](/es/agenteye/evaluations)**: puntuaciones de calidad producidas por tu propio servicio evaluador, con el razonamiento por puntuación. -- **[Auditorías](/es/agenteye/audits)**: investigaciones recurrentes que detectan patrones de fallo entre sesiones. -- **[Alertas](/es/agenteye/alerts)** e **[incidentes](/es/agenteye/incidents)**: reglas de umbral que te notifican, más un flujo de trabajo de incidentes para gestionarlos. +- **[Consultas](/es/cloud/queries)** y **[paneles](/es/cloud/dashboards)**: SQL guardado sobre tus eventos y evaluaciones, representado en paneles compartidos con ámbito de organización. +- **[Evaluaciones](/es/cloud/evaluations)**: puntuaciones de calidad producidas por tu propio servicio evaluador, con el razonamiento por puntuación. +- **[Auditorías](/es/cloud/audits)**: investigaciones recurrentes que detectan patrones de fallo entre sesiones. +- **[Alertas](/es/cloud/alerts)** e **[incidentes](/es/cloud/incidents)**: reglas de umbral que te notifican, más un flujo de trabajo de incidentes para gestionarlos. **Interfaces** (accede a tus datos a tu manera): -- **[CLI](/es/agenteye/cli-and-agents)**: gestiona todo tu despliegue desde el terminal o un script, y deja que un agente de codificación lo haga por ti en lenguaje natural. -- **[Asistente de IA](/es/agenteye/assistant)**: haz preguntas sobre tus agentes en lenguaje natural, directamente desde el panel de control. -- **REST API**: todo lo que hacen el panel y la CLI está respaldado por una REST API que puedes llamar directamente con una [clave de API](/es/agenteye/api-keys) con ámbito definido — ingesta eventos, consulta sesiones y evaluaciones, y gestiona paneles, alertas, auditorías, usuarios y claves, para poder integrar Failproof AI Observability en tus propias herramientas. +- **[CLI](/es/cloud/cli)**: gestiona todo tu despliegue desde el terminal o un script, y deja que un agente de codificación lo haga por ti en lenguaje natural. +- **[Asistente de IA](/es/cloud/assistant)**: haz preguntas sobre tus agentes en lenguaje natural, directamente desde el panel de control. +- **REST API**: todo lo que hacen el panel y la CLI está respaldado por una REST API que puedes llamar directamente con una [clave de API](/es/cloud/access) con ámbito definido — ingesta eventos, consulta sesiones y evaluaciones, y gestiona paneles, alertas, auditorías, usuarios y claves, para poder integrar FailproofAI Cloud en tus propias herramientas. **Administrar** (gestiónalo para tu equipo): -- **[Claves de API](/es/agenteye/api-keys)**: tokens con ámbito para el colector, el panel y el asistente. +- **[Claves de API](/es/cloud/access)**: tokens con ámbito para el colector, el panel y el asistente. - **Usuarios**: inicio de sesión sin contraseña, basado en correo electrónico, con lista de permitidos. - **Configuración**: configuración por organización, incluidas las anulaciones de ventana de contexto de los modelos. @@ -91,18 +91,18 @@ Los datos fluyen en una sola dirección, desde el código de tu agente hasta el - **Panel de control**: donde exploras todo. - **Servicios opcionales**: un servicio de puntuación (evaluaciones) y un servicio de asistente de IA (el chat integrado en el panel). -Para el vocabulario utilizado en toda la documentación (*evento, sesión, evaluación, auditoría, hallazgo, incidente*), consulta [Conceptos](/es/agenteye/concepts). +Para el vocabulario utilizado en toda la documentación (*evento, sesión, evaluación, auditoría, hallazgo, incidente*), consulta [Conceptos](/es/concepts). --- -## Cómo obtener Failproof AI Observability +## Cómo obtener FailproofAI Cloud -Failproof AI Observability es un producto empresarial de Failproof AI, y funciona junto con Failproof AI Enforcement — el producto de políticas y barreras de seguridad — bajo la marca Failproof AI. Se ejecuta completamente en tu propio entorno. Si aún no tienes acceso a los paquetes, solicita una demo y te ayudamos a ponerte en marcha: escribe a [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +FailproofAI Cloud es un producto empresarial de Failproof AI, y funciona junto con FailproofAI guardrails — el producto de políticas y barreras de seguridad — bajo la marca Failproof AI. Se ejecuta completamente en tu propio entorno. Si aún no tienes acceso a los paquetes, solicita una demo y te ayudamos a ponerte en marcha: escribe a [nikita@befailproof.ai](mailto:nikita@befailproof.ai). --- ## Próximos pasos -- [Conceptos](/es/agenteye/concepts): el vocabulario de Failproof AI Observability en un solo lugar. -- [Observabilidad](/es/agenteye/observability): sigue lo que hacen tus agentes, ejecución a ejecución. -- [Seguridad](/es/agenteye/security): cómo Failproof AI Observability mantiene tus datos aislados y bajo tu control. \ No newline at end of file +- [Conceptos](/es/concepts): el vocabulario de FailproofAI Cloud en un solo lugar. +- [Observabilidad](/es/cloud/overview): sigue lo que hacen tus agentes, ejecución a ejecución. +- [Seguridad](/es/cloud/security): cómo FailproofAI Cloud mantiene tus datos aislados y bajo tu control. \ No newline at end of file diff --git a/docs/es/agenteye/telemetry.mdx b/docs/es/cloud/performance.mdx similarity index 81% rename from docs/es/agenteye/telemetry.mdx rename to docs/es/cloud/performance.mdx index 55b69a24..5a1181c7 100644 --- a/docs/es/agenteye/telemetry.mdx +++ b/docs/es/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "Detecta al instante cuándo tus modelos, herramientas o hooks rale Detecta al instante cuándo tus modelos, herramientas o hooks ralentizan el sistema o disparan la factura, y anticipa un pico de latencia de cola antes de que tus usuarios lo noten. Tres páginas dedicadas convierten los tiempos brutos en p50, p95 y p99 que puedes leer de un vistazo. -![La página de Modelos con un mapa de calor de latencia, una banda de percentiles y datos de tokens, coste y ventana de contexto por modelo](/agenteye/images/models.png) +![La página de Modelos con un mapa de calor de latencia, una banda de percentiles y datos de tokens, coste y ventana de contexto por modelo](/cloud/images/models.png) *La página de Modelos: un mapa de calor de latencia, una banda de percentiles y, por modelo, tokens, coste estimado y ocupación de la ventana de contexto.* ## Deja de permitir que los promedios oculten tus peores ejecuciones @@ -24,13 +24,13 @@ Un crosshair de hover compartido vincula el mapa de calor y la banda, de modo qu La página de Modelos (mostrada arriba) responde las dos preguntas que siempre plantea una factura: qué modelo y cuánto. Además de la vista de latencia compartida, añade el **consumo de tokens por modelo**, el **coste estimado** y la **ocupación de la ventana de contexto**, de modo que el crecimiento desbocado de los prompts y una compactación inminente son visibles antes de que te sorprendan. -Failproof AI Observability reconoce los IDs de modelos más comunes automáticamente. Si una ventana aparece incorrecta o ejecutas un modelo privado propio, corrígelo o añade uno en **Settings**, en **model context windows**, y las lecturas de ocupación se actualizarán en consecuencia. +FailproofAI Cloud reconoce los IDs de modelos más comunes automáticamente. Si una ventana aparece incorrecta o ejecutas un modelo privado propio, corrígelo o añade uno en **Settings**, en **model context windows**, y las lecturas de ocupación se actualizarán en consecuencia. ## Herramientas: distingue lo lento de lo roto Una llamada a una herramienta puede ser lenta o puede estar fallando silenciosamente, y quieres saberlo en segundos, no después de revisar logs. -![La página de Herramientas con el mapa de calor de latencia y la banda de percentiles compartidos junto a un desglose de éxitos y fallos y una barra de distribución de herramientas](/agenteye/images/tools.png) +![La página de Herramientas con el mapa de calor de latencia y la banda de percentiles compartidos junto a un desglose de éxitos y fallos y una barra de distribución de herramientas](/cloud/images/tools.png) *La página de Herramientas: el mismo mapa de calor y banda de percentiles, más un desglose de éxitos y fallos y una barra de distribución de herramientas.* Junto a la vista de latencia compartida, la página de Herramientas añade un **desglose de éxitos y fallos** y una **barra de distribución de herramientas**, para que veas de un vistazo qué herramientas usas más y cuáles están consumiendo tu presupuesto de errores. @@ -39,14 +39,14 @@ Junto a la vista de latencia compartida, la página de Herramientas añade un ** Cuando un hook de ciclo de vida ralentiza una ejecución, "los hooks son lentos" no es algo sobre lo que puedas actuar. La página de Hooks te lleva directamente al que importa. -![La página de Hooks con la latencia desglosada por nombre de hook y evento disparador sobre el mapa de calor y la banda de percentiles compartidos](/agenteye/images/hooks.png) +![La página de Hooks con la latencia desglosada por nombre de hook y evento disparador sobre el mapa de calor y la banda de percentiles compartidos](/cloud/images/hooks.png) *La página de Hooks: latencia desglosada por nombre de hook y evento disparador.* Sobre el mismo mapa de calor de latencia y banda de percentiles, la página de Hooks desglosa la actividad por **nombre de hook** y **evento disparador**, para que llegues al hook concreto y al evento concreto que necesitan atención. ## Relacionado -- [Flujo de eventos](/es/agenteye/event-stream): el rastro en vivo con código de colores de cada evento. -- [Sesiones](/es/agenteye/sessions): agrupa los eventos en una fila por ejecución y abre su grafo de ejecución. -- [Seguimiento de errores](/es/agenteye/error-tracking): una única superficie de triaje para todo lo que el dashboard marca en rojo. -- [Dashboards](/es/agenteye/dashboards): vistas agregadas de toda tu flota. \ No newline at end of file +- [Flujo de eventos](/es/cloud/event-stream): el rastro en vivo con código de colores de cada evento. +- [Sesiones](/es/cloud/sessions): agrupa los eventos en una fila por ejecución y abre su grafo de ejecución. +- [Seguimiento de errores](/es/cloud/errors): una única superficie de triaje para todo lo que el dashboard marca en rojo. +- [Dashboards](/es/cloud/dashboards): vistas agregadas de toda tu flota. \ No newline at end of file diff --git a/docs/es/agenteye/queries.mdx b/docs/es/cloud/queries.mdx similarity index 84% rename from docs/es/agenteye/queries.mdx rename to docs/es/cloud/queries.mdx index 97c6f6b5..23e440d5 100644 --- a/docs/es/agenteye/queries.mdx +++ b/docs/es/cloud/queries.mdx @@ -6,7 +6,7 @@ description: "Haz cualquier pregunta sobre los datos de tu agente y obtén una r Haz cualquier pregunta sobre los datos de tu agente y obtén una respuesta en segundos. La observabilidad de Failproof AI te ofrece una biblioteca de consultas guardadas y listas para ejecutar sobre tus eventos y evaluaciones, para que partas de un ejemplo funcional en lugar de un editor SQL en blanco. -![La biblioteca de consultas guardadas: una cuadrícula de consultas reutilizables, tanto presets integrados como personalizados](/agenteye/images/queries.png) +![La biblioteca de consultas guardadas: una cuadrícula de consultas reutilizables, tanto presets integrados como personalizados](/cloud/images/queries.png) *Tu biblioteca de consultas guardadas en `//queries`: presets integrados junto a las consultas que tu equipo ha guardado.* @@ -22,7 +22,7 @@ Encuéntrala en `//queries`. Abre cualquier consulta y aterrizará en el compositor SQL, donde puedes modificarla y ver la respuesta de inmediato: sin exportaciones, sin viajes de ida y vuelta, sin esperar a nadie. -![El compositor de consultas SQL ejecutando una consulta guardada, con una barra lateral del esquema y una cuadrícula de resultados en vivo](/agenteye/images/query-lab.png) +![El compositor de consultas SQL ejecutando una consulta guardada, con una barra lateral del esquema y una cuadrícula de resultados en vivo](/cloud/images/query-lab.png) *El compositor SQL: tu consulta a la izquierda, una barra lateral del esquema para que nunca tengas que adivinar el nombre de una columna, y una cuadrícula de resultados en vivo debajo.* @@ -43,14 +43,14 @@ agenteye query list # las mismas consultas guardadas, desde tu t agenteye query run errs --arg prod # ejecuta una e imprime las filas (añade --json para redirigirla) ``` - Consulta [CLI y agentes](/es/agenteye/cli-and-agents) para ver el conjunto completo de comandos. + Consulta [CLI y agentes](/es/cloud/cli) para ver el conjunto completo de comandos. -- **Desde el asistente de IA.** ¿No sabes cómo formular el SQL? Pregúntale al [asistente de IA](/es/agenteye/assistant) dentro del dashboard en lenguaje natural y redactará la consulta y la guardará en tu biblioteca por ti. +- **Desde el asistente de IA.** ¿No sabes cómo formular el SQL? Pregúntale al [asistente de IA](/es/cloud/assistant) dentro del dashboard en lenguaje natural y redactará la consulta y la guardará en tu biblioteca por ti. Ejecutar una consulta guardada requiere el permiso `queries:run`, separado de los permisos para crear o eliminar consultas, para que puedas otorgar acceso de lectura sin permitir que todos reescriban la biblioteca. ## Relacionado -- [Dashboards](/es/agenteye/dashboards): fija los resultados de consultas en gráficos compartidos para toda la organización. -- [Asistente de IA](/es/agenteye/assistant): haz preguntas en lenguaje natural y obtén una consulta como respuesta. -- [CLI y agentes](/es/agenteye/cli-and-agents): ejecuta y guarda las mismas consultas desde tu terminal. \ No newline at end of file +- [Dashboards](/es/cloud/dashboards): fija los resultados de consultas en gráficos compartidos para toda la organización. +- [Asistente de IA](/es/cloud/assistant): haz preguntas en lenguaje natural y obtén una consulta como respuesta. +- [CLI y agentes](/es/cloud/cli): ejecuta y guarda las mismas consultas desde tu terminal. \ No newline at end of file diff --git a/docs/es/agenteye/python-sdk.mdx b/docs/es/cloud/sdk.mdx similarity index 93% rename from docs/es/agenteye/python-sdk.mdx rename to docs/es/cloud/sdk.mdx index 5cf34244..90b36214 100644 --- a/docs/es/agenteye/python-sdk.mdx +++ b/docs/es/cloud/sdk.mdx @@ -4,11 +4,11 @@ description: "Ve exactamente qué hicieron tus agentes de IA en producción: cad --- -Ve exactamente qué hicieron tus agentes de IA en producción: cada ejecución de agente, llamada a herramienta, solicitud al modelo, hook e intervención humana. El SDK de Observabilidad de Failproof AI para Python registra ese rastro desde dentro del código de tu agente para que puedas depurar, auditar y evaluar lo que ocurrió. Úsalo siempre que quieras que Failproof AI Observability observe tus agentes. +Ve exactamente qué hicieron tus agentes de IA en producción: cada ejecución de agente, llamada a herramienta, solicitud al modelo, hook e intervención humana. El SDK de Observabilidad de Failproof AI para Python registra ese rastro desde dentro del código de tu agente para que puedas depurar, auditar y evaluar lo que ocurrió. Úsalo siempre que quieras que FailproofAI Cloud observe tus agentes. Internamente, el SDK escribe eventos estructurados en archivos JSONL locales, y el daemon recolector los recoge y los envía a la plataforma de forma automática. No necesitas gestionar esos archivos tú mismo. -> **Sugerencia:** ¿Eres nuevo en Failproof AI Observability? Esta página es la referencia completa de eventos del SDK. +> **Sugerencia:** ¿Eres nuevo en FailproofAI Cloud? Esta página es la referencia completa de eventos del SDK.
@@ -26,7 +26,7 @@ Una vez instalado, confirma que lo tienes: python -c "import agenteye; print(agenteye.__version__)" ``` -¿Prefieres dejar que un agente de programación haga toda la integración? El [Python SDK Agent Skill](/es/agenteye/python-sdk-skill) conoce la ruta de instalación, planifica los puntos de instrumentación, los escribe y verifica que los eventos lleguen correctamente. +¿Prefieres dejar que un agente de programación haga toda la integración? El [Python SDK Agent Skill](/es/cloud/agent-skills) conoce la ruta de instalación, planifica los puntos de instrumentación, los escribe y verifica que los eventos lleguen correctamente. --- @@ -60,7 +60,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### Instrumentando una llamada real -En la práctica, envuelves tu código de agente existente. Enmarca una llamada al modelo con `model_request` antes y `model_response` después, de modo que los dos eventos abarquen la solicitud real y Failproof AI Observability pueda emparejarlos: +En la práctica, envuelves tu código de agente existente. Enmarca una llamada al modelo con `model_request` antes y `model_response` después, de modo que los dos eventos abarquen la solicitud real y FailproofAI Cloud pueda emparejarlos: ```python import anthropic @@ -99,7 +99,7 @@ Envuelve las llamadas a herramientas de la misma forma con `tool_use` y `tool_re Así es como se ven esos eventos una vez que llegan al panel de control, con código de colores por tipo y filtrables por entorno, agente y sesión: -![El flujo de eventos en vivo, con código de colores por tipo de evento y filtrable por entorno, agente y sesión](/agenteye/images/events-stream.png) +![El flujo de eventos en vivo, con código de colores por tipo de evento y filtrable por entorno, agente y sesión](/cloud/images/events-stream.png) --- @@ -156,9 +156,9 @@ Esto convierte la redacción en tu elección y tu responsabilidad. Si un prompt ## Referencia de eventos -La mayoría de los eventos vienen en pares inicio/fin que comparten un ID de correlación: `tool_use` y `tool_result` comparten un `tool_call_id`, `hook_triggered` y `hook_completed` comparten un `hook_id`, y `human_wait` y `human_input` comparten un `input_id`. Emite el evento de inicio, realiza el trabajo y luego emite el evento de fin con el mismo ID. Failproof AI Observability empareja los dos y calcula `duration_ms` por ti, por lo que nunca debes pasar `duration_ms` tú mismo. +La mayoría de los eventos vienen en pares inicio/fin que comparten un ID de correlación: `tool_use` y `tool_result` comparten un `tool_call_id`, `hook_triggered` y `hook_completed` comparten un `hook_id`, y `human_wait` y `human_input` comparten un `input_id`. Emite el evento de inicio, realiza el trabajo y luego emite el evento de fin con el mismo ID. FailproofAI Cloud empareja los dos y calcula `duration_ms` por ti, por lo que nunca debes pasar `duration_ms` tú mismo. -![El grafo de ejecución estilo git de una sesión junto a su línea de tiempo de eventos, reconstruido a partir de los eventos emparejados, con el panel de desglose de herramientas/modelo/hook](/agenteye/images/session-detail.png) +![El grafo de ejecución estilo git de una sesión junto a su línea de tiempo de eventos, reconstruido a partir de los eventos emparejados, con el panel de desglose de herramientas/modelo/hook](/cloud/images/session-detail.png) Todos los métodos de evento requieren estos dos campos: @@ -432,5 +432,5 @@ Cada archivo se escribe de forma atómica: el SDK escribe en un archivo temporal ## Próximos pasos -- [Flujo de eventos](/es/agenteye/event-stream): observa cómo llegan estos eventos en vivo, con código de colores y filtrables por entorno, agente y sesión. -- [Sesiones](/es/agenteye/sessions): ve cómo los eventos emparejados reconstruyen cada ejecución del agente como un grafo de ejecución y una línea de tiempo. \ No newline at end of file +- [Flujo de eventos](/es/cloud/event-stream): observa cómo llegan estos eventos en vivo, con código de colores y filtrables por entorno, agente y sesión. +- [Sesiones](/es/cloud/sessions): ve cómo los eventos emparejados reconstruyen cada ejecución del agente como un grafo de ejecución y una línea de tiempo. \ No newline at end of file diff --git a/docs/es/cloud/security.mdx b/docs/es/cloud/security.mdx new file mode 100644 index 00000000..61a243c7 --- /dev/null +++ b/docs/es/cloud/security.mdx @@ -0,0 +1,68 @@ +--- +title: "Seguridad" +description: "FailproofAI Cloud está diseñado para situarse cerca de tus agentes en producción, lo que significa que tiene acceso a tus prompts, entradas de herramientas y salidas." +--- + + +FailproofAI Cloud está diseñado para situarse cerca de tus agentes en producción, lo que significa que tiene acceso a tus prompts, entradas de herramientas y salidas. Esta página explica cómo mantiene esos datos aislados, bajo control y en tus manos. Si estás evaluando FailproofAI Cloud para una revisión de seguridad, comienza aquí. + +--- + +## Tus datos permanecen en tu entorno + +FailproofAI Cloud es autoalojado. Los eventos, prompts, respuestas del modelo y las analíticas se almacenan en tus propias bases de datos, en tu propio entorno. Nada se envía a un SaaS de terceros para su almacenamiento, y tus datos permanecen en tu propia cuenta en la nube. + +--- + +## Aislamiento de inquilinos + +Una sola instancia de FailproofAI Cloud puede alojar muchas organizaciones, y cada una está aislada a nivel de la capa de almacenamiento — aplicado por la base de datos, no solo por la interfaz de usuario: + +- Los datos operativos de una organización (usuarios, claves, paneles, consultas guardadas) están delimitados a esa organización, y las lecturas entre organizaciones están bloqueadas por la propia base de datos. +- Cada evento ingestado lleva el sello de la organización propietaria, por lo que los eventos de una organización nunca pueden ser leídos por otra. + +Cada ruta del panel está delimitada bajo un slug de organización (`//…`). + +--- + +## Inicio de sesión + +FailproofAI Cloud utiliza inicio de sesión sin contraseña, basado en correo electrónico. No hay contraseña que pueda ser objeto de phishing o filtrarse. Un usuario solicita un código de un solo uso (o un enlace mágico de un clic), que se envía por correo electrónico y expira rápidamente. El inicio de sesión está controlado por una **lista de permitidos**: solo las direcciones de correo electrónico (o dominios) que tú autorices pueden autenticarse. + +![La pantalla de inicio de sesión de FailproofAI Cloud, que envía un código de uso único a tu correo electrónico](/cloud/images/login.png) + +--- + +## Acceso delimitado con claves de API + +Cada cliente se autentica con una clave de API que lleva permisos granulares de mínimo privilegio. Un recopilador solo necesita `events:add`; una clave de panel o asistente puede ser de solo lectura; las acciones destructivas (eliminar, regenerar) son permisos separados que tú decides incluir. + +![La página de claves de API: los permisos de cada clave, codificados por color según el alcance de lectura, escritura y destructivo](/cloud/images/api-keys.png) + +Conserva la clave de arranque de administrador para la configuración, y emite claves con permisos reducidos para todo lo demás. Consulta [Claves de API](/es/cloud/access). + +--- + +## Un asistente de solo lectura con aprobación previa + +El [asistente de IA](/es/cloud/assistant) del panel responde preguntas sobre tus datos, pero está restringido por diseño: + +- Es **de solo lectura por defecto**: su SQL se ejecuta a través de un guardián que solo permite consultas `SELECT`/`WITH`, de una sola instrucción, con un límite de filas. +- Todo lo que crea (una consulta guardada, un panel) requiere **aprobación previa**: tú revisas y apruebas cada escritura antes de que ocurra. +- **Nunca puede eliminar**. + +Así, un compañero de equipo puede preguntar "¿qué agentes tuvieron más errores esta semana?" y actuar sobre la respuesta, sin que el asistente pueda modificar o eliminar tus datos por su cuenta. + +--- + +## En tránsito + +Todo el tráfico circula a través de HTTPS. Tú terminas el TLS con tus propios certificados, por lo que el tráfico entre el recopilador y el servidor, y entre el navegador y el servidor, está cifrado en tránsito. + +--- + +## Próximos pasos + +- [Descripción general](/es/cloud/overview): cómo encaja FailproofAI Cloud en conjunto. +- [Claves de API](/es/cloud/access): delimita el acceso para el recopilador, el panel y el asistente. +- [Observabilidad](/es/cloud/overview): qué captura FailproofAI Cloud de tus agentes. \ No newline at end of file diff --git a/docs/es/agenteye/sessions.mdx b/docs/es/cloud/sessions.mdx similarity index 82% rename from docs/es/agenteye/sessions.mdx rename to docs/es/cloud/sessions.mdx index 430ec88e..c230b923 100644 --- a/docs/es/agenteye/sessions.mdx +++ b/docs/es/cloud/sessions.mdx @@ -5,7 +5,7 @@ description: "Cada evento de una ejecución, resumido en una fila legible y repr Deja de adivinar por qué falló una ejecución. La Observabilidad de Failproof AI consolida cada evento de una ejecución en una fila legible y luego representa la ejecución completa como un diagrama al estilo git que puedes interpretar en segundos, para que veas exactamente qué hizo tu agente, paso a paso. -![La lista de Sesiones: una fila por ejecución, a través de entornos y agentes, con indicadores de estado y etiquetas de puntuación de evaluación](/agenteye/images/sessions-list.png) +![La lista de Sesiones: una fila por ejecución, a través de entornos y agentes, con indicadores de estado y etiquetas de puntuación de evaluación](/cloud/images/sessions-list.png) *Una fila por ejecución: el indicador de estado te dice cómo terminó la ejecución de un vistazo, y una etiqueta de puntuación aparece en cuanto conectas un evaluador.* @@ -29,7 +29,7 @@ Una vez que conectas un evaluador, cada ejecución completada recibe una puntuac ## Lee la ejecución completa como un diagrama -![El gráfico de ejecución al estilo git de una sesión junto a su cronología de eventos, con el panel de desglose de herramientas, modelos y hooks](/agenteye/images/session-detail.png) +![El gráfico de ejecución al estilo git de una sesión junto a su cronología de eventos, con el panel de desglose de herramientas, modelos y hooks](/cloud/images/session-detail.png) *El gráfico de ejecución (izquierda) aparece junto a la cronología de eventos; el panel derecho desglosa las herramientas, modelos, hooks y el consumo de tokens de la ejecución.* @@ -37,7 +37,7 @@ Haz clic en cualquier sesión para abrir su gráfico de ejecución: una vista al El panel derecho te ofrece el desglose por ejecución: qué herramientas y modelos se ejecutaron, qué hooks se activaron y cuántos tokens consumió la ejecución. Esa es la respuesta a "¿por qué costó tanto esta ejecución?" o "¿cuál es la herramienta más lenta?", justo al lado del gráfico que lo originó. -Los eventos individuales tienen su propia dirección, así que puedes pasarle a alguien un enlace a un momento concreto en lugar de "la sesión, más o menos a dos tercios". Copia el enlace desde cualquier evento, o síguelo desde un hallazgo de [auditoría](/es/agenteye/audits) o un error, y la sesión se abre con ese evento seleccionado y desplazado hasta él. Esto funciona también en ejecuciones muy largas: la cronología carga una ventana acotada por el bien de tu navegador, y un enlace que apunte más allá de esa ventana igualmente encontrará su evento en lugar de llevarte al inicio. Si el evento ha superado tu ventana de retención, la página te lo indica en lugar de seleccionar nada de forma silenciosa. +Los eventos individuales tienen su propia dirección, así que puedes pasarle a alguien un enlace a un momento concreto en lugar de "la sesión, más o menos a dos tercios". Copia el enlace desde cualquier evento, o síguelo desde un hallazgo de [auditoría](/es/cloud/audits) o un error, y la sesión se abre con ese evento seleccionado y desplazado hasta él. Esto funciona también en ejecuciones muy largas: la cronología carga una ventana acotada por el bien de tu navegador, y un enlace que apunte más allá de esa ventana igualmente encontrará su evento en lugar de llevarte al inicio. Si el evento ha superado tu ventana de retención, la página te lo indica en lugar de seleccionar nada de forma silenciosa. --- @@ -45,12 +45,12 @@ Los eventos individuales tienen su propia dirección, así que puedes pasarle a Cada página del panel de control está dentro del alcance de tu organización (`//…`). Sesiones se encuentra en **Observe** en la barra lateral izquierda, junto a Eventos, con los filtros de rango de fechas, entorno, agente y sesión en la parte superior de la lista. Cada fila está a un clic de su gráfico de ejecución completo. -Para activar las etiquetas de puntuación y el filtrado por rango de puntuación, conecta un evaluador: consulta [Evaluaciones](/es/agenteye/evaluations). +Para activar las etiquetas de puntuación y el filtrado por rango de puntuación, conecta un evaluador: consulta [Evaluaciones](/es/cloud/evaluations). --- ## Relacionado -- [Flujo de eventos](/es/agenteye/event-stream): el registro en bruto por paso del que se compila cada sesión. -- [Evaluaciones](/es/agenteye/evaluations): conecta un evaluador para que cada ejecución obtenga una etiqueta de puntuación por la que puedas filtrar. -- [Telemetría](/es/agenteye/telemetry): cómo pasan las ejecuciones de tu agente a estas sesiones. \ No newline at end of file +- [Flujo de eventos](/es/cloud/event-stream): el registro en bruto por paso del que se compila cada sesión. +- [Evaluaciones](/es/cloud/evaluations): conecta un evaluador para que cada ejecución obtenga una etiqueta de puntuación por la que puedas filtrar. +- [Telemetría](/es/cloud/performance): cómo pasan las ejecuciones de tu agente a estas sesiones. \ No newline at end of file diff --git a/docs/es/concepts.mdx b/docs/es/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/es/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/es/daemon.mdx b/docs/es/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/es/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/es/dashboard.mdx b/docs/es/dashboard.mdx index fa9b207e..dfcdbc75 100644 --- a/docs/es/dashboard.mdx +++ b/docs/es/dashboard.mdx @@ -69,7 +69,7 @@ Un informe con personalidad sobre cómo se ha comportado realmente tu agente a l 4. **Cómo mejorar** — lista de filas, una por política prescrita: nombre de la política en blanco, descripción de una línea, comando de instalación + botón de copiar a la derecha. El encabezado de la sección muestra `enable all N → projected · ` (la puntuación que alcanzarías con todas las correcciones aplicadas), y su botón `[install all]` copia el comando combinado `failproofai policy add a b c …` para cada política prescrita. 5. **Vuelve mejor** — dos tarjetas lado a lado. Izquierda: establece un recordatorio (selector de cadencia `3d` / `7d` / `14d` / `30d`; persiste mediante `/api/auth/reminder` una vez autenticado). Derecha: desbloquea ventajas de failproof — `invite a friend` abre un modal que acepta una lista separada por comas/espacios/saltos de línea de correos electrónicos de amigos (máximo 10 por envío), los envía mediante POST a `/api/audit/invite`, que los reenvía al `POST /v0/invite` del api-server. El api-server envía un correo electrónico por destinatario desde `invite@failproof.ai` con el remitente en Cc y `Reply-To` configurado, para que el destinatario vea quién lo invitó y el remitente reciba una copia en su bandeja de entrada. Los usuarios anónimos son redirigidos primero a través del `AuthDialog` para que el correo del remitente sea conocido antes de que salgan las invitaciones. El cumplimiento de derechos/ventajas es un seguimiento pendiente. -Impulsado por el runtime de `failproofai audit` — consulta [Audit CLI](/es/cli/audit) para el motor de escaneo subyacente, los flags compatibles y los invariantes de caché por transcripción. El dashboard almacena en caché el último resultado en `~/.failproofai/audit-dashboard.json` (modo `0600`, una sola ranura, las nuevas ejecuciones sobreescriben) para que las revisitas sean instantáneas; **tanto la caché por transcripción como la caché del resultado completo se rechazan al leerlas una vez que tienen más de 7 días**, por lo que el dashboard nunca sirve silenciosamente un resultado de una semana — pasado el TTL, `/audit` cae a su estado vacío y solicita una nueva ejecución. Al hacer clic en `[ re-audit now ]` cerca de la parte inferior del informe se envía un POST a `/api/audit/run` con `noCache: true` — la re-auditoría omite la caché por transcripción y vuelve a escanear cada transcripción desde cero en lugar de devolver silenciosamente el resultado en caché — y el dashboard consulta `/api/audit/status` a 1 Hz hasta que la ejecución finaliza; una barra de progreso rosa pegajosa se fija en la parte superior del viewport durante la ejecución con un temporizador transcurrido, y el nuevo resultado reemplaza al anterior en el lugar cuando tiene éxito (sin recarga de página completa; una re-auditoría fallida deja el informe anterior intacto). En caso de fallo, la barra se vuelve roja con texto basado en el `RerunError.kind` (`timeout` / `network` / `post_failed`). El estado vacío (sin caché o caducada) y el estado de cero sesiones (caché existe pero el escaneo no encontró transcripciones) se muestran por separado. +Impulsado por el runtime de `failproofai audit` — consulta [Audit CLI](/es/audit) para el motor de escaneo subyacente, los flags compatibles y los invariantes de caché por transcripción. El dashboard almacena en caché el último resultado en `~/.failproofai/audit-dashboard.json` (modo `0600`, una sola ranura, las nuevas ejecuciones sobreescriben) para que las revisitas sean instantáneas; **tanto la caché por transcripción como la caché del resultado completo se rechazan al leerlas una vez que tienen más de 7 días**, por lo que el dashboard nunca sirve silenciosamente un resultado de una semana — pasado el TTL, `/audit` cae a su estado vacío y solicita una nueva ejecución. Al hacer clic en `[ re-audit now ]` cerca de la parte inferior del informe se envía un POST a `/api/audit/run` con `noCache: true` — la re-auditoría omite la caché por transcripción y vuelve a escanear cada transcripción desde cero en lugar de devolver silenciosamente el resultado en caché — y el dashboard consulta `/api/audit/status` a 1 Hz hasta que la ejecución finaliza; una barra de progreso rosa pegajosa se fija en la parte superior del viewport durante la ejecución con un temporizador transcurrido, y el nuevo resultado reemplaza al anterior en el lugar cuando tiene éxito (sin recarga de página completa; una re-auditoría fallida deja el informe anterior intacto). En caso de fallo, la barra se vuelve roja con texto basado en el `RerunError.kind` (`timeout` / `network` / `post_failed`). El estado vacío (sin caché o caducada) y el estado de cero sesiones (caché existe pero el escaneo no encontró transcripciones) se muestran por separado. ### Políticas diff --git a/docs/es/architecture.mdx b/docs/es/how-it-works.mdx similarity index 100% rename from docs/es/architecture.mdx rename to docs/es/how-it-works.mdx diff --git a/docs/es/introduction.mdx b/docs/es/introduction.mdx index aafd134f..901e0d48 100644 --- a/docs/es/introduction.mdx +++ b/docs/es/introduction.mdx @@ -54,4 +54,4 @@ failproofai policies --install # enable policies (or skip — `failproofai` wi failproofai # launch the dashboard ``` -Consulta la guía de [Primeros pasos](/es/getting-started) para ver el tutorial completo. \ No newline at end of file +Consulta la guía de [Primeros pasos](/es/quickstart) para ver el tutorial completo. \ No newline at end of file diff --git a/docs/es/policies.mdx b/docs/es/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/es/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/es/getting-started.mdx b/docs/es/quickstart.mdx similarity index 100% rename from docs/es/getting-started.mdx rename to docs/es/quickstart.mdx diff --git a/docs/es/reference/files.mdx b/docs/es/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/es/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/for-agents.mdx b/docs/for-agents.mdx index 13e0f785..4635c860 100644 --- a/docs/for-agents.mdx +++ b/docs/for-agents.mdx @@ -15,13 +15,14 @@ npx skills add https://docs.befailproof.ai | Area | What's included | |------|----------------| -| Policies | Built-in policy names, event types, parameters, enable/disable | +| Policies | All 39 built-in policy names, event types, parameters, enable/disable | | Custom policies | `customPolicies.add()`, match filters, `allow`/`deny`/`instruct` API | | Context object | `ctx.eventType`, `ctx.toolName`, `ctx.toolInput`, `ctx.session` | | Configuration | `policies-config.json` structure, scope merging, `policyParams` | -| CLI | `failproofai policies --install`, `--uninstall`, `--custom`, scopes | -| Dashboard | Session viewer, policy activity, environment variables | -| Architecture | Hook handler flow, exit codes, stdin/stdout contract | +| CLI | Every `failproofai` command and flag, including `config`, `harness`, `backfill` | +| Agent support | Which CLIs exist, and what a deny can actually block on each | +| Cloud | Connecting a machine, managed policies, the observability surfaces | +| Mechanics | Hook flow, the daemon, exit codes, the stdin/stdout contract | ## Is the skill complete? @@ -36,3 +37,9 @@ npx skills add https://docs.befailproof.ai/custom-policies # Just the built-in policies npx skills add https://docs.befailproof.ai/built-in-policies ``` + + + Looking for skills that let an agent *operate* FailproofAI Cloud, instrument your own + agents, or build an evaluator? Those are separate, purpose-built skills — see [Agent + skills](/cloud/agent-skills). + diff --git a/docs/fr/agent-support.mdx b/docs/fr/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/fr/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/fr/agenteye/cli-and-agents.mdx b/docs/fr/agenteye/cli-and-agents.mdx deleted file mode 100644 index 3010ba79..00000000 --- a/docs/fr/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,79 +0,0 @@ ---- -title: "CLI" -description: "Tout votre déploiement Failproof AI Observability, à portée d'une seule commande." ---- - -Tout votre déploiement Failproof AI Observability, à portée d'une seule commande. Vérifiez la production, créez une clé API ou acquittez un incident sans quitter votre terminal, puis scriptez n'importe quelle opération dans votre CI, ou laissez un agent de code s'en charger en langage naturel. - -```bash -pipx install agenteye -agenteye login --email vous@exemple.com # un code à 6 chiffres arrive dans votre boîte mail -agenteye --json sessions --since 24h # toutes les exécutions d'agents des dernières 24h, les plus récentes en premier -``` - -*Le CLI `agenteye` communique avec votre tableau de bord. C'est un outil distinct du collecteur, qui achemine les événements vers le serveur.* - -## Tout votre déploiement, une seule commande suffit - -Fini de jongler entre les onglets pour répondre à une simple question. Le CLI `agenteye` lit vos données et administre votre organisation depuis un seul binaire : une vérification qui nécessitait auparavant de naviguer dans le tableau de bord devient une ligne que vous pouvez relancer, mettre en alias ou coller dans un runbook. Quatre surfaces sont à votre disposition : - -- **Lire vos données :** `sessions`, `events`, `evals` et `errors`, filtrés par plage horaire, agent et environnement. -- **Gérer votre organisation :** `keys`, `users`, `settings`, `alerts` et `incidents`. -- **Lancer des analyses :** requêtes SQL enregistrées et un runner `query` ad hoc sur vos données d'événements. -- **Interroger l'assistant :** `agent ask` accède au même analyste en lecture seule que celui disponible dans le tableau de bord. - -Installez-le une fois avec `pipx`, connectez-vous via un code à 6 chiffres reçu par e-mail, et vous êtes prêt. La session dure environ une journée ; relancez `agenteye login` à son expiration. Utilisez-le pour contrôler la production, provisionner une clé ou trier un incident actif, sans jamais ouvrir un navigateur : - -```bash -agenteye errors --since 24h --aggregate # ce qui est en erreur, regroupé par type -agenteye incidents list --state firing # ce qui est en feu en ce moment -agenteye keys create ci --add events:add # une clé qui ne peut qu'envoyer des événements, secret affiché une seule fois -``` - -Un point important à retenir : les options globales comme `--json` se placent avant la commande. `agenteye --json sessions` est correct ; `agenteye sessions --json` ne l'est pas. - -## Scriptez-le, intégrez-le dans votre CI - -Chaque commande accepte `--json`, et cela change tout. Le JSON brut part sur stdout tandis que les messages de statut et les avertissements destinés à l'humain vont sur stderr — une capture avec `--json` s'envoie donc directement dans `jq` sans ligne parasite à éliminer. C'est ce qui rend le CLI aussi efficace pour vous à l'invite de commande que pour un agent de code qui parse les sorties : - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -Il est conçu pour fonctionner sans surveillance. Les confirmations interactives sont automatiquement ignorées lorsqu'aucun terminal n'est attaché, rien ne bloque donc dans un pipeline, et chaque commande retourne un code de sortie explicite : `0` succès, `4` non connecté, `5` permission manquante (le message la précise, par exemple `alerts:write`), `3` tableau de bord inaccessible. Un script peut brancher sur un `4` pour se réauthentifier, ou sur un `5` pour vous indiquer exactement ce qu'il faut demander à un administrateur — au lieu d'échouer silencieusement. - -## Laissez un agent de code piloter le CLI en langage naturel - -Mieux encore, vous ne devriez pas avoir à retenir tous ces drapeaux. La **compétence CLI** est un petit dossier Agent Skill nommé `agenteye-cli` qui apprend à un agent de code tel que Claude Code ou Codex à piloter le CLI à partir de requêtes en langage naturel. Demandez « est-ce que quelque chose est cassé aujourd'hui ? » et l'agent choisit la commande, l'exécute en votre nom et répond en prose. - -Pour Claude Code, déposez le dossier `agenteye-cli` dans `~/.claude/skills/` et il est découvert automatiquement. Failproof AI Observability fournit le dossier ; il n'y a rien de supplémentaire à installer, car il se contente de piloter le CLI que vous avez déjà installé. Connectez-vous vous-même au préalable : la compétence ne peut pas effectuer à votre place la connexion par code reçu par e-mail. - -Comme l'agent exécute le CLI en votre nom, il peut tout faire dans les limites de votre session — lectures et écritures comprises : créer des clés, modifier des paramètres, résoudre des incidents. La confirmation interactive du CLI ne se déclenche pas pour un agent, c'est pourquoi la compétence est conçue pour indiquer la commande exacte et attendre votre accord avant tout changement. Vous êtes l'étape de confirmation. - -```text -vous Pourquoi la session run-001 a-t-elle échoué ? - -agent Exécution : agenteye --json events --session-id run-001 --all - Le checkout-agent a rencontré une TimeoutError lors de son troisième appel d'outil. -``` - -Les lectures restent instantanées, et chaque écriture marque une pause pour vous : - -```text -vous Donne à CI une clé qui ne peut qu'envoyer des événements. - -agent Ceci va créer une clé API. Je vais exécuter : - agenteye keys create ci --add events:add - Confirmer ? - -vous oui - -agent Fait. Clé "ci" créée avec events:add uniquement. Le secret est affiché une seule fois, enregistrez-le maintenant. -``` - -## En relation - -- [Référence CLI](/fr/agenteye/cli) : toutes les commandes, options et structures JSON. -- [Recettes CLI pour agents](/fr/agenteye/cli-recipes) : patterns `jq` prêts à l'emploi et gestion des codes de sortie. -- [Compétence agent CLI](/fr/agenteye/cli-skill) : installation et utilisation de la compétence `agenteye-cli`. -- [Assistant IA](/fr/agenteye/assistant) : l'analyste intégré au tableau de bord que `agent ask` interroge. \ No newline at end of file diff --git a/docs/fr/agenteye/cli-skill.mdx b/docs/fr/agenteye/cli-skill.mdx deleted file mode 100644 index 89bb85da..00000000 --- a/docs/fr/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Compétence d'agent CLI Failproof AI Observability" -description: "Demandez à votre agent de développement « est-ce que quelque chose est cassé aujourd'hui ? » et laissez-le répondre à partir de vos données Failproof AI Observability en direct, sans aucune commande à mémoriser." ---- - - -Demandez à votre agent de développement *« est-ce que quelque chose est cassé aujourd'hui ? »* et laissez-le répondre à partir de vos données Failproof AI Observability en direct, sans aucune commande à mémoriser. La **compétence CLI Failproof AI Observability** (`agenteye-cli`) est une *compétence d'agent* : un petit dossier d'instructions qu'un agent de développement comme Claude Code ou Codex charge à la demande. Elle apprend à l'agent à piloter votre déploiement Observability via le [CLI `agenteye`](/fr/agenteye/cli) à partir de requêtes en langage naturel comme *« donne à la CI une clé qui ne peut qu'envoyer des événements »* ou *« acquitte l'incident en cours et assigne-le moi »*. - -Il **ne s'agit pas** d'un service ni d'un binaire distinct ; il n'y a rien à déployer. La compétence s'appuie sur le CLI déjà installé : l'agent exécute `agenteye --json …`, analyse le JSON propre renvoyé et vous répond en texte clair. Tout ce qu'elle peut faire, vous pourriez le faire vous-même en tapant les mêmes commandes. - ---- - -## Relation avec les autres interfaces Failproof AI Observability - -Failproof AI Observability vous offre quatre façons d'accéder aux mêmes données et contrôles. Elles se complètent : - -| Interface | Description | Où elle s'exécute | Utilisez-la quand | -|---|---|---|---| -| **[CLI](/fr/agenteye/cli)** | La référence des commandes et options pour `agenteye` | Votre terminal | Vous voulez exécuter ou scripter une commande précise | -| **[Recettes CLI](/fr/agenteye/cli-recipes)** | Modèles `jq`/pipeline à copier-coller | Votre terminal / scripts | Vous intégrez le CLI dans de l'automatisation | -| **Compétence CLI** (ce document) | Une porte d'entrée en langage naturel sur le CLI | Votre agent de développement, sur votre poste | Vous voulez *poser la question* et laisser l'agent choisir la commande | -| **[Compétence Evaluator](/fr/agenteye/evaluator-skill)** | Une compétence jumelle qui conçoit et construit votre service de scoring | Votre agent de développement, sur votre poste | Vous voulez *produire* des scores d'évaluation plutôt que les lire | -| **[Compétence SDK Python](/fr/agenteye/python-sdk-skill)** | Une compétence jumelle qui instrumente votre agent pour qu'il émette de la télémétrie | Votre agent de développement, sur votre poste | Vous voulez que votre agent *produise* les événements que cette compétence lit | -| **[Assistant IA intégré au tableau de bord](/fr/agenteye/assistant)** | Un chat intégré au tableau de bord | Côté serveur (dans le tableau de bord) | Vous voulez des questions-réponses sur vos données directement dans le tableau de bord | - -La compétence elle-même n'a aucun privilège propre ; elle se contente de transformer vos mots en appels CLI qui s'exécutent en tant que vous : - -```mermaid -flowchart TD - YOU["vous : 'acquitte l'incident en cours'"] --> AGENT["agent de développement (Claude Code / Codex)
charge la compétence agenteye-cli"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|votre session CLI authentifiée| API["API du tableau de bord Observability"] -``` - -### vs. l'assistant IA intégré au tableau de bord : une distinction importante - -Ce sont deux outils différents avec des périmètres d'action très différents : - -- L'**assistant IA intégré au tableau de bord** ([assistant IA](/fr/agenteye/assistant)) est un chat intégré au tableau de bord, alimenté par le service d'agent. Il est **en lecture seule avec création soumise à validation** : il peut rédiger des requêtes sauvegardées et des tableaux de bord, mais chaque écriture s'arrête pour demander votre approbation explicite, et il ne supprime jamais rien. Il est conditionné à la permission `agent:use` et ne voit jamais que les données de l'organisation que vous consultez. -- La **compétence CLI** s'exécute sur *votre* poste, dans *votre* agent de développement, et pilote le CLI `agenteye` en tant que **vous**. Elle peut utiliser **toute la surface du CLI, y compris les mutations** (créer/alterner/désactiver des clés API, modifier les paramètres d'organisation, résoudre des incidents, supprimer des requêtes sauvegardées), limitée uniquement par les permissions de votre connexion CLI. Traitez-la exactement avec la même prudence que si vous tapiez ces commandes vous-même. - ---- - -## Prérequis - -1. Le **CLI `agenteye` installé** et dans le `PATH` (voir la référence [CLI](/fr/agenteye/cli) : `pipx install agenteye`). -2. Votre **URL de tableau de bord** configurée (`AGENTEYE_DASHBOARD_URL`, ou l'agent passe `--base-url`). -3. Une **session connectée** : exécutez `agenteye login` vous-même au préalable. La compétence **ne peut pas** effectuer la connexion par code à usage unique envoyé par e-mail à votre place ; elle vous indiquera d'exécuter `agenteye login` si la session est manquante ou expirée (code de sortie CLI `4`). - ---- - -## Où la trouver - -La compétence est publiée dans la collection publique de compétences de Failproof AI : - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -Rien n'est restreint — le dépôt est public et la compétence n'a besoin d'aucun identifiant propre, car elle ne fait que piloter le CLI `agenteye` **public** contre *votre* tableau de bord, en utilisant la session avec laquelle *vous* vous êtes connecté. Vous n'avez besoin de la demander à personne. - -Notez qu'elle est distribuée dans son propre dossier et **n'est pas** incluse dans le paquet `pipx install agenteye`, donc ne la cherchez pas là. - -## Installation de la compétence - -Le chemin le plus rapide est le CLI [`skills`](https://skills.sh), qui récupère le dossier et le place là où votre agent le cherche : - -```bash -# Claude Code, ce projet uniquement -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# tous les projets (installe dans ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex à la place -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -Gérez-la ensuite comme n'importe quelle autre compétence : - -```bash -npx skills list -a claude-code # ce qui est installé -npx skills update agenteye-cli # récupérer la dernière version -npx skills remove agenteye-cli # la supprimer -``` - -Vous préférez installer manuellement ? Une compétence d'agent est simplement un dossier contenant un `SKILL.md` (plus des références optionnelles), donc la copier fonctionne également : - -- **Claude Code** : placez le dossier `agenteye-cli/` dans `~/.claude/skills/` (tous les projets) ou `/.claude/skills/` (ce dépôt uniquement). Claude Code la détecte automatiquement — vérifiez avec la liste `/skills`, ou posez simplement une question correspondant à sa description. -- **Codex (OpenAI)** : Codex lit le même `SKILL.md`. Le fichier `agents/openai.yaml` inclus définit `allow_implicit_invocation: true`, donc Codex sélectionne automatiquement la compétence quand une tâche correspond ; sinon, invoquez-la explicitement avec `$agenteye-cli`. - ---- - -## Sécurité : les mutations ne demandent PAS confirmation quand un agent exécute le CLI - -> **Avertissement :** Lisez ceci avant de laisser un agent effectuer des modifications. - -Le CLI `agenteye` demande normalement *« êtes-vous sûr ? »* avant une action destructive. Il **saute automatiquement cette confirmation dès qu'il n'est pas attaché à un terminal (ce qui correspond exactement à la façon dont un agent de développement l'exécute), et `--json` la saute également.** Ainsi, la demande de confirmation **ne se déclenchera pas** pour l'agent. - -La compétence est conçue pour compenser : elle est instruite d'énoncer la commande exacte qu'elle va exécuter et d'obtenir votre **accord explicite avant tout changement d'état**. Maintenez cette discipline. Lorsque vous pilotez Failproof AI Observability via un agent, *vous* êtes l'étape de confirmation. Les commandes modifiant l'état à surveiller : - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- les sous-commandes `incidents` en écriture : `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -Tout ce qui relève de la section **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) est en lecture seule et ne modifie rien. - -L'agent agissant en tant que **vous**, il ne peut faire que ce que votre connexion est autorisée à faire ; les permissions sont résolues **par organisation** (voir [Clés API](/fr/agenteye/api-keys)). Une commande pour laquelle vous n'avez pas la permission renvoie le code de sortie `5` avec le nom exact de la permission manquante, ce qui permet à l'agent de vous indiquer précisément ce qu'il faut demander à un administrateur plutôt que d'échouer silencieusement. - ---- - -## Ce que vous pouvez lui demander - -Voici à quoi ressemble un vrai échange — une lecture, puis une modification qui attend votre accord : - -```text -vous ▸ Est-ce que quelque chose est cassé ces dernières 24 heures ? - -agent ▸ Exécution : agenteye --json errors --since 24h --aggregate - 3 types d'erreurs dans les dernières 24h. Le plus fréquent est TimeoutError - (18 occurrences, toutes sur le checkout-agent), puis RateLimited (5). - Voulez-vous les sessions en échec pour TimeoutError ? - -vous ▸ Donne à la CI une clé qui ne peut qu'envoyer des événements. - -agent ▸ Cela crée une clé API. Je vais exécuter : - agenteye keys create ci --add events:add - D'accord pour continuer ? - -vous ▸ oui - -agent ▸ Fait. Clé « ci » créée avec events:add uniquement. - Le secret n'est affiché qu'une seule fois, enregistrez-le maintenant. Je ne peux pas le réafficher. -``` - -La compétence associe chaque intention exprimée en langage naturel à la bonne commande `agenteye`, en découvrant d'abord les valeurs valides (`list `, `whoami`) pour ne pas deviner, et en énonçant la commande exacte avant tout changement. Quelques exemples supplémentaires : - -- *« Est-ce que quelque chose est cassé / en échec ces dernières 24 heures ? »* → `errors --since 24h --aggregate`, puis un récapitulatif. -- *« Pourquoi la session `run-001` a-t-elle échoué ? »* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *« Comment évolue la qualité cette semaine ? »* → `evals --aggregate --since 7d`, puis exploration des exécutions avec les scores les plus bas. -- *« Donne à la CI une clé qui ne peut qu'envoyer des événements. »* → `keys create ci --add events:add` (elle énonce la commande, puis la crée et capture le secret à usage unique). -- *« Qui a accès ? Mets Dana en lecture seule. »* → `users list` → `users update dana@… --permission-set read-only` (après confirmation de votre part). -- *« Acquitte l'incident en cours et assigne-le moi. »* → `incidents list --state firing` → `incidents ack ` / `incidents assign vous@…`. - -Pour les commandes exactes, les options et les structures JSON correspondantes, consultez la référence [CLI](/fr/agenteye/cli) et les [recettes CLI pour agents](/fr/agenteye/cli-recipes). - ---- - -## Prochaines étapes - -- **[CLI](/fr/agenteye/cli)** : référence complète des commandes et options pour `agenteye`. -- **[Recettes CLI pour agents](/fr/agenteye/cli-recipes)** : modèles `jq` à copier-coller et gestion des codes de sortie. -- **[Compétence d'agent Evaluator](/fr/agenteye/evaluator-skill)** : la compétence jumelle, pour construire l'évaluateur dont les scores sont lus par `agenteye evals`. -- **[Compétence d'agent SDK Python](/fr/agenteye/python-sdk-skill)** : la compétence jumelle, pour instrumenter un agent afin qu'il émette la télémétrie lue par `agenteye`. -- **[Assistant IA](/fr/agenteye/assistant)** : l'assistant intégré au tableau de bord (à ne pas confondre avec cette compétence en ligne de commande). -- **[Clés API](/fr/agenteye/api-keys)** : le modèle de permissions par organisation qui délimite ce que la compétence peut faire. \ No newline at end of file diff --git a/docs/fr/agenteye/codex-capture.mdx b/docs/fr/agenteye/codex-capture.mdx deleted file mode 100644 index 22017495..00000000 --- a/docs/fr/agenteye/codex-capture.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "Capture de session Codex" -description: "Transmettez les sessions OpenAI Codex locales de votre équipe vers AgentEye sous forme de sessions et d'événements ordinaires — sans modifier leur façon d'utiliser Codex." ---- - -Vos ingénieurs utilisent déjà OpenAI Codex au quotidien. La capture de sessions Codex importe ces sessions de codage dans AgentEye sous forme de sessions et d'événements ordinaires, afin que vous puissiez les rechercher, les rejouer et les évaluer aux côtés de tout ce que vous observez par ailleurs. Cette fonctionnalité complète le [SDK Python](/fr/agenteye/python-sdk) : le SDK instrumente les agents que vous écrivez, tandis que la capture récupère le travail Codex que votre équipe effectue déjà — sans aucune modification de leur façon de l'utiliser. - -Un petit collecteur en arrière-plan lit les transcripts de sessions locaux de Codex au fur et à mesure de leur écriture et les envoie vers AgentEye. Un seul collecteur par machine capture simultanément toutes les surfaces Codex locales — aucune configuration par surface n'est nécessaire. - -Ce même collecteur capture également d'autres agents — voir [OpenClaw](/fr/agenteye/openclaw-capture) et [Hermes](/fr/agenteye/hermes-capture). Activez chacun de ceux que vous utilisez ; un seul collecteur peut en capturer plusieurs à la fois. - ---- - -## Ce qui est capturé - -Chaque surface Codex fonctionnant **localement** produit les mêmes transcripts de session sur disque, et le collecteur les récupère tous : - -- le **CLI** Codex et `codex exec` -- l'**extension VS Code / IDE** -- l'**application de bureau**, lorsqu'elle exécute une session localement - -Chaque session Codex devient une [session](/fr/agenteye/sessions) AgentEye ; ses messages utilisateur et assistant, son raisonnement, ses appels d'outils, ses résultats d'outils et son utilisation des tokens deviennent les [événements](/fr/agenteye/event-stream) correspondants. La surface d'origine de chaque session (CLI, IDE ou bureau) est enregistrée, ce qui vous permet de les distinguer. - -> **Les sessions cloud ne sont pas capturées.** L'application de bureau exécute de plus en plus de sessions dans le cloud Codex et ne conserve que leurs métadonnées sur la machine — il n'existe aucun transcript local à lire. Seules les sessions exécutées localement sont capturées. - ---- - -## Activation - -La capture est désactivée jusqu'à ce que vous l'activiez. Installez le collecteur avec une clé API disposant de la permission `events:add` (voir [Clés API](/fr/agenteye/api-keys)), puis activez la capture Codex : - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -Cette commande installe le collecteur, l'enregistre en tant que service en arrière-plan et démarre la capture. Vérifiez qu'il fonctionne : - -```bash -agenteye-collector health -``` - -Au premier démarrage, vos sessions Codex existantes sont importées rétroactivement une seule fois, puis la nouvelle activité est transmise en quelques secondes. Les fichiers de Codex ne sont qu'en lecture seule — ils ne sont jamais modifiés, déplacés ni supprimés — et chaque session est envoyée exactement une fois, même en cas de redémarrage. - ---- - -## Où retrouver les données - -Les sessions capturées apparaissent dans **Sessions**, et leurs événements dans le flux **Events**, de la même façon que tout autre agent observé — ainsi, la [relecture de session](/fr/agenteye/sessions), la [recherche](/fr/agenteye/queries), les [évaluations](/fr/agenteye/evaluations) et les [alertes](/fr/agenteye/alerts) fonctionnent toutes avec elles. Filtrez par agent Codex pour les afficher séparément. - ---- - -## Confidentialité - -Les transcripts Codex contiennent l'intégralité de la session — y compris les sorties de commandes, le contenu des fichiers et tout ce que Codex a lu ou écrit — et peuvent contenir des secrets. Les sessions capturées sont transmises telles quelles ; n'activez donc la capture que sur les machines et pour les équipes pour lesquelles la centralisation de ce contenu dans AgentEye est appropriée, et fournissez au collecteur une clé dont la portée se limite à `events:add`. Consultez [Sécurité](/fr/agenteye/security) pour en savoir plus sur l'isolation de vos données. \ No newline at end of file diff --git a/docs/fr/agenteye/concepts.mdx b/docs/fr/agenteye/concepts.mdx deleted file mode 100644 index 9343dadf..00000000 --- a/docs/fr/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "Concepts" -description: "Le vocabulaire de Failproof AI Observability — événements, sessions, évaluations, audits, findings et incidents — défini en un seul endroit." ---- - - -Cette page définit le vocabulaire utilisé par Failproof AI Observability. Si un terme vous est inconnu dans un autre guide, il est défini ici. Inutile de la lire en entier : parcourez-la en diagonale, ou revenez-y dès qu'un mot mérite d'être précisé. - ---- - -## Le modèle de données - -**Événement** -La plus petite unité de données. Un événement enregistre une seule étape effectuée par votre agent : un `tool_use`, un `model_request`, un `hook_completed`, une `error`, etc. Votre agent émet des événements via le [Python SDK](/fr/agenteye/python-sdk) ; ils apparaissent en temps réel sur la page **Events**. - -**Session** -Une exécution d'agent, identifiée par un `session_id`. Une session regroupe tous les événements partageant cet identifiant, consolidés en une seule ligne sur la page **Sessions** et représentés sous forme de graphe d'exécution sur sa page de détail. Une session commence généralement par `agent_start` et se termine par `agent_end`. - -**Agent** -Un acteur nommé au sein d'une exécution, identifié par un `agent_id`. Une exécution peut impliquer plusieurs agents : par exemple, un planificateur qui instancie un sous-agent de synthèse. Les sous-agents portent un `parent_id`, ce qui permet à Failproof AI Observability de les représenter sur leurs propres pistes dans le graphe d'exécution. - -**Environnement** -Un libellé indiquant où s'est déroulée l'exécution : `production`, `staging`, `dev`. Vous le définissez une seule fois lors de la configuration du SDK. Presque toutes les pages du tableau de bord permettent de filtrer par environnement. - -**Taux de remplissage de la fenêtre de contexte** -Le pourcentage de la fenêtre de contexte d'un modèle consommé par une réponse. Failproof AI Observability l'horodate sur les événements `model_response` pour les modèles qu'il reconnaît, rendant ainsi visibles la croissance des prompts et les compactions imminentes directement dans le flux d'événements. - ---- - -## Qualité - -**Évaluation** -Un score de qualité pour une session terminée, produit par un service de notation que vous exécutez. Les évaluations sont optionnelles : tant que vous ne connectez pas d'évaluateur, les sessions sont enregistrées mais pas notées. Chaque évaluation peut comporter plusieurs scores nommés (par exemple `helpfulness`, `factuality`, `tool_efficiency`), chacun accompagné d'une courte note explicative. Voir [Evaluation suite](/fr/agenteye/evaluation-suite). - -**Clé de score** -Le nom d'une dimension rapportée par un évaluateur, comme `helpfulness`. Les alertes et les audits peuvent surveiller une clé de score spécifique dans le temps. - -**Évaluateur** -Votre service de notation. Failproof AI Observability lui envoie via POST la transcription d'une exécution terminée et stocke les scores renvoyés. Aucun évaluateur par défaut n'est fourni ; la logique de notation vous appartient. - ---- - -## Identifier et corriger les défaillances - -**Hook** -Un garde-fou ou un effet secondaire que votre framework d'agent exécute autour d'une étape : une vérification de sécurité du contenu, une anonymisation des données personnelles, un contrôle budgétaire. Les hooks émettent des événements `hook_triggered` / `hook_completed` avec un `outcome` (allow, deny, modify) et disposent de leur propre page d'observation. - -**Règle d'alerte** -Une règle qui se déclenche lorsqu'une métrique dépasse un seuil que vous définissez : taux d'erreur, latence p95, coût en tokens ou score d'un évaluateur. Lorsqu'une règle se déclenche, elle ouvre un incident et notifie les canaux que vous avez choisis (e-mail, Slack, webhook, tableau de bord). Voir [Alerts](/fr/agenteye/alerts). - -**Incident** -Un problème ouvert créé lorsqu'une règle d'alerte se déclenche. Les incidents suivent un cycle de vie (accusé de réception, assignation, résolution) et disposent d'une chronologie d'activité enregistrant chaque action. Vous pouvez également en ouvrir un manuellement. - -**Audit** -Une investigation récurrente (toutes les heures à une fois par semaine) qui analyse vos journaux *à travers* les sessions pour détecter des patterns de défaillance pour lesquels vous n'avez pas encore écrit de règle : clusters d'erreurs, scores faibles, valeurs aberrantes de latence, boucles d'appels d'outils et exécutions n'ayant jamais abouti. Là où une alerte surveille une métrique que vous connaissez déjà, un audit vous indique ce sur quoi vous devriez vous pencher ensuite. Voir [Audits](/fr/agenteye/audits). - -**Finding** -Un résultat classé et étayé par des preuves, issu d'une exécution d'audit. Un finding nomme un pattern, renvoie aux sessions exactes qui le sous-tendent et suit un cycle de vie de triage (accusé de réception, résolution, mise en sourdine, rejet). Failproof AI Observability déduplique les findings d'une exécution à l'autre, de sorte qu'un pattern connu est mis à jour plutôt que de s'accumuler. - -**L'assistant IA** -Le chat intégré au tableau de bord qui répond en langage naturel à vos questions sur vos agents, en s'appuyant sur vos propres données. Il est en lecture seule par défaut ; tout ce qu'il crée (une requête sauvegardée, un tableau de bord) nécessite une approbation, et il ne peut jamais supprimer quoi que ce soit. Voir [AI assistant](/fr/agenteye/assistant). - ---- - -## Fonctionnement - -**Organisation (tenant)** -Un espace de travail isolé. Une instance Failproof AI Observability peut héberger plusieurs organisations, chacune avec ses propres utilisateurs, clés et données. Chaque URL du tableau de bord est rattachée à votre slug d'organisation (`//…`). - -**Collector** -`agenteye-collector`, le démon léger qui s'exécute sur chaque machine agent, regroupe les événements écrits sur disque par le SDK et les envoie au serveur. - -**Clé API** -Un token à périmètre défini qui authentifie un client auprès du serveur. Les clés portent des permissions granulaires (par exemple `events:add` pour le collector, des périmètres en lecture seule pour une clé de tableau de bord). Voir [API keys](/fr/agenteye/api-keys). - -**Serveur** -Le service d'ingestion et d'API. Il ingère les événements, stocke l'état opérationnel dans vos bases de données et sert le tableau de bord ainsi que la CLI. - -**Tableau de bord** -L'interface web. Chaque page est rattachée à une organisation et lit les données via l'API du serveur. - ---- - -## Étapes suivantes - -- [Overview](/fr/agenteye/overview) : comment ces éléments s'articulent entre eux. -- [Observability](/fr/agenteye/observability) : les surfaces d'observation (Events, Sessions, Models, Tools, Hooks, Errors). \ No newline at end of file diff --git a/docs/fr/agenteye/evaluator-skill.mdx b/docs/fr/agenteye/evaluator-skill.mdx deleted file mode 100644 index 0841bb94..00000000 --- a/docs/fr/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: "Compétence d'agent évaluateur Failproof AI Observability" -description: "Passez de « je pense que notre agent est parfois mauvais » à un service de scoring déployé, votre agent de codage se chargeant à la fois de la conception et de la construction." ---- - - -Passez de *« je pense que notre agent est parfois mauvais »* à un service de scoring déployé, votre agent de codage se chargeant à la fois de la conception et de la construction. La **compétence évaluateur Failproof AI Observability** (`agenteye-evaluator`) est une *Agent Skill* : un petit dossier d'instructions qu'un agent de codage tel que Claude Code ou Codex charge à la demande. Elle apprend à l'agent à déterminer quelles dimensions de qualité méritent d'être suivies pour *votre* agent, puis à écrire, tester et déployer le [service évaluateur](/fr/agenteye/evaluation-suite) qui les note. - -Il ne s'agit **pas** d'un scoring hébergé, d'un registre vers lequel vous téléversez du contenu, ni d'un système de plugins. Votre évaluateur reste votre propre service HTTP sur votre propre infrastructure, exactement comme décrit dans le guide [Evaluation suite](/fr/agenteye/evaluation-suite). La compétence apprend simplement à votre agent à le construire correctement — tout ce qu'elle fait, vous pourriez le faire vous-même en écrivant le même code. - ---- - -## La partie difficile, c'est de décider quoi noter - -La surface du SDK est réduite — un décorateur et deux modèles — et un agent peut l'écrire à partir du seul [contrat](/fr/agenteye/evaluation-suite#http-contract). Ce n'est pas là que les évaluateurs échouent. Ils échouent parce qu'ils mesurent la mauvaise chose, et un évaluateur qui mesure la mauvaise chose est pire qu'aucun : il produit un tableau de bord que tout le monde apprend à ignorer. - -L'essentiel de la compétence concerne donc ce qui précède tout code. Elle fait interviewer l'agent (*« décrivez une exécution qui s'est bien passée ; maintenant une qui s'est mal passée »*), puis lui fait parcourir vos vraies sessions via la [CLI `agenteye`](/fr/agenteye/cli) et les lire de bout en bout. Ces deux sources divergent généralement, et l'écart est justement le point central : ce que vous avez l'intention de mesurer par rapport à ce que vos transcriptions peuvent réellement étayer. Une dimension ne survit que si elle est **calculable** à partir des événements et **discriminante** — si elle donne 0,9 à la fois sur votre bonne exécution et sur la mauvaise, elle n'enseigne rien et est supprimée. - -Ce qui en ressort est une proposition de 2 à 4 dimensions avec le raisonnement associé, que vous devez valider avant qu'une seule ligne ne soit écrite. - -```mermaid -flowchart TD - YOU["vous : 'je veux des évals pour mon bot de support'"] --> AGENT["agent de codage (Claude Code / Codex)
charge la compétence agenteye-evaluator"] - AGENT -->|"interview : à quoi ressemble le bon vs le mauvais ?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["vos vraies sessions
ce qui se passe réellement"] - DATA --> DIMS["2-4 dimensions, vous validez"] - DIMS --> SVC["votre service évaluateur
SDK agenteye-evaluator"] - SVC --> SCORES["les scores apparaissent dans le tableau de bord
et agenteye evals"] -``` - ---- - -## Relation avec les autres composants d'évaluation - -Quatre pages couvrent le scoring, et se relaient dans l'ordre : - -| Page | Ce que c'est | À utiliser quand | -|---|---|---| -| **[Evaluations](/fr/agenteye/evaluations)** | La fonctionnalité : scores sur la grille de sessions, tableaux de bord, réévaluation | Vous voulez savoir ce que le scoring automatique vous apporte | -| **[Evaluation suite](/fr/agenteye/evaluation-suite)** | Le contrat HTTP, le SDK, les variables d'environnement serveur | Vous implémentez ou déboguez vous-même l'évaluateur | -| **Compétence évaluateur** (ce doc) | Une entrée en langage naturel pour concevoir *et* construire le scorer | Vous voulez passer de « je veux des évals » à un service opérationnel | -| **[CLI skill](/fr/agenteye/cli-skill)** | Une entrée en langage naturel sur la CLI `agenteye` | Vous voulez *lire* les scores que vous avez déjà | -| **[Python SDK skill](/fr/agenteye/python-sdk-skill)** | Une entrée en langage naturel pour instrumenter votre agent | Votre agent n'émet pas encore de sessions — il n'y a rien à noter | - -### Par rapport à la CLI skill : construire versus lire - -Les deux compétences sont délibérément sans chevauchement, et les installer toutes les deux est la configuration habituelle — l'agent choisit entre elles en fonction de ce que vous demandez : - -- **`agenteye-evaluator`** (ce doc) construit ce qui *produit* les scores. Sa mission se termine quand les scores arrivent pour la première fois. -- **[`agenteye-cli`](/fr/agenteye/cli-skill)** lit les scores déjà existants (`agenteye evals`). *« La qualité a-t-elle baissé cette semaine ? »* est sa question, pas celle de cette compétence. - ---- - -## Prérequis - -1. **La CLI `agenteye` installée et connectée** (`pipx install agenteye`, puis `agenteye login`). La compétence s'appuie dessus à deux reprises : pour récupérer les vraies sessions sur lesquelles elle se base lors de la conception, et pour confirmer que vos scores sont bien arrivés à la fin. Votre connexion nécessite `events:read`, plus `evaluations:read` pour cette vérification finale. Comme avec la CLI skill, elle **ne peut pas** compléter la connexion par code à usage unique envoyé par e-mail à votre place. -2. **Un endroit où héberger l'évaluateur.** Il est construit dans une image et exécuté en tant que service de longue durée, il a donc besoin d'un vrai dépôt, pas d'un fichier temporaire. Les évaluateurs vivent souvent dans leur propre dépôt, séparé de l'agent évalué — la compétence cherche un dépôt existant et demande avant d'en créer un nouveau. -3. **La roue SDK `agenteye-evaluator`** — lisez la section suivante avant que votre agent commence à taper des commandes `pip`. - ---- - -## Où l'obtenir - -La compétence est publiée dans la collection publique de compétences de Failproof AI : - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -Le dépôt est public et la compétence n'a pas besoin de ses propres identifiants — elle pilote uniquement la CLI `agenteye` avec la session *sur laquelle vous êtes connecté*, et écrit du code dans *votre* dépôt. Notez qu'elle est livrée dans son propre dossier et n'est **pas** incluse dans le package `pipx install agenteye`, donc ne la cherchez pas là. - -## Installer la compétence - -Le chemin le plus rapide passe par la CLI [`skills`](https://skills.sh), qui récupère le dossier et le place là où votre agent le cherche : - -```bash -# Claude Code, ce projet uniquement -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# tous les projets (installe dans ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codex à la place -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -Puis gérez-la comme n'importe quelle autre compétence : - -```bash -npx skills list -a claude-code # ce qui est installé -npx skills update agenteye-evaluator # récupérer la dernière version -npx skills remove agenteye-evaluator # la supprimer -``` - -Vous préférez installer manuellement ? Une Agent Skill est juste un dossier contenant un `SKILL.md` (plus des références optionnelles), donc la copier fonctionne aussi : - -- **Claude Code** : placez le dossier `agenteye-evaluator/` dans `~/.claude/skills/` (tous les projets) ou `/.claude/skills/` (ce dépôt uniquement). Claude Code le découvre automatiquement — vérifiez avec la liste `/skills`, ou demandez simplement des évals. -- **Codex (OpenAI)** : Codex lit le même `SKILL.md`. Le fichier `agents/openai.yaml` fourni définit `allow_implicit_invocation: true`, donc Codex sélectionne automatiquement la compétence quand une tâche correspond ; sinon invoquez-la explicitement avec `$agenteye-evaluator`. - ---- - -## Le SDK n'est pas sur PyPI public - -> **Avertissement :** Lisez ceci avant de laisser un agent installer le SDK. - -La compétence est publique ; le SDK qu'elle pilote ne l'est pas. `agenteye-evaluator` est livré uniquement comme artefact de version privée, et contrairement à `agenteye`, le nom est **non revendiqué sur PyPI public** — donc un simple `pip install agenteye-evaluator` pourrait installer le package d'un inconnu dans le service qui lit vos transcriptions de production. C'est un problème de chaîne d'approvisionnement, pas une faute de frappe. - -La compétence le sait et parcourt à la place une liste d'installation par priorité, s'arrêtant à la première qui s'applique : la source du monodépôt si vous êtes dans le dépôt AgentEye, sinon la roue de version privée depuis GitHub Releases (nécessite un accès), et si aucune n'est accessible, elle **s'arrête et vous indique de demander la roue à votre contact Failproof AI** plutôt que d'improviser. - -Donc si votre agent propose un simple `pip install agenteye-evaluator` depuis PyPI public, c'est le signe que la compétence n'a jamais été chargée. Arrêtez-vous et vérifiez qu'elle est bien installée. - ---- - -## Ce que vous pouvez lui demander - -Un vrai aller-retour commence par une demande vague et se termine par une conception validée, pas par du code : - -```text -vous ▸ Je veux des évals pour notre bot de support. Je pense qu'il est parfois mauvais. - -agent ▸ Avant d'écrire quoi que ce soit : décrivez une exécution qui s'est bien - passée, et une qui s'est mal passée. - -vous ▸ Bien : il répond à partir de la documentation. Mal : il invente des choses, - ou abandonne et dit « contactez le support ». - -agent ▸ Exécution : agenteye --json sessions --status error --since 7d - Lecture de 5 sessions, dont deux que vous qualifieriez de mauvaises. - - Vos mots évoquent la factualité et le décrochage. Les transcriptions - confirment le décrochage (2 sur 5 se terminent par un transfert au support, - clairement visible dans model_response.content), mais la factualité - nécessite une vérité terrain que vos événements ne contiennent pas. - - Je mesurerais donc trois choses : - - deflection règles, peu coûteux, distingue vos deux exécutions - - tool_efficiency règles, détecte la boucle de réessai dans run-004 - - groundedness juge LLM : chaque affirmation est-elle dans les docs récupérés ? - - « factualité » nécessiterait une vérité terrain que nous n'avons pas ; - groundedness est ce que vos transcriptions peuvent réellement étayer. - D'accord avec ces trois noms ? Les renommer plus tard divise l'historique. - -vous ▸ oui - -agent ▸ Pas d'évaluateur dans ce dépôt. Dois-je en créer un ici, ou en avez-vous - un ailleurs ? -``` - -À partir de là, il écrit d'abord les dimensions basées sur des règles (gratuites, instantanées, déterministes), les teste contre une vraie session capturée incluant les sessions vides et inachevées qui font planter les évaluateurs naïfs, et n'utilise un juge LLM que pour la dimension subjective. Il connaît les [limites du dispatcher](/fr/agenteye/evaluation-suite#configuring-the-server) — un délai de requête de 30s et 8 appels simultanés dans tout le déploiement — donc si le juge ne s'y adapte pas de manière fiable, il passe en asynchrone avec `JobPending` plutôt que de laisser votre juge être annulé et réessayé cinq fois au quintuple du coût. - -Ensuite il déploie, définit les deux variables d'environnement serveur, et confirme avec `agenteye --json evals --session-id ` que les scores sont bien arrivés. L'arrivée des scores est la seule preuve. - ---- - -## Ce à quoi faire attention - -- **Les noms de dimensions sont quasi permanents.** Les clés de score sont des chaînes arbitraires et la plateforme suit les tendances de tout ce que vous envoyez, ce qui signifie que rien en aval ne corrige un mauvais choix. Renommer plus tard divise l'historique : les anciennes sessions conservent l'ancienne clé et la tendance se brise. C'est pourquoi la compétence obtient une validation explicite avant d'écrire du code — prenez cette invite au sérieux. -- **Les fixtures sont de vraies transcriptions de production.** Concevoir à partir de vraies sessions signifie les télécharger sur le disque, et elles peuvent contenir des données clients. La compétence demande avant de les committer dans git ; en cas de doute, gardez `fixtures/` hors du dépôt et faites récupérer les siennes à chaque développeur. -- **L'agent écrit et déploie un service qui lit chaque transcription.** Il agit en votre nom, limité par les permissions de votre connexion CLI, mais examinez l'évaluateur comme n'importe quel autre code qui touche des données de production. - ---- - -## Prochaines étapes - -- **[Evaluation suite](/fr/agenteye/evaluation-suite)** : le contrat HTTP, le SDK et les variables d'environnement serveur que la compétence configure. -- **[Evaluations](/fr/agenteye/evaluations)** : là où les scores s'affichent une fois qu'ils arrivent. -- **[CLI skill](/fr/agenteye/cli-skill)** : la compétence jumelle, pour lire les résultats plutôt que construire le scorer. -- **[CLI](/fr/agenteye/cli)** : la référence des commandes derrière les données de session sur lesquelles la compétence se base. \ No newline at end of file diff --git a/docs/fr/agenteye/hermes-capture.mdx b/docs/fr/agenteye/hermes-capture.mdx deleted file mode 100644 index 7a8cc9f4..00000000 --- a/docs/fr/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Capture de sessions Hermes" -description: "Intégrez les sessions de votre passerelle Hermes — Slack, Telegram, CLI et exécutions planifiées — dans AgentEye sous forme de sessions et d'événements ordinaires." ---- - -[Hermes](https://hermes-agent.nousresearch.com) répond à votre équipe depuis n'importe quel outil de travail — Slack, Telegram, la CLI, des exécutions planifiées. La capture de sessions Hermes intègre l'ensemble de ces interactions dans AgentEye sous forme de sessions et d'événements ordinaires, afin que l'assistant que votre équipe utilise au quotidien soit aussi observable que les agents que vous développez vous-même. - -Un petit collecteur en arrière-plan lit le dépôt de sessions local de Hermes au fur et à mesure de son écriture, puis transmet les sessions à AgentEye. Son fonctionnement est identique à celui des captures [Codex](/fr/agenteye/codex-capture) et [OpenClaw](/fr/agenteye/openclaw-capture), et un seul collecteur peut en capturer plusieurs simultanément. - ---- - -## Ce qui est capturé - -Toutes les sessions Hermes présentes sur la machine sont capturées, quel que soit le canal d'origine. Chacune devient une [session](/fr/agenteye/sessions) AgentEye ; ses messages utilisateur et assistant, ses appels d'outils et leurs résultats deviennent les [événements](/fr/agenteye/event-stream) correspondants. - -Le canal depuis lequel une session a démarré — Slack, Telegram, CLI ou une exécution planifiée — est enregistré sur la session, ce qui vous permet de les distinguer et de filtrer sur un seul canal à la fois. Sont également consignés : le modèle utilisé par la session, le chat et la personne à l'origine de son démarrage, ainsi que, lorsqu'une session en a engendré une autre, le lien vers sa session parente. - -Les sessions apparaissent dès que Hermes les démarre, qu'un message ait été échangé ou non, et la réponse d'un tour ainsi que ses appels d'outils sont conservés dans l'ordre réel des événements. Lorsqu'une session se termine, vous obtenez également la raison de sa fin, son coût et le nombre de tokens consommés. - ---- - -## Activation - -La capture est désactivée par défaut. Installez le collecteur avec une clé API disposant de la permission `events:add` (voir [Clés API](/fr/agenteye/api-keys)), puis activez la capture Hermes : - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -Cette commande installe le collecteur, l'enregistre en tant que service en arrière-plan et lance la capture. Pour vérifier qu'il est en cours d'exécution : - -```bash -agenteye-collector health -``` - -Vous capturez plusieurs agents sur la même machine ? Ajoutez le flag de chacun à la même commande — par exemple `--hermes-enabled --codex-enabled`. - -Au premier lancement, vos sessions Hermes existantes sont importées rétroactivement en une seule fois, puis la nouvelle activité est transmise en quelques secondes. Les données de Hermes sont uniquement lues — jamais modifiées ni supprimées — et chaque message est transmis une seule fois, même après des redémarrages. - -`health` vous indique également si tout ce que le collecteur a capturé a bien atteint AgentEye. Si un lot n'a pas pu être livré, il est conservé et réessayé plutôt que supprimé, et la vérification signale un état non sain tant que des données sont encore en attente — ainsi, « sain » signifie que vos données sont bien arrivées, et pas seulement que le processus est en vie. - ---- - -## Où les retrouver - -Les sessions capturées apparaissent dans **Sessions**, et leurs événements dans le flux **Events**, comme pour tout autre agent observé — ainsi, la [relecture de session](/fr/agenteye/sessions), la [recherche](/fr/agenteye/queries), les [évaluations](/fr/agenteye/evaluations) et les [alertes](/fr/agenteye/alerts) fonctionnent toutes sur ces données. Filtrez par l'agent Hermes pour les visualiser de manière isolée. - ---- - -## Confidentialité - -Les sessions Hermes contiennent la transcription complète — y compris les sorties de commandes, le contenu des fichiers et tout ce que l'agent a lu ou écrit — et peuvent contenir des secrets. Les sessions capturées sont transmises telles quelles ; n'activez donc la capture que dans les contextes où la centralisation de ce contenu dans AgentEye est appropriée, et donnez au collecteur une clé limitée à la seule permission `events:add`. Consultez [Sécurité](/fr/agenteye/security) pour en savoir plus sur l'isolation de vos données. \ No newline at end of file diff --git a/docs/fr/agenteye/observability.mdx b/docs/fr/agenteye/observability.mdx deleted file mode 100644 index 577af02e..00000000 --- a/docs/fr/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Observer" -description: "Les surfaces d'observation permettent de surveiller en temps réel ce que font vos agents et d'explorer chaque exécution en détail." ---- - - -Les surfaces d'observation permettent de surveiller en temps réel ce que font vos agents et d'explorer chaque exécution en détail. Tout ici est en direct, limité à votre organisation, et filtrable par plage de dates, environnement, agent et session — vous passez de « quelque chose cloche » à l'exécution exacte en quelques secondes. - -![Le flux d'événements en direct, coloré par type et filtrable par environnement, agent et session](/agenteye/images/events-stream.png) - -Quatre surfaces, chacune avec sa propre page : - -- **[Flux d'événements](/fr/agenteye/event-stream)** : le suivi en direct, étape par étape, de chaque exécution pour tous les agents, du plus récent au plus ancien. La page d'accueil de votre organisation et premier point de triage. -- **[Sessions et graphe d'exécution](/fr/agenteye/sessions)** : ces événements regroupés en une ligne par exécution, accompagnés d'une représentation visuelle de type git montrant comment chaque exécution s'est déroulée. -- **[Métriques de performance](/fr/agenteye/telemetry)** : cartes de chaleur de latence et indicateurs p50/p95/p99 pour vos modèles, outils et hooks, afin de distinguer les pics extrêmes de la médiane. -- **[Suivi des erreurs](/fr/agenteye/error-tracking)** : une surface de triage unique pour tout ce qui a mal tourné, à un clic d'une alerte déclenchée vers l'exécution responsable. - -## Liens connexes - -- [Évaluations](/fr/agenteye/evaluations) : notez chaque exécution selon la qualité. -- [Alertes](/fr/agenteye/alerts) : transformez n'importe quel seuil en règle de notification. -- [Audits](/fr/agenteye/audits) : laissez Failproof AI Observability identifier automatiquement les schémas d'échec entre les sessions. -- [CLI et agents](/fr/agenteye/cli-and-agents) : la même observabilité depuis votre terminal. \ No newline at end of file diff --git a/docs/fr/agenteye/openclaw-capture.mdx b/docs/fr/agenteye/openclaw-capture.mdx deleted file mode 100644 index bf25828f..00000000 --- a/docs/fr/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "Capture de session OpenClaw" -description: "Transmettez les sessions OpenClaw locales de votre équipe vers AgentEye en tant que sessions et événements ordinaires — sans aucune modification de votre façon d'utiliser OpenClaw." ---- - -Si votre équipe utilise [OpenClaw](https://docs.openclaw.ai), la capture de session OpenClaw importe ces sessions dans AgentEye en tant que sessions et événements ordinaires, afin que vous puissiez les rechercher, les rejouer et les évaluer aux côtés de tout ce que vous observez. Cette fonctionnalité complète le [SDK Python](/fr/agenteye/python-sdk) : le SDK instrumente les agents que vous développez, tandis que la capture OpenClaw enregistre le travail que votre équipe réalise déjà — sans aucune modification de leur façon de l'exécuter. - -Un petit collecteur en arrière-plan lit les transcripts de session locaux d'OpenClaw au fur et à mesure de leur écriture et les envoie vers AgentEye. Il fonctionne de la même manière que la [capture Codex](/fr/agenteye/codex-capture), et un seul collecteur peut capturer les deux simultanément. - ---- - -## Ce qui est capturé - -Chaque agent configuré dans l'installation OpenClaw d'une machine est capturé par le collecteur de cette machine — aucune configuration par agent n'est nécessaire. - -Chaque session OpenClaw devient une [session](/fr/agenteye/sessions) AgentEye ; ses messages utilisateur et assistant, ses appels d'outils et les résultats de ces appels deviennent les [événements](/fr/agenteye/event-stream) correspondants. - ---- - -## Activation - -La capture est désactivée jusqu'à ce que vous l'activiez. Installez le collecteur avec une clé API disposant de la permission `events:add` (voir [Clés API](/fr/agenteye/api-keys)), puis activez la capture OpenClaw : - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -Cette commande installe le collecteur, l'enregistre en tant que service en arrière-plan et démarre la capture. Vérifiez qu'il est bien en cours d'exécution : - -```bash -agenteye-collector health -``` - -Vous capturez plusieurs agents sur la même machine ? Ajoutez le flag de chacun à la même commande — par exemple `--openclaw-enabled --codex-enabled`. - -Au premier démarrage, vos sessions OpenClaw existantes sont importées une seule fois, puis la nouvelle activité est transmise en quelques secondes. Les fichiers d'OpenClaw sont uniquement lus — jamais modifiés, déplacés ou supprimés — et chaque session est envoyée exactement une fois, même lors des redémarrages. - ---- - -## Où retrouver les données - -Les sessions capturées apparaissent dans **Sessions**, et leurs événements dans le flux **Events**, comme pour tout autre agent que vous observez — ainsi, le [replay de session](/fr/agenteye/sessions), la [recherche](/fr/agenteye/queries), les [évaluations](/fr/agenteye/evaluations) et les [alertes](/fr/agenteye/alerts) fonctionnent tous sur ces données. Filtrez par agent OpenClaw pour les afficher séparément. - ---- - -## Confidentialité - -Les transcripts OpenClaw contiennent l'intégralité de la session — y compris la sortie des commandes, le contenu des fichiers, et tout ce que l'agent a lu ou écrit — et peuvent contenir des secrets. Les sessions capturées sont transmises telles quelles, donc n'activez la capture que sur les machines et pour les équipes pour lesquelles la centralisation de ces données dans AgentEye est appropriée, et accordez au collecteur une clé limitée au seul scope `events:add`. Consultez la section [Sécurité](/fr/agenteye/security) pour en savoir plus sur l'isolation de vos données. \ No newline at end of file diff --git a/docs/fr/agenteye/python-sdk-skill.mdx b/docs/fr/agenteye/python-sdk-skill.mdx deleted file mode 100644 index 010b3a6d..00000000 --- a/docs/fr/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,135 +0,0 @@ ---- -title: "Compétence Agent du SDK Python Failproof AI Observability" -description: "Passez d'un agent non instrumenté à des événements visibles, votre agent de codage trouvant les points d'instrumentation, les écrivant et prouvant qu'ils ont bien été intégrés." ---- - -Dites à votre agent de codage *« ajoute Failproof AI Observability à cet agent »* et laissez-le lire votre boucle, déterminer où placer l'instrumentation, l'écrire et vérifier les événements avant de considérer le travail terminé. - -La **compétence SDK Python** (`agenteye-python-sdk`) est une *Agent Skill* : un dossier d'instructions qu'un agent de codage tel que Claude Code ou Codex charge à la demande lorsqu'une tâche lui correspond. Elle apprend à l'agent à utiliser le [SDK Python](/fr/agenteye/python-sdk) — ce n'est pas une bibliothèque, et elle ne modifie en rien le fonctionnement du SDK. - -## L'instrumentation est facile à écrire et facile à rater silencieusement - -Le SDK est minimaliste : treize méthodes d'événements, toutes avec des paramètres nommés uniquement. Un agent de codage peut lire la référence du [SDK Python](/fr/agenteye/python-sdk) et produire une instrumentation plausible en une minute. - -Le problème, c'est que ce SDK ne lève pas d'exception en cas d'erreur, et une mauvaise instrumentation ressemble exactement à une bonne instrumentation jusqu'à ce que quelqu'un ouvre un tableau de bord et le trouve vide. Les erreurs qui font perdre du temps sont toutes des silences : - -| L'erreur | Ce que vous voyez | -|---|---| -| Pas de `agent_start` | Tous les événements arrivent. Zéro session. | -| Environnement jamais défini | Tout fonctionne, classé sous `dev`. | -| `outcome="failure"` | L'exécution s'affiche en vert — seuls `failed`, `error`, `timeout`, `rejected` comptent. | -| Un nom de champ mal orthographié | Accepté et stocké comme nouveau champ. | -| Événements émis depuis un pool de threads | Silencieusement abandonnés. | - -Aucun de ces cas ne lève d'exception. Aucun n'apparaît dans les tests. Chacun est documenté dans la compétence, énoncé comme un contrat avec la vérification qui le détecte. - -## Ce qu'elle fait, dans l'ordre - -La compétence exécute les trois mêmes étapes qu'un ingénieur rigoureux suivrait : - -1. **Planifier.** Elle lit votre boucle d'agent et pose les deux questions auxquelles vous seul pouvez répondre : ce qui constitue une exécution (votre `session_id`), et qui sont les acteurs distinguables (votre `agent_id`). Elle obtient un accord sur ces points avant d'écrire du code, car les modifier plus tard divise votre historique et casse les tendances. -2. **Écrire.** Elle lie l'identité une seule fois par exécution plutôt que de la propager à travers chaque point d'appel, et elle choisit une forme sûre pour la concurrence — un détail qui compte, car le raccourci évident mélange silencieusement deux exécutions simultanées en une seule session. -3. **Vérifier.** Elle exécute votre agent et lit les fichiers d'événements résultants, en vérifiant que `agent_start` est présent, que l'environnement est correct et qu'une exécution a produit une session. - -Cette troisième étape est celle que les gens ignorent. Le SDK écrit les événements dans des fichiers locaux, donc une intégration complète peut être prouvée sur un ordinateur portable sans serveur, sans clé API et sans réseau — c'est précisément pourquoi la compétence insiste pour le faire. - -## Son rapport aux autres compétences - -Trois compétences, une séparation nette : - -| Compétence | À utiliser quand | Ce qu'elle modifie | -|---|---|---| -| **Compétence SDK Python** (cette page) | Vous voulez que votre agent *émette* de la télémétrie — « ajoute de l'observabilité », « pourquoi mon agent n'apparaît pas ? » | Écrit du code dans le dépôt de votre agent. Ne lit rien. | -| **[Compétence Evaluator](/fr/agenteye/evaluator-skill)** | Vous voulez *noter* les exécutions — « que devrions-nous même mesurer ? » | Écrit du code dans votre dépôt ; lit la télémétrie | -| **[Compétence CLI](/fr/agenteye/cli-skill)** | Vous voulez *lire* ce qui s'est passé, ou opérer votre déploiement | Pilote la CLI en votre nom, y compris les modifications | - -Elles se relaient dans cet ordre : cette compétence fait circuler les événements, l'évaluateur les note, la CLI les relit. Il n'y a rien à évaluer et rien à lire tant que votre agent n'émet pas de sessions — donc si vous partez de zéro, commencez ici. - -## Prérequis - -1. **Python 3.10+** et la base de code de l'agent que vous souhaitez instrumenter. -2. **Le SDK.** Il est distribué aux clients sous forme de wheel privé plutôt que depuis un index public — votre intégration couvre comment l'obtenir et l'installer. La compétence connaît le chemin d'installation et vous demandera plutôt que de deviner si elle ne le trouve pas. -3. **Rien d'autre.** Pas de connexion au tableau de bord, pas de clé API, pas de réseau. La compétence vérifie à partir des fichiers d'événements que le SDK écrit, elle peut donc terminer et prouver son travail hors ligne. - -## Où l'obtenir - -La compétence se trouve dans la collection publique [`FailproofAI/skills`](https://github.com/FailproofAI/skills) : - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -Ajoutez `-g` pour l'installer pour tous les projets plutôt que pour le seul projet en cours, et `--copy` si votre environnement ne suit pas les liens symboliques. Pour Codex, passez `-a codex`. - -## Installation manuelle - -Les Agent Skills sont des dossiers contenant un `SKILL.md` et des références associées. Si vous préférez ne pas utiliser l'installateur : - -- **Claude Code** : copiez le dossier `agenteye-python-sdk/` dans `~/.claude/skills/` (tous les projets) ou `/.claude/skills/` (ce dépôt uniquement). Claude Code le découvre automatiquement — vérifiez la liste `/skills`, ou posez simplement une question qui lui correspond. -- **Codex** : Codex lit le même `SKILL.md`. Le fichier `agents/openai.yaml` inclus définit `allow_implicit_invocation: true`, il est donc auto-sélectionné quand une tâche lui correspond ; sinon invoquez-le avec `$agenteye-python-sdk`. - -Exécutez votre agent **dans le dépôt contenant le code que vous souhaitez instrumenter** — la compétence lit votre boucle d'agent avant de proposer quoi que ce soit. - -## À quoi ressemble une session - -```text -vous ▸ Ajoute Failproof AI Observability à cet agent. - -agent ▸ J'ai lu votre boucle — un dispatcheur d'outils dans tools.py, un - wrapper Anthropic dans llm.py. Deux questions avant d'écrire quoi - que ce soit. - - Qu'est-ce qui constitue une exécution ? Vous gérez les requêtes - en parallèle, donc j'utiliserais l'id de requête que vous avez - déjà dans ctx.request_id — ainsi une session ici et une requête - dans vos logs sont la même chaîne. - - Et qui sont les acteurs ? Je vois un chemin planificateur et un - chemin chercheur. Les étiqueter séparément, ou c'est un seul - agent pour vous ? - -vous ▸ L'id de requête, c'est bon. C'est un seul agent, appelez-le "assistant". - -agent ▸ Terminé. J'ai enveloppé le dispatcheur et le client LLM ; - agent_start et agent_end encadrent handle_request, y compris le - chemin d'erreur. J'ai utilisé un contexte local pour l'identité - — votre dispatcheur exécute les outils dans un pool de threads, - et l'approche plus simple aurait mélangé des exécutions - simultanées en une seule session sans se plaindre. - - Vérifié sur une vraie exécution : 11 événements dans 4 fichiers — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, une session - par requête, durées appariées. - - Une chose que je n'ai PAS faite : votre pool de workers dans - batch.py soumet directement à l'executor, donc les événements - venant de là seraient perdus. Voulez-vous que je corrige ça aussi ? -``` - -Le schéma à remarquer : il a lu le code avant de proposer, n'a posé que les questions auxquelles vous seul pouvez répondre, a réutilisé un id que vous aviez déjà, a choisi la forme sûre pour la concurrence *parce qu'* il a vu un pool de threads, et **a vérifié en lisant les événements réels** plutôt qu'en déclarant le succès — puis a signalé l'endroit qu'il savait qui échouerait silencieusement. - -## Ce que vous pouvez lui demander - -- *« Pourquoi mon agent n'apparaît pas sur le tableau de bord ? »* → parcourt l'échelle : les événements sont-ils écrits, `agent_start` est-il présent, l'environnement est-il correct, le collecteur lit-il au bon endroit. -- *« Tout arrive sous dev. »* → l'environnement n'a jamais été défini, ou a été réinitialisé par un appel ultérieur. -- *« Ajoute le suivi des tokens. »* → trouve votre wrapper LLM et enregistre le modèle, la raison d'arrêt et l'utilisation. -- *« Instrumente aussi les sous-agents. »* → une session, des étiquettes d'agent distinctes, imbriqués sous leur parent. -- *« Écris des tests pour l'instrumentation. »* → pointe le SDK vers un répertoire temporaire et effectue des assertions sur les événements qu'il a écrits. - -## Points de vigilance - -**Laissez-le vérifier.** L'étape qui rend cette compétence utile est la dernière — exécuter votre agent et relire les événements. Un agent qui écrit l'instrumentation et s'arrête a fait la moitié facile, et la moitié qui échoue silencieusement, c'est l'autre. - -**Convenez des noms avant le code.** `session_id` et `agent_id` sont les axes selon lesquels chaque surface regroupe les données. Les renommer plus tard divise l'historique : les anciennes exécutions conservent les anciennes étiquettes et vos tendances se cassent. La compétence posera la question ; la réponse mérite une minute de réflexion. - -**Si votre agent propose d'installer le SDK depuis un index public, la compétence n'a pas été chargée.** Le SDK est distribué en privé. Cette proposition est un signe révélateur que votre agent de codage improvise plutôt que de suivre la compétence — arrêtez-le là et vérifiez que la compétence est installée. - -En dehors de cela, son rayon d'action est limité : elle écrit du code dans votre répertoire de travail et des fichiers d'événements là où vous lui indiquez. Elle ne lit rien de votre déploiement et n'y change rien. - -## Étapes suivantes - -- **[SDK Python](/fr/agenteye/python-sdk)** : la référence complète des événements — chaque type d'événement et chaque champ — derrière ce que cette compétence automatise. -- **[Sessions](/fr/agenteye/sessions)** : ce que produit votre instrumentation une fois les événements reçus. -- **[Agent Skill Evaluator](/fr/agenteye/evaluator-skill)** : l'étape suivante une fois que les exécutions arrivent — les noter. -- **[Agent Skill CLI](/fr/agenteye/cli-skill)** : relire votre télémétrie. \ No newline at end of file diff --git a/docs/fr/agenteye/security.mdx b/docs/fr/agenteye/security.mdx deleted file mode 100644 index 4b1954c1..00000000 --- a/docs/fr/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "Sécurité" -description: "Failproof AI Observability est conçu pour fonctionner au plus près de vos agents en production, ce qui signifie qu'il voit vos prompts, les entrées des outils et leurs sorties." ---- - - -Failproof AI Observability est conçu pour fonctionner au plus près de vos agents en production, ce qui signifie qu'il voit vos prompts, les entrées des outils et leurs sorties. Cette page explique comment vos données restent isolées, contrôlées et entre vos mains. Si vous évaluez Failproof AI Observability dans le cadre d'une revue de sécurité, commencez ici. - ---- - -## Vos données restent dans votre environnement - -Failproof AI Observability est auto-hébergé. Les événements, prompts, réponses des modèles et analyses sont stockés dans vos propres bases de données, dans votre propre environnement. Rien n'est envoyé à un service SaaS tiers pour y être stocké, et vos données demeurent dans votre propre compte cloud. - ---- - -## Isolation des locataires - -Une instance Failproof AI Observability peut héberger plusieurs organisations, chacune étant isolée au niveau de la couche de stockage — appliqué par la base de données elle-même, et pas seulement par l'interface : - -- Les données opérationnelles d'une organisation (utilisateurs, clés, tableaux de bord, requêtes sauvegardées) sont limitées à cette organisation, et les lectures inter-organisations sont bloquées par la base de données elle-même. -- Chaque événement ingéré est marqué avec l'organisation à laquelle il appartient, de sorte qu'une organisation ne peut jamais lire les événements d'une autre. - -Chaque route de tableau de bord est délimitée sous un slug d'organisation (`//…`). - ---- - -## Connexion - -Failproof AI Observability utilise une connexion sans mot de passe, par e-mail. Il n'y a pas de mot de passe à hameçonner ou à divulguer. Un utilisateur demande un code à usage unique (ou un lien magique en un clic), qui lui est envoyé par e-mail et expire rapidement. La connexion est contrôlée par une **liste d'autorisation** : seules les adresses e-mail (ou domaines) que vous autorisez peuvent s'authentifier. - -![L'écran de connexion de Failproof AI Observability, qui envoie un code à usage unique à votre adresse e-mail](/agenteye/images/login.png) - ---- - -## Accès délimité avec des clés API - -Chaque client s'authentifie avec une clé API dotée de permissions granulaires et à moindre privilège. Un collecteur n'a besoin que de `events:add` ; une clé de tableau de bord ou d'assistant peut être en lecture seule ; les actions destructives (suppression, regénération) sont des droits distincts que vous choisissez d'inclure. - -![La page des clés API : les permissions accordées à chaque clé, avec un code couleur par portée lecture, écriture et destructive](/agenteye/images/api-keys.png) - -Conservez la clé d'amorçage administrateur pour la configuration, et créez des clés restreintes pour tout le reste. Voir [Clés API](/fr/agenteye/api-keys). - ---- - -## Un assistant en lecture seule avec validation obligatoire - -L'[assistant IA](/fr/agenteye/assistant) intégré au tableau de bord répond à vos questions sur vos données, mais il est limité par conception : - -- Il est **en lecture seule par défaut** : son SQL passe par un garde-fou qui n'autorise que les requêtes `SELECT`/`WITH`, à instruction unique, avec un plafond de lignes. -- Tout ce qu'il crée (une requête sauvegardée, un tableau de bord) est soumis à **validation** : vous examinez et approuvez chaque écriture avant qu'elle ne se produise. -- Il **ne peut jamais supprimer**. - -Ainsi, un membre de l'équipe peut demander « quels agents ont généré le plus d'erreurs cette semaine ? » et agir sur la réponse, sans que l'assistant puisse modifier ou supprimer vos données de son propre chef. - ---- - -## En transit - -Tout le trafic passe par HTTPS. Vous terminez le TLS avec vos propres certificats, de sorte que le trafic collecteur-vers-serveur et navigateur-vers-serveur est chiffré en transit. - ---- - -## Étapes suivantes - -- [Vue d'ensemble](/fr/agenteye/overview) : comment Failproof AI Observability s'articule. -- [Clés API](/fr/agenteye/api-keys) : délimitez l'accès pour le collecteur, le tableau de bord et l'assistant. -- [Observabilité](/fr/agenteye/observability) : ce que Failproof AI Observability capture depuis vos agents. \ No newline at end of file diff --git a/docs/fr/cli/audit.mdx b/docs/fr/audit.mdx similarity index 100% rename from docs/fr/cli/audit.mdx rename to docs/fr/audit.mdx diff --git a/docs/fr/cli/backfill.mdx b/docs/fr/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/fr/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/fr/cli/config.mdx b/docs/fr/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/fr/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/fr/cli/flush.mdx b/docs/fr/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/fr/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/fr/cli/harness.mdx b/docs/fr/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/fr/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/fr/cli/migrate.mdx b/docs/fr/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/fr/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/fr/cli/uninstall.mdx b/docs/fr/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/fr/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/fr/cli/update.mdx b/docs/fr/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/fr/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/fr/agenteye/api-keys.mdx b/docs/fr/cloud/access.mdx similarity index 98% rename from docs/fr/agenteye/api-keys.mdx rename to docs/fr/cloud/access.mdx index 5fe70198..292cc8a9 100644 --- a/docs/fr/agenteye/api-keys.mdx +++ b/docs/fr/cloud/access.mdx @@ -82,7 +82,7 @@ Le serveur applique un catalogue fixe de permissions ; chacune conditionne l'acc Ces permissions alimentent la page **Utilisateurs** du tableau de bord, où les portées accordées à chaque membre s'affichent sous forme de puces : -![La page Utilisateurs : une carte par utilisateur du tableau de bord avec son e-mail, les permissions accordées et les contrôles de modification/désactivation](/agenteye/images/users.png) +![La page Utilisateurs : une carte par utilisateur du tableau de bord avec son e-mail, les permissions accordées et les contrôles de modification/désactivation](/cloud/images/users.png) ### Paramètres opérationnels @@ -91,7 +91,7 @@ Ces permissions alimentent la page **Utilisateurs** du tableau de bord, où les | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | Afficher les paramètres opérationnels gérés par le tableau de bord et leurs métadonnées ; lister les remplacements de fenêtre de contexte par modèle ; et résoudre la fenêtre effective pour un modèle. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | Modifier les paramètres opérationnels et ajouter, modifier ou supprimer les remplacements de fenêtre de contexte par modèle. Les modifications s'appliquent aux nouveaux événements sans redémarrage du serveur. | -![La page Paramètres : paramètres opérationnels gérés par le tableau de bord tels que les connexions autorisées et les durées de vie des sessions/OTP, modifiables sans redémarrage](/agenteye/images/settings.png) +![La page Paramètres : paramètres opérationnels gérés par le tableau de bord tels que les connexions autorisées et les durées de vie des sessions/OTP, modifiables sans redémarrage](/cloud/images/settings.png) ### Alertes et incidents @@ -247,7 +247,7 @@ La page **Clés** du tableau de bord fournit une interface utilisateur pour tout Lorsque vous créez une clé depuis le tableau de bord, vous ne fournissez pas le secret ; le tableau de bord génère un secret fort pour vous et l'affiche **une seule fois** à la création. Copiez-le immédiatement et stockez-le de manière sécurisée ; il ne sera plus jamais affiché, exactement comme lors d'une régénération. Vous pouvez toujours choisir les permissions de la clé directement, ou les initialiser depuis un ensemble de permissions (voir ci-dessous). -![La page Clés API : une carte par clé affichant son nom, les permissions accordées et la date de création, avec les actions de régénération et de désactivation ; les clés protégées comme `admin` sont marquées](/agenteye/images/api-keys.png) +![La page Clés API : une carte par clé affichant son nom, les permissions accordées et la date de création, avec les actions de régénération et de désactivation ; les clés protégées comme `admin` sont marquées](/cloud/images/api-keys.png) --- @@ -276,5 +276,5 @@ Ces notes ne sont nécessaires que si vous mettez à niveau une instance existan ## Étapes suivantes -- [SDK Python](/fr/agenteye/python-sdk) : comment votre code d'agent s'authentifie lors de l'envoi d'événements. -- [Sécurité](/fr/agenteye/security) : comment fonctionnent la connexion, le contrôle d'accès et l'isolation des données par organisation. \ No newline at end of file +- [SDK Python](/fr/cloud/sdk) : comment votre code d'agent s'authentifie lors de l'envoi d'événements. +- [Sécurité](/fr/cloud/security) : comment fonctionnent la connexion, le contrôle d'accès et l'isolation des données par organisation. \ No newline at end of file diff --git a/docs/fr/cloud/agent-skills.mdx b/docs/fr/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/fr/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/fr/agenteye/alerts.mdx b/docs/fr/cloud/alerts.mdx similarity index 84% rename from docs/fr/agenteye/alerts.mdx rename to docs/fr/cloud/alerts.mdx index 0f4ffa6d..e2cda656 100644 --- a/docs/fr/agenteye/alerts.mdx +++ b/docs/fr/cloud/alerts.mdx @@ -6,7 +6,7 @@ description: "Soyez informé dès qu'un seuil est franchi, sur le canal déjà u Soyez informé dès qu'un seuil est franchi, sur le canal déjà utilisé par votre équipe, plutôt que de l'apprendre d'un client. Définissez une règle une fois, et l'observabilité Failproof AI la vérifie selon un planning, puis vous alerte par e-mail, Slack, webhook ou directement dans le tableau de bord. -![La page Alertes : une grille de cartes de règles d'alerte, chacune affichant son déclencheur, sa fenêtre d'évaluation, ses canaux et un badge de sévérité info, avertissement ou critique](/agenteye/images/alerts.png) +![La page Alertes : une grille de cartes de règles d'alerte, chacune affichant son déclencheur, sa fenêtre d'évaluation, ses canaux et un badge de sévérité info, avertissement ou critique](/cloud/images/alerts.png) *Toutes les règles d'alerte en un coup d'œil : ce qu'elles surveillent, à quelle fréquence, où elles notifient et leur niveau d'urgence.* ## Soyez alerté des problèmes avant vos utilisateurs @@ -24,7 +24,7 @@ Combinez n'importe lesquels sur une même règle, et la sévérité (info, avert Vous décrivez ce que signifie « en erreur » dans un formulaire, et l'observabilité Failproof AI génère la règle sous-jacente pour vous. La spec JSON n'est que ce que ce formulaire produit en coulisses, vous pouvez la lire pour comprendre une règle, mais vous la saisissez rarement manuellement. -![Le formulaire de nouvelle alerte : nom et description, un interrupteur d'activation et un sélecteur de déclencheur proposant seuil de métrique, SQL personnalisé, score d'évaluation, évaluation composée et conditions par événement](/agenteye/images/alert-new.png) +![Le formulaire de nouvelle alerte : nom et description, un interrupteur d'activation et un sélecteur de déclencheur proposant seuil de métrique, SQL personnalisé, score d'évaluation, évaluation composée et conditions par événement](/cloud/images/alert-new.png) *Choisissez un déclencheur et le formulaire affiche les bons champs ; Enregistrer écrit la règle.* Le chemin classique est rapide : nommez-la, choisissez un **déclencheur** (ce qu'il faut surveiller), définissez le **seuil et la fenêtre** (quelle gravité, sur quelle durée), associez au moins un **canal**, puis **Enregistrez** et cliquez sur **Tester** pour déclencher une notification synthétique et vérifier que chaque destination est bien configurée. En coulisses, cela produit une petite spec comme : @@ -43,7 +43,7 @@ Vous n'êtes pas limité à un seul type de signal. Choisissez le déclencheur q | **Évaluation composée** | plusieurs vérifications de scores se combinent avec une logique any, all ou au-moins-N, pour détecter une régression qui n'apparaît qu'à travers plusieurs scores | | **Par événement** | un événement correspondant survient : un agent spécifique, un type d'erreur spécifique ou une sous-chaîne de message | -Vous êtes déjà en train d'examiner une défaillance sur la [page Erreurs](/fr/agenteye/error-tracking) ? Chaque ligne dispose d'un bouton **+ alerte** qui ouvre ce même formulaire pré-rempli pour détecter exactement cette défaillance à l'avenir, de sorte que l'incident que vous venez de traiter devient celui qui vous alertera la prochaine fois. +Vous êtes déjà en train d'examiner une défaillance sur la [page Erreurs](/fr/cloud/errors) ? Chaque ligne dispose d'un bouton **+ alerte** qui ouvre ce même formulaire pré-rempli pour détecter exactement cette défaillance à l'avenir, de sorte que l'incident que vous venez de traiter devient celui qui vous alertera la prochaine fois. **Où le trouver :** Les alertes se trouvent à `//alerts`. La création, la modification, la suppression et le test des règles nécessitent **`alerts:write`** ; `alerts:read` suffit pour consulter. Le sélecteur de destinataires liste les membres de votre organisation par nom, vous pouvez donc notifier une personne sans quitter le formulaire. @@ -53,11 +53,11 @@ Une mauvaise mesure ne devrait pas vous réveiller. Le filtre anti-bruit **M sur ## Ce qui se passe quand une alerte se déclenche -Un dépassement ouvre un **incident** et notifie vos canaux une fois. À partir de là, votre équipe le reconnaît, lui assigne un responsable, en discute et le résout, le tout dans un journal clair et attribué. Ce workflow de triage a son propre espace : voir [Incidents](/fr/agenteye/incidents). +Un dépassement ouvre un **incident** et notifie vos canaux une fois. À partir de là, votre équipe le reconnaît, lui assigne un responsable, en discute et le résout, le tout dans un journal clair et attribué. Ce workflow de triage a son propre espace : voir [Incidents](/fr/cloud/incidents). ## Voir aussi -- [Incidents](/fr/agenteye/incidents) : suivez une alerte déclenchée de l'ouverture à l'acquittement jusqu'à la résolution. -- [Suivi des erreurs](/fr/agenteye/error-tracking) : regroupez les défaillances des agents et transformez-en une en alerte en un clic. -- [Tableaux de bord](/fr/agenteye/dashboards) : consultez les tableaux partagés d'où proviennent les seuils que vous alertez. -- [CLI et agents](/fr/agenteye/cli-and-agents) : créez des alertes et acquittez des incidents depuis votre terminal, ou intégrez-les dans votre CI. \ No newline at end of file +- [Incidents](/fr/cloud/incidents) : suivez une alerte déclenchée de l'ouverture à l'acquittement jusqu'à la résolution. +- [Suivi des erreurs](/fr/cloud/errors) : regroupez les défaillances des agents et transformez-en une en alerte en un clic. +- [Tableaux de bord](/fr/cloud/dashboards) : consultez les tableaux partagés d'où proviennent les seuils que vous alertez. +- [CLI et agents](/fr/cloud/cli) : créez des alertes et acquittez des incidents depuis votre terminal, ou intégrez-les dans votre CI. \ No newline at end of file diff --git a/docs/fr/agenteye/assistant.mdx b/docs/fr/cloud/assistant.mdx similarity index 84% rename from docs/fr/agenteye/assistant.mdx rename to docs/fr/cloud/assistant.mdx index 2289c322..f89cd29a 100644 --- a/docs/fr/agenteye/assistant.mdx +++ b/docs/fr/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "Posez une question sur vos données d'agent en langage naturel et --- -Posez une question sur vos données d'agent en langage naturel et obtenez une réponse qui renvoie directement aux preuves. Pas de SQL à écrire, pas de tableaux de bord à parcourir — l'assistant **Failproof AI Observability** est le moyen le plus rapide pour n'importe quel membre de votre équipe d'obtenir des réponses sur vos agents. +Posez une question sur vos données d'agent en langage naturel et obtenez une réponse qui renvoie directement aux preuves. Pas de SQL à écrire, pas de tableaux de bord à parcourir — l'assistant **FailproofAI Cloud** est le moyen le plus rapide pour n'importe quel membre de votre équipe d'obtenir des réponses sur vos agents. -![L'assistant Failproof AI Observability répondant à une question en langage naturel dans le tableau de bord, affichant un tableau d'activité des agents en direct, une répartition de l'utilisation des modèles par agent, et des conclusions rédigées, avec les requêtes exécutées affichées en ligne](/agenteye/images/assistant.png) +![L'assistant FailproofAI Cloud répondant à une question en langage naturel dans le tableau de bord, affichant un tableau d'activité des agents en direct, une répartition de l'utilisation des modèles par agent, et des conclusions rédigées, avec les requêtes exécutées affichées en ligne](/cloud/images/assistant.png) *Posez votre question en langage naturel et obtenez une réponse construite à partir de vos propres données. Ici, l'assistant décompose quels agents sont les plus actifs et quels modèles ils utilisent, et affiche les requêtes exécutées pour que vous puissiez vérifier chaque chiffre.* Rien à apprendre. Ouvrez le chat, tapez ce que vous voulez savoir et suivez les liens qu'il vous renvoie : @@ -36,7 +36,7 @@ Lorsqu'une réponse mérite d'être conservée, demandez à l'assistant de la sa Sur la page **Queries**, il va encore plus loin et devient un auteur SQL : décrivez la requête souhaitée (« afficher le taux d'erreur par agent sur les 7 derniers jours ») et il diffuse le SQL directement dans l'éditeur, en ouvrant une vue diff afin que vous puissiez **Accepter** ou **Rejeter** la modification avant qu'elle ne soit appliquée. -![La page Queries d'Observability et son éditeur SQL](/agenteye/images/query-lab.png) +![La page Queries d'FailproofAI Cloud et son éditeur SQL](/cloud/images/query-lab.png) *La page Queries : cet éditeur est l'endroit où l'assistant diffuse un brouillon de requête en lecture seule que vous acceptez ou rejetez.* La création de SQL par cette méthode utilise la permission `queries:run`, la même que celle du bouton **Run** de l'éditeur. Le chat partout ailleurs nécessite `agent:use`. @@ -49,7 +49,7 @@ Vous pouvez ouvrir l'assistant à tous sans vous inquiéter de ce qu'il pourrait - **Chaque écriture attend votre confirmation.** Les requêtes sauvegardées et les tableaux de bord ne sont créés qu'après votre clic explicite sur Approuver, et aucun paramètre ne désactive cette validation. - **Il ne peut jamais rien supprimer.** Aucun outil de suppression n'est exposé et l'assistant ne détient aucune permission de suppression. Les suppressions restent entre vos mains, dans le tableau de bord. - **Il reste dans votre organisation.** L'assistant ne voit que l'organisation que vous consultez actuellement. -- **Vos questions vous appartiennent.** Les invites et les réponses sont stockées dans votre propre base de données Observability ; l'analytique produit n'enregistre que les métadonnées d'utilisation, jamais le texte de vos invites. +- **Vos questions vous appartiennent.** Les invites et les réponses sont stockées dans votre propre base de données FailproofAI Cloud ; l'analytique produit n'enregistre que les métadonnées d'utilisation, jamais le texte de vos invites. ## Où le trouver @@ -57,7 +57,7 @@ L'assistant est présent sur le bord droit de chaque page sous votre organisatio ## Voir aussi -- [CLI et agents](/fr/agenteye/cli-and-agents) -- [Queries](/fr/agenteye/queries) -- [Tableaux de bord](/fr/agenteye/dashboards) -- [Suite d'évaluation](/fr/agenteye/evaluation-suite) \ No newline at end of file +- [CLI et agents](/fr/cloud/cli) +- [Queries](/fr/cloud/queries) +- [Tableaux de bord](/fr/cloud/dashboards) +- [Suite d'évaluation](/fr/cloud/evaluators) \ No newline at end of file diff --git a/docs/fr/agenteye/audits.mdx b/docs/fr/cloud/audits.mdx similarity index 82% rename from docs/fr/agenteye/audits.mdx rename to docs/fr/cloud/audits.mdx index eb0df4e6..e3539cce 100644 --- a/docs/fr/agenteye/audits.mdx +++ b/docs/fr/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "Audits : votre analyste de fiabilité automatique" -description: "Failproof AI Observability détecte les défaillances pour lesquelles vous n'avez jamais défini de règle et vous remet une liste de priorités classées, étayées par des preuves, indiquant précisément quoi corriger." +description: "FailproofAI Cloud détecte les défaillances pour lesquelles vous n'avez jamais défini de règle et vous remet une liste de priorités classées, étayées par des preuves, indiquant précisément quoi corriger." --- -Failproof AI Observability détecte les défaillances pour lesquelles vous n'avez jamais défini de règle et vous remet une liste de priorités classées, étayées par des preuves, indiquant précisément quoi corriger. C'est comme avoir un analyste qui parcourt vos logs chaque nuit et vous dépose un résumé sur le bureau chaque matin. +FailproofAI Cloud détecte les défaillances pour lesquelles vous n'avez jamais défini de règle et vous remet une liste de priorités classées, étayées par des preuves, indiquant précisément quoi corriger. C'est comme avoir un analyste qui parcourt vos logs chaque nuit et vous dépose un résumé sur le bureau chaque matin.
@@ -12,7 +12,7 @@ Failproof AI Observability détecte les défaillances pour lesquelles vous n'ave *Un tour d'horizon en deux minutes : d'une exécution planifiée à une correction sur laquelle vous pouvez agir.* -![La page Audits : des tâches récurrentes qui analysent vos sessions à la recherche de schémas d'échec, chacune avec une planification et une sensibilité](/agenteye/images/audits.png) +![La page Audits : des tâches récurrentes qui analysent vos sessions à la recherche de schémas d'échec, chacune avec une planification et une sensibilité](/cloud/images/audits.png) *Chaque audit est une tâche récurrente qui fouille vos sessions et rédige des recommandations classées et étayées par des preuves.* ## Arrêtez de deviner quoi corriger ensuite @@ -48,7 +48,7 @@ Les audits se trouvent dans le tableau de bord à **`//audits`** (barr ## Voir aussi -- [Alerts](/fr/agenteye/alerts) : soyez notifié dès qu'un seuil que vous connaissez déjà est franchi. -- [Evaluations](/fr/agenteye/evaluations) : notez chaque exécution afin que les régressions de qualité remontent d'elles-mêmes. -- [Error tracking](/fr/agenteye/error-tracking) : regroupez et suivez les erreurs que vos agents génèrent. -- [Incidents](/fr/agenteye/incidents) : suivez un problème détecté par un audit jusqu'à sa résolution. \ No newline at end of file +- [Alerts](/fr/cloud/alerts) : soyez notifié dès qu'un seuil que vous connaissez déjà est franchi. +- [Evaluations](/fr/cloud/evaluations) : notez chaque exécution afin que les régressions de qualité remontent d'elles-mêmes. +- [Error tracking](/fr/cloud/errors) : regroupez et suivez les erreurs que vos agents génèrent. +- [Incidents](/fr/cloud/incidents) : suivez un problème détecté par un audit jusqu'à sa résolution. \ No newline at end of file diff --git a/docs/fr/cloud/capture.mdx b/docs/fr/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/fr/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/fr/agenteye/cli-recipes.mdx b/docs/fr/cloud/cli-recipes.mdx similarity index 91% rename from docs/fr/agenteye/cli-recipes.mdx rename to docs/fr/cloud/cli-recipes.mdx index 34943c0d..a3177143 100644 --- a/docs/fr/agenteye/cli-recipes.mdx +++ b/docs/fr/cloud/cli-recipes.mdx @@ -4,9 +4,9 @@ description: "Patterns de requêtes à copier-coller et recettes jq qui transfor --- -Récupérez les données de sessions, d'événements et d'évaluations (et déclenchez des réévaluations) directement depuis un script ou un agent de codage, avec du JSON propre sur stdout qui s'enchaîne directement dans `jq`. Ces recettes transforment les données de Failproof AI Observability en quelque chose qu'un utilisateur de terminal ou un agent de codage IA (Claude Code, Cursor) peut interroger et automatiser, sans cliquer dans le tableau de bord. +Récupérez les données de sessions, d'événements et d'évaluations (et déclenchez des réévaluations) directement depuis un script ou un agent de codage, avec du JSON propre sur stdout qui s'enchaîne directement dans `jq`. Ces recettes transforment les données de FailproofAI Cloud en quelque chose qu'un utilisateur de terminal ou un agent de codage IA (Claude Code, Cursor) peut interroger et automatiser, sans cliquer dans le tableau de bord. -Les patterns ci-dessous sont prêts à être copiés-collés pour la CLI Failproof AI Observability (`agenteye`). Pour l'installation, l'authentification et la liste complète des options, consultez [CLI](/fr/agenteye/cli) ; exécutez `agenteye -h` ou `agenteye -h` pour l'aide intégrée. +Les patterns ci-dessous sont prêts à être copiés-collés pour la CLI FailproofAI Cloud (`agenteye`). Pour l'installation, l'authentification et la liste complète des options, consultez [CLI](/fr/cloud/cli) ; exécutez `agenteye -h` ou `agenteye -h` pour l'aide intégrée. ## Règles d'or @@ -173,7 +173,7 @@ Le `--fields` de chaque commande accepte exactement les noms de champs de ses pr ## Étapes suivantes -- [CLI](/fr/agenteye/cli) : installation, authentification et référence complète des options pour chaque commande. -- [Compétence CLI pour agent](/fr/agenteye/cli-skill) : regroupez ces recettes en une compétence que votre agent de codage peut charger. -- [Clés API](/fr/agenteye/api-keys) : créez et délimitez les clés avec lesquelles la CLI, le SDK et le collecteur s'authentifient. -- [SDK Python](/fr/agenteye/python-sdk) : envoyez des événements dans Failproof AI Observability pour que ces recettes aient des données à interroger. \ No newline at end of file +- [CLI](/fr/cloud/cli) : installation, authentification et référence complète des options pour chaque commande. +- [Compétence CLI pour agent](/fr/cloud/agent-skills) : regroupez ces recettes en une compétence que votre agent de codage peut charger. +- [Clés API](/fr/cloud/access) : créez et délimitez les clés avec lesquelles la CLI, le SDK et le collecteur s'authentifient. +- [SDK Python](/fr/cloud/sdk) : envoyez des événements dans FailproofAI Cloud pour que ces recettes aient des données à interroger. \ No newline at end of file diff --git a/docs/fr/agenteye/cli.mdx b/docs/fr/cloud/cli.mdx similarity index 97% rename from docs/fr/agenteye/cli.mdx rename to docs/fr/cloud/cli.mdx index f9b7259a..f4446cdf 100644 --- a/docs/fr/agenteye/cli.mdx +++ b/docs/fr/cloud/cli.mdx @@ -338,13 +338,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | Authentifié, mais votre compte ne dispose pas de la permission requise (le message la nomme) | | 6 | La ressource demandée est introuvable (ex. session ou id d'incident inconnu) | -Ces codes rendent la CLI sûre à scripter : un agent de code peut brancher sur un `4` pour vous inviter à vous ré-authentifier, ou sur un `5` pour signaler la permission manquante. Voir [Recettes CLI pour les agents](/fr/agenteye/cli-recipes) pour les modèles de gestion des codes de sortie et les structures de sortie JSON. +Ces codes rendent la CLI sûre à scripter : un agent de code peut brancher sur un `4` pour vous inviter à vous ré-authentifier, ou sur un `5` pour signaler la permission manquante. Voir [Recettes CLI pour les agents](/fr/cloud/cli-recipes) pour les modèles de gestion des codes de sortie et les structures de sortie JSON. --- ## Prochaines étapes -- **[Recettes CLI pour les agents](/fr/agenteye/cli-recipes)** : modèles de requêtes à copier-coller, one-liners `jq`, projections `--fields`, gestion des codes de sortie et structures de sortie JSON, écrits pour les agents de code qui pilotent la CLI. -- **[Compétence CLI pour agent](/fr/agenteye/cli-skill)** : packagée cette CLI comme une *compétence* installable Claude Code / Codex afin qu'un agent de code pilote l'Observabilité Failproof AI à partir de requêtes en langage naturel. -- **[Clés API](/fr/agenteye/api-keys)** : le modèle de permissions derrière `keys create --add …`. -- **[Assistant IA](/fr/agenteye/assistant)** : activation de l'assistant qu'`agent ask` utilise. \ No newline at end of file +- **[Recettes CLI pour les agents](/fr/cloud/cli-recipes)** : modèles de requêtes à copier-coller, one-liners `jq`, projections `--fields`, gestion des codes de sortie et structures de sortie JSON, écrits pour les agents de code qui pilotent la CLI. +- **[Compétence CLI pour agent](/fr/cloud/agent-skills)** : packagée cette CLI comme une *compétence* installable Claude Code / Codex afin qu'un agent de code pilote l'Observabilité Failproof AI à partir de requêtes en langage naturel. +- **[Clés API](/fr/cloud/access)** : le modèle de permissions derrière `keys create --add …`. +- **[Assistant IA](/fr/cloud/assistant)** : activation de l'assistant qu'`agent ask` utilise. \ No newline at end of file diff --git a/docs/fr/cloud/connect.mdx b/docs/fr/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/fr/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/fr/agenteye/dashboards.mdx b/docs/fr/cloud/dashboards.mdx similarity index 71% rename from docs/fr/agenteye/dashboards.mdx rename to docs/fr/cloud/dashboards.mdx index eaf690cc..de56294c 100644 --- a/docs/fr/agenteye/dashboards.mdx +++ b/docs/fr/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "Transformez vos données d'agents en temps réel en une vue partag Transformez vos données d'agents en temps réel en une vue partagée que toute votre équipe consulte. Épinglez les requêtes importantes sous forme de graphiques, et tout le monde accède instantanément aux mêmes chiffres, sans avoir à relancer une seule requête. -![Un tableau de bord construit à partir de requêtes sauvegardées : une courbe d'événements par heure, un histogramme des erreurs par type, un graphique en aire de la latence, et une répartition des tokens par modèle](/agenteye/images/dashboard-fleet.png) +![Un tableau de bord construit à partir de requêtes sauvegardées : une courbe d'événements par heure, un histogramme des erreurs par type, un graphique en aire de la latence, et une répartition des tokens par modèle](/cloud/images/dashboard-fleet.png) *Un tableau de bord, quatre requêtes sauvegardées : événements par heure, erreurs par type, latence et tokens par modèle.* @@ -25,15 +25,15 @@ Vous trouverez vos tableaux de bord à `//dashboards`. ## Épinglez les requêtes que vous avez déjà sauvegardées -Chaque vignette commence par une requête sauvegardée. Créez et sauvegardez la requête qui vous intéresse dans la bibliothèque [Requêtes](/fr/agenteye/queries) (préréglages intégrés et requêtes personnalisées, sur vos événements et évaluations), puis épinglez-la sur un tableau de bord sous la forme du graphique adapté à vos données : une **courbe** pour les tendances dans le temps, un **histogramme** pour comparer des catégories, une **aire** pour les volumes, ou un **camembert** pour une répartition en parts. +Chaque vignette commence par une requête sauvegardée. Créez et sauvegardez la requête qui vous intéresse dans la bibliothèque [Requêtes](/fr/cloud/queries) (préréglages intégrés et requêtes personnalisées, sur vos événements et évaluations), puis épinglez-la sur un tableau de bord sous la forme du graphique adapté à vos données : une **courbe** pour les tendances dans le temps, un **histogramme** pour comparer des catégories, une **aire** pour les volumes, ou un **camembert** pour une répartition en parts. Puisqu'une vignette n'est que votre requête sauvegardée affichée sous forme de graphique, rien n'est à synchroniser manuellement. Mettez à jour la requête une fois, et tous les tableaux de bord qui l'utilisent se mettent à jour automatiquement. ## Surveillez la qualité, pas seulement le volume -Le volume vous indique que les agents sont actifs. La qualité vous indique qu'ils font réellement leur travail. Orientez un tableau de bord vers vos [scores d'évaluation](/fr/agenteye/evaluations) et vous obtenez un tableau qui suit la qualité des exécutions dans le temps : une régression de qualité apparaît comme un creux sur un graphique, plutôt que comme une mauvaise surprise venue d'un client. +Le volume vous indique que les agents sont actifs. La qualité vous indique qu'ils font réellement leur travail. Orientez un tableau de bord vers vos [scores d'évaluation](/fr/cloud/evaluations) et vous obtenez un tableau qui suit la qualité des exécutions dans le temps : une régression de qualité apparaît comme un creux sur un graphique, plutôt que comme une mauvaise surprise venue d'un client. -![Un tableau de bord axé sur la qualité, construit à partir de requêtes d'évaluation sauvegardées](/agenteye/images/dashboard-quality.png) +![Un tableau de bord axé sur la qualité, construit à partir de requêtes d'évaluation sauvegardées](/cloud/images/dashboard-quality.png) *Un tableau de bord qualité garde vos scores d'évaluation au premier plan, juste à côté des métriques opérationnelles.* @@ -41,6 +41,6 @@ Maintenez un tableau de bord opérationnel et un tableau de bord qualité côte ## Voir aussi -- [Requêtes](/fr/agenteye/queries) : créez et sauvegardez les requêtes qui deviendront vos vignettes. -- [Évaluations](/fr/agenteye/evaluations) : scorez vos exécutions pour pouvoir suivre la qualité dans le temps. -- [Alertes](/fr/agenteye/alerts) : transformez un seuil sur n'importe laquelle de ces métriques en une notification. \ No newline at end of file +- [Requêtes](/fr/cloud/queries) : créez et sauvegardez les requêtes qui deviendront vos vignettes. +- [Évaluations](/fr/cloud/evaluations) : scorez vos exécutions pour pouvoir suivre la qualité dans le temps. +- [Alertes](/fr/cloud/alerts) : transformez un seuil sur n'importe laquelle de ces métriques en une notification. \ No newline at end of file diff --git a/docs/fr/agenteye/error-tracking.mdx b/docs/fr/cloud/errors.mdx similarity index 66% rename from docs/fr/agenteye/error-tracking.mdx rename to docs/fr/cloud/errors.mdx index 178ca8ce..7574dca6 100644 --- a/docs/fr/agenteye/error-tracking.mdx +++ b/docs/fr/cloud/errors.mdx @@ -5,14 +5,14 @@ description: "Visualisez en un seul endroit toutes les défaillances de vos agen Visualisez en un seul endroit toutes les défaillances de vos agents, regroupées pour qu'une rafale d'erreurs apparaisse comme un problème unique. Vous disposez d'un accès en un clic entre « quelque chose est rouge » et l'exécution exacte qui a échoué, sans avoir à parcourir un flux en direct pour la retrouver. -![La page Erreurs : un histogramme des défaillances au fil du temps au-dessus de lignes d'erreurs rouges groupées, chacune avec un bouton « + alert » en un clic](/agenteye/images/errors.png) +![La page Erreurs : un histogramme des défaillances au fil du temps au-dessus de lignes d'erreurs rouges groupées, chacune avec un bouton « + alert » en un clic](/cloud/images/errors.png) *La page Erreurs : un histogramme des défaillances au fil du temps, avec les erreurs répétées regroupées en une seule ligne par incident.* ## Toutes les défaillances, déjà collectées pour vous Quand un agent tombe en panne, vous ne devriez pas avoir à parcourir un flux d'événements en direct en espérant repérer les lignes rouges avant qu'elles disparaissent. La page **Errors** se charge de la collecte à votre place. Elle rassemble tout ce que le tableau de bord afficherait en rouge dans une interface de triage unique, de sorte que la première chose que vous voyez est ce qui échoue, et non l'endroit où chercher. -Et elle détecte bien plus que les erreurs évidentes. En plus des événements `error` explicites, Failproof AI Observability remonte également les défaillances silencieuses : tout `tool_result`, `hook_completed` ou `agent_end` dont le contenu indique un échec apparaît ici. Un outil ayant retourné une erreur, ou un hook s'étant terminé de manière anormale, ne passe plus inaperçu simplement parce qu'aucune exception bruyante n'a été levée. +Et elle détecte bien plus que les erreurs évidentes. En plus des événements `error` explicites, FailproofAI Cloud remonte également les défaillances silencieuses : tout `tool_result`, `hook_completed` ou `agent_end` dont le contenu indique un échec apparaît ici. Un outil ayant retourné une erreur, ou un hook s'étant terminé de manière anormale, ne passe plus inaperçu simplement parce qu'aucune exception bruyante n'a été levée. En haut de la page, un histogramme trace l'évolution des erreurs dans le temps. Un simple coup d'œil vous indique s'il s'agit d'un filet constant en arrière-plan ou d'un pic apparu il y a quelques minutes, vous permettant de décider immédiatement si vous devez tout laisser tomber. @@ -22,19 +22,19 @@ Comme toutes les surfaces d'observation, la page Errors est limitée à votre or Une dépendance défaillante peut déclencher la même erreur des centaines de fois par minute. Sans regroupement, cela donne un mur de lignes quasi identiques qui noie l'information dont vous avez vraiment besoin. -Failproof AI Observability regroupe les défaillances répétées partageant la même session et le même type d'erreur en une seule ligne. Une rafale apparaît comme un seul incident. Vous comptez des problèmes, pas des lignes de log, et le signal qui compte reste en évidence au lieu d'être noyé par son propre volume. +FailproofAI Cloud regroupe les défaillances répétées partageant la même session et le même type d'erreur en une seule ligne. Une rafale apparaît comme un seul incident. Vous comptez des problèmes, pas des lignes de log, et le signal qui compte reste en évidence au lieu d'être noyé par son propre volume. ## De « quelque chose est rouge » à l'événement exact Cliquez sur n'importe quelle ligne pour accéder directement à la session de cette exécution, positionné sur l'événement exact qui a échoué. Pas besoin de copier des identifiants de session ni de faire défiler pour trouver le moment de la rupture : vous arrivez directement dessus, avec le graphe d'exécution complet à portée de regard pour voir ce que l'agent faisait dans les instants précédant la défaillance. -Si vous disposez de `alerts:write`, chaque ligne comporte également un bouton **+ alert**. Cliquez dessus et Observability ouvre une nouvelle règle d'alerte déjà configurée pour détecter ce même type de défaillance. L'incident que vous venez de traiter deviendra celui qui vous alerte la prochaine fois, au lieu de vous surprendre une deuxième fois. +Si vous disposez de `alerts:write`, chaque ligne comporte également un bouton **+ alert**. Cliquez dessus et FailproofAI Cloud ouvre une nouvelle règle d'alerte déjà configurée pour détecter ce même type de défaillance. L'incident que vous venez de traiter deviendra celui qui vous alerte la prochaine fois, au lieu de vous surprendre une deuxième fois. **Où le trouver :** la page **Errors** se trouve dans la section observe du tableau de bord, à l'adresse `//errors`. ## Ressources associées -- [Alerts](/fr/agenteye/alerts) : transformez n'importe quelle défaillance en règle d'alerte. -- [Incidents](/fr/agenteye/incidents) : suivez une alerte déclenchée de son ouverture à sa résolution. -- [Sessions](/fr/agenteye/sessions) : ouvrez l'exécution complète derrière n'importe quelle erreur. -- [Audits](/fr/agenteye/audits) : laissez Observability identifier les schémas de défaillance dans vos exécutions. \ No newline at end of file +- [Alerts](/fr/cloud/alerts) : transformez n'importe quelle défaillance en règle d'alerte. +- [Incidents](/fr/cloud/incidents) : suivez une alerte déclenchée de son ouverture à sa résolution. +- [Sessions](/fr/cloud/sessions) : ouvrez l'exécution complète derrière n'importe quelle erreur. +- [Audits](/fr/cloud/audits) : laissez FailproofAI Cloud identifier les schémas de défaillance dans vos exécutions. \ No newline at end of file diff --git a/docs/fr/agenteye/evaluations.mdx b/docs/fr/cloud/evaluations.mdx similarity index 67% rename from docs/fr/agenteye/evaluations.mdx rename to docs/fr/cloud/evaluations.mdx index d442dd0f..1bd5322b 100644 --- a/docs/fr/agenteye/evaluations.mdx +++ b/docs/fr/cloud/evaluations.mdx @@ -3,15 +3,15 @@ title: "Évaluations" description: "Les problèmes de qualité viennent à vous, au lieu d'en entendre parler dans une réclamation utilisateur." --- -Les problèmes de qualité viennent à vous, au lieu d'en entendre parler dans une réclamation utilisateur. Connectez votre propre service de scoring une seule fois et Failproof AI Observability note chaque exécution terminée automatiquement — ainsi, une baisse d'utilité ou une hausse des hallucinations apparaît d'elle-même, avant qu'un client ne le ressente. +Les problèmes de qualité viennent à vous, au lieu d'en entendre parler dans une réclamation utilisateur. Connectez votre propre service de scoring une seule fois et FailproofAI Cloud note chaque exécution terminée automatiquement — ainsi, une baisse d'utilité ou une hausse des hallucinations apparaît d'elle-même, avant qu'un client ne le ressente. -![La grille des sessions avec une colonne de scores : chaque exécution porte un badge d'état d'évaluation et des indicateurs codés par couleur pour l'utilité, la factualité et l'efficacité des outils](/agenteye/images/sessions-list.png) +![La grille des sessions avec une colonne de scores : chaque exécution porte un badge d'état d'évaluation et des indicateurs codés par couleur pour l'utilité, la factualité et l'efficacité des outils](/cloud/images/sessions-list.png) *Chaque exécution dans la grille des sessions affiche ses scores ; les badges rouges, ambrés et verts font ressortir les exécutions faibles sans que vous ayez à ouvrir une seule transcription.* ## Arrêtez de contrôler manuellement les exécutions -Vous vérifiez encore quelques exécutions au hasard en espérant que le reste est correct. Désormais, chaque session terminée est scorée au moment où elle se termine, selon les dimensions qui vous importent : utilité, efficacité des outils, factualité, sécurité, quel que soit votre seuil de qualité. Vous définissez les clés de score ; Failproof AI Observability stocke, suit les tendances et affiche tout ce que votre évaluateur renvoie. Aucune exécution ne passe sans être scorée, et vous n'apprendrez plus une régression via un ticket de support. +Vous vérifiez encore quelques exécutions au hasard en espérant que le reste est correct. Désormais, chaque session terminée est scorée au moment où elle se termine, selon les dimensions qui vous importent : utilité, efficacité des outils, factualité, sécurité, quel que soit votre seuil de qualité. Vous définissez les clés de score ; FailproofAI Cloud stocke, suit les tendances et affiche tout ce que votre évaluateur renvoie. Aucune exécution ne passe sans être scorée, et vous n'apprendrez plus une régression via un ticket de support. Les scores apparaissent dans la grille des sessions à **`//sessions`** (barre latérale → *observe* → *sessions*), avec un groupe de badges par ligne. Vous voulez uniquement les exécutions en dessous du seuil ? Filtrez la grille par plage de scores — par exemple, une utilité inférieure à 0,5 — pour afficher exactement les exécutions qui méritent d'être lues. La consultation des scores nécessite la permission `evaluations:read`. @@ -19,7 +19,7 @@ Les scores apparaissent dans la grille des sessions à **`//sessions`* Un chiffre vous indique qu'une exécution était faible ; la page de session vous explique pourquoi. Ouvrez n'importe quelle exécution et le panneau de droite commence par le résumé principal, puis affiche une barre par dimension avec le raisonnement de votre évaluateur sous chacune — ainsi, vous passez de « cette exécution a obtenu 0,4 en factualité » à l'affirmation exacte qui était incorrecte en quelques secondes. -![Le panneau droit d'une session : le résumé de l'évaluation en haut, puis des barres de score par dimension avec une ligne de raisonnement pour chacune, à côté de la chronologie complète des événements](/agenteye/images/session-detail.png) +![Le panneau droit d'une session : le résumé de l'évaluation en haut, puis des barres de score par dimension avec une ligne de raisonnement pour chacune, à côté de la chronologie complète des événements](/cloud/images/session-detail.png) *La vue détaillée d'une session : résumé, barres de score par dimension et le raisonnement derrière chaque score, juste à côté de la chronologie des événements de l'exécution.* @@ -29,7 +29,7 @@ Vous avez déployé un évaluateur plus précis, ou vous regardez une exécution Une exécution avec un score faible est du bruit ; toute une cohorte qui glisse est un signal. Les tableaux de bord sauvegardés transforment vos scores en une tendance que vous pouvez surveiller d'un coup d'œil : utilité moyenne cette semaine par rapport à la semaine dernière, par agent, par environnement. -![Un tableau de bord qualité : barres de score moyen par dimension d'évaluation accompagnées d'une tendance dans le temps](/agenteye/images/dashboard-quality.png) +![Un tableau de bord qualité : barres de score moyen par dimension d'évaluation accompagnées d'une tendance dans le temps](/cloud/images/dashboard-quality.png) *Un tableau de bord qualité sauvegardé suit les clés de score que vous mettez en avant, afin qu'une dérive progressive soit évidente bien avant de devenir un incident.* @@ -37,14 +37,14 @@ Les tableaux de bord se trouvent à **`//dashboards`** (barre latéral ## Connecter un évaluateur une seule fois -Le scoring est optionnel et reste complètement désactivé jusqu'à ce que vous pointiez Failproof AI Observability vers un scorer. Vous déployez un petit service HTTP (Observability fournit une référence fonctionnelle que vous pouvez copier), définissez deux valeurs sur votre serveur, et chaque exécution à partir de ce moment est scorée pour vous. Le guide complet, le contrat de scoring et le SDK se trouvent dans le guide approfondi. +Le scoring est optionnel et reste complètement désactivé jusqu'à ce que vous pointiez FailproofAI Cloud vers un scorer. Vous déployez un petit service HTTP (FailproofAI Cloud fournit une référence fonctionnelle que vous pouvez copier), définissez deux valeurs sur votre serveur, et chaque exécution à partir de ce moment est scorée pour vous. Le guide complet, le contrat de scoring et le SDK se trouvent dans le guide approfondi. -Vous ne savez pas quelles dimensions valent la peine d'être scorées ? La [compétence d'agent évaluateur](/fr/agenteye/evaluator-skill) fait travailler votre agent de code pour les déterminer à partir de vos propres sessions, puis construire et déployer le service. +Vous ne savez pas quelles dimensions valent la peine d'être scorées ? La [compétence d'agent évaluateur](/fr/cloud/agent-skills) fait travailler votre agent de code pour les déterminer à partir de vos propres sessions, puis construire et déployer le service. ## Liens connexes -- [Suite d'évaluation](/fr/agenteye/evaluation-suite) : connecter votre évaluateur, le contrat de scoring et le SDK. -- [Compétence d'agent évaluateur](/fr/agenteye/evaluator-skill) : laissez un agent de code choisir vos dimensions de score et construire l'évaluateur. -- [Sessions](/fr/agenteye/sessions) : la grille exécution par exécution où les scores apparaissent. -- [Tableaux de bord](/fr/agenteye/dashboards) : sauvegardez et partagez les tendances de qualité dans votre organisation. -- [Audits](/fr/agenteye/audits) : l'autre fonctionnalité de qualité automatique d'Observability, pour les investigations inter-sessions. \ No newline at end of file +- [Suite d'évaluation](/fr/cloud/evaluators) : connecter votre évaluateur, le contrat de scoring et le SDK. +- [Compétence d'agent évaluateur](/fr/cloud/agent-skills) : laissez un agent de code choisir vos dimensions de score et construire l'évaluateur. +- [Sessions](/fr/cloud/sessions) : la grille exécution par exécution où les scores apparaissent. +- [Tableaux de bord](/fr/cloud/dashboards) : sauvegardez et partagez les tendances de qualité dans votre organisation. +- [Audits](/fr/cloud/audits) : l'autre fonctionnalité de qualité automatique d'FailproofAI Cloud, pour les investigations inter-sessions. \ No newline at end of file diff --git a/docs/fr/agenteye/evaluation-suite.mdx b/docs/fr/cloud/evaluators.mdx similarity index 89% rename from docs/fr/agenteye/evaluation-suite.mdx rename to docs/fr/cloud/evaluators.mdx index df6567ac..6a9ce7d5 100644 --- a/docs/fr/agenteye/evaluation-suite.mdx +++ b/docs/fr/cloud/evaluators.mdx @@ -1,20 +1,20 @@ --- title: "Suite d'évaluation" -description: "Failproof AI Observability peut noter automatiquement chaque exécution d'agent terminée pour en évaluer la qualité : vous fournissez un petit service de notation, et Observability s'occupe du reste." +description: "FailproofAI Cloud peut noter automatiquement chaque exécution d'agent terminée pour en évaluer la qualité : vous fournissez un petit service de notation, et FailproofAI Cloud s'occupe du reste." --- -Failproof AI Observability peut noter automatiquement chaque exécution d'agent terminée pour en évaluer la qualité : vous fournissez un petit service de notation, et Observability s'occupe du reste. Utilisez-le pour suivre les dimensions qui vous importent (utilité, efficacité des outils, factualité, sécurité — vous choisissez), détecter les régressions tôt et comparer des agents ou des environnements en un coup d'œil. La notation est optionnelle : le pipeline ne fait rien tant que vous n'avez pas défini `EVALUATOR_ENDPOINT` sur le serveur. +FailproofAI Cloud peut noter automatiquement chaque exécution d'agent terminée pour en évaluer la qualité : vous fournissez un petit service de notation, et FailproofAI Cloud s'occupe du reste. Utilisez-le pour suivre les dimensions qui vous importent (utilité, efficacité des outils, factualité, sécurité — vous choisissez), détecter les régressions tôt et comparer des agents ou des environnements en un coup d'œil. La notation est optionnelle : le pipeline ne fait rien tant que vous n'avez pas défini `EVALUATOR_ENDPOINT` sur le serveur. -> **Remarque :** Vous définissez vous-même les dimensions de notation. Votre évaluateur peut retourner les clés numériques de son choix ; Observability stocke, suit les tendances et affiche tout ce que vous renvoyez. +> **Remarque :** Vous définissez vous-même les dimensions de notation. Votre évaluateur peut retourner les clés numériques de son choix ; FailproofAI Cloud stocke, suit les tendances et affiche tout ce que vous renvoyez. ## En bref -1. **Écrivez un évaluateur.** Déployez un petit service HTTP qui lit la transcription d'une session et retourne des scores. Observability inclut une référence fonctionnelle que vous pouvez copier. Voir [Écrire un évaluateur avec le SDK](#writing-an-evaluator-with-the-sdk). -2. **Pointez Observability vers ce service.** Définissez `EVALUATOR_ENDPOINT` (et un `EVALUATOR_TOKEN` partagé) sur le processus serveur. +1. **Écrivez un évaluateur.** Déployez un petit service HTTP qui lit la transcription d'une session et retourne des scores. FailproofAI Cloud inclut une référence fonctionnelle que vous pouvez copier. Voir [Écrire un évaluateur avec le SDK](#writing-an-evaluator-with-the-sdk). +2. **Pointez FailproofAI Cloud vers ce service.** Définissez `EVALUATOR_ENDPOINT` (et un `EVALUATOR_TOKEN` partagé) sur le processus serveur. 3. **Regardez les scores arriver.** Chaque session terminée est notée automatiquement ; les résultats apparaissent sur la page de détail de la session, la grille des sessions et les tableaux de bord sauvegardés. -![Vue de détail d'une session avec le résumé de l'évaluation, les barres de score par dimension et le texte de justification dans le rail droit](/agenteye/images/session-detail.png) +![Vue de détail d'une session avec le résumé de l'évaluation, les barres de score par dimension et le texte de justification dans le rail droit](/cloud/images/session-detail.png) *Une fois un évaluateur configuré, chaque exécution terminée est notée et les résultats apparaissent dans le rail droit de la session : le résumé en haut, puis les barres de score par dimension avec leur justification.* @@ -24,7 +24,7 @@ Failproof AI Observability peut noter automatiquement chaque exécution d'agent ```mermaid flowchart LR - ING["ingest /events
agent_end"] --> SRV["Observability server"] + ING["ingest /events
agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -32,26 +32,26 @@ flowchart LR SRV --> RES["evaluations
terminal results"] ``` -Lorsque le SDK Observability émet un événement `agent_end` pour une session, le serveur +Lorsque le SDK FailproofAI Cloud émet un événement `agent_end` pour une session, le serveur planifie une évaluation. Il envoie ensuite en POST la transcription complète des événements à votre service d'évaluation, qui peut alors : - **Retourner le résultat immédiatement** avec `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`. Le résultat est ajouté à la chronologie d'évaluation de la session. `reasoning` et `summary` sont optionnels. -- **Différer** avec `{"status":"pending", "job_id":"abc-123"}`. Observability appelle alors +- **Différer** avec `{"status":"pending", "job_id":"abc-123"}`. FailproofAI Cloud appelle alors `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` jusqu'à ce que votre évaluateur retourne `{"status":"done", ...}` ou `{"status":"error", "error":"..."}`. La cadence de polling est par tâche : une réponse `pending` peut inclure - `next_poll_secs` pour la surcharger ; sinon Observability utilise la valeur + `next_poll_secs` pour la surcharger ; sinon FailproofAI Cloud utilise la valeur `default_poll_interval_secs` issue de `GET /config` ; sinon le serveur se rabat sur `EVALUATOR_POLLING_INTERVAL_SECS` (défaut : 10 s). Toutes les valeurs sont limitées à [1 s, 1 h]. Les sessions qui n'émettent jamais `agent_end` (par exemple, un processus d'agent planté) peuvent également être traitées : le `GET /config` de l'évaluateur peut retourner -`{"inactivity_timeout_secs": 1800}`, et Observability évaluera toute session +`{"inactivity_timeout_secs": 1800}`, et FailproofAI Cloud évaluera toute session restée inactive pendant ce délai. Définissez le champ à `null` ou omettez-le pour désactiver ce comportement de secours. @@ -75,7 +75,7 @@ en file d'attente. Les échecs transitoires (5xx, 429, délais d'expiration, erreurs réseau) font l'objet de nouvelles tentatives avec backoff exponentiel jusqu'à `EVALUATOR_MAX_ATTEMPTS` ; les réponses 4xx sont -terminales. Observability fonctionne en toute sécurité avec plusieurs instances de serveur à +terminales. FailproofAI Cloud fonctionne en toute sécurité avec plusieurs instances de serveur à échelle horizontale ; le travail est partitionné de sorte qu'une même session ne soit jamais traitée deux fois simultanément. @@ -86,7 +86,7 @@ traitée deux fois simultanément. Toutes les routes authentifiées utilisent **l'authentification par jeton bearer**. La même valeur doit être configurée des deux côtés : -- Serveur Observability : variable d'environnement `EVALUATOR_TOKEN` +- Serveur FailproofAI Cloud : variable d'environnement `EVALUATOR_TOKEN` - Service d'évaluation : configuré de la même façon (le SDK `agenteye-evaluator` lit `EVALUATOR_TOKEN` par convention) @@ -170,7 +170,7 @@ Vous n'avez pas à implémenter le contrat HTTP manuellement. Le package Python `agenteye-evaluator` vous fournit un wrapper FastAPI typé qui gère l'authentification, le routage et les formats requête/réponse à votre place. -Failproof AI Observability inclut également un **évaluateur de référence fonctionnel** qui +FailproofAI Cloud inclut également un **évaluateur de référence fonctionnel** qui note `helpfulness`, `tool_efficiency` et `factuality` à partir de la forme de la transcription. Copiez-le comme point de départ et remplacez-y votre propre logique : un juge LLM, un moteur de règles, ou tout ce qui correspond à vos critères de qualité. @@ -197,7 +197,7 @@ def run(req: EvalRequest) -> EvalResponse: L'instance `app` s'exécute sous n'importe quel serveur ASGI, donc `uvicorn module:app` suffit à la démarrer. Pour les évaluateurs qui ont besoin de différer un traitement coûteux, retournez `JobPending` -à la place et enregistrez un handler `@app.job_lookup` ; le serveur Observability interroge +à la place et enregistrez un handler `@app.job_lookup` ; le serveur FailproofAI Cloud interroge `GET /evaluate/{job_id}` jusqu'à ce que vous retourniez un statut terminal ou que le plafond `EVALUATOR_MAX_POLL_DURATION_SECS` (défaut : 1 h) soit atteint. @@ -208,7 +208,7 @@ le README du SDK `agenteye-evaluator`. ## Exécuter votre évaluateur -L'évaluateur est **votre service** — Failproof AI Observability ne fournit pas d'évaluateur +L'évaluateur est **votre service** — FailproofAI Cloud ne fournit pas d'évaluateur par défaut, vous devez donc le créer et l'exécuter là où vous déployez vos propres services. Il s'exécute sous n'importe quel serveur ASGI (par exemple `uvicorn my_evaluator:app`) ; exposez les routes `/health`, `/config` et `/evaluate` du @@ -327,7 +327,7 @@ L'administrateur bootstrap (`ADMIN_KEY`, `ADMIN_EMAIL`) reçoit automatiquement d'évaluation et les scores de chaque session en un coup d'œil. - **`/dashboards`** : vues de santé d'évaluation sauvegardées (voir [Tableaux de bord](#dashboards) ci-dessous). -![La grille Sessions avec des pastilles de statut d'évaluation par session et des badges de score colorés (helpfulness, factuality, tool_efficiency, safety, coherence)](/agenteye/images/sessions-list.png) +![La grille Sessions avec des pastilles de statut d'évaluation par session et des badges de score colorés (helpfulness, factuality, tool_efficiency, safety, coherence)](/cloud/images/sessions-list.png) *La grille des sessions affiche le statut d'évaluation et les scores de chaque exécution en un coup d'œil ; les badges rouge/orange/vert font ressortir les scores faibles.* @@ -354,7 +354,7 @@ page des sessions pré-filtrée sur exactement cette tranche. Les métriques son côté serveur sur l'ensemble correspondant (via `GET /evaluations/aggregate`), les chiffres sont donc exacts plutôt qu'échantillonnés. -![Un tableau de bord de santé d'évaluation avec des barres de score moyen par dimension d'évaluateur, une répartition outil ok/erreur, les meilleurs outils et une tendance d'événements par heure](/agenteye/images/dashboard-quality.png) +![Un tableau de bord de santé d'évaluation avec des barres de score moyen par dimension d'évaluateur, une répartition outil ok/erreur, les meilleurs outils et une tendance d'événements par heure](/cloud/images/dashboard-quality.png) **Permissions :** la consultation nécessite à la fois `dashboards:read` et `evaluations:read` ; la création et la modification nécessitent `dashboards:write` ; la suppression nécessite `dashboards:delete`. @@ -395,7 +395,7 @@ de plus de temps que la valeur par défaut. ## Prochaines étapes -- [Compétence d'agent évaluateur](/fr/agenteye/evaluator-skill) : demandez à un agent de codage de concevoir vos dimensions à partir de sessions réelles et de créer ce service pour vous. -- [SDK Python](/fr/agenteye/python-sdk) : émettez les événements `agent_end` qui déclenchent la notation. -- [Clés API](/fr/agenteye/api-keys) : les permissions `evaluations:read` et `evaluations:trigger`. -- [Audits](/fr/agenteye/audits) : l'autre fonctionnalité de contrôle qualité automatisé d'Observability, pour la revue basée sur des politiques. \ No newline at end of file +- [Compétence d'agent évaluateur](/fr/cloud/agent-skills) : demandez à un agent de codage de concevoir vos dimensions à partir de sessions réelles et de créer ce service pour vous. +- [SDK Python](/fr/cloud/sdk) : émettez les événements `agent_end` qui déclenchent la notation. +- [Clés API](/fr/cloud/access) : les permissions `evaluations:read` et `evaluations:trigger`. +- [Audits](/fr/cloud/audits) : l'autre fonctionnalité de contrôle qualité automatisé d'FailproofAI Cloud, pour la revue basée sur des politiques. \ No newline at end of file diff --git a/docs/fr/agenteye/event-stream.mdx b/docs/fr/cloud/event-stream.mdx similarity index 85% rename from docs/fr/agenteye/event-stream.mdx rename to docs/fr/cloud/event-stream.mdx index 6cfe42ef..8c5524ec 100644 --- a/docs/fr/agenteye/event-stream.mdx +++ b/docs/fr/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "Au moment où votre agent agit, vous le voyez." Au moment où votre agent agit, vous le voyez. Le flux d'événements est votre pouls en direct sur chaque agent en production : pas d'attente, pas de recherche dans les logs, pas de devinettes sur ce qui vient de se passer. -![Le flux d'événements en direct : lignes d'événements colorées défilant en temps réel, filtrables par environnement, agent, session, type d'événement et texte libre](/agenteye/images/events-stream.png) +![Le flux d'événements en direct : lignes d'événements colorées défilant en temps réel, filtrables par environnement, agent, session, type d'événement et texte libre](/cloud/images/events-stream.png) *Chaque événement de chaque agent de votre organisation, du plus récent au plus ancien, mis à jour au fil de l'eau.* @@ -43,8 +43,8 @@ C'est la source de vérité brute sur laquelle s'appuient toutes les autres surf ## Voir aussi -- [Sessions](/fr/agenteye/sessions) : les mêmes événements regroupés en une ligne par exécution, avec un graphe d'exécution de style git. -- [Telemetry](/fr/agenteye/telemetry) : ce que vos agents envoient et comment les événements parviennent au flux. -- [Suivi des erreurs](/fr/agenteye/error-tracking) : une surface de triage unique pour tout ce qui a mal tourné. -- [Alertes](/fr/agenteye/alerts) : transformez n'importe quel seuil en règle de notification. -- [CLI et agents](/fr/agenteye/cli-and-agents) : la même trace en direct depuis votre terminal. \ No newline at end of file +- [Sessions](/fr/cloud/sessions) : les mêmes événements regroupés en une ligne par exécution, avec un graphe d'exécution de style git. +- [Telemetry](/fr/cloud/performance) : ce que vos agents envoient et comment les événements parviennent au flux. +- [Suivi des erreurs](/fr/cloud/errors) : une surface de triage unique pour tout ce qui a mal tourné. +- [Alertes](/fr/cloud/alerts) : transformez n'importe quel seuil en règle de notification. +- [CLI et agents](/fr/cloud/cli) : la même trace en direct depuis votre terminal. \ No newline at end of file diff --git a/docs/fr/cloud/fleet.mdx b/docs/fr/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/fr/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/fr/agenteye/incidents.mdx b/docs/fr/cloud/incidents.mdx similarity index 84% rename from docs/fr/agenteye/incidents.mdx rename to docs/fr/cloud/incidents.mdx index e85743fa..45531f16 100644 --- a/docs/fr/agenteye/incidents.mdx +++ b/docs/fr/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "Dès qu'une alerte se déclenche, chacun peut voir que l'incident Dès qu'une alerte se déclenche, la première question est toujours « qui s'en occupe ? » Les incidents y répondent : à l'instant où un seuil est franchi, tout le monde peut voir que l'incident est ouvert, qui en est propriétaire, et exactement ce qui s'est passé jusqu'ici, avec un historique propre et attribué que vous pouvez transmettre directement à un post-mortem. -![La boîte de réception des incidents : cartes d'incidents liés à des alertes et ouverts manuellement, regroupées par état, chacune avec un badge de sévérité et un assigné](/agenteye/images/incidents.png) +![La boîte de réception des incidents : cartes d'incidents liés à des alertes et ouverts manuellement, regroupées par état, chacune avec un badge de sévérité et un assigné](/cloud/images/incidents.png) *La boîte de réception regroupe les incidents ouverts par état et permet de filtrer par sévérité et par assigné, afin que vous voyiez immédiatement ce qui nécessite une intervention humaine.* ## Savoir qui s'en occupe, d'un coup d'œil @@ -17,10 +17,10 @@ Fini les « est-ce que quelqu'un regarde ça ? » dans un fil de discussion. Un Quand l'incident est terminé, le compte rendu est déjà prêt. Ouvrez n'importe quel incident et vous obtenez les preuves du dépassement, ses assignés et abonnés, un fil de commentaires pour coordonner sur place, et une chronologie d'activité en ajout seul. -![Une vue détaillée d'un incident : l'alerte parente et le résumé du dépassement, les assignés et abonnés, une chronologie d'activité attribuée, et un fil de commentaires](/agenteye/images/incident-detail.png) +![Une vue détaillée d'un incident : l'alerte parente et le résumé du dépassement, les assignés et abonnés, une chronologie d'activité attribuée, et un fil de commentaires](/cloud/images/incident-detail.png) *Tout ce qui s'est passé, dans l'ordre, chaque ligne signée par celui qui l'a effectuée.* -Chaque action (ouverture, acquittement, résolution, etc.) est écrite dans cette chronologie et n'est jamais modifiée. Chaque entrée est attribuée : à l'opérateur qui l'a effectuée, par e-mail, ou à **automated** pour tout ce que Failproof AI Observability a fait de manière autonome, comme l'ouverture de l'incident lors du dépassement. Rien n'est anonyme et rien n'est perdu, si bien que le post-mortem s'écrit en grande partie tout seul. +Chaque action (ouverture, acquittement, résolution, etc.) est écrite dans cette chronologie et n'est jamais modifiée. Chaque entrée est attribuée : à l'opérateur qui l'a effectuée, par e-mail, ou à **automated** pour tout ce que FailproofAI Cloud a fait de manière autonome, comme l'ouverture de l'incident lors du dépassement. Rien n'est anonyme et rien n'est perdu, si bien que le post-mortem s'écrit en grande partie tout seul. ## Comment un incident évolue @@ -45,6 +45,6 @@ Les incidents se trouvent à `//incidents`. La consultation nécessite ## Voir aussi -- [Alertes](/fr/agenteye/alerts) : les règles qui ouvrent ces incidents lorsqu'un seuil est franchi. -- [Suivi des erreurs](/fr/agenteye/error-tracking) : consultez tous les échecs en un seul endroit et promouvez-en un en alerte. -- [Audits](/fr/agenteye/audits) : l'analyste planifié qui détecte les défaillances qu'aucune règle ne surveillait. \ No newline at end of file +- [Alertes](/fr/cloud/alerts) : les règles qui ouvrent ces incidents lorsqu'un seuil est franchi. +- [Suivi des erreurs](/fr/cloud/errors) : consultez tous les échecs en un seul endroit et promouvez-en un en alerte. +- [Audits](/fr/cloud/audits) : l'analyste planifié qui détecte les défaillances qu'aucune règle ne surveillait. \ No newline at end of file diff --git a/docs/fr/cloud/managed-policies.mdx b/docs/fr/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/fr/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/fr/agenteye/overview.mdx b/docs/fr/cloud/overview.mdx similarity index 51% rename from docs/fr/agenteye/overview.mdx rename to docs/fr/cloud/overview.mdx index ae189ffd..dc4a2ec7 100644 --- a/docs/fr/agenteye/overview.mdx +++ b/docs/fr/cloud/overview.mdx @@ -1,16 +1,16 @@ --- title: "Failproof AI : Observez vos agents pour détecter les défaillances" -description: "Failproof AI Observability est une plateforme auto-hébergée pour observer, évaluer et améliorer vos agents IA en production." +description: "FailproofAI Cloud est une plateforme auto-hébergée pour observer, évaluer et améliorer vos agents IA en production." --- -Failproof AI Observability est une plateforme auto-hébergée pour observer, évaluer et améliorer vos agents IA en production. Elle enregistre tout ce que font vos agents (chaque appel d'outil, requête de modèle, hook et erreur), note la qualité de chaque exécution, et met en évidence les défaillances que vous n'auriez pas su chercher — le tout dans un tableau de bord que vous faites tourner dans votre propre infrastructure. +FailproofAI Cloud est une plateforme auto-hébergée pour observer, évaluer et améliorer vos agents IA en production. Elle enregistre tout ce que font vos agents (chaque appel d'outil, requête de modèle, hook et erreur), note la qualité de chaque exécution, et met en évidence les défaillances que vous n'auriez pas su chercher — le tout dans un tableau de bord que vous faites tourner dans votre propre infrastructure. -Si vous déployez des agents IA et que vous en avez assez de deviner pourquoi une exécution a mal tourné, c'est par ici qu'il faut commencer. Cette page explique ce que Failproof AI Observability vous apporte et comment les différentes pièces s'articulent, avant même que vous n'installiez quoi que ce soit. +Si vous déployez des agents IA et que vous en avez assez de deviner pourquoi une exécution a mal tourné, c'est par ici qu'il faut commencer. Cette page explique ce que FailproofAI Cloud vous apporte et comment les différentes pièces s'articulent, avant même que vous n'installiez quoi que ce soit. -> **Failproof AI Observability est un produit entreprise de Failproof AI.** Vous voulez le voir en action ? Demandez une démo : écrivez à [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +> **FailproofAI Cloud est un produit entreprise de Failproof AI.** Vous voulez le voir en action ? Demandez une démo : écrivez à [nikita@befailproof.ai](mailto:nikita@befailproof.ai). -![Une session Failproof AI Observability représentée sous forme de graphe d'exécution à la git, à côté de sa chronologie d'événements, avec un détail par exécution des outils, modèles et hooks dans le panneau de droite](/agenteye/images/session-detail.png) +![Une session FailproofAI Cloud représentée sous forme de graphe d'exécution à la git, à côté de sa chronologie d'événements, avec un détail par exécution des outils, modèles et hooks dans le panneau de droite](/cloud/images/session-detail.png) *Chaque exécution d'agent est représentée sous forme de graphe d'exécution à la git (gauche), à côté de sa chronologie d'événements. Les sous-agents parallèles ont chacun leur propre couloir ; le panneau de droite détaille les outils, modèles, hooks et la consommation de tokens pour l'exécution.* @@ -30,52 +30,52 @@ Deux courtes vidéos illustrent les deux choses que les équipes recherchent en
-*Failproof Audit : laissez Failproof AI Observability analyser vos logs sur l'ensemble des sessions et vous indiquer ce qu'il faut corriger.* +*Failproof Audit : laissez FailproofAI Cloud analyser vos logs sur l'ensemble des sessions et vous indiquer ce qu'il faut corriger.* --- ## Pourquoi les équipes l'utilisent - **Voyez ce que votre agent a réellement fait.** Chaque exécution devient un graphe d'exécution lisible à la git : quels outils ont fonctionné en parallèle, quels sous-agents ont divergé, où l'exécution s'est bloquée, et ce qu'elle a consommé. -- **Détectez automatiquement les régressions de qualité.** Connectez un petit service de notation et Failproof AI Observability note chaque exécution terminée — une baisse d'utilité ou une hausse des hallucinations apparaît d'elle-même. +- **Détectez automatiquement les régressions de qualité.** Connectez un petit service de notation et FailproofAI Cloud note chaque exécution terminée — une baisse d'utilité ou une hausse des hallucinations apparaît d'elle-même. - **Trouvez les défaillances pour lesquelles vous n'avez écrit aucune règle.** Des audits récurrents analysent vos logs sur l'ensemble des sessions pour repérer des clusters d'erreurs, des valeurs aberrantes de latence, des scores faibles et des exécutions bloquées, puis vous remettent des résultats classés et étayés par des preuves. - **Soyez alerté quand ça compte vraiment.** Des règles de seuil se déclenchent sur le taux d'erreur, la latence, le coût ou les scores d'évaluation, et ouvrent des incidents que vous pouvez prendre en charge, assigner et résoudre. - **Posez des questions en langage naturel.** Un assistant IA intégré au tableau de bord répond à des questions comme « comment évolue la qualité en production cette semaine ? » en s'appuyant sur vos propres données. Toute modification qu'il propose est soumise à validation. -- **Gardez la maîtrise de vos données.** Failproof AI Observability est auto-hébergé : les événements, les prompts et les analyses restent dans une infrastructure que vous contrôlez. +- **Gardez la maîtrise de vos données.** FailproofAI Cloud est auto-hébergé : les événements, les prompts et les analyses restent dans une infrastructure que vous contrôlez. --- ## Ce que vous obtenez -Failproof AI Observability s'articule autour de trois idées (**observer**, **analyser** et **administrer**), reflétées dans la barre latérale gauche du tableau de bord. +FailproofAI Cloud s'articule autour de trois idées (**observer**, **analyser** et **administrer**), reflétées dans la barre latérale gauche du tableau de bord. **Observer** (la réalité brute de ce qui s'est passé) : -- **[Flux d'événements](/fr/agenteye/event-stream)** : la trace en direct, étape par étape, de chaque exécution (appels d'outils, appels de modèles, hooks, erreurs). -- **[Sessions](/fr/agenteye/sessions)** : ces événements regroupés en une ligne par exécution, chacune prête à être notée, avec un graphe d'exécution à la git. -- **[Métriques de performance](/fr/agenteye/telemetry)** : cartes thermiques de latence par surface et indicateurs p50/p95/p99 pour les modèles, outils et hooks, pour qu'une valeur aberrante en queue de distribution ressorte clairement par rapport à la médiane. -- **[Suivi des erreurs](/fr/agenteye/error-tracking)** : une surface de triage unique pour tout ce qui a dysfonctionné, à un clic d'une alerte déclenchée. +- **[Flux d'événements](/fr/cloud/event-stream)** : la trace en direct, étape par étape, de chaque exécution (appels d'outils, appels de modèles, hooks, erreurs). +- **[Sessions](/fr/cloud/sessions)** : ces événements regroupés en une ligne par exécution, chacune prête à être notée, avec un graphe d'exécution à la git. +- **[Métriques de performance](/fr/cloud/performance)** : cartes thermiques de latence par surface et indicateurs p50/p95/p99 pour les modèles, outils et hooks, pour qu'une valeur aberrante en queue de distribution ressorte clairement par rapport à la médiane. +- **[Suivi des erreurs](/fr/cloud/errors)** : une surface de triage unique pour tout ce qui a dysfonctionné, à un clic d'une alerte déclenchée. -![La page d'observation des outils : une carte thermique de latence, une bande de percentiles et un graphique de distribution des outils sur 24 plages temporelles](/agenteye/images/tools.png) +![La page d'observation des outils : une carte thermique de latence, une bande de percentiles et un graphique de distribution des outils sur 24 plages temporelles](/cloud/images/tools.png) *Chaque surface d'observation associe une sparkline et des indicateurs p50/p95/p99 à une carte thermique de latence et une bande de percentiles. Ici : Outils.* **Analyser** (transformer l'activité en réponses) : -- **[Requêtes](/fr/agenteye/queries)** et **[tableaux de bord](/fr/agenteye/dashboards)** : du SQL sauvegardé sur vos événements et évaluations, représenté sous forme de graphiques dans des tableaux de bord partagés à l'échelle de l'organisation. -- **[Évaluations](/fr/agenteye/evaluations)** : scores de qualité produits par votre propre service d'évaluation, avec le raisonnement associé à chaque score. -- **[Audits](/fr/agenteye/audits)** : investigations récurrentes qui font remonter les patterns de défaillance sur l'ensemble des sessions. -- **[Alertes](/fr/agenteye/alerts)** et **[incidents](/fr/agenteye/incidents)** : règles de seuil qui vous notifient, accompagnées d'un workflow d'incidents pour les trier. +- **[Requêtes](/fr/cloud/queries)** et **[tableaux de bord](/fr/cloud/dashboards)** : du SQL sauvegardé sur vos événements et évaluations, représenté sous forme de graphiques dans des tableaux de bord partagés à l'échelle de l'organisation. +- **[Évaluations](/fr/cloud/evaluations)** : scores de qualité produits par votre propre service d'évaluation, avec le raisonnement associé à chaque score. +- **[Audits](/fr/cloud/audits)** : investigations récurrentes qui font remonter les patterns de défaillance sur l'ensemble des sessions. +- **[Alertes](/fr/cloud/alerts)** et **[incidents](/fr/cloud/incidents)** : règles de seuil qui vous notifient, accompagnées d'un workflow d'incidents pour les trier. **Interfaces** (accédez à vos données à votre façon) : -- **[CLI](/fr/agenteye/cli-and-agents)** : pilotez l'ensemble de votre déploiement depuis le terminal ou un script, et laissez un agent de développement le faire pour vous en langage naturel. -- **[Assistant IA](/fr/agenteye/assistant)** : posez des questions sur vos agents en langage naturel, directement depuis le tableau de bord. -- **API REST** : tout ce que font le tableau de bord et la CLI est soutenu par une API REST que vous pouvez appeler directement avec une [clé API](/fr/agenteye/api-keys) à portée limitée — ingérer des événements, interroger des sessions et des évaluations, et gérer des tableaux de bord, alertes, audits, utilisateurs et clés, pour intégrer Failproof AI Observability dans vos propres outils. +- **[CLI](/fr/cloud/cli)** : pilotez l'ensemble de votre déploiement depuis le terminal ou un script, et laissez un agent de développement le faire pour vous en langage naturel. +- **[Assistant IA](/fr/cloud/assistant)** : posez des questions sur vos agents en langage naturel, directement depuis le tableau de bord. +- **API REST** : tout ce que font le tableau de bord et la CLI est soutenu par une API REST que vous pouvez appeler directement avec une [clé API](/fr/cloud/access) à portée limitée — ingérer des événements, interroger des sessions et des évaluations, et gérer des tableaux de bord, alertes, audits, utilisateurs et clés, pour intégrer FailproofAI Cloud dans vos propres outils. **Administrer** (faites-le tourner pour votre équipe) : -- **[Clés API](/fr/agenteye/api-keys)** : tokens à portée limitée pour le collecteur, le tableau de bord et l'assistant. +- **[Clés API](/fr/cloud/access)** : tokens à portée limitée pour le collecteur, le tableau de bord et l'assistant. - **Utilisateurs** : connexion sans mot de passe, par e-mail avec liste d'autorisation. - **Paramètres** : configuration par organisation, y compris les surcharges de fenêtre de contexte des modèles. @@ -91,18 +91,18 @@ Les données circulent dans un seul sens, de votre code d'agent vers le tableau - **Tableau de bord** : l'endroit où vous explorez tout. - **Services optionnels** : un service de notation (évaluations) et un service d'assistant IA (le chat intégré au tableau de bord). -Pour le vocabulaire utilisé tout au long de la documentation (*event, session, evaluation, audit, finding, incident*), consultez [Concepts](/fr/agenteye/concepts). +Pour le vocabulaire utilisé tout au long de la documentation (*event, session, evaluation, audit, finding, incident*), consultez [Concepts](/fr/concepts). --- -## Obtenir Failproof AI Observability +## Obtenir FailproofAI Cloud -Failproof AI Observability est un produit entreprise de Failproof AI, et fonctionne en complément de Failproof AI Enforcement — le produit de politiques et de garde-fous — sous la marque Failproof AI. Il fonctionne entièrement dans votre propre environnement. Si vous n'avez pas encore accès aux packages, demandez une démo et nous vous aiderons à démarrer : écrivez à [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +FailproofAI Cloud est un produit entreprise de Failproof AI, et fonctionne en complément de FailproofAI guardrails — le produit de politiques et de garde-fous — sous la marque Failproof AI. Il fonctionne entièrement dans votre propre environnement. Si vous n'avez pas encore accès aux packages, demandez une démo et nous vous aiderons à démarrer : écrivez à [nikita@befailproof.ai](mailto:nikita@befailproof.ai). --- ## Prochaines étapes -- [Concepts](/fr/agenteye/concepts) : le vocabulaire de Failproof AI Observability en un seul endroit. -- [Observabilité](/fr/agenteye/observability) : suivez ce que font vos agents, exécution par exécution. -- [Sécurité](/fr/agenteye/security) : comment Failproof AI Observability maintient vos données isolées et sous votre contrôle. \ No newline at end of file +- [Concepts](/fr/concepts) : le vocabulaire de FailproofAI Cloud en un seul endroit. +- [Observabilité](/fr/cloud/overview) : suivez ce que font vos agents, exécution par exécution. +- [Sécurité](/fr/cloud/security) : comment FailproofAI Cloud maintient vos données isolées et sous votre contrôle. \ No newline at end of file diff --git a/docs/fr/agenteye/telemetry.mdx b/docs/fr/cloud/performance.mdx similarity index 81% rename from docs/fr/agenteye/telemetry.mdx rename to docs/fr/cloud/performance.mdx index fa47768e..b988bdd6 100644 --- a/docs/fr/agenteye/telemetry.mdx +++ b/docs/fr/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "Détectez à l'instant précis où vos modèles, outils ou hooks r Détectez à l'instant précis où vos modèles, outils ou hooks ralentissent ou font grimper la facture, et interceptez un pic de latence en queue de distribution avant que vos utilisateurs ne le ressentent. Trois pages dédiées transforment les mesures brutes en p50, p95 et p99 lisibles en un coup d'œil. -![La page Models affichant une carte de chaleur de latence, une bande de percentiles et des chiffres de tokens, coût et fenêtre de contexte par modèle](/agenteye/images/models.png) +![La page Models affichant une carte de chaleur de latence, une bande de percentiles et des chiffres de tokens, coût et fenêtre de contexte par modèle](/cloud/images/models.png) *La page Models : une carte de chaleur de latence, une bande de percentiles et, par modèle, le nombre de tokens, le coût estimé et le remplissage de la fenêtre de contexte.* ## Arrêtez de laisser les moyennes masquer vos pires exécutions @@ -24,13 +24,13 @@ Un réticule de survol partagé relie la carte de chaleur et la bande, de sorte La page Models (illustrée ci-dessus) répond aux deux questions qu'une facture soulève invariablement : quel modèle, et combien. En plus de la vue de latence partagée, elle ajoute la **consommation de tokens par modèle**, le **coût estimé** et le **remplissage de la fenêtre de contexte**, afin que la croissance incontrôlée des prompts et une compaction imminente soient visibles avant de vous surprendre. -Failproof AI Observability reconnaît automatiquement les identifiants de modèles courants. Si une fenêtre semble incorrecte, ou si vous utilisez un modèle privé, corrigez-la ou ajoutez-en un depuis **Settings**, dans **model context windows** — les indicateurs de remplissage se mettront à jour en conséquence. +FailproofAI Cloud reconnaît automatiquement les identifiants de modèles courants. Si une fenêtre semble incorrecte, ou si vous utilisez un modèle privé, corrigez-la ou ajoutez-en un depuis **Settings**, dans **model context windows** — les indicateurs de remplissage se mettront à jour en conséquence. ## Tools : distinguez la lenteur de la défaillance Un appel d'outil peut être lent, ou il peut échouer silencieusement — et vous voulez savoir lequel en quelques secondes, pas après avoir fouillé des logs. -![La page Tools affichant la carte de chaleur et la bande de percentiles partagées, à côté d'une répartition succès/échecs et d'une barre de distribution des outils](/agenteye/images/tools.png) +![La page Tools affichant la carte de chaleur et la bande de percentiles partagées, à côté d'une répartition succès/échecs et d'une barre de distribution des outils](/cloud/images/tools.png) *La page Tools : la même carte de chaleur et bande de percentiles, plus une répartition succès/échecs et une barre de distribution des outils.* En complément de la vue de latence partagée, la page Tools ajoute une **répartition succès/échecs** et une **barre de distribution des outils**, afin de voir en un coup d'œil quels outils vous sollicitez le plus et lesquels grignotent votre budget d'erreurs. @@ -39,14 +39,14 @@ En complément de la vue de latence partagée, la page Tools ajoute une **répar Quand un hook de cycle de vie alourdit une exécution, constater que « les hooks sont lents » n'est pas exploitable. La page Hooks vous amène directement à celui qui pose problème. -![La page Hooks affichant la latence décomposée par nom de hook et événement déclencheur, sur la carte de chaleur et la bande de percentiles partagées](/agenteye/images/hooks.png) +![La page Hooks affichant la latence décomposée par nom de hook et événement déclencheur, sur la carte de chaleur et la bande de percentiles partagées](/cloud/images/hooks.png) *La page Hooks : la latence décomposée par nom de hook et événement déclencheur.* Au-dessus de la même carte de chaleur et bande de percentiles, la page Hooks décompose l'activité par **nom de hook** et **événement déclencheur**, afin de cibler précisément le hook unique et l'événement unique qui nécessitent votre attention. ## Voir aussi -- [Flux d'événements](/fr/agenteye/event-stream) : la trace en direct, colorée, de chaque événement. -- [Sessions](/fr/agenteye/sessions) : regroupez les événements en une ligne par exécution et ouvrez son graphe d'exécution. -- [Suivi des erreurs](/fr/agenteye/error-tracking) : une surface de triage unique pour tout ce que le tableau de bord affiche en rouge. -- [Tableaux de bord](/fr/agenteye/dashboards) : vues agrégées sur l'ensemble de votre flotte. \ No newline at end of file +- [Flux d'événements](/fr/cloud/event-stream) : la trace en direct, colorée, de chaque événement. +- [Sessions](/fr/cloud/sessions) : regroupez les événements en une ligne par exécution et ouvrez son graphe d'exécution. +- [Suivi des erreurs](/fr/cloud/errors) : une surface de triage unique pour tout ce que le tableau de bord affiche en rouge. +- [Tableaux de bord](/fr/cloud/dashboards) : vues agrégées sur l'ensemble de votre flotte. \ No newline at end of file diff --git a/docs/fr/agenteye/queries.mdx b/docs/fr/cloud/queries.mdx similarity index 78% rename from docs/fr/agenteye/queries.mdx rename to docs/fr/cloud/queries.mdx index 903c50f3..1066a560 100644 --- a/docs/fr/agenteye/queries.mdx +++ b/docs/fr/cloud/queries.mdx @@ -4,9 +4,9 @@ description: "Posez n'importe quelle question sur les données de vos agents et --- -Posez n'importe quelle question sur les données de vos agents et obtenez une réponse en quelques secondes. Failproof AI Observability vous propose une bibliothèque de requêtes sauvegardées, prêtes à l'emploi, sur vos événements et évaluations — vous partez ainsi d'un exemple fonctionnel plutôt que d'un éditeur SQL vide. +Posez n'importe quelle question sur les données de vos agents et obtenez une réponse en quelques secondes. FailproofAI Cloud vous propose une bibliothèque de requêtes sauvegardées, prêtes à l'emploi, sur vos événements et évaluations — vous partez ainsi d'un exemple fonctionnel plutôt que d'un éditeur SQL vide. -![La bibliothèque de requêtes sauvegardées : une grille de requêtes réutilisables, qu'il s'agisse de préréglages intégrés ou de requêtes personnalisées](/agenteye/images/queries.png) +![La bibliothèque de requêtes sauvegardées : une grille de requêtes réutilisables, qu'il s'agisse de préréglages intégrés ou de requêtes personnalisées](/cloud/images/queries.png) *Votre bibliothèque de requêtes sauvegardées à l'adresse `//queries` : les préréglages intégrés côtoient les requêtes enregistrées par votre équipe.* @@ -22,7 +22,7 @@ Accédez-y à l'adresse `//queries`. Ouvrez n'importe quelle requête et elle s'affiche dans le compositeur SQL, où vous pouvez la modifier et obtenir la réponse immédiatement : sans export, sans aller-retour, sans attendre quelqu'un d'autre. -![Le compositeur de requêtes SQL exécutant une requête sauvegardée, avec un panneau latéral de schéma et une grille de résultats en direct](/agenteye/images/query-lab.png) +![Le compositeur de requêtes SQL exécutant une requête sauvegardée, avec un panneau latéral de schéma et une grille de résultats en direct](/cloud/images/query-lab.png) *Le compositeur SQL : votre requête à gauche, un panneau latéral de schéma pour ne jamais avoir à deviner un nom de colonne, et une grille de résultats en direct en dessous.* @@ -43,14 +43,14 @@ agenteye query list # les mêmes requêtes sauvegardées, depuis agenteye query run errs --arg prod # exécutez-en une et affichez les lignes (ajoutez --json pour la rediriger) ``` - Consultez [CLI and agents](/fr/agenteye/cli-and-agents) pour l'ensemble complet des commandes. + Consultez [CLI and agents](/fr/cloud/cli) pour l'ensemble complet des commandes. -- **Depuis l'assistant IA.** Vous ne savez pas comment formuler le SQL ? Demandez à l'[assistant IA](/fr/agenteye/assistant) intégré au tableau de bord en langage naturel — il rédigera la requête et la sauvegardera dans votre bibliothèque. +- **Depuis l'assistant IA.** Vous ne savez pas comment formuler le SQL ? Demandez à l'[assistant IA](/fr/cloud/assistant) intégré au tableau de bord en langage naturel — il rédigera la requête et la sauvegardera dans votre bibliothèque. L'exécution d'une requête sauvegardée est contrôlée par la permission `queries:run`, distincte des permissions de création ou de suppression de requêtes, ce qui vous permet d'accorder un accès en lecture sans laisser tout le monde réécrire la bibliothèque. ## Voir aussi -- [Dashboards](/fr/agenteye/dashboards) : épinglez les résultats de requêtes dans des graphiques partagés à l'échelle de l'organisation. -- [AI assistant](/fr/agenteye/assistant) : posez vos questions en langage naturel et recevez une requête en retour. -- [CLI and agents](/fr/agenteye/cli-and-agents) : exécutez et sauvegardez les mêmes requêtes depuis votre terminal. \ No newline at end of file +- [Dashboards](/fr/cloud/dashboards) : épinglez les résultats de requêtes dans des graphiques partagés à l'échelle de l'organisation. +- [AI assistant](/fr/cloud/assistant) : posez vos questions en langage naturel et recevez une requête en retour. +- [CLI and agents](/fr/cloud/cli) : exécutez et sauvegardez les mêmes requêtes depuis votre terminal. \ No newline at end of file diff --git a/docs/fr/agenteye/python-sdk.mdx b/docs/fr/cloud/sdk.mdx similarity index 94% rename from docs/fr/agenteye/python-sdk.mdx rename to docs/fr/cloud/sdk.mdx index e94062ea..f958e5c0 100644 --- a/docs/fr/agenteye/python-sdk.mdx +++ b/docs/fr/cloud/sdk.mdx @@ -4,11 +4,11 @@ description: "Observez exactement ce que vos agents IA ont fait en production : --- -Observez exactement ce que vos agents IA ont fait en production : chaque exécution d'agent, appel d'outil, requête de modèle, hook et intervention humaine. Le SDK Python d'observabilité Failproof AI enregistre cette trace depuis l'intérieur de votre code d'agent afin que vous puissiez déboguer, auditer et évaluer ce qui s'est passé. Utilisez-le chaque fois que vous souhaitez que Failproof AI Observability observe vos agents. +Observez exactement ce que vos agents IA ont fait en production : chaque exécution d'agent, appel d'outil, requête de modèle, hook et intervention humaine. Le SDK Python d'observabilité Failproof AI enregistre cette trace depuis l'intérieur de votre code d'agent afin que vous puissiez déboguer, auditer et évaluer ce qui s'est passé. Utilisez-le chaque fois que vous souhaitez que FailproofAI Cloud observe vos agents. En coulisses, le SDK écrit des événements structurés dans des fichiers JSONL locaux, et le daemon collecteur les récupère et les envoie automatiquement vers la plateforme. Vous n'avez pas à gérer ces fichiers vous-même. -> **Conseil :** Vous découvrez Failproof AI Observability ? Cette page est la référence complète des événements du SDK. +> **Conseil :** Vous découvrez FailproofAI Cloud ? Cette page est la référence complète des événements du SDK.
@@ -26,7 +26,7 @@ Une fois installé, vérifiez qu'il est bien présent : python -c "import agenteye; print(agenteye.__version__)" ``` -Vous préférez laisser un agent de codage gérer toute l'intégration ? Le [Python SDK Agent Skill](/fr/agenteye/python-sdk-skill) connaît le chemin d'installation, planifie les points d'instrumentation, les implémente et vérifie que les événements arrivent bien. +Vous préférez laisser un agent de codage gérer toute l'intégration ? Le [Python SDK Agent Skill](/fr/cloud/agent-skills) connaît le chemin d'installation, planifie les points d'instrumentation, les implémente et vérifie que les événements arrivent bien. --- @@ -60,7 +60,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### Instrumenter un appel réel -En pratique, vous enveloppez votre code d'agent existant. Encadrez un appel de modèle avec `model_request` avant et `model_response` après, afin que les deux événements couvrent la requête réelle et que Failproof AI Observability puisse les associer : +En pratique, vous enveloppez votre code d'agent existant. Encadrez un appel de modèle avec `model_request` avant et `model_response` après, afin que les deux événements couvrent la requête réelle et que FailproofAI Cloud puisse les associer : ```python import anthropic @@ -99,7 +99,7 @@ Enveloppez les appels d'outils de la même manière avec `tool_use` et `tool_res Voici à quoi ressemblent ces événements une fois qu'ils arrivent dans le tableau de bord, codés par couleur selon leur type et filtrables par environnement, agent et session : -![Le flux d'événements en direct, codé par couleur selon le type d'événement et filtrable par environnement, agent et session](/agenteye/images/events-stream.png) +![Le flux d'événements en direct, codé par couleur selon le type d'événement et filtrable par environnement, agent et session](/cloud/images/events-stream.png) --- @@ -156,9 +156,9 @@ La suppression des données sensibles est donc votre choix et votre responsabili ## Référence des événements -La plupart des événements viennent par paires début/fin partageant un identifiant de corrélation : `tool_use` et `tool_result` partagent un `tool_call_id`, `hook_triggered` et `hook_completed` partagent un `hook_id`, et `human_wait` et `human_input` partagent un `input_id`. Émettez l'événement de début, effectuez le travail, puis émettez l'événement de fin avec le même identifiant. Failproof AI Observability associe la paire et calcule `duration_ms` pour vous, vous n'avez donc jamais à passer `duration_ms` vous-même. +La plupart des événements viennent par paires début/fin partageant un identifiant de corrélation : `tool_use` et `tool_result` partagent un `tool_call_id`, `hook_triggered` et `hook_completed` partagent un `hook_id`, et `human_wait` et `human_input` partagent un `input_id`. Émettez l'événement de début, effectuez le travail, puis émettez l'événement de fin avec le même identifiant. FailproofAI Cloud associe la paire et calcule `duration_ms` pour vous, vous n'avez donc jamais à passer `duration_ms` vous-même. -![Le graphe d'exécution de style git d'une session à côté de sa chronologie d'événements, reconstruit à partir des événements associés, avec le panneau de répartition outil/modèle/hook](/agenteye/images/session-detail.png) +![Le graphe d'exécution de style git d'une session à côté de sa chronologie d'événements, reconstruit à partir des événements associés, avec le panneau de répartition outil/modèle/hook](/cloud/images/session-detail.png) Toutes les méthodes d'événement requièrent ces deux champs : @@ -432,5 +432,5 @@ Chaque fichier est écrit de manière atomique : le SDK écrit dans un fichier t ## Étapes suivantes -- [Flux d'événements](/fr/agenteye/event-stream) : regardez ces événements arriver en direct, codés par couleur et filtrables par environnement, agent et session. -- [Sessions](/fr/agenteye/sessions) : découvrez comment les événements associés reconstituent chaque exécution d'agent sous forme de graphe d'exécution et de chronologie. \ No newline at end of file +- [Flux d'événements](/fr/cloud/event-stream) : regardez ces événements arriver en direct, codés par couleur et filtrables par environnement, agent et session. +- [Sessions](/fr/cloud/sessions) : découvrez comment les événements associés reconstituent chaque exécution d'agent sous forme de graphe d'exécution et de chronologie. \ No newline at end of file diff --git a/docs/fr/cloud/security.mdx b/docs/fr/cloud/security.mdx new file mode 100644 index 00000000..dbb1ebb3 --- /dev/null +++ b/docs/fr/cloud/security.mdx @@ -0,0 +1,68 @@ +--- +title: "Sécurité" +description: "FailproofAI Cloud est conçu pour fonctionner au plus près de vos agents en production, ce qui signifie qu'il voit vos prompts, les entrées des outils et leurs sorties." +--- + + +FailproofAI Cloud est conçu pour fonctionner au plus près de vos agents en production, ce qui signifie qu'il voit vos prompts, les entrées des outils et leurs sorties. Cette page explique comment vos données restent isolées, contrôlées et entre vos mains. Si vous évaluez FailproofAI Cloud dans le cadre d'une revue de sécurité, commencez ici. + +--- + +## Vos données restent dans votre environnement + +FailproofAI Cloud est auto-hébergé. Les événements, prompts, réponses des modèles et analyses sont stockés dans vos propres bases de données, dans votre propre environnement. Rien n'est envoyé à un service SaaS tiers pour y être stocké, et vos données demeurent dans votre propre compte cloud. + +--- + +## Isolation des locataires + +Une instance FailproofAI Cloud peut héberger plusieurs organisations, chacune étant isolée au niveau de la couche de stockage — appliqué par la base de données elle-même, et pas seulement par l'interface : + +- Les données opérationnelles d'une organisation (utilisateurs, clés, tableaux de bord, requêtes sauvegardées) sont limitées à cette organisation, et les lectures inter-organisations sont bloquées par la base de données elle-même. +- Chaque événement ingéré est marqué avec l'organisation à laquelle il appartient, de sorte qu'une organisation ne peut jamais lire les événements d'une autre. + +Chaque route de tableau de bord est délimitée sous un slug d'organisation (`//…`). + +--- + +## Connexion + +FailproofAI Cloud utilise une connexion sans mot de passe, par e-mail. Il n'y a pas de mot de passe à hameçonner ou à divulguer. Un utilisateur demande un code à usage unique (ou un lien magique en un clic), qui lui est envoyé par e-mail et expire rapidement. La connexion est contrôlée par une **liste d'autorisation** : seules les adresses e-mail (ou domaines) que vous autorisez peuvent s'authentifier. + +![L'écran de connexion de FailproofAI Cloud, qui envoie un code à usage unique à votre adresse e-mail](/cloud/images/login.png) + +--- + +## Accès délimité avec des clés API + +Chaque client s'authentifie avec une clé API dotée de permissions granulaires et à moindre privilège. Un collecteur n'a besoin que de `events:add` ; une clé de tableau de bord ou d'assistant peut être en lecture seule ; les actions destructives (suppression, regénération) sont des droits distincts que vous choisissez d'inclure. + +![La page des clés API : les permissions accordées à chaque clé, avec un code couleur par portée lecture, écriture et destructive](/cloud/images/api-keys.png) + +Conservez la clé d'amorçage administrateur pour la configuration, et créez des clés restreintes pour tout le reste. Voir [Clés API](/fr/cloud/access). + +--- + +## Un assistant en lecture seule avec validation obligatoire + +L'[assistant IA](/fr/cloud/assistant) intégré au tableau de bord répond à vos questions sur vos données, mais il est limité par conception : + +- Il est **en lecture seule par défaut** : son SQL passe par un garde-fou qui n'autorise que les requêtes `SELECT`/`WITH`, à instruction unique, avec un plafond de lignes. +- Tout ce qu'il crée (une requête sauvegardée, un tableau de bord) est soumis à **validation** : vous examinez et approuvez chaque écriture avant qu'elle ne se produise. +- Il **ne peut jamais supprimer**. + +Ainsi, un membre de l'équipe peut demander « quels agents ont généré le plus d'erreurs cette semaine ? » et agir sur la réponse, sans que l'assistant puisse modifier ou supprimer vos données de son propre chef. + +--- + +## En transit + +Tout le trafic passe par HTTPS. Vous terminez le TLS avec vos propres certificats, de sorte que le trafic collecteur-vers-serveur et navigateur-vers-serveur est chiffré en transit. + +--- + +## Étapes suivantes + +- [Vue d'ensemble](/fr/cloud/overview) : comment FailproofAI Cloud s'articule. +- [Clés API](/fr/cloud/access) : délimitez l'accès pour le collecteur, le tableau de bord et l'assistant. +- [Observabilité](/fr/cloud/overview) : ce que FailproofAI Cloud capture depuis vos agents. \ No newline at end of file diff --git a/docs/fr/agenteye/sessions.mdx b/docs/fr/cloud/sessions.mdx similarity index 80% rename from docs/fr/agenteye/sessions.mdx rename to docs/fr/cloud/sessions.mdx index 918cf10c..6201ab90 100644 --- a/docs/fr/agenteye/sessions.mdx +++ b/docs/fr/cloud/sessions.mdx @@ -6,7 +6,7 @@ description: "Chaque événement d'une exécution regroupé en une ligne lisible Fini les suppositions sur la cause d'un échec. L'observabilité Failproof AI regroupe chaque événement d'une exécution en une ligne lisible, puis représente l'ensemble sous forme d'un schéma à la git que vous pouvez déchiffrer en quelques secondes — vous voyez exactement ce que votre agent a fait, étape par étape. -![La liste des Sessions : une ligne par exécution, tous environnements et agents confondus, avec des pastilles de statut et des badges de score d'évaluation](/agenteye/images/sessions-list.png) +![La liste des Sessions : une ligne par exécution, tous environnements et agents confondus, avec des pastilles de statut et des badges de score d'évaluation](/cloud/images/sessions-list.png) *Une ligne par exécution : la pastille de statut vous indique en un coup d'œil comment s'est terminée l'exécution, et un badge de score apparaît dès qu'un évaluateur est connecté.* @@ -30,7 +30,7 @@ Une fois un évaluateur connecté, chaque exécution terminée est automatiqueme ## Lire l'intégralité d'une exécution sous forme de schéma -![Le graphe d'exécution à la git d'une session à côté de sa chronologie d'événements, avec le panneau de détail des outils, modèles et hooks](/agenteye/images/session-detail.png) +![Le graphe d'exécution à la git d'une session à côté de sa chronologie d'événements, avec le panneau de détail des outils, modèles et hooks](/cloud/images/session-detail.png) *Le graphe d'exécution (à gauche) se trouve à côté de la chronologie des événements ; le rail de droite détaille les outils, modèles, hooks et la consommation de tokens pour l'exécution.* @@ -38,7 +38,7 @@ Cliquez sur n'importe quelle session pour ouvrir son graphe d'exécution : une v Le rail de droite vous offre la ventilation par exécution : quels outils et modèles ont été utilisés, quels hooks se sont déclenchés, et ce que l'exécution a consommé en tokens. C'est la réponse à «pourquoi cette exécution a-t-elle coûté si cher ?» ou «quel outil est le plus lent ?», placée juste à côté du graphe qui en est la cause. -Les événements individuels sont adressables, vous pouvez donc envoyer à quelqu'un un lien vers un moment précis plutôt que «la session, environ aux deux tiers». Copiez le lien depuis n'importe quel événement, ou suivez-en un depuis un constat d'[audit](/fr/agenteye/audits) ou une erreur, et la session s'ouvre avec cet événement sélectionné et visible à l'écran. Cela vaut aussi pour les exécutions très longues : la chronologie charge une fenêtre délimitée pour préserver les performances de votre navigateur, et un lien pointant au-delà de cette fenêtre retrouvera quand même son événement plutôt que de vous déposer au début. Si l'événement a dépassé votre fenêtre de rétention, la page vous l'indique explicitement au lieu de ne rien sélectionner silencieusement. +Les événements individuels sont adressables, vous pouvez donc envoyer à quelqu'un un lien vers un moment précis plutôt que «la session, environ aux deux tiers». Copiez le lien depuis n'importe quel événement, ou suivez-en un depuis un constat d'[audit](/fr/cloud/audits) ou une erreur, et la session s'ouvre avec cet événement sélectionné et visible à l'écran. Cela vaut aussi pour les exécutions très longues : la chronologie charge une fenêtre délimitée pour préserver les performances de votre navigateur, et un lien pointant au-delà de cette fenêtre retrouvera quand même son événement plutôt que de vous déposer au début. Si l'événement a dépassé votre fenêtre de rétention, la page vous l'indique explicitement au lieu de ne rien sélectionner silencieusement. --- @@ -46,12 +46,12 @@ Les événements individuels sont adressables, vous pouvez donc envoyer à quelq Chaque page du tableau de bord est limitée à votre organisation (`//…`). Sessions se trouve sous **Observe** dans la barre latérale gauche, à côté d'Events, avec les filtres de plage de dates, d'environnement, d'agent et de session en haut de la liste. Chaque ligne est à un clic de son graphe d'exécution complet. -Pour activer les badges de score et le filtrage par plage de scores, connectez un évaluateur : voir [Evaluations](/fr/agenteye/evaluations). +Pour activer les badges de score et le filtrage par plage de scores, connectez un évaluateur : voir [Evaluations](/fr/cloud/evaluations). --- ## En rapport -- [Event stream](/fr/agenteye/event-stream) : le journal brut, étape par étape, dont chaque session est le regroupement. -- [Evaluations](/fr/agenteye/evaluations) : connectez un évaluateur pour que chaque exécution reçoive un badge de score filtrable. -- [Telemetry](/fr/agenteye/telemetry) : comment les exécutions transitent de votre agent vers ces sessions. \ No newline at end of file +- [Event stream](/fr/cloud/event-stream) : le journal brut, étape par étape, dont chaque session est le regroupement. +- [Evaluations](/fr/cloud/evaluations) : connectez un évaluateur pour que chaque exécution reçoive un badge de score filtrable. +- [Telemetry](/fr/cloud/performance) : comment les exécutions transitent de votre agent vers ces sessions. \ No newline at end of file diff --git a/docs/fr/concepts.mdx b/docs/fr/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/fr/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/fr/daemon.mdx b/docs/fr/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/fr/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/fr/dashboard.mdx b/docs/fr/dashboard.mdx index 4922603d..5dd37d29 100644 --- a/docs/fr/dashboard.mdx +++ b/docs/fr/dashboard.mdx @@ -69,7 +69,7 @@ Un rapport à personnalité qui rend compte du comportement réel de votre agent 4. **Comment s'améliorer** — liste calme, une entrée par politique prescrite : nom de la politique en blanc, description en une ligne, commande d'installation + bouton de copie sur le côté droit. L'en-tête de section indique `enable all N → projected · ` (le score que vous atteindriez avec tous les correctifs appliqués), et son bouton `[install all]` copie la commande combinée `failproofai policy add a b c …` pour chaque politique prescrite. 5. **Revenez meilleur** — deux cartes côte à côte. À gauche : définir un rappel (sélecteur de cadence `3d` / `7d` / `14d` / `30d` ; persiste via `/api/auth/reminder` une fois authentifié). À droite : débloquer des avantages failproof — `invite a friend` ouvre une fenêtre modale acceptant une liste d'adresses e-mail d'amis séparées par des virgules/espaces/sauts de ligne (10 maximum par envoi), les envoie via POST à `/api/audit/invite`, qui les transmet au `POST /v0/invite` du serveur API. Le serveur API envoie un e-mail par destinataire depuis `invite@failproof.ai` avec l'expéditeur en Cc et `Reply-To` défini, de sorte que le destinataire voit qui l'a invité et l'expéditeur reçoit une copie dans sa boîte de réception. Les utilisateurs anonymes sont d'abord redirigés via `AuthDialog` afin que l'adresse e-mail de l'expéditeur soit connue avant l'envoi des invitations. La gestion des droits / avantages est prévue dans une prochaine étape. -Alimenté par le moteur d'exécution `failproofai audit` — voir [CLI Audit](/fr/cli/audit) pour le moteur d'analyse sous-jacent, les indicateurs pris en charge et les invariants de cache par transcription. Le dashboard met en cache le dernier résultat dans `~/.failproofai/audit-dashboard.json` (mode `0600`, emplacement unique, les nouvelles exécutions écrasent) afin que les revisites soient instantanées ; **les caches par transcription et par résultat complet sont tous deux rejetés à la lecture s'ils ont plus de 7 jours**, ainsi le dashboard ne sert jamais silencieusement un résultat vieux d'une semaine — passé la TTL, `/audit` retombe sur son état vide et invite à relancer une analyse. Cliquer sur `[ re-audit now ]` près du bas du rapport envoie un POST `/api/audit/run` avec `noCache: true` — la ré-analyse contourne le cache par transcription et réanalyse chaque transcription depuis le début plutôt que de retourner silencieusement le résultat mis en cache — et le dashboard interroge `/api/audit/status` à 1 Hz jusqu'à la fin de l'exécution ; une bande de progression rose épinglée s'affiche en haut du viewport pendant l'exécution avec un minuteur écoulé, et le nouveau résultat remplace l'ancien en place en cas de succès (sans rechargement de page ; une ré-analyse échouée laisse le rapport précédent intact). En cas d'échec, la bande devient rouge avec un message basé sur `RerunError.kind` (`timeout` / `network` / `post_failed`). L'état vide (pas de cache ou expiré) et l'état zéro session (le cache existe mais l'analyse n'a trouvé aucune transcription) sont affichés séparément. +Alimenté par le moteur d'exécution `failproofai audit` — voir [CLI Audit](/fr/audit) pour le moteur d'analyse sous-jacent, les indicateurs pris en charge et les invariants de cache par transcription. Le dashboard met en cache le dernier résultat dans `~/.failproofai/audit-dashboard.json` (mode `0600`, emplacement unique, les nouvelles exécutions écrasent) afin que les revisites soient instantanées ; **les caches par transcription et par résultat complet sont tous deux rejetés à la lecture s'ils ont plus de 7 jours**, ainsi le dashboard ne sert jamais silencieusement un résultat vieux d'une semaine — passé la TTL, `/audit` retombe sur son état vide et invite à relancer une analyse. Cliquer sur `[ re-audit now ]` près du bas du rapport envoie un POST `/api/audit/run` avec `noCache: true` — la ré-analyse contourne le cache par transcription et réanalyse chaque transcription depuis le début plutôt que de retourner silencieusement le résultat mis en cache — et le dashboard interroge `/api/audit/status` à 1 Hz jusqu'à la fin de l'exécution ; une bande de progression rose épinglée s'affiche en haut du viewport pendant l'exécution avec un minuteur écoulé, et le nouveau résultat remplace l'ancien en place en cas de succès (sans rechargement de page ; une ré-analyse échouée laisse le rapport précédent intact). En cas d'échec, la bande devient rouge avec un message basé sur `RerunError.kind` (`timeout` / `network` / `post_failed`). L'état vide (pas de cache ou expiré) et l'état zéro session (le cache existe mais l'analyse n'a trouvé aucune transcription) sont affichés séparément. ### Politiques diff --git a/docs/fr/architecture.mdx b/docs/fr/how-it-works.mdx similarity index 100% rename from docs/fr/architecture.mdx rename to docs/fr/how-it-works.mdx diff --git a/docs/fr/introduction.mdx b/docs/fr/introduction.mdx index ce69bd5d..14fbbd51 100644 --- a/docs/fr/introduction.mdx +++ b/docs/fr/introduction.mdx @@ -54,4 +54,4 @@ failproofai policies --install # enable policies (or skip — `failproofai` wi failproofai # launch the dashboard ``` -Consultez le guide [Premiers pas](/fr/getting-started) pour le parcours complet. \ No newline at end of file +Consultez le guide [Premiers pas](/fr/quickstart) pour le parcours complet. \ No newline at end of file diff --git a/docs/fr/policies.mdx b/docs/fr/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/fr/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/fr/getting-started.mdx b/docs/fr/quickstart.mdx similarity index 100% rename from docs/fr/getting-started.mdx rename to docs/fr/quickstart.mdx diff --git a/docs/fr/reference/files.mdx b/docs/fr/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/fr/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/getting-started.mdx b/docs/getting-started.mdx deleted file mode 100644 index 92aafcab..00000000 --- a/docs/getting-started.mdx +++ /dev/null @@ -1,201 +0,0 @@ ---- -title: Getting started -description: "Install failproofai, enable policies, and let your agents run reliably" -icon: rocket ---- - -## Requirements - -- **Node.js** >= 20.9.0 -- **Bun** >= 1.3.0 (optional - only needed for building from source) - ---- - -## Installation - - - -```bash npm -npm install -g failproofai -``` - -```bash bun -bun add -g failproofai -``` - - - ---- - -## Quick start - - - - Policies are rules that run before and after every agent tool call. They catch destructive commands, secret leakage, and other failure modes before they cause damage. - - ```bash - failproofai policies --install - ``` - - This writes hook entries into your installed agent CLIs (Claude Code's `~/.claude/settings.json`, OpenAI Codex's `~/.codex/hooks.json`, GitHub Copilot CLI's `~/.copilot/hooks/failproofai.json`, Cursor Agent's `~/.cursor/hooks.json`, OpenCode's generated plugin shim at `~/.config/opencode/plugins/failproofai.mjs` plus a registration entry in `~/.config/opencode/opencode.json`'s `plugin` array, Pi's `~/.pi/agent/settings.json`, Hermes's `~/.hermes/config.yaml`, OpenClaw's `~/.openclaw/openclaw.json`, Factory Droid's `~/.factory/hooks.json`, Devin CLI's `~/.config/devin/config.json`, Antigravity CLI's `~/.gemini/config/hooks.json`, or Goose's auto-discovered plugin dir at `~/.agents/plugins/failproofai/hooks/hooks.json`). When more than one is present you'll be prompted; pass `--cli claude codex copilot cursor opencode pi hermes openclaw factory devin antigravity goose` (any subset) to skip the prompt. - - GitHub Copilot CLI, Cursor Agent, OpenCode, and Pi support are **beta** — install with `--cli copilot`, `--cli cursor`, `--cli opencode`, or `--cli pi`. Hermes (hermes-agent, a Slack/Telegram gateway) installs user-scope with `--cli hermes` and is **also** an offline audit source. OpenClaw (openclaw gateway, a self-hosted multi-channel assistant) installs user-scope with `--cli openclaw` — enforcement runs through its in-process plugin hooks (`before_agent_finalize` is a real turn-end gate, so the `require-*-before-stop` builtins enforce) — and is **also** an offline audit source. Factory Droid (`droid`) installs with `--cli factory` (user + project scope) and is **also** an offline audit source. Devin CLI (`devin`, Cognition) installs with `--cli devin` (user + project scope) and is **also** an offline audit source. Antigravity CLI (`agy`) installs with `--cli antigravity` (user + project scope) and is **also** an offline audit source. Goose (codename goose, Block) installs with `--cli goose` (user + project scope) — the installer just drops a plugin dir at `~/.agents/plugins/failproofai/` that Goose auto-discovers, and it is **also** an offline audit source. - - ```bash - failproofai policies --install --scope project - failproofai policies --install --cli codex --scope project - failproofai policies --install --cli copilot --scope project - failproofai policies --install --cli cursor --scope project - failproofai policies --install --cli opencode --scope project - failproofai policies --install --cli pi --scope project - failproofai policies --install --cli hermes --scope user - failproofai policies --install --cli openclaw --scope user - failproofai policies --install --cli factory --scope project - failproofai policies --install --cli devin --scope project - failproofai policies --install --cli antigravity --scope project - failproofai policies --install --cli goose --scope project - failproofai policies --install block-sudo block-rm-rf sanitize-api-keys - ``` - - - ```bash - failproofai policies - ``` - - Shows every policy, whether it's enabled, and any configured parameters. - - - ```bash - failproofai - ``` - - Opens a local dashboard at `http://localhost:8020` where you can browse sessions, inspect tool calls, and manage policies. - - - Start Claude Code as usual. If the agent tries something risky, failproofai intercepts it automatically. Leave it running unattended and review what happened in the dashboard. - - - ---- - -## How policies work - -Every time an agent runs a tool, Claude Code calls failproofai as a subprocess: - -```text -Claude Code → failproofai --hook PreToolUse → reads stdin JSON - evaluates policies - writes decision to stdout -``` - -Each policy returns one of three decisions: - -- **allow** - the agent proceeds normally -- **deny** - the action is blocked, the agent is told why -- **instruct** - extra context is added to the agent's prompt - - -Policies run in your local process. Nothing is sent to a remote service. - - ---- - -## Set up team policies with convention-based policies - -The fastest way to establish quality standards across your team is the `.failproofai/policies/` convention. Drop policy files into this directory and they're loaded automatically — no flags, no config changes, no install commands. - - - - ```bash - mkdir -p .failproofai/policies - ``` - - - Copy the starter examples or write your own: - - ```bash - cp node_modules/failproofai/examples/convention-policies/*.mjs .failproofai/policies/ - ``` - - Or create a new one: - - ```js - // .failproofai/policies/team-policies.mjs - import { customPolicies, allow, deny, instruct } from "failproofai"; - - customPolicies.add({ - name: "test-before-commit", - match: { events: ["PreToolUse"] }, - fn: async (ctx) => { - if (ctx.toolName !== "Bash") return allow(); - if (/git\s+commit/.test(ctx.toolInput?.command ?? "")) { - return instruct("Run tests before committing."); - } - return allow(); - }, - }); - ``` - - - ```bash - git add .failproofai/policies/ - git commit -m "Add team quality policies" - ``` - - Every team member who has failproofai installed picks up these policies automatically. No per-developer setup needed. - - - - -Commit `.failproofai/policies/` to your repo so the whole team shares the same standards. As your team discovers new failure modes, add policies and push — everyone gets the update on their next `git pull`. Over time these policies become a living quality standard that keeps improving. - - ---- - -## Data storage - -All configuration and logs stay on your machine: - -| Path | What it stores | -|------|----------------| -| `~/.failproofai/policies-config.json` | Global policy config | -| `~/.failproofai/policies/` | Your own policies — drop `*-policies.mjs` in, no config needed | -| `~/.failproofai/policies/cloud-policies/` | Policies deployed to this machine by your organisation | -| `~/.failproofai/hook-activity/` | Hook execution history (paged JSONL) | -| `~/.failproofai/logs/` | Debug logs for custom hook errors | -| `.failproofai/policies-config.json` | Per-project config (committed) | -| `.failproofai/policies-config.local.json` | Personal overrides (gitignored) | - ---- - -## Uninstalling - -```bash -failproofai policies --uninstall -``` - -Removes hook entries from `~/.claude/settings.json`. Config files in `~/.failproofai/` are kept. - ---- - -## Next steps - - - - - Scopes and config file format - - - - All 26 policies with parameters - - - - Write your own policies in JavaScript - - - - Monitor sessions and review policy activity - - - diff --git a/docs/he/agent-support.mdx b/docs/he/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/he/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/he/agenteye/cli-and-agents.mdx b/docs/he/agenteye/cli-and-agents.mdx deleted file mode 100644 index 1da70ee1..00000000 --- a/docs/he/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "כל הפריסה של Failproof AI Observability שלך, במרחק פקודה אחת." ---- - - -כל הפריסה של Failproof AI Observability שלך, במרחק פקודה אחת. בדוק את הייצור, צור מפתח API, או אשר תקלה מבלי לעזוב את הטרמינל שלך, ואז כתוב סקריפט לכל זה ל-CI, או תן לסוכן קוד לעשות זאת בעברית פשוטה. - -```bash -pipx install agenteye -agenteye login --email you@example.com # a 6-digit code lands in your inbox -agenteye --json sessions --since 24h # every agent run from the last day, newest first -``` - -*ה-CLI של `agenteye` מדבר עם הדשבורד שלך. זהו כלי שונה מהאספן, שמשדר אירועים לשרת.* - -## כל הפריסה שלך, במרחק פקודה אחת - -הפסק לדלג בין כרטיסיות כדי לענות על שאלה מהירה. ה-CLI של `agenteye` קורא את הנתונים שלך ומנהל את הארגון שלך מקובץ בינארי אחד, כך שבדיקה שפעם הייתה דורשת לחיצה דרך הדשבורד הופכת לשורה אחת שאתה יכול להפעיל מחדש, ליצור כינוי, או להדביק לתוך runbook. אתה מקבל ארבע ממשקים: - -- **קרא את הנתונים שלך:** `sessions`, `events`, `evals`, ו-`errors`, מסוננים לפי זמן, סוכן וסביבה. -- **נהל את הארגון שלך:** `keys`, `users`, `settings`, `alerts`, ו-`incidents`. -- **הרץ ניתוח:** SQL שמור בתוספת מריץ `query` אד-הוק על נתוני האירוע שלך. -- **שאל את העוזר:** `agent ask` מגיע לאותו אנליסט בקריאה בלבד שאתה משוחח איתו בדשבורד. - -התקן אותו פעם אחת עם `pipx`, היכנס עם קוד בן 6 ספרות שנשלח בדוא"ל, ואתה מוכן. ההפעלה נמשכת כיום; הרץ את `agenteye login` מחדש כאשר היא תפוג. השתמש בו כדי לבדוק את הייצור, לספק מפתח, או לטפל בתקלה שזורקת, הכל ללא פתיחת דפדפן: - -```bash -agenteye errors --since 24h --aggregate # what is breaking, grouped by error type -agenteye incidents list --state firing # what is on fire right now -agenteye keys create ci --add events:add # a key that can only push events, secret shown once -``` - -הרגל אחד שכדאי לדעת: אפשרויות גלובליות כמו `--json` מופיעות לפני הפקודה. `agenteye --json sessions` נכון; `agenteye sessions --json` לא. - -## כתוב סקריפט, חברו ל-CI - -כל פקודה מקבלת `--json`, וזה משנה הכל. JSON נקי עובר ל-stdout בעוד מצב ואזהרות לבני אדם עוברות ל-stderr, כך שתיעוד `--json` מופעל ישר ל-`jq` ללא שורה תועה לפירוק. זה מה שהופך את ה-CLI לטוב באותה מידה עבורך בהנחיה ועבור סוכן קוד שמנתח פלט: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -זה בנוי להפעלה ללא השגחה. בקשות אישור דלג אוטומטי כאשר אין טרמינל מצורף, כך שלא משהו תלוי בצינור, וכל פקודה מחזירה קוד יציאה משמעותי: `0` הצלחה, `4` לא מחובר, `5` חסר הרשאה (ההודעה שמה שם, למשל `alerts:write`), `3` דשבורד לא ניתן להשגה. סקריפט יכול להתחלק על `4` כדי לאמת מחדש או על `5` כדי לומר לך בדיוק מה לבקש ממנהל. - -## תן לסוכן קוד להנהיג אותו בעברית פשוטה - -עדיף עדיין, לא צריך לזכור את דגלים אלה כלל. **ה-CLI skill** הוא תיקייה Skill סוכן קטנה בשם `agenteye-cli` שמלמדת סוכן קוד כמו Claude Code או Codex להנהיג את ה-CLI מבקשות בעברית פשוטה. שאל "יש משהו שבור היום?" והסוכן בוחר את הפקודה, מריץ אותה כמוך, וענה בפרוזה. - -עבור Claude Code, שחרר את תיקיית `agenteye-cli` ל-`~/.claude/skills/` והיא מגלה אוטומטי. Failproof AI Observability מספק את התיקייה; אין שום דבר נוסף להתקנה, מכיוון שזה רק מנהיג את ה-CLI שכבר התקנת. היכנס בעצמך תחילה: הskill לא יכול להשלים את הכניסה לקוד דוא"ל עבורך. - -מכיוון שהסוכן מנהיג את ה-CLI כמוך, הוא יכול לעשות הכל שההתחברות שלך מאפשרת, קריאה וכתיבה כאחד: צור מפתחות, שנה הגדרות, פתור תקלות. בקשת "האם אתה בטוח?" ב-CLI לא מתחדשת עבור סוכן, כך שהskill כתוב כדי להצהיר על הפקודה המדויקת ולהמתין לאישור שלך לפני כל שינוי. אתה שלב האישור. - -```text -you Why did session run-001 fail? - -agent Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -הקריאות נשארות מיידיות, וכל כתיבה עוצרת עבורך: - -```text -you Give CI a key that can only push events. - -agent This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you yes - -agent Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## קשור - -- [CLI reference](/he/agenteye/cli): כל פקודה, דגל וצורת JSON. -- [CLI recipes for agents](/he/agenteye/cli-recipes): עותקים והדבקות של דפוסי `jq` וטיפול קוד יציאה. -- [CLI agent skill](/he/agenteye/cli-skill): התקן והרץ את ה-`agenteye-cli` skill. -- [AI assistant](/he/agenteye/assistant): האנליסט בדשבורד שה-`agent ask` מדבר אליו. \ No newline at end of file diff --git a/docs/he/agenteye/cli-skill.mdx b/docs/he/agenteye/cli-skill.mdx deleted file mode 100644 index e645f111..00000000 --- a/docs/he/agenteye/cli-skill.mdx +++ /dev/null @@ -1,160 +0,0 @@ ---- ---- -title: "כישורון CLI Observability של Failproof AI" -description: "שאל את סוכן הקוד שלך \"האם משהו שבור היום?\" והנח לו לענות מנתוני Failproof AI Observability השידוריים שלך, ללא צורך לשנן פקודות." ---- - - -שאל את סוכן הקוד שלך *"האם משהו שבור היום?"* והנח לו לענות מנתוני Failproof AI Observability השידוריים שלך, ללא צורך לשנן פקודות. **כישורון CLI Observability של Failproof AI** (`agenteye-cli`) הוא *Agent Skill*: תיקייה קטנה של הוראות שסוכן קוד כגון Claude Code או Codex טוען לפי הצורך. היא מלמדת את הסוכן להפעיל את התפוצה של Observability שלך דרך ה-[`agenteye` CLI](/he/agenteye/cli) מבקשות בעברית רגילה כמו *"תן ל-CI מפתח שיכול רק לדחוף אירועים"* או *"אשר את האירוע הפועל והקצה אותו אלי."* - -זה **לא** שירות או בינארי נפרד; אין כלום לפרוס. זה עובד על גבי ה-CLI שכבר התקנת: הסוכן שדרג אל `agenteye --json …`, מנתח את ה-JSON הנקי, והשיב לך בטקסט. כל דבר שהוא יכול לעשות, אתה יכול לעשות בעצמך בהקלדת אותן פקודות. - ---- - -## איך זה קשור לממשקים אחרים של Failproof AI Observability - -Failproof AI Observability נותן לך ארבע דרכים להגיע לאותם נתונים ובקרות. הם משלימים זה את זה: - -| ממשק | מה זה | איפה זה רץ | הגש אליו כאשר | -|---|---|---|---| -| **[CLI](/he/agenteye/cli)** | ההתייחסות לפקודה/דגל עבור `agenteye` | הטרמינל שלך | אתה רוצה להריץ או לתסריט פקודה ספציפית | -| **[CLI recipes](/he/agenteye/cli-recipes)** | דוגמות `jq`/pipeline להעתקה-הדבקה | הטרמינל / סקריפטים שלך | אתה מחברת את ה-CLI לאוטומציה | -| **כישורון CLI** (מסמך זה) | דלת חזיתית בשפה טבעית ל-CLI | סוכן הקוד שלך, בתחנת העבודה שלך | אתה רוצה לשאול ולתת לסוכן לבחור את הפקודה | -| **[כישורון Evaluator](/he/agenteye/evaluator-skill)** | כישורון אחות שתכנן ובונה את שירות הניקוד שלך | סוכן הקוד שלך, בתחנת העבודה שלך | אתה רוצה **לייצר** ניקוד eval במקום לקרוא אותו | -| **[כישורון Python SDK](/he/agenteye/python-sdk-skill)** | כישורון אחות שמכשיר את הסוכן שלך כך שהוא פולט טלמטריה כלל | סוכן הקוד שלך, בתחנת העבודה שלך | אתה רוצה שהסוכן שלך **ייצור** את האירועים שכישורון זה קורא | -| **[עוזר AI בתוך הלוח](/he/agenteye/assistant)** | צ'אט משובץ בלוח המחוונים | צד שרת (בלוח המחוונים) | אתה רוצה שאלות ותשובות בתוך לוח המחוונים על הנתונים שלך | - -לכישורון עצמו אין הרשאות שלו; הוא רק הופך את המילים שלך לקריאות CLI שרצות כך: - -```mermaid -flowchart TD - YOU["אתה: 'אשר את האירוע הפועל'"] --> AGENT["סוכן קוד (Claude Code / Codex)
טוען את כישורון agenteye-cli"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|הפעלת CLI המאומתת שלך| API["API לוח Observability"] -``` - -### לעומת עוזר ה-AI בתוך הלוח: הבחנה חשובה - -אלה שני כלים שונים עם טווחי פיצוץ שונים מאוד: - -- **עוזר ה-AI בתוך הלוח** ([AI assistant](/he/agenteye/assistant)) הוא צ'אט משובץ בלוח המחוונים, בגיבוי שירות הסוכן. זה **קריאה בלבד בתוספת כתיבה שאושרה**: הוא יכול לטיוטה שאלות שמורות ולוחות, אך כל כתיבה עוצרת לאישור ההקלקה המפורש שלך, והוא לעולם לא מוחק. זה נשער על ידי ההרשאה `agent:use` ורק אי פעם רואה נתונים עבור הארגון שאתה צופה בו. -- **כישורון CLI** רץ על *תחנת העבודה שלך* בתוך *סוכן הקוד שלך* ומנהל את `agenteye` CLI כ-**אתה**. הוא יכול לבצע את **המשטח המלא של ה-CLI, כולל מוטציות** (יצור/סיבוב/הפסקה של מפתחות API, שנה הגדרות ארגון, פתור אירועים, מחק שאלות שמורות), מוגבל רק בהרשאות ההתחברות שלך ל-CLI. התייחס אליו בדיוק כפי שהיית מתייחס להרצת אותן פקודות ביד. - ---- - -## דרישות ראשוניות - -1. **ה-`agenteye` CLI מותקן** ו-`PATH` (ראה את התייחסות [CLI](/he/agenteye/cli): `pipx install agenteye`). -2. **כתובת ה-URL של לוח המחוונים שלך** מוגדרת (`AGENTEYE_DASHBOARD_URL`, או הסוכן עובר `--base-url`). -3. **הפעלה שנכנסה**: הרץ `agenteye login` בעצמך קודם לכן. הכישורון **לא יכול** להשלים את הכניסה לקוד חד-פעמי בדוא״ל עבורך; זה אמר לך להרוץ `agenteye login` אם ההפעלה חסרה או פג תוקף (קוד יציאת CLI `4`). - ---- - -## איפה להשיג את זה - -הכישורון פורסם באוסף הכישורונים הציבורי של Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -שום דבר בו לא משוער — המאגר ציבורי והכישורון לא זקוק לעדות משלו משום שהוא רק מנהל את `agenteye` CLI **הציבורי** מול לוח המחוונים שלך, תוך שימוש בהפעלה **שהתחברת אליה**. אתה לא צריך לשאול אף אחד על זה. - -שימו לב שהוא משתלח כתיקייה משלו והוא **לא** בתוך חבילת `pipx install agenteye`, כך שלא תחפש אותו שם. - -## התקנת הכישורון - -הנתיב המהיר ביותר הוא CLI [`skills`](https://skills.sh), אשר אחזר את התיקייה ושוחק אותה כאשר הסוכן שלך מחפש: - -```bash -# Claude Code, פרויקט זה בלבד -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# כל פרויקט (מתקין ל-~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex במקום זאת -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -לאחר מכן נהל אותו כמו כל כישורון אחר: - -```bash -npx skills list -a claude-code # מה שהותקן -npx skills update agenteye-cli # משוך את הגרסה העדכנית -npx skills remove agenteye-cli # הסר אותו -``` - -מעדיף להתקין ביד? Agent Skill הוא רק תיקייה המכילה `SKILL.md` (בתוספת התייחסויות אופציונליות), כך שהעתקה פועלת גם: - -- **Claude Code**: שים את תיקיית `agenteye-cli/` ב-`~/.claude/skills/` (כל פרויקט) או `/.claude/skills/` (רק אותו רפו). Claude Code מגלה זאת באופן אוטומטי — אמת עם רשימת `/skills`, או פשוט שאל שאלה התואמת את התיאור שלו. -- **Codex (OpenAI)**: Codex קורא את אותה `SKILL.md`. ה-`agents/openai.yaml` המצורף קובע `allow_implicit_invocation: true`, כך ש-Codex בוחר באופן אוטומטי את הכישורון כאשר משימה תואמת; אחרת הפעל אותו באופן מפורש כ-`$agenteye-cli`. - ---- - -## בטיחות: מוטציות **לא** מבקשות כאשר סוכן מריץ את ה-CLI - -> **אזהרה:** קרא זאת לפני שאתה נותן לסוכן לבצע שינויים. - -ה-CLI `agenteye` בדרך כלל שואל *"האם אתה בטוח?"* לפני פעולה הרסנית. זה **דילוג אוטומטי על אישור זה בכל פעם שלא מוצמד לטרמינל (שהוא בדיוק איך סוכן קוד מריץ אותו), ו-`--json` דילוג עליו גם.** אז הנושא הבטיחות **לא** יופעל עבור הסוכן. - -הכישורון כתוב לפיצוי: הוא מוּעד להצהיר על הפקודה המדויקת שהוא יריץ ולהשיג את ה-**אישור המפורש שלך לפני כל שינוי מצב**. השמור על המשמעת הזו. כאשר אתה מנהל את Failproof AI Observability דרך סוכן, *אתה* הצעד האישור. הפקודות המשנות מצב שצריך להיזהר מהן: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- תת-הפקודות הכתיבה `incidents`: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -הכל תחת **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) הוא קריאה בלבד ולא משנה שום דבר. - -מכיוון שהסוכן פועל כ-**אתה**, הוא יכול לעשות רק מה שההתחברות שלך מורשת לעשות; הרשאות מתבררות **לכל ארגון** (ראה [API keys](/he/agenteye/api-keys)). פקודה שאתה חסר הרשאה עבורה מחזירה קוד יציאה `5` עם ההרשאה המדויקת שנקראה, כך שהסוכן יכול לומר לך בדיוק למה לבקש מ-admin במקום להיכשל בעלום. - ---- - -## מה אתה יכול לשאול אותו - -הנה איך נראית סיבוב בעולם האמת, קריאה ואחריה שינוי שמחכה לאישור שלך: - -```text -אתה ▸ האם משהו שבור בבחינה האחרונה? - -סוכן ▸ ריצה: agenteye --json errors --since 24h --aggregate - 3 סוגי שגיאה בבחינה האחרונה. הגדול ביותר הוא TimeoutError - (18 עוקבים, כולם ב-checkout-agent), ואחריו RateLimited (5). - רוצה את ההפעלות הנכשלות עבור TimeoutError? - -אתה ▸ תן ל-CI מפתח שיכול רק לדחוף אירועים. - -סוכן ▸ זה יוצר מפתח API. אני אריץ: - agenteye keys create ci --add events:add - בסדר להמשיך? - -אתה ▸ כן - -סוכן ▸ בוצע. מפתח "ci" נוצר עם events:add בלבד. - הסוד מוצג רק פעם אחת, כך שאחסן אותו עכשיו. אני לא יכול להדפיס מחדש. -``` - -הכישורון ממפה כל כוונה בשפה טבעית לפקודת `agenteye` הנכונה, תוך גילוי ערכים חוקיים קודם (`list `, `whoami`) כך שהוא לא מנחש, והצהיר על הפקודה המדויקת לפני כל שינוי. עוד דוגמאות: - -- *"האם משהו שבור / נכשל בבחינה האחרונה?"* → `errors --since 24h --aggregate`, ואחריו פירוט. -- *"למה הפעלה `run-001` נכשלה?"* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *"איך האיכות מתגברת בשבוע זה?"* → `evals --aggregate --since 7d`, ואחריו קדרילה לתוך ריצות בציון נמוך. -- *"תן ל-CI מפתח שיכול רק לדחוף אירועים."* → `keys create ci --add events:add` (זה מצהיר על הפקודה, ואחריו יוצר אותה ותופס את הסוד החד-פעמי). -- *"מי יש גישה? הפוך את Dana לקריאה בלבד."* → `users list` → `users update dana@… --permission-set read-only` (לאחר אישור איתך). -- *"אשר את האירוע הפועל והקצה אותו אלי."* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`. - -עבור הפקודות המדויקות, הדגלים, וצורות JSON שמאחוריהן, ראה את התייחסות [CLI](/he/agenteye/cli) ו-[CLI recipes for agents](/he/agenteye/cli-recipes). - ---- - -## שלבים הבאים - -- **[CLI](/he/agenteye/cli)**: ההתייחסות המלאה לפקודה ודגל עבור `agenteye`. -- **[CLI recipes for agents](/he/agenteye/cli-recipes)**: דוגמות `jq` להעתקה-הדבקה וטיפול בקוד יציאה. -- **[כישורון סוכן Evaluator](/he/agenteye/evaluator-skill)**: הכישורון אחות, לבניית ה-evaluator שאותו ניקוד `agenteye evals` קורא. -- **[כישורון סוכן Python SDK](/he/agenteye/python-sdk-skill)**: הכישורון אחות, להכשרת סוכן כך שהוא פולט את הטלמטריה שקוראת `agenteye`. -- **[עוזר AI](/he/agenteye/assistant)**: העוזר בתוך הלוח (לא להתבלבל עם כישורון הטרמינל הזה). -- **[API keys](/he/agenteye/api-keys)**: מודל ההרשאה לכל ארגון שמגביל מה הכישורון יכול לעשות. \ No newline at end of file diff --git a/docs/he/agenteye/codex-capture.mdx b/docs/he/agenteye/codex-capture.mdx deleted file mode 100644 index d4faa449..00000000 --- a/docs/he/agenteye/codex-capture.mdx +++ /dev/null @@ -1,56 +0,0 @@ ---- ---- -title: "Codex session capture" -description: "Tail your team's local OpenAI Codex sessions into AgentEye as ordinary sessions and events — with no change to how they run Codex." ---- - -המהנדסים שלך כבר משתמשים ב-OpenAI Codex כל יום. Codex session capture מביא את הסשנים של קידוד אלה לתוך AgentEye כסשנים ואירועים רגילים, כך שתוכל לחפש, להשמיע שוב ולהעריך אותם לצד כל שאר מה שאתה צופה בו. זה משלים את [Python SDK](/he/agenteye/python-sdk): ה-SDK מחוממי אגנטים שאתה כותב, בעוד שזה תופס את עבודת ה-Codex שהצוות שלך כבר עושה — ללא שום שינוי בדרך שהם משתמשים בו. - -collector בעלי רקע קטן קורא Codex local session transcripts כשהם נכתבים ושולח אותם ל-AgentEye. collector אחד לכל מכונה תופס כל Codex surface מקומי בו זמנית — אין הגדרה לכל משטח. - -אותו collector תופס אגנטים אחרים גם — ראה [OpenClaw](/he/agenteye/openclaw-capture) ו-[Hermes](/he/agenteye/hermes-capture). הפוך כל אחד שאתה מריץ; collector יחיד יכול להשתמע למספר בו זמנית. - ---- - -## מה זה תופס - -כל Codex surface שמריץ **locally** מייצר את אותו on-disk session transcripts, ו-collector תופס את כולם: - -- ה-Codex **CLI** ו-`codex exec` -- ה-**VS Code / IDE extension** -- ה-**desktop app**, כשהוא מריץ סשן locally - -כל Codex session הופך ל-AgentEye [session](/he/agenteye/sessions); ההודעות של user ו-assistant שלו, reasoning, tool calls, tool results, ו-token usage הופכים ל-[events](/he/agenteye/event-stream) התואמים. ה-surface שכל סשן הגיע ממנה (CLI, IDE, או desktop) נרשם, כך שתוכל להבחין ביניהם. - -> **Cloud sessions לא תופסים.** ה-desktop app בהולך וגדל מריץ סשנים בענן Codex ושומר רק את metadata שלהם במכונה — אין local transcript לקרוא. רק סשנים המתורגמים locally תופסים. - ---- - -## הפוך זה פעיל - -Capture כבוי עד שתהפוך אותו פעיל. התקן את ה-collector עם API key שיש לו את הרשות `events:add` (ראה [API keys](/he/agenteye/api-keys)), והפוך את ה-Codex capture פעיל: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -זה מתקין את ה-collector, משנה אותו כ-background service, ומתחיל ללכוד. אשר שהוא פועל: - -```bash -agenteye-collector health -``` - -בהפעלה הראשונה, הסשנים ה-Codex הקיימים שלך מתמלאים חזרה פעם אחת ופעילות חדשה ואז זורמת תוך שניות. הקבצים שלהם של Codex עצמם נקראים בלבד — אף פעם לא משונים, מועברים, או מחוקים — וכל סשן נשלח בדיוק פעם אחת, אפילו על פני restarts. - ---- - -## היכן זה מופיע - -סשנים תפוסים מופיעים ב-**Sessions**, והאירועים שלהם בזרם **Events**, כמו כל אגנט אחר שאתה צופה בו — כך שה-[session replay](/he/agenteye/sessions), [search](/he/agenteye/queries), [evaluations](/he/agenteye/evaluations), ו-[alerts](/he/agenteye/alerts) כולם עובדים עליהם. סנן לפי ה-Codex agent כדי לראות אותם בעצמם. - ---- - -## Privacy - -Codex transcripts מכילים את הסשן המלא — כולל command output, file contents, וכל מה ש-Codex קרא או כתב — ויכול להכיל סודות. סשנים תפוסים נשלחים כמו שהם, אז הפוך את ה-capture פעיל רק במכונות וצוותים שבהם ריכוז תוכן זה ב-AgentEye הוא מתאים, ותן ל-collector key בהיקף `events:add` בלבד. ראה [Security](/he/agenteye/security) להבנת איך הנתונים שלך מוחזקים מבודדים. \ No newline at end of file diff --git a/docs/he/agenteye/concepts.mdx b/docs/he/agenteye/concepts.mdx deleted file mode 100644 index 131473d6..00000000 --- a/docs/he/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "קונספטים" -description: "אוצר המילים של Failproof AI Observability — אירועים, סשנים, הערכות, ביקורות, ממצאים, וכרונות — מוגדרים במקום אחד." ---- - - -עמוד זה מגדיר את אוצר המילים שבו משתמשת Failproof AI Observability. אם מונח בגיד אחר לא מוכר לך, הוא מוגדר כאן. אתה לא חייב לקרוא את זה מתחילה עד סוף: התסקור, או חזור בעת שתיתקל במילה שאתה רוצה להבהיר. - ---- - -## מודל הנתונים - -**Event** -יחידת הנתונים הקטנה ביותר. אירוע אחד רושם צעד יחיד שהסוכן שלך ביצע: `tool_use`, `model_request`, `hook_completed`, `error`, וכדומה. הסוכן שלך פולט אירועים דרך ה-[Python SDK](/he/agenteye/python-sdk); הם מופיעים בזמן אמת בעמוד **Events**. - -**Session** -ריצה אחת של סוכן, המזוהה על ידי `session_id`. סשן הוא כל האירועים החולקים את המזהה הזה, ממוקדים בשורה יחידה בעמוד **Sessions** וצויירו כגרף ביצוע בעמוד הפרטים שלו. סשן בדרך כלל מתחיל עם `agent_start` ומסתיים עם `agent_end`. - -**Agent** -שחקן בעל שם בתוך ריצה, המזוהה על ידי `agent_id`. ריצה יכולה לכלול כמה סוכנים: מתכננן שיוצר תת-סוכן מסכם, לדוגמה. תת-סוכנים נושאים `parent_id`, וזה מה שמאפשר ל-Failproof AI Observability לצייר אותם בנתיבים שלהם בגרף הביצוע. - -**Environment** -תווית למקום בו התרחשה הריצה: `production`, `staging`, `dev`. אתה מגדיר את זה פעם אחת כשאתה מגדיר את ה-SDK. כמעט כל עמוד בלוח הבקרה יכול לסנן לפי סביבה. - -**Context-window fill** -אחוז חלון ההקשר של מודל שתגובה צרכה. Failproof AI Observability חוצצה אותו על אירועי `model_response` עבור מודלים שהוא מזהה, כך שגדילת ההנחיה והעימות קרוב יהיו גלויים ממש בזרם האירועים. - ---- - -## איכות - -**Evaluation** -ציון איכות לסשן שהסתיים, שמופק על ידי שירות ניקוד שאתה מריץ. הערכות הן אופציונליות: עד שאתה מחבר מערך, סשנים מתועדים אך לא מדורגים. כל הערכה יכולה להכיל כמה ציונים בעלי שם (לדוגמה `helpfulness`, `factuality`, `tool_efficiency`), כל אחד עם הערה קצרה של הנמקה. ראה [Evaluation suite](/he/agenteye/evaluation-suite). - -**Score key** -שם של ממד אחד שמערך דיווח עליו, כגון `helpfulness`. התראות וביקורות יכולות להסתכל על מפתח ציון ספציפי לאורך זמן. - -**Evaluator** -שירות הניקוד שלך. Failproof AI Observability משדרת את התמלול של ריצה שהסתיימה אליו ושומרת את הציונים שהוא מחזיר. זה לא משדר מערך ברירת מחדל; לוגיקת הניקוד היא שלך. - ---- - -## מציאה ותיקון כשלים - -**Hook** -מגן או תופעת לוואי שמסגרת הסוכן שלך מריצה סביב צעד: בדיקת בטיחות תוכן, עריכת PII, שמורת תקציב. Hooks פולטות אירועי `hook_triggered` / `hook_completed` עם `outcome` (allow, deny, modify), ומקבלות את עמוד ההתבוננות שלהן. - -**Alert rule** -כלל שנכנס לפעולה כאשר מטרי חוצה סף שהגדרת: שיעור שגיאות, p95 latency, עלות אסימונים, או ציון מערך. כאשר כלל נכנס לפעולה, הוא פותח כרונה ומודיע לערוצים שבחרת (דוא"ל, Slack, webhook, בתוך לוח הבקרה). ראה [Alerts](/he/agenteye/alerts). - -**Incident** -בעיה פתוחה שנוצרה כאשר כלל התראה נכנס לפעולה. לכרונות יש מחזור חיים (קבל, הקצה, פתור) וציר זמן פעילות שרושם כל פעולה. אתה יכול גם לפתוח אחת ידנית. - -**Audit** -חקירה חוזרת (כל שעה עד שבועית) שחופרת את היומנים שלך *על פני* סשנים לחיפוש דפוסי כשל שלא כתבת כלל עבורם: אשכולות שגיאות, ציונים נמוכים, חריגות latency, לולאות קריאת כלים, וריצות שלא הסתיימו. איפה שהתראה שומרת על מטרי שאתה כבר יודע עליו, ביקורת אומרת לך למה להסתכל הבא. ראה [Audits](/he/agenteye/audits). - -**Finding** -תוצאה אחת דורגת וגיבוי ראיות מריצת ביקורת. מציאה מכנה דפוס, מקשרת להפעלות המדויקות מאחוריו, וממלאה מחזור חיים בדיקה (קבל, פתור, השתק, בטל). Failproof AI Observability מסלקת מציאות פעם על פעם כך שדפוס ידוע מתעדכן במקום להצטבר. - -**The AI assistant** -הצ'אט בתוך לוח הבקרה שמענה לשאלות על הסוכנים שלך באנגלית רגילה, על הנתונים שלך שלך. הוא קריאה בלבד כברירת מחדל; כל דבר שהוא יוצר (שאילתה שמורה, לוח בקרה) מאושר בשער, והוא לא יכול לעולם למחוק. ראה [AI assistant](/he/agenteye/assistant). - ---- - -## הפעלה - -**Organization (tenant)** -סביבת עבודה מבודדת. מופע אחד של Failproof AI Observability יכול להנחות ארגונים רבים, כל אחד עם המשתמשים, המפתחות, וההנתונים שלו. כל URL של לוח בקרה מסודר בהיקף של ה-slug הארגוני שלך (`//…`). - -**Collector** -`agenteye-collector`, הדמון הקל שרץ בכל מכונת סוכן, מקבץ את האירועים שה-SDK כותב לדיסק, ומשדר אותם לשרת. - -**API key** -אסימון בהיקף זה מאמת לקוח כנגד השרת. מפתחות נושאים הרשאות דקיקות (לדוגמה `events:add` עבור הקולט, היקפי קריאה בלבד עבור מפתח לוח בקרה). ראה [API keys](/he/agenteye/api-keys). - -**Server** -שירות ההשקעה וה-API. הוא משקיע אירועים, שומר מצב תפעולי בבסיסי הנתונים שלך, ומשרת את לוח הבקרה וה-CLI. - -**Dashboard** -ממשק המשתמש של האינטרנט. כל עמוד מסודר לארגון ו קורא דרך API של השרת. - ---- - -## צעדים הבאים - -- [Overview](/he/agenteye/overview): איך החלקים האלה מתאימים יחד. -- [Observability](/he/agenteye/observability): משטחי ההתבוננות (Events, Sessions, Models, Tools, Hooks, Errors). \ No newline at end of file diff --git a/docs/he/agenteye/evaluator-skill.mdx b/docs/he/agenteye/evaluator-skill.mdx deleted file mode 100644 index f5902d58..00000000 --- a/docs/he/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,168 +0,0 @@ ---- ---- -title: "כישרון סוכן הערכה של Failproof AI Observability" -description: "עבור מ\"אני חושב שהסוכן שלנו לפעמים רע\" לשירות ניקוד פרוס, כשהסוכן הקוד שלך עושה גם את ההחלטה וגם את הבנייה." ---- - - -עבור מ*\"אני חושב שהסוכן שלנו לפעמים רע\"* לשירות ניקוד פרוס, כשהסוכן הקוד שלך עושה גם את ההחלטה וגם את הבנייה. **כישרון Failproof AI Observability evaluator** (`agenteye-evaluator`) הוא *Agent Skill*: תיקייה קטנה של הוראות שסוכן קוד כמו Claude Code או Codex טוען לפי דרישה. זה מלמד את הסוכן לעבוד ולברר אילו מימדי איכות כדאי לעקוב עבור *הסוכן שלך*, ואז לכתוב, לבדוק ולפרוס את [שירות ה-evaluator](/he/agenteye/evaluation-suite) שמדרג אותם. - -זה **לא** מדרג מתארח, רישום שאתה מעלה אליו, או מערכת תוספים. ה-evaluator שלך נשאר שירות HTTP שלך בתשתית שלך, בדיוק כما מתואר בהדרכה [Evaluation suite](/he/agenteye/evaluation-suite). הכישרון רק מלמד את הסוכן שלך לבנות זאת טוב, כך שכל מה שהוא עושה, אתה יכול לעשות בעצמך על ידי כתיבת אותו קוד. - ---- - -## החלק הקשה הוא להחליט מה לדרג - -משטח ה-SDK קטן — דקורטור ושני מודלים — וסוכן יכול לכתוב את זה מ[החוזה](/he/agenteye/evaluation-suite#http-contract) לבד. זה לא המקום שבו evaluators נכשלים. הם נכשלים כי הם דורגים את הדבר הלא נכון, וה-evaluator שדורג את הדבר הלא נכון הוא גרוע מכלום: הוא מייצר לוח מחוונים שכולם למדו להתעלם ממנו. - -אז רוב הכישרון הוא החלק לפני שקוד כלשהו קיים. יש לסוכן לראיין אותך (*\"תאר הפעלה שהלכה טוב; עכשיו אחת שהלכה בצורה רעה\"*), ואז לשוך את הסשנים האמיתיים שלך דרך [`agenteye` CLI](/he/agenteye/cli) וקרא אותם מקצה לקצה. שתי החצאים האלה בדרך כלל לא מסכימים, והפער הוא הנקודה: מה אתה מתכוון למדוד בעבור מה שהתמלילים שלך יכולים להתמוך בו. מימד שורד רק אם הוא **ניתן לחישוב** מהאירועים ו**מבדיל** — אם הוא מדרג 0.9 גם בהפעלה הטובה שלך וגם בהרעה, הוא לא מלמד כלום ומתחלק. - -מה שחוזר הוא הצעה של 2-4 מימדים כשהנימוק מצורף, בשבילך לאשר לפני שכתוב שורה אחת. - -```mermaid -flowchart TD - YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent (Claude Code / Codex)
loads the agenteye-evaluator skill"] - AGENT -->|"interview: what does good vs bad look like?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["your real sessions
what actually happens"] - DATA --> DIMS["2-4 dimensions, you sign off"] - DIMS --> SVC["your evaluator service
agenteye-evaluator SDK"] - SVC --> SCORES["scores land in the dashboard
and agenteye evals"] -``` - ---- - -## איך זה קשור לחלקי ההערכה האחרים - -ארבע מסמכים מכסים ניקוד, והם מוסרים זה לזה בסדר: - -| עמוד | מה זה | הגע אליו כאשר | -|---|---|---| -| **[Evaluations](/he/agenteye/evaluations)** | התכונה: ניקודים בגריד הסשנים, לוחות מחוונים, הערכה מחדש | אתה רוצה לדעת מה ניקוד אוטומטי מקבל לך | -| **[Evaluation suite](/he/agenteye/evaluation-suite)** | החוזה HTTP, ה-SDK, משתני סביבת השרת | אתה מיישם או ניפוי באגים ב-evaluator בעצמך | -| **Evaluator skill** (מסמך זה) | דלת קדמית בשפה טבעית לעיצוב *וגם* בנייה של המדרג | אתה רוצה להעבור מ"אני רוצה evals" לשירות שרץ | -| **[CLI skill](/he/agenteye/cli-skill)** | דלת קדמית בשפה טבעית על ה-`agenteye` CLI | אתה רוצה *לקרוא* את הניקודים שכבר יש לך | -| **[Python SDK skill](/he/agenteye/python-sdk-skill)** | דלת קדמית בשפה טבעית על כלי הסוכן שלך | הסוכן שלך עדיין לא משדר סשנים — אין שום דבר לדרג | - -### לעומת CLI skill: בנייה לעומת קריאה - -שני הכישרונות מכוונים במכוון שאינם חופפים, והתקנת שניהם היא ההגדרה הרגילה — הסוכן בוחר ביניהם על סמך מה שאתה שואל: - -- **`agenteye-evaluator`** (מסמך זה) בונה את הדבר שמייצר ניקודים. עבודתו מסתיימת כאשר ניקודים נוחתים בפעם הראשונה. -- **[`agenteye-cli`](/he/agenteye/cli-skill)** קורא ניקודים שכבר קיימים (`agenteye evals`). *"האם איכות ירדה השבוע?"* היא השאלה שלה, לא של הכישרון הזה. - ---- - -## דרישות מוקדמות - -1. **`agenteye` CLI מותקן ומחובר** (`pipx install agenteye`, ואז `agenteye login`). הכישרון מסתמך עליו פעמיים: לשוך את הסשנים האמיתיים שהוא מעצב בהם, ולאשר שהניקודים שלך נוחתו בסוף. הכניסה שלך צריכה `events:read`, בתוספת `evaluations:read` לאישור סופי זה. כמו CLI skill, היא **לא יכולה** להשלים את כניסת קוד חד-פעמית שנשלחה בדוא\"ל עבורך. -2. **מקום ל-evaluator לחיות בו.** הוא מובנה לתמונה ורץ כשירות ממושך, אז הוא צריך ריפו אמיתי, לא קובץ סקראץ'. Evaluators לעתים קרובות חיים בריפו שלהם, נפרדים מהסוכן שנדרג — הכישרון חפש אחד קיים ושואל לפני סיבוך חדש. -3. **גלגל `agenteye-evaluator` SDK** — קרא את הסעיף הבא לפני שהסוכן שלך מתחיל להקליד `pip` פקודות. - ---- - -## איפה להשיג זאת - -הכישרון פורסם בקולקציית הכישרונות הציבורית של Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -המחסן ציבורי והכישרון לא זקוק לכל אישור משלו — הוא רק מנהל את `agenteye` CLI עם הסשן *שלך* התחברת אליו, וכותב קוד בריפו *שלך*. שים לב שהוא מסופק כתיקייה משלו ו**לא** בתוך חבילת `pipx install agenteye`, אז אל תחפש אותו שם. - -## התקנת הכישרון - -הנתיב המהיר ביותר הוא [`skills`](https://skills.sh) CLI, המביא את התיקייה וזורקת אותה למקום שהסוכן שלך מחפש: - -```bash -# Claude Code, this project only -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# every project (installs to ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codex instead -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -אז נהל זאת כמו כל כישרון אחר: - -```bash -npx skills list -a claude-code # what's installed -npx skills update agenteye-evaluator # pull the latest version -npx skills remove agenteye-evaluator # remove it -``` - -מעדיף להתקין ביד? Agent Skill הוא רק תיקייה המכילה `SKILL.md` (בתוספת הפניות אופציונליות), אז הועתקה עובדת גם: - -- **Claude Code**: הצב את תיקייה `agenteye-evaluator/` ב-`~/.claude/skills/` (כל פרויקט) או `/.claude/skills/` (הריפו הזה בלבד). Claude Code גילוי אוטומטי שלה — אימות עם רשימה `/skills`, או פשוט שאל אותו evals. -- **Codex (OpenAI)**: Codex קורא אותו `SKILL.md`. ה-`agents/openai.yaml` המצורף מגדיר `allow_implicit_invocation: true`, אז Codex בחירה אוטומטית בכישרון כאשר משימה תואמת; אחרת הזמן זאת בפירוש כ-`$agenteye-evaluator`. - ---- - -## ה-SDK לא ב-PyPI הציבורי - -> **Warning:** קרא את זה לפני שאתה מעביר סוכן להתקין את ה-SDK. - -הכישרון ציבורי; ה-SDK שהוא מנהל אינו. `agenteye-evaluator` משודר רק כיצירה ריליז פרטית, ובשונה מ-`agenteye`, השם הוא **לא תבועה על PyPI ציבורי** — אז `pip install agenteye-evaluator` חשוף יכול לשוך חבילה של זר לתוך השירות שקורא את התמלילים של הייצור שלך. זה בעיה של שרשרת אספקה, לא טעות הקלדה. - -הכישרון יודע זאת ועוסק בסולם התקנה במקום, עוצר בשלב הראשון החל: ה-monorepo מקור אם אתה בתוך ריפו AgentEye, אחרת גלגל ריליז פרטי מ-GitHub Releases (צריך גישה), ואם כלום לא נגיע זה **עוצר ואומר לך לשאול את איש הקשר Failproof AI שלך לגלגל** במקום improvising. - -אז אם הסוכן שלך מציע `pip install agenteye-evaluator` חשוף מ-PyPI ציבורי, זה הדבר שהכישרון לא היה נטען. עצור שם בדוק שהוא מותקן. - ---- - -## מה אתה יכול לשאול אותו - -סיבוב טיול אמיתי מתחיל בשאלה עמומה ומסתיים בעיצוב שחתום, לא עם קוד: - -```text -you ▸ I want evals for our support bot. I think it's sometimes bad. - -agent ▸ Before I write anything: describe a run that went well, - and one that went badly. - -you ▸ Good: it answers from the docs. Bad: it makes stuff up, - or gives up and says "contact support". - -agent ▸ Running: agenteye --json sessions --status error --since 7d - Reading 5 sessions, including two you'd call bad. - - Your words say factuality and deflection. The transcripts - agree on deflection (2 of 5 end with a support hand-off, - and it's plain in model_response.content), but factuality - needs a ground truth your events don't carry. - - So I'd score three things: - - deflection rules, cheap, separates your two runs - - tool_efficiency rules, catches the retry loop in run-004 - - groundedness LLM judge: is every claim in the docs it retrieved? - - "factuality" would need ground truth we don't have; groundedness - is the closest thing your transcripts can actually support. - OK with those three names? Renaming later splits the history. - -you ▸ yes - -agent ▸ No evaluator in this repo. Should I scaffold one here, or do - you have one elsewhere? -``` - -משם זה כותב את המימדים המבוססים על כללים קודם (חינם, מיידי, דטרמיניסטי), בודק אותם כנגד סשן שלכד אמיתי כולל הריקים והעולם לא בסוף שמתרסקים naive evaluators, ורק מגיע לשופט LLM בממד הסובייקטיבי. זה יודע את [מגבלות ה-dispatcher](/he/agenteye/evaluation-suite#configuring-the-server) — timeout בקשה 30 שניות ו-8 שיחות בו-זמנית פריסה-רחבה — אז אם השופט לא יתאים בהצלחה, זה הולך async עם `JobPending` במקום להפוך את השופט שלך לחצוי וחזור חמש פעמים בחמש פעמים העלות. - -אז זה פורס, מגדיר את שני משתני סביבת השרת, ומאשר עם `agenteye --json evals --session-id ` ש-scores בעצם נוחתו. ניקודים נחתו הוא ההוכחה היחידה. - ---- - -## מה להשגיח על - -- **שמות מימדים קרובים לקבוע.** מפתחות ניקוד הם מחרוזות שרירותיות והפלטפורמה עולה כל דבר שאתה שולח, מה שאומר שום דבר במורד הזרם מתקן בחירה רעה. שנה קורא ובמימדים נפרדים: סשנים ישנים שמור המפתח הישן והטרנד שבר. זו הסיבה שהכישרון מקבל חתימה מוגדרת לפני קוד כתיבה — קח את ההנחיה ברצינות. -- **Fixtures הם תמלילי ייצור אמיתיים.** עיצוב כנגד סשנים אמיתיים אומר שוך אותם לדיסק, והם יכולים להכיל נתוני לקוח. הכישרון שואל לפני התחייב שלהם לגיט; אם בספק, שמור `fixtures/` מחוץ לריפו ויש כל מפתח לשוך שלהם שלהם. -- **הסוכן כותב ופורס שירות שקורא כל תמליל.** זה עובד כמוך, מחובר לפי ההרשאות של כניסת ה-CLI שלך, אבל סקור את ה-evaluator כמו כל קוד אחר שנוגע לנתוני ייצור. - ---- - -## הצעדים הבאים - -- **[Evaluation suite](/he/agenteye/evaluation-suite)**: החוזה HTTP, ה-SDK, ומשתני סביבת השרת שהכישרון מגדיר. -- **[Evaluations](/he/agenteye/evaluations)**: איפה הניקודים מופיעים ברגע שהם נוחתו. -- **[CLI skill](/he/agenteye/cli-skill)**: הכישרון האחות, לקריאת תוצאות במקום בנייה של המדרג. -- **[CLI](/he/agenteye/cli)**: הנושא הפקודה מאחורי נתוני הסשן שהכישרון עיצוב כנגדו. \ No newline at end of file diff --git a/docs/he/agenteye/hermes-capture.mdx b/docs/he/agenteye/hermes-capture.mdx deleted file mode 100644 index b67d0c6d..00000000 --- a/docs/he/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,54 +0,0 @@ ---- ---- -title: "Hermes session capture" -description: "הביאו את ישיבות Hermes gateway של הצוות שלכם — Slack, Telegram, CLI, והרצות מתוזמנות — ל-AgentEye כישיבות ואירועים רגילים." ---- - -[Hermes](https://hermes-agent.nousresearch.com) עונה לצוות שלכם מכל מקום שבו הם כבר עובדים — Slack, Telegram, ה-CLI, הרצות מתוזמנות. Hermes session capture מביא הכל ל-AgentEye כישיבות ואירועים רגילים, כך שהעוזר שהצוות מדבר איתו כל יום ניתן להצפה בדיוק כמו ה-agents שאתם כותבים בעצמכם. - -אספן רקע קטן קורא את חנות הישיבות המקומית של Hermes כשהיא נכתבת ומשדר ישיבות ל-AgentEye. זה עובד באותו אופן כמו [Codex](/he/agenteye/codex-capture) ו-[OpenClaw](/he/agenteye/openclaw-capture) capture, ואספן אחד יכול ללכוד כמה בו זמנית. - ---- - -## מה זה תופס - -כל ישיבת Hermes במכונה תופסת, באיזה ערוץ שהיא הגיעה. כל אחת הופכת ל-[session](/he/agenteye/sessions) ב-AgentEye; הודעות המשתמש והעוזר שלה, קריאות כלים ותוצאות כלים הופכות ל-[events](/he/agenteye/event-stream) התואמים. - -הערוץ שממנו התחילה ישיבה — Slack, Telegram, CLI, או הרצה מתוזמנת — מתועד בישיבה, כך שאתה יכול להבחין בהם ולסנן לאחד בכל פעם. לצידו מגיע המודל שעליו רצה הישיבה, הצ'אט והאדם שממנו הוא הוקם, ובכל פעם שישיבה יצרה שנייה, הקישור חזרה להורה שלה. - -ישיבות מופיעות ברגע שב-Hermes הם מתחילים אותם, בין אם משהו נאמר או לא, וההשגה של תור וקריאות הכלים שלו נשארות בסדר שבו הן בעצם קרו. כשישיבה מסתיימת אתה גם מקבל למה היא הסתיימה, מה היא עלתה, וכמה tokenים היא השתמשה. - ---- - -## הפעלתו - -ה-capture כבוי עד שאתה מפעיל אותו. התקן את האספן עם מפתח API שיש לו את ההרשאה `events:add` (ראה [API keys](/he/agenteye/api-keys)), והפעל Hermes capture: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -זה מתקין את האספן, רושם אותו כשירות רקע, ומתחיל ללכוד. אשר שהוא פועל: - -```bash -agenteye-collector health -``` - -תופסים יותר מ-agent אחד באותה מכונה? הוסף את הדגל של כל אחד לאותה הפקודה — למשל `--hermes-enabled --codex-enabled`. - -בהרצה הראשונה, הישיבות Hermes הקיימות שלך מלאות חזרה פעם אחת והפעילות החדשה אז נשדרת תוך שניות. נתוני Hermes שלהם נקראים בלבד — לעולם לא שונו או נמחקו — וכל הודעה משודרת פעם אחת, גם על פני הפעלות מחדש. - -`health` גם אומר לך אם הכל שהאספן תפס בעצם הגיע ל-AgentEye. אם קבוצה לא יכלה להיות מסופקת היא נשמרת ובוחנת שוב במקום להיהנות, והבדיקה מדווחת בריאה אם משהו עדיין בהמתנה — כך "בריא" פירושו שהנתונים שלך הגיעו, לא רק שהתהליך קיים. - ---- - -## היכן זה מופיע - -ישיבות שנתפסו מופיעות ב-**Sessions**, והאירועים שלהם בזרם **Events**, בדיוק כמו כל agent אחר שאתה צופה בו — כך [session replay](/he/agenteye/sessions), [search](/he/agenteye/queries), [evaluations](/he/agenteye/evaluations), ו-[alerts](/he/agenteye/alerts) כולם עובדים עליהם. סנן לפי ה-Hermes agent כדי לראות אותם בעצמם. - ---- - -## פרטיות - -ישיבות Hermes מכילות את השיחה המלאה — כולל פלט פקודה, תכנים של קבצים, וכל דבר שהעוזר קרא או כתב — ויכולות להכיל סודות. ישיבות שנתפסו משודרות כמו שהן, כך להפעיל capture רק במקום שבו ריכוז תכנים זה AgentEye הוא מתאים, ותן לאספן מפתח מוגבל ל-`events:add` בלבד. ראה [Security](/he/agenteye/security) לאופן שבו הנתונים שלך נשמרים בידוד. \ No newline at end of file diff --git a/docs/he/agenteye/observability.mdx b/docs/he/agenteye/observability.mdx deleted file mode 100644 index 74bf1c8a..00000000 --- a/docs/he/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "צפייה" -description: "משטחי הצפייה הם המקום שבו אתה רואה מה האג'נטים שלך עושים כרגע וחוקר כל ריצה בודדת." ---- - - -משטחי הצפייה הם המקום שבו אתה רואה מה האג'נטים שלך עושים כרגע וחוקר כל ריצה בודדת. הכל כאן הוא בזמן אמת, מסוגנן לארגון שלך, וניתן לסינון לפי טווח תאריכים, סביבה, אג'נט וסשן, כך שאתה עובר מ"משהו לא בסדר" להריצה המדויקת תוך שניות. - -![ה-Event Stream בזמן אמת, מעוצב בצבעים לפי סוג וניתן לסינון לפי סביבה, אג'נט וסשן](/agenteye/images/events-stream.png) - -ארבעה משטחים, כל אחד עם הדף שלו: - -- **[זרם אירועים](/he/agenteye/event-stream)**: שביל בזמן אמת, לפי שלב, של כל ריצה בכל אג'נט, החדש ביותר ראשון. בית הארגון שלך והתחנה הראשונה לטריאז'. -- **[סשנים וגרף ביצוע](/he/agenteye/sessions)**: אירועים אלה מתוקבצים לשורה אחת לכל ריצה, בתוספת תמונה בסגנון git של איך כל ריצה התגלגלה. -- **[מטריקות ביצועים](/he/agenteye/telemetry)**: מפות חום של שהיות וקריטיקלים p50/p95/p99 עבור המודלים, הכלים והוקים שלך, כך שנקודה בחלק העליון בולטת מהחציון. -- **[עקבוי שגיאות](/he/agenteye/error-tracking)**: משטח טריאז' יחיד לכל מה שהשתבש, קליק אחד מהתראה שנשלחה לריצה שקרסה. - -## קשור - -- [הערכות](/he/agenteye/evaluations): דרג כל ריצה על איכות. -- [התראות](/he/agenteye/alerts): הפוך כל סף לכלל דיוור. -- [ביקורות](/he/agenteye/audits): תן ל-Failproof AI Observability למצוא דפוסי כשל בסשנים בשבילך. -- [CLI ואג'נטים](/he/agenteye/cli-and-agents): אותה צפיפות מהמסוף שלך. \ No newline at end of file diff --git a/docs/he/agenteye/openclaw-capture.mdx b/docs/he/agenteye/openclaw-capture.mdx deleted file mode 100644 index 057c0616..00000000 --- a/docs/he/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,50 +0,0 @@ ---- ---- -title: "תיעוד הפגישות של OpenClaw" -description: "עקוב אחרי פגישות OpenClaw המקומיות של הצוות שלך ב-AgentEye כפגישות ואירועים רגילים — ללא שום שינוי בדרך שבה OpenClaw פועל." ---- - -אם הצוות שלך מריץ [OpenClaw](https://docs.openclaw.ai), תיעוד הפגישות של OpenClaw מביא את הפגישות האלה ל-AgentEye כפגישות ואירועים רגילים, כך שאתה יכול לחפש, להשמיע שוב, והערכה שלהם לצד כל שאר מה שאתה צופה. זה משלים את [Python SDK](/he/agenteye/python-sdk): ה-SDK מתחקה אחרי agents שאתה כותב, בעוד שזה תוקף את עבודת OpenClaw שהצוות שלך כבר עושה — ללא שום שינוי בדרך שהם מריצים אותה. - -אספן רקע קטן קורא את תמלול הפגישות המקומיות של OpenClaw כשהם נכתבים ושולח אותם ל-AgentEye. זה עובד בדיוק באותו אופן כמו [Codex capture](/he/agenteye/codex-capture), ואספן אחד יכול ללכוד גם את שניהם בו-זמנית. - ---- - -## מה זה תוקף - -כל agent שהוגדר בהגדרת OpenClaw של מכונה מוקלט על ידי אספן המכונה של אותה מכונה — אין כל הגדרה לכל agent. - -כל פגישת OpenClaw הופכת ל-[session](/he/agenteye/sessions) של AgentEye; ההודעות שלה של המשתמש והעוזר, קריאות הכלים, ותוצאות הכלים הופכות ל-[events](/he/agenteye/event-stream) המתאימים. - ---- - -## הפעלה - -התיעוד כבוי עד שתפעיל אותו. התקן את האספן עם מפתח API שיש לו הרשאה `events:add` (ראה [API keys](/he/agenteye/api-keys)), והפעל את תיעוד OpenClaw: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -זה מתקין את האספן, משלם אותו כשירות רקע, ומתחיל לתעד. אשר שהוא פועל: - -```bash -agenteye-collector health -``` - -תיעוד של יותר מ-agent אחד באותה מכונה? הוסף את הדגל של כל אחד לאותה פקודה — לדוגמה `--openclaw-enabled --codex-enabled`. - -בהרצה הראשונה, הפגישות הקיימות של OpenClaw שלך משמשות כמילוי פעם אחת ופעילות חדשה זורמת לאחר מכן תוך שניות. קבצים של OpenClaw קוראים בלבד — לעולם לא משונים, מועברים, או מחוקים — וכל פגישה משלוחה בדיוק פעם אחת, גם על פני הפעלות מחדש. - ---- - -## היכן זה מופיע - -פגישות שתועדו מופיעות ב-**Sessions**, והאירועים שלהן בזרם **Events**, בדיוק כמו כל agent אחר שאתה צופה — כך [session replay](/he/agenteye/sessions), [search](/he/agenteye/queries), [evaluations](/he/agenteye/evaluations), ו-[alerts](/he/agenteye/alerts) כולם עובדים עליהם. סנן לפי ה-agent של OpenClaw כדי לראות אותם בעצמם. - ---- - -## פרטיות - -תמלול של OpenClaw מכיל את הפגישה המלאה — כולל פלט פקודה, תוכן קבצים, וכל דבר שה-agent קרא או כתב — ויכול להכיל סודות. פגישות שתועדו משלוחות כשהן, אז הפעל תיעוד רק על מכונות ועבור צוותים שבהם ריכוז התוכן הזה ב-AgentEye מתאים, ותן לאספן מפתח שמתוחם ל-`events:add` בלבד. ראה [Security](/he/agenteye/security) כדי להבין כיצד הנתונים שלך מובדלים. \ No newline at end of file diff --git a/docs/he/agenteye/python-sdk-skill.mdx b/docs/he/agenteye/python-sdk-skill.mdx deleted file mode 100644 index 52732c36..00000000 --- a/docs/he/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,132 +0,0 @@ ---- ---- -title: "Failproof AI Observability Python SDK Agent Skill" -description: "מעבר מסוכן שלא מכיל instrumentationליוצרי אירועים שבהם אתה יכול לראות, כאשר סוכן הקידוד שלך מוצא את נקודות ה-instrumentation, כותב אותן, ומוכיח שהן הגיעו." ---- - -אמור לסוכן הקידוד שלך *"הוסף Failproof AI Observability לסוכן זה"* וברשתך לקרוא את הלולאה שלך, להבין לאן ה-instrumentation צריך להישתייך, לכתוב אותו, ולאמת את האירועים לפני שהוא משלים את העבודה. - -ה-**Python SDK skill** (`agenteye-python-sdk`) הוא *Agent Skill*: תיקייה של הוראות שסוכן קידוד כמו Claude Code או Codex טוען לפי דרישה כאשר משימה תואמת אותו. הוא מלמד את הסוכן להשתמש ב-[Python SDK](/he/agenteye/python-sdk) — זה לא ספרייה, והוא לא משנה שום דבר בדרך שה-SDK פועלת. - -## Instrumentation קל לכתיבה וקל להשגיאה בשקט - -ה-SDK קטן: שלוש עשרה שיטות אירועים, כולן keyword-only. סוכן קידוד יכול לקרוא את ה-[Python SDK](/he/agenteye/python-sdk) reference וליצור instrumentation סביר בדקה. - -הבעיה היא שה-SDK הזה לא זורק כשאתה טועה, וinstrumentation שגוי נראה בדיוק כמו instrumentation נכון עד שמישהו פותח דאשבורד ומוצא שהוא ריק. הטעויות שעולות בזמן אמיתי הן כולן שתיקות: - -| הטעות | מה אתה רואה | -|---|---| -| No `agent_start` | כל אירוע מגיע. אפס sessions. | -| Environment לא הוגדר | הכל עובד, מוגדר תחת `dev`. | -| `outcome="failure"` | הריצה מוצגת בירוק — רק `failed`, `error`, `timeout`, `rejected` נחשבים. | -| שם שדה עם typo | מקובל ומאוחסן כשדה חדש. | -| אירועים נפלטים מ-thread pool | מושמטים בשקט. | - -אחד מאלה לא זורק. אחד לא מופיע בבדיקות. כל אחד בטוב בskill, המוצהר כחוזה עם הבדיקה שתופסת אותה. - -## מה הוא עושה, לפי הסדר - -ה-skill מריץ אותם שלושה שלבים שמהנדס זהיר היה עושה: - -1. **Plan.** הוא קורא את לולאת הסוכן שלך ושואל שתי שאלות שרק אתה יכול לענות: מה נחשב לריצה אחת (`session_id` שלך), ומיהם השחקנים הבחינים (`agent_id` שלך). הוא מקבל את ההסכמה לפני כתיבת קוד, כי שינוי אותם מאוחר יותר חותך את ההיסטוריה שלך ושובר את התמיהות. -2. **Write.** הוא קושר זהות פעם אחת לכל ריצה ולא מעבירה דרך כל אתר קריאה, והוא בוחר צורה בטוחה לחוזקות — פרט שחשוב, כי הדרך המקוצרת הברורה מערבבת בשקט שתי ריצות חופפות לסשן אחד. -3. **Verify.** הוא מריץ את הסוכן שלך וקורא את קבצי האירועים שנוצרו, בודק ש-`agent_start` קיים, הסביבה נכונה, וריצה אחת הפיקה סשן אחד. - -השלב השלישי הוא אותו שאנשים מדלגים. ה-SDK כותב אירועים לקבצים מקומיים, כך שintegration שלם יכול להיות מוכח על נייד ללא שרת, ללא API key, וללא רשת — שזה בדיוק למה ה-skill מнастаיває על עשיית זה. - -## איך זה קשור לטכנולוגיות האחרות - -שלוש skills, חלוקה נקייה אחת: - -| Skill | הגע אליו כאשר | מה זה נוגע | -|---|---|---| -| **Python SDK skill** (דף זה) | אתה רוצה שהסוכן שלך *יפלוט* telemetry — "הוסף observability", "למה הסוכן שלי לא מופיע?" | כותב קוד במאגר הסוכן שלך. לא קורא שום דבר. | -| **[Evaluator skill](/he/agenteye/evaluator-skill)** | אתה רוצה *לדרוג* ריצות — "מה כבר צריך למדוד?" | כותב קוד במאגר שלך; קורא telemetry | -| **[CLI skill](/he/agenteye/cli-skill)** | אתה רוצה *לקרוא* מה קרה, או להפעיל את ה-deployment שלך | מנהל את ה-CLI כמוך, כולל שינויים | - -הם עוברים בסדר הזה: skill זה מקבל אירועים לזרימה, המדרג מדרג אותם, ה-CLI קורא אותם חזרה. אין שום דבר להערכה ואין שום דבר לקרוא עד שהסוכן שלך פולט sessions, כך שאם אתה מתחיל מ scratch, התחל כאן. - -## דרישות מקדימות - -1. **Python 3.10+** ובסיס הקוד של הסוכן שאתה רוצה לעבודת את המכשיר. -2. **ה-SDK.** הוא מופץ ללקוחות כ wheel פרטי ולא מאינדקס ציבורי — ה-onboarding שלך מכסה כיצד להשיג אותו ולהתקין אותו. ה-skill יודע את נתיב ההתקנה ויבקש ממך במקום לנחש אם הוא לא יכול למצוא אותו. -3. **כום דבר אחר.** אין כניסה לדאשבורד, אין API key, אין רשת. ה-skill מאמת לעומת קבצי האירועים שה-SDK כותב, כך שהוא יכול לסיים ולהוכיח את עבודתו offline. - -## איפה להשיגו - -ה-skill גר בקולקציה ציבורית [`FailproofAI/skills`](https://github.com/FailproofAI/skills): - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -הוסף `-g` להתקנתו לכל פרויקט במקום רק זה הנוכחי, ו-`--copy` אם הסביבה שלך לא עוקבת אחר symlinks. עבור Codex, העבור `-a codex`. - -## התקנתו ביד - -Agent Skills הן תיקיות המכילות `SKILL.md` בתוספת הפניות. אם אתה מעדיף לא להשתמש בהתקנה: - -- **Claude Code**: העתק את תיקיית `agenteye-python-sdk/` ל-`~/.claude/skills/` (כל פרויקט) או `/.claude/skills/` (רק המאגר הזה). Claude Code מגלה אותה באופן אוטומטי — בדוק את רשימת `/skills`, או פשוט שאל משהו שמתאים אליה. -- **Codex**: Codex קורא את אותו `SKILL.md`. ה-`agents/openai.yaml` המלווה מגדיר `allow_implicit_invocation: true`, כך שהוא נבחר אוטומטי כאשר משימה תואמת; אחרת קרא אליו כ-`$agenteye-python-sdk`. - -הפעל את הסוכן שלך **במאגר המחזיק את הקוד שאתה רוצה לעבודת את המכשיר** — ה-skill קורא את לולאת הסוכן שלך לפני שהוא מציע משהו. - -## איך session נראה - -```text -you ▸ Add Failproof AI Observability to this agent. - -agent ▸ Read your loop — one tool dispatcher in tools.py, one Anthropic - wrapper in llm.py. Two questions before I write anything. - - What counts as one run? You handle requests concurrently, so I'd - use the request id you already have in ctx.request_id — then a - session here and a request in your logs are the same string. - - And who are the actors? I see a planner path and a researcher - path. Label them separately, or is that one agent to you? - -you ▸ Request id is right. They're one agent, call it "assistant". - -agent ▸ Done. Wrapped the dispatcher and the LLM client; agent_start and - agent_end bracket handle_request, including the error path. Used - a context-local for identity — your dispatcher runs tools in a - thread pool, and the simpler approach would have mixed concurrent - runs into one session without complaining. - - Verified against a real run: 11 events across 4 files — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, one session - per request, durations paired. - - One thing I did NOT do: your worker pool in batch.py submits - straight to the executor, so events from there would be dropped. - Want me to fix those too? -``` - -הדפוס להבחנה: הוא קרא את הקוד לפני הצעה, שאל רק את השאלות שאתה יכול לענות, שימש שנית ID שכבר היה לך, בחר את הצורה הבטוחה לחוזקות *כי* ראה thread pool, ו**אימת על ידי קריאת האירועים בפועל** במקום הצהרה על הצלחה — ואז דגל במקום היחיד בו ידע שיכול להיכשל בשקט. - -## מה אתה יכול לבקש ממנו - -- *"Why isn't my agent showing up on the dashboard?"* → הולך בסולם: אם אירועים נכתבים, אם `agent_start` שם, אם הסביבה נכונה, אם הקלט קורא באותו מקום. -- *"Everything's landing under dev."* → הסביבה לא הוגדרה, או אופסה על ידי קריאה מאוחרת יותר. -- *"Add token tracking."* → מוצא את עטיפת ה-LLM שלך ורושם modularizer, stop reason, ו-usage. -- *"Instrument the sub-agents too."* → סשן אחד, תוויות סוכן ברורות, קן תחת הורם. -- *"Write tests for the instrumentation."* → מפנה את ה-SDK לתיקייה זמנית וטוען על האירועים שהוא כתב. - -## מה להביט - -**תן לו לאמת.** השלב שהופך את ה-skill הזה שווה להשתמש בו הוא האחרון — הפעלת הסוכן שלך וקריאת האירועים חזרה. סוכן שכותב instrumentation ועוצר עשה את החצי הקל, וחצי זה נכשל בשקט הוא השני. - -**הסכימו על השמות לפני הקוד.** `session_id` ו-`agent_id` הם הצירים שכל משטח קובץ לפי. שינוי שם להם מאוחר יותר חותך את ההיסטוריה: ריצות ישנות שמרו התוויות הישנות והתמיהות שלך שובקות. ה-skill ישאל; התשובה שווה דקה של מחשבה. - -**אם הסוכן שלך מציע התקנת ה-SDK מאינדקס ציבורי, ה-skill לא טען.** ה-SDK מופץ באופן פרטי. ההצעה הזו היא סימן אמין שסוכן הקידוד שלך מנחש במקום לעקוב אחר ה-skill — עצור אותו שם ובדוק אם ה-skill מותקן. - -מעבר לכך, רדיוס הנפץ שלו קטן: הוא כותב קוד בספריית העבודה שלך וקבצי אירועים שבהם אתה אומר לו. הוא לא קורא שום דבר מה-deployment שלך ולא משנה שום דבר בעולם. - -## שלבים הבאים - -- **[Python SDK](/he/agenteye/python-sdk)**: ה-event reference השלם — כל סוג אירוע ושדה — מאחורי מה ה-skill הזה אוטומטי. -- **[Sessions](/he/agenteye/sessions)**: מה ה-instrumentation שלך מייצר כאשר אירועים מגיעים. -- **[Evaluator Agent Skill](/he/agenteye/evaluator-skill)**: השלב הבא כאשר ריצות מגיעות — ניקודן. -- **[CLI Agent Skill](/he/agenteye/cli-skill)**: קריאת ה-telemetry שלך חזרה. \ No newline at end of file diff --git a/docs/he/agenteye/security.mdx b/docs/he/agenteye/security.mdx deleted file mode 100644 index dee38a73..00000000 --- a/docs/he/agenteye/security.mdx +++ /dev/null @@ -1,67 +0,0 @@ ---- -title: "אבטחה" -description: "Failproof AI Observability בנוי כך שיעמוד קרוב לאגנטים הייצור שלך, מה שאומר שהוא רואה את ההנמקות שלך, קלטי הכלים, והפלטים שלהם." ---- - -Failproof AI Observability בנוי כך שיעמוד קרוב לאגנטים הייצור שלך, מה שאומר שהוא רואה את ההנמקות שלך, קלטי הכלים, והפלטים שלהם. דף זה מסביר כיצד הוא משמר את הנתונים הללו בצורה מבודדת, מבוקרת, וברשותך. אם אתה בתהליך הערכה של Failproof AI Observability לסקירת אבטחה, התחל כאן. - ---- - -## הנתונים שלך נשארים בסביבתך - -Failproof AI Observability הוא self-hosted. אירועים, הנמקות, תגובות מודל, וניתוחים מאוחסנים בבסיסי הנתונים שלך, בסביבתך שלך. שום דבר לא נשלח ל-SaaS של צד שלישי לאחסון, והנתונים שלך נשארים בחשבון הענן שלך. - ---- - -## בידוד דיירים - -מופע אחד של Failproof AI Observability יכול להנחות ארגונים רבים, וכל אחד מבודד בשכבת האחסון — מאופשר על ידי מסד הנתונים, לא רק על ידי ממשק המשתמש: - -- הנתונים התפעוליים של ארגון (משתמשים, מפתחות, לוחות מחוונים, שאילתות שמורות) מוגבלים לארגון זה, וקריאות חוצות-ארגוניות חסומות על ידי מסד הנתונים עצמו. -- כל אירוע שנקלט מוקלד עם הארגון שבעליו, כך שאירועים של ארגון אחד לעולם לא יוכלו להיקרא על ידי ארגון אחר. - -כל נתיב לוח מחוונים מוגבל תחת slug ארגוני (`//…`). - ---- - -## כניסה למערכת - -Failproof AI Observability משתמש בכניסה ללא ססמה, מבוססת דוא״ל. אין ססמה שאפשר לתפוס או לדלוף. משתמש מבקש קוד חד-פעמי (או קישור קסום של לחיצה אחת), שנשלח להם בדוא״ל ותוקפו פוקע במהירות. הכניסה מוגדרת על ידי **רשימת אישור**: רק כתובות דוא״ל (או דומיינים) שאתה מאשר יכולות להתחקות. - -![מסך הכניסה של Failproof AI Observability, המשדר קוד חד-פעמי לדוא״ל שלך](/agenteye/images/login.png) - ---- - -## גישה מוגבלת עם מפתחות API - -כל לקוח מתחקה עם מפתח API שנושא הרשאות דקות, עם עקרון הפחות-הרשאות. קולקטור צריך רק `events:add`; מפתח לוח מחוונים או עוזר יכול להיות קריאה בלבד; פעולות הרסניות (מחיקה, יצירה מחדש) הן הנחות נפרדות שאתה בוחר להכליל. - -![דף מפתחות ה-API: הנחות ההרשאות של כל מפתח, בקודים צבע לפי היקף קריאה, כתיבה, והרסני](/agenteye/images/api-keys.png) - -שמור על מפתח bootstrap המנהל להגדרה, והנפק מפתחות צרים לכל השאר. ראה [מפתחות API](/he/agenteye/api-keys). - ---- - -## עוזר קריאה-בלבד, בשער אישור - -[העוזר בלוח המחוונים](/he/agenteye/assistant) משובץ מענה על שאלות על הנתונים שלך, אך הוא מוגבל בעיצוב: - -- הוא **קריאה-בלבד כברירת מחדל**: SQL שלו עובר דרך שומר שמותר רק `SELECT`/`WITH` שאילתות, הצהרה יחידה, עם מכסה שורות. -- כל דבר שהוא יוצר (שאילתה שמורה, לוח מחוונים) הוא **בשער אישור**: אתה סוקר ומאשר כל כתיבה לפני שזה קורה. -- הוא **לא יכול למחוק לעולם**. - -אז חברה יכולה לשאול "אילו אגנטים השגיאו הכי הרבה השבוע?" ולפעול על פי התשובה, ללא שהעוזר יכול לשנות או להסיר את הנתונים שלך בעצמו. - ---- - -## בדרך - -כל התעבורה עובדת על HTTPS. אתה מסיים TLS עם התעודות שלך, כך שתעבורת קולקטור-לשרת ודפדפן-לשרת מוצפנת בדרך. - ---- - -## הצעדים הבאים - -- [סקירה כללית](/he/agenteye/overview): כיצד Failproof AI Observability מתחברים ביחד. -- [מפתחות API](/he/agenteye/api-keys): הגבל גישה לקולקטור, לוח מחוונים, ועוזר. -- [Observability](/he/agenteye/observability): מה Failproof AI Observability לוקח מהאגנטים שלך. \ No newline at end of file diff --git a/docs/he/cli/audit.mdx b/docs/he/audit.mdx similarity index 100% rename from docs/he/cli/audit.mdx rename to docs/he/audit.mdx diff --git a/docs/he/cli/backfill.mdx b/docs/he/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/he/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/he/cli/config.mdx b/docs/he/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/he/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/he/cli/flush.mdx b/docs/he/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/he/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/he/cli/harness.mdx b/docs/he/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/he/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/he/cli/migrate.mdx b/docs/he/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/he/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/he/cli/uninstall.mdx b/docs/he/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/he/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/he/cli/update.mdx b/docs/he/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/he/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/he/agenteye/api-keys.mdx b/docs/he/cloud/access.mdx similarity index 95% rename from docs/he/agenteye/api-keys.mdx rename to docs/he/cloud/access.mdx index 3326c725..10207ab9 100644 --- a/docs/he/agenteye/api-keys.mdx +++ b/docs/he/cloud/access.mdx @@ -1,9 +1,9 @@ --- title: "מפתחות API" -description: "מפתחות API שולטים על מי ומה יכול להגיע לשרת Failproof AI Observability שלך, כך שקולקטור יכול לשלוח אירועים מבלי להשיג אי פעם הרשאות קריאה או admin." +description: "מפתחות API שולטים על מי ומה יכול להגיע לשרת FailproofAI Cloud שלך, כך שקולקטור יכול לשלוח אירועים מבלי להשיג אי פעם הרשאות קריאה או admin." --- -מפתחות API שולטים על מי ומה יכול להגיע לשרת Failproof AI Observability שלך, כך שקולקטור יכול לשלוח אירועים מבלי להשיג אי פעם הרשאות קריאה או admin. כל מפתח נושא הרשאה אחת או יותר, וכל הרשאה שולטת במסלולי שרת ספציפיים; אתה מעניק רק את אלה שעבודה זקוקה להם. רוב ההפעלות יוצרות רק שלוש סוגי מפתחות. +מפתחות API שולטים על מי ומה יכול להגיע לשרת FailproofAI Cloud שלך, כך שקולקטור יכול לשלוח אירועים מבלי להשיג אי פעם הרשאות קריאה או admin. כל מפתח נושא הרשאה אחת או יותר, וכל הרשאה שולטת במסלולי שרת ספציפיים; אתה מעניק רק את אלה שעבודה זקוקה להם. רוב ההפעלות יוצרות רק שלוש סוגי מפתחות. ## 3 המפתחות שרוב ההפעלות צריכות @@ -81,7 +81,7 @@ description: "מפתחות API שולטים על מי ומה יכול להגיע הרשאות אלה תומכות בעמוד **Users** של ה-dashboard, שם ההיקפים שניתנו של כל חבר מוצגים כ-chips: -![עמוד Users: כרטיס לכל משתמש dashboard עם דוא"ל שלהם, הרשאות שניתנו, ובקרות עריכה/ביטול](/agenteye/images/users.png) +![עמוד Users: כרטיס לכל משתמש dashboard עם דוא"ל שלהם, הרשאות שניתנו, ובקרות עריכה/ביטול](/cloud/images/users.png) ### הגדרות תפעוליות @@ -90,7 +90,7 @@ description: "מפתחות API שולטים על מי ומה יכול להגיע | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | צפה בהגדרות תפעוליות המנוהלות ב-dashboard ובמטה-דטה שלהן; רשימת overrides context-window per-model; וסגור את החלון האפקטיבי למודל. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | ערוך הגדרות תפעוליות והוסף, שנה, או הסר per-model context-window overrides. השינויים משפיעים על אירועים חדשים ללא restart של השרת. | -![עמוד Settings: הגדרות תפעוליות המנוהלות ב-dashboard כגון sign-ins מורשים וחיי session/OTP, ניתנים לעריכה ללא restart](/agenteye/images/settings.png) +![עמוד Settings: הגדרות תפעוליות המנוהלות ב-dashboard כגון sign-ins מורשים וחיי session/OTP, ניתנים לעריכה ללא restart](/cloud/images/settings.png) ### alerts ו-incidents @@ -246,7 +246,7 @@ curl -s -X POST http://your-server/keys//regenerate \ כאשר אתה יוצר מפתח מה-dashboard אתה לא מספק את הסוד; ה-dashboard יוצר סוד חזק בשבילך ומציג אותו **פעם אחת** ביצירה. העתק אותו מיד ואחסן אותו בבטחה; הוא לעולם לא מוצג שוב, בדיוק כמו עם regenerate. אתה עדיין יכול לבחור את הרשאות המפתח ישירות, או לזרוע אותם מערכת הרשאות (ראה למטה). -![עמוד API Keys: כרטיס לכל מפתח המציג את שמו, הרשאות שניתנו, וזמן יצירה, עם regenerate ו-disable פעולות; מפתחות מוגנים כמו `admin` מסומנים](/agenteye/images/api-keys.png) +![עמוד API Keys: כרטיס לכל מפתח המציג את שמו, הרשאות שניתנו, וזמן יצירה, עם regenerate ו-disable פעולות; מפתחות מוגנים כמו `admin` מסומנים](/cloud/images/api-keys.png) --- @@ -275,5 +275,5 @@ curl -s -X POST http://your-server/keys//regenerate \ ## צעדים הבאים -- [Python SDK](/he/agenteye/python-sdk): כיצד קוד ה-agent שלך מטפל בהנחה כאשר שולח אירועים. -- [Security](/he/agenteye/security): כיצד sign-in, access control, ו-per-organization data isolation עובדים. \ No newline at end of file +- [Python SDK](/he/cloud/sdk): כיצד קוד ה-agent שלך מטפל בהנחה כאשר שולח אירועים. +- [Security](/he/cloud/security): כיצד sign-in, access control, ו-per-organization data isolation עובדים. \ No newline at end of file diff --git a/docs/he/cloud/agent-skills.mdx b/docs/he/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/he/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/he/agenteye/alerts.mdx b/docs/he/cloud/alerts.mdx similarity index 77% rename from docs/he/agenteye/alerts.mdx rename to docs/he/cloud/alerts.mdx index 7fa63cd2..bef46ab5 100644 --- a/docs/he/agenteye/alerts.mdx +++ b/docs/he/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "גלה ברגע שמשהו חוצה את הגבול שלך, בער --- -גלה ברגע שמשהו חוצה את הגבול שלך, בערוץ שהצוות שלך כבר צופה בו, במקום לשמוע על זה מלקוח. הגדר כלל פעם אחת ו-Failproof AI Observability בודק אותו לפי לוח זמנים, ואז שולח לך התראה בדוא"ל, Slack, webhook, או ישירות בלוח הבקרה. +גלה ברגע שמשהו חוצה את הגבול שלך, בערוץ שהצוות שלך כבר צופה בו, במקום לשמוע על זה מלקוח. הגדר כלל פעם אחת ו-FailproofAI Cloud בודק אותו לפי לוח זמנים, ואז שולח לך התראה בדוא"ל, Slack, webhook, או ישירות בלוח הבקרה. -![עמוד ההתראות: רשת של כרטיסי כללי התראה, כל אחד מציג את ההגדרה שלו, חלון ההערכה, ערוצים, ותג חומרה של מידע, אזהרה או קריטי](/agenteye/images/alerts.png) +![עמוד ההתראות: רשת של כרטיסי כללי התראה, כל אחד מציג את ההגדרה שלו, חלון ההערכה, ערוצים, ותג חומרה של מידע, אזהרה או קריטי](/cloud/images/alerts.png) *כל כלל התראה בהצצה: מה הוא מוקד, בכמה תדירות, לאן זה שולח התראות, ועד כמה זה דחוף.* ## קבל ידיעה על בעיות לפני המשתמשים שלך @@ -22,9 +22,9 @@ description: "גלה ברגע שמשהו חוצה את הגבול שלך, בער ## בנה את הכלל בטופס, לא ב-JSON -אתה מתאר מה "שבור" אומר בטופס, ו-Failproof AI Observability כותב את הכלל הבסיסי בשבילך. מפרט ה-JSON הוא רק מה שהטופס הזה מייצר בעמקי המערכת, כך שאתה יכול לקרוא אותו כדי להבין כלל אבל בדרך כלל לא תקליד אותו. +אתה מתאר מה "שבור" אומר בטופס, ו-FailproofAI Cloud כותב את הכלל הבסיסי בשבילך. מפרט ה-JSON הוא רק מה שהטופס הזה מייצר בעמקי המערכת, כך שאתה יכול לקרוא אותו כדי להבין כלל אבל בדרך כלל לא תקליד אותו. -![טופס ההתראה החדשה: שם ותיאור, כפתור הפעלה, ובוררי הגדרה המציעים סף מטרי, SQL מותאם אישית, ציון הערכה, eval מורכב, ותנאים לכל אירוע](/agenteye/images/alert-new.png) +![טופס ההתראה החדשה: שם ותיאור, כפתור הפעלה, ובוררי הגדרה המציעים סף מטרי, SQL מותאם אישית, ציון הערכה, eval מורכב, ותנאים לכל אירוע](/cloud/images/alert-new.png) *בחר הגדרה והטופס מחליף לשדות הנכונים; שמור כותב את הכלל.* הנתיב הטוב הוא מהיר: תן לו שם, בחר **הגדרה** (מה לצפות בו), קבע **סף וחלון** (כמה רע, על פני כמה זמן), צרף לפחות ערוץ **אחד**, ואז **שמור** ולחץ על **בדיקה** כדי לשלוח התראה סינתטית ולאשר שכל יעד חוברה. בעמקי המערכת זה יוצר spec קטן כמו: @@ -43,7 +43,7 @@ description: "גלה ברגע שמשהו חוצה את הגבול שלך, בער | **Eval מורכב** | מספר בדיקות ציון משולבות עם any, all, או at-least-N logic, כדי לתפוס רגרסיה שמופיעה רק על פני ציונים | | **לכל אירוע** | אירוע תואם יחיד נוחת: agent ספציפי, סוג שגיאה ספציפי, או substring הודעה | -כבר בעיניים על כשל בעמוד ה-[Errors](/he/agenteye/error-tracking)? כל שורה שם יש לה כפתור **+ alert** שפותח את אותו טופס עם מילוי מראש כדי לתפוס את הכשל המדויק הזה שוב, כך שהתקרית שזה עתה טיפלת בה הופכת לאחד שישדר לך בפעם הבאה. +כבר בעיניים על כשל בעמוד ה-[Errors](/he/cloud/errors)? כל שורה שם יש לה כפתור **+ alert** שפותח את אותו טופס עם מילוי מראש כדי לתפוס את הכשל המדויק הזה שוב, כך שהתקרית שזה עתה טיפלת בה הופכת לאחד שישדר לך בפעם הבאה. **איפה למצוא אותו:** התראות נמצאות ב-`//alerts`. יצירה, עריכה, מחיקה, ובדיקת כללים דורשים **`alerts:write`**; `alerts:read` מספיק להסתכלות. בוררי הנמענה מפרטים את חברי הארגון שלך בשם, כך שתוכל להתריע לאדם מבלי להשאיר את הטופס. @@ -53,11 +53,11 @@ description: "גלה ברגע שמשהו חוצה את הגבול שלך, בער ## מה קורה כשהתראה משדרת -הפרה פותחת **תקרית** ומשדרת את הערוצים שלך פעם אחת. משם הצוות שלך מכיר בה, מקצה בעלים, דן בה, ופותר אותה, הכל מול רקורד נקי ומיוחסו. לזרימת העבודה של טריאז 'הזו יש בית משלו: ראה [Incidents](/he/agenteye/incidents). +הפרה פותחת **תקרית** ומשדרת את הערוצים שלך פעם אחת. משם הצוות שלך מכיר בה, מקצה בעלים, דן בה, ופותר אותה, הכל מול רקורד נקי ומיוחסו. לזרימת העבודה של טריאז 'הזו יש בית משלו: ראה [Incidents](/he/cloud/incidents). ## קשור -- [Incidents](/he/agenteye/incidents): עקוב אחר התראה משדרת מפתח לממומנע לנפתר. -- [Error tracking](/he/agenteye/error-tracking): קבץ כשלי agent והעלה אחד להתראה בלחיצה. -- [Dashboards](/he/agenteye/dashboards): צפה בלוחות המשותפים שהספים שאתה משדר עליהם מגיעים מהם. -- [CLI and agents](/he/agenteye/cli-and-agents): צור התראות וack תקריות מהטרמינל שלך, או script אותן ל-CI. \ No newline at end of file +- [Incidents](/he/cloud/incidents): עקוב אחר התראה משדרת מפתח לממומנע לנפתר. +- [Error tracking](/he/cloud/errors): קבץ כשלי agent והעלה אחד להתראה בלחיצה. +- [Dashboards](/he/cloud/dashboards): צפה בלוחות המשותפים שהספים שאתה משדר עליהם מגיעים מהם. +- [CLI and agents](/he/cloud/cli): צור התראות וack תקריות מהטרמינל שלך, או script אותן ל-CI. \ No newline at end of file diff --git a/docs/he/agenteye/assistant.mdx b/docs/he/cloud/assistant.mdx similarity index 84% rename from docs/he/agenteye/assistant.mdx rename to docs/he/cloud/assistant.mdx index bf37cb18..12932c9d 100644 --- a/docs/he/agenteye/assistant.mdx +++ b/docs/he/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "שאל את נתוני הסוכן שלך שאלה באנגלית --- -שאל את נתוני הסוכן שלך שאלה באנגלית פשוטה וקבל תשובה המקושרת ישירות להוכחה. אין SQL לכתוב, אין לוחות מחוונים לדפדף דרכם — עוזר **Failproof AI Observability** הוא הדרך המהירה ביותר לכל אחד בצוות שלך לקבל תשובות על הסוכנים שלך. +שאל את נתוני הסוכן שלך שאלה באנגלית פשוטה וקבל תשובה המקושרת ישירות להוכחה. אין SQL לכתוב, אין לוחות מחוונים לדפדף דרכם — עוזר **FailproofAI Cloud** הוא הדרך המהירה ביותר לכל אחד בצוות שלך לקבל תשובות על הסוכנים שלך. -![עוזר Failproof AI Observability משיב לשאלה באנגלית פשוטה בתוך לוח המחוונים, המציג טבלת Agent Activity חיה, פירוט שימוש בדגם לכל סוכן, ותובנות כתובות, עם השאילתות שהוא הריץ המוצגות בשורה](/agenteye/images/assistant.png) +![עוזר FailproofAI Cloud משיב לשאלה באנגלית פשוטה בתוך לוח המחוונים, המציג טבלת Agent Activity חיה, פירוט שימוש בדגם לכל סוכן, ותובנות כתובות, עם השאילתות שהוא הריץ המוצגות בשורה](/cloud/images/assistant.png) *שאל באנגלית פשוטה וקבל תשובה שנבנתה מנתונים משלך. כאן הוא מפרק אילו סוכנים עסוקים ביותר ואילו דגמים הם משתמשים בהם, ומציג את השאילתות שהוא הריץ כדי שתוכל לאמת כל מספר.* אין מה ללמוד. פתח את הצ'אט, הקלד מה שאתה רוצה לדעת, וקבע את הקישורים שהוא מחזיר: @@ -36,7 +36,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a בעמוד **Queries** הוא הולך צעד קדימה הופך ללוחור SQL: תאר את השאילתה שאתה רוצה ("הצג שיעור שגיאה לפי סוכן במשך 7 הימים האחרונים") והוא זורם SQL ישר לעורך, ופוצה תצוגת diff כדי שתוכל **Accept** או **Reject** את השינוי לפני שהוא נוחת. -![עמוד Observability Queries ועורך SQL שלו](/agenteye/images/query-lab.png) +![עמוד FailproofAI Cloud Queries ועורך SQL שלו](/cloud/images/query-lab.png) *עמוד Queries: עורך זה הוא המקום שבו העוזר זורם רק לקריאה שאילתה בדעת לך לקבל או לדחות.* לשם SQL על ידי שאילה כאן משתמש בהרשאה `queries:run`, אותה שלידה כפתור **Run** של העורך. צ'אט בכל מקום אחר זקוק `agent:use`. @@ -49,7 +49,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a - **כל כתיבה מחכה לך.** שאילתות שמורות ולוחות מחוונים נוצרים רק לאחר לחיצת Approve מפורשת, ואין הגדרה שהופכת את השער הזה. - **זה לעולם לא יכול למחוק שום דבר.** אין כלי מחיקה חשוף ללעוזר אין הרשאת מחיקה. מחיקות נשארות בידיך, בלוח המחוונים. - **זה נשאר בתוך הארגון שלך.** העוזר רואה רק את הארגון שאתה צופה כרגע. -- **השאלות שלך נשארות שלך.** הנושאים והתשובות חיים בנתוני Observability שלך; רק ניתוחי המוצר מתעדים מטא -דטה שימוש, לעולם לא טקסט הנושא שלך. +- **השאלות שלך נשארות שלך.** הנושאים והתשובות חיים בנתוני FailproofAI Cloud שלך; רק ניתוחי המוצר מתעדים מטא -דטה שימוש, לעולם לא טקסט הנושא שלך. ## איפה למצוא אותו @@ -57,7 +57,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a ## קשור -- [CLI and agents](/he/agenteye/cli-and-agents) -- [Queries](/he/agenteye/queries) -- [Dashboards](/he/agenteye/dashboards) -- [Evaluation suite](/he/agenteye/evaluation-suite) \ No newline at end of file +- [CLI and agents](/he/cloud/cli) +- [Queries](/he/cloud/queries) +- [Dashboards](/he/cloud/dashboards) +- [Evaluation suite](/he/cloud/evaluators) \ No newline at end of file diff --git a/docs/he/agenteye/audits.mdx b/docs/he/cloud/audits.mdx similarity index 82% rename from docs/he/agenteye/audits.mdx rename to docs/he/cloud/audits.mdx index b1af7a32..f3908054 100644 --- a/docs/he/agenteye/audits.mdx +++ b/docs/he/cloud/audits.mdx @@ -1,9 +1,9 @@ --- title: "審査: מנתח אמינות אוטומטי שלך" -description: "Failproof AI Observability חוקר את הכשלים שלא כתבת עבורם כלל כלל, ומסר לך רשימת עדיפויות מדורגת ומבוססת ראיות של בדיוק מה לתקן." +description: "FailproofAI Cloud חוקר את הכשלים שלא כתבת עבורם כלל כלל, ומסר לך רשימת עדיפויות מדורגת ומבוססת ראיות של בדיוק מה לתקן." --- -Failproof AI Observability חוקר את הכשלים שלא כתבת עבורם כלל כלל, ומסר לך רשימת עדיפויות מדורגת ומבוססת ראיות של בדיוק מה לתקן. זה כמו שיש לך אנליסט שמסרק את הלוגים שלך כל לילה, ואז משאיר את הרשימה הקצרה על השולחן שלך בבוקר. +FailproofAI Cloud חוקר את הכשלים שלא כתבת עבורם כלל כלל, ומסר לך רשימת עדיפויות מדורגת ומבוססת ראיות של בדיוק מה לתקן. זה כמו שיש לך אנליסט שמסרק את הלוגים שלך כל לילה, ואז משאיר את הרשימה הקצרה על השולחן שלך בבוקר.
@@ -11,7 +11,7 @@ Failproof AI Observability חוקר את הכשלים שלא כתבת עבורם *סיור של שתי דקות: מריצה מתוזמנת לתיקון שאתה יכול לפעול לפיו.* -![דף הAudits: עבודות חוזרות שסורקות את ההפעלות שלך לדפוסי כשל, כל אחת עם לוח זמנים והרגישות](/agenteye/images/audits.png) +![דף הAudits: עבודות חוזרות שסורקות את ההפעלות שלך לדפוסי כשל, כל אחת עם לוח זמנים והרגישות](/cloud/images/audits.png) *כל 审查 היא עבודה חוזרת שחוקרת את ההפעלות שלך וכותבת המלצות מדורגות ומבוססות ראיות.* ## הפסק להנחש מה לתקן הבא @@ -47,7 +47,7 @@ Audits חיים בלוח הבקרה ב **`//audits`** (צד לאנלי ## קשורה -- [Alerts](/he/agenteye/alerts): קבל דף בנקודה הן סף שאתה כבר יודע על חצתה. -- [Evaluations](/he/agenteye/evaluations): קלע כל ריצה כך רגרסיות איכות על פני השטח בעצמם. -- [Error tracking](/he/agenteye/error-tracking): קבוצה ועקוב אחר השגיאות agents שלך לזרוק. -- [Incidents](/he/agenteye/incidents): עקוב אחרי בעיה審查 הופכת עד לתיקון שלה. \ No newline at end of file +- [Alerts](/he/cloud/alerts): קבל דף בנקודה הן סף שאתה כבר יודע על חצתה. +- [Evaluations](/he/cloud/evaluations): קלע כל ריצה כך רגרסיות איכות על פני השטח בעצמם. +- [Error tracking](/he/cloud/errors): קבוצה ועקוב אחר השגיאות agents שלך לזרוק. +- [Incidents](/he/cloud/incidents): עקוב אחרי בעיה審查 הופכת עד לתיקון שלה. \ No newline at end of file diff --git a/docs/he/cloud/capture.mdx b/docs/he/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/he/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/he/agenteye/cli-recipes.mdx b/docs/he/cloud/cli-recipes.mdx similarity index 91% rename from docs/he/agenteye/cli-recipes.mdx rename to docs/he/cloud/cli-recipes.mdx index b78709a1..5ba68390 100644 --- a/docs/he/agenteye/cli-recipes.mdx +++ b/docs/he/cloud/cli-recipes.mdx @@ -4,9 +4,9 @@ description: "דוגמאות query וקומנדות jq שהניתנות להעת --- -משוך נתוני session, event וערכת ערכים (והפעל הערכות מחדש) ישירות מסקריפט או סוכן קוד, עם JSON נקי ב-stdout שמופנה ישירות ל-`jq`. המתכונים האלה משנים נתונים של Failproof AI Observability למשהו שמשתמש בטרמינל או סוכן קוד AI (Claude Code, Cursor) יכול לשאול וליישם אוטומציה, ללא לחיצה דרך ה-dashboard. +משוך נתוני session, event וערכת ערכים (והפעל הערכות מחדש) ישירות מסקריפט או סוכן קוד, עם JSON נקי ב-stdout שמופנה ישירות ל-`jq`. המתכונים האלה משנים נתונים של FailproofAI Cloud למשהו שמשתמש בטרמינל או סוכן קוד AI (Claude Code, Cursor) יכול לשאול וליישם אוטומציה, ללא לחיצה דרך ה-dashboard. -ההוראות למטה מוכנות להעתקה ישירה לממשק הפקודה של Failproof AI Observability (`agenteye`). להתקנה, אימות וקائמת האפשרויות המלאה ראה [CLI](/he/agenteye/cli); הרץ `agenteye -h` או `agenteye -h` לעזרה המובנית. +ההוראות למטה מוכנות להעתקה ישירה לממשק הפקודה של FailproofAI Cloud (`agenteye`). להתקנה, אימות וקائמת האפשרויות המלאה ראה [CLI](/he/cloud/cli); הרץ `agenteye -h` או `agenteye -h` לעזרה המובנית. ## כללים זהב @@ -173,7 +173,7 @@ esac ## שלבים הבאים -- [CLI](/he/agenteye/cli): התקנה, אימות וההתייחסות המלאה לאפשרויות לכל פקודה. -- [CLI agent skill](/he/agenteye/cli-skill): אפס את המתכונים האלה כמו מיומנות שסוכן הקוד שלך יכול לטעון. -- [API keys](/he/agenteye/api-keys): צור ותחום את המפתחות שעם ה-CLI, SDK והאספן מתאמתים. -- [Python SDK](/he/agenteye/python-sdk): שלח events ל-Failproof AI Observability כדי שיהיו נתונים כדי שהמתכונים האלה יכלו לשאול. \ No newline at end of file +- [CLI](/he/cloud/cli): התקנה, אימות וההתייחסות המלאה לאפשרויות לכל פקודה. +- [CLI agent skill](/he/cloud/agent-skills): אפס את המתכונים האלה כמו מיומנות שסוכן הקוד שלך יכול לטעון. +- [API keys](/he/cloud/access): צור ותחום את המפתחות שעם ה-CLI, SDK והאספן מתאמתים. +- [Python SDK](/he/cloud/sdk): שלח events ל-FailproofAI Cloud כדי שיהיו נתונים כדי שהמתכונים האלה יכלו לשאול. \ No newline at end of file diff --git a/docs/he/agenteye/cli.mdx b/docs/he/cloud/cli.mdx similarity index 91% rename from docs/he/agenteye/cli.mdx rename to docs/he/cloud/cli.mdx index 19a792a3..9ff36240 100644 --- a/docs/he/agenteye/cli.mdx +++ b/docs/he/cloud/cli.mdx @@ -1,9 +1,9 @@ --- title: "CLI" -description: "נהל את כל Failproof AI Observability מהטרמינל או מסקריפט: ללא צורך בגלישה בדashboard." +description: "נהל את כל FailproofAI Cloud מהטרמינל או מסקריפט: ללא צורך בגלישה בדashboard." --- -נהל את כל Failproof AI Observability מהטרמינל או מסקריפט: ללא צורך בגלישה בדashboard. ה-CLI של `agenteye` שואל על הנתונים שלך (sessions, event logs, evaluations) וממנהל את הארגון שלך (API keys, users, settings, alerts, incidents, saved queries), אז הפנה אליו כאשר אתה רוצה להוסיף בדיקה אוטומטית, לחבר Observability ל-CI, או להשאיר לagent לבדוק production. כל פקודה תומכת בדגל `--json`, כך שהיא עובדת באותה מידה טובה בשבילך בשורת הפקודה או לagent שמריץ ודורס את התוצאה. +נהל את כל FailproofAI Cloud מהטרמינל או מסקריפט: ללא צורך בגלישה בדashboard. ה-CLI של `agenteye` שואל על הנתונים שלך (sessions, event logs, evaluations) וממנהל את הארגון שלך (API keys, users, settings, alerts, incidents, saved queries), אז הפנה אליו כאשר אתה רוצה להוסיף בדיקה אוטומטית, לחבר FailproofAI Cloud ל-CI, או להשאיר לagent לבדוק production. כל פקודה תומכת בדגל `--json`, כך שהיא עובדת באותה מידה טובה בשבילך בשורת הפקודה או לagent שמריץ ודורס את התוצאה. עם בינארי אחד אתה יכול: @@ -68,7 +68,7 @@ agenteye --version agenteye --help ``` -> **הערה:** ה-Python SDK של Failproof AI Observability משתמש גם בשם ההפצה `agenteye`. התקנת ה-CLI עם `pipx` או `uv tool` (במקום `pip install` לתוך virtualenv משותף) מונעת התנגשות בין השניים. `pip install agenteye` פשוט בסדר רק אם ה-SDK לא מותקן באותה סביבה. +> **הערה:** ה-Python SDK של FailproofAI Cloud משתמש גם בשם ההפצה `agenteye`. התקנת ה-CLI עם `pipx` או `uv tool` (במקום `pip install` לתוך virtualenv משותף) מונעת התנגשות בין השניים. `pip install agenteye` פשוט בסדר רק אם ה-SDK לא מותקן באותה סביבה. --- @@ -90,7 +90,7 @@ agenteye logout # revoke the session and clear the stored token `whoami` לעולם לא נכשל בsession חסר או פג; הוא מדווח על `logged_in: false` במקום זאת, כך שסקריפט או agent יכול לבדוק את מצב ה-auth בבטחה (הוא עדיין יכול להיכשל עם non-zero אם לא מוגדר base URL או ה-dashboard לא זמין). -**דרישות:** הדואר שלך חייב להיות מורשה להתחבר לdashboard (שאל את מנהל Failproof AI Observability שלך), וה-dashboard חייב להיות זמין ב-base URL שלו (ראה [Configuration](#configuration)). אם אתה מבקש קוד וכלום לא מגיע, הדואר שלך כנראה עדיין לא מופעל לגישה לdashboard. +**דרישות:** הדואר שלך חייב להיות מורשה להתחבר לdashboard (שאל את מנהל FailproofAI Cloud שלך), וה-dashboard חייב להיות זמין ב-base URL שלו (ראה [Configuration](#configuration)). אם אתה מבקש קוד וכלום לא מגיע, הדואר שלך כנראה עדיין לא מופעל לגישה לdashboard. --- @@ -337,13 +337,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | Authenticated, אבל החשבון שלך חסר את ההרשאה הנדרשת (ההודעה קורא את זה) | | 6 | משאב המבוקש לא היה found (לדוגמה session לא ידוע או incident id) | -אלה עושים את ה-CLI בטוח לsript: coding agent יכול branch על `4` להנושא אותך re-authenticate, או `5` to surface החסרה הרשאה. ראה [CLI recipes לagents](/he/agenteye/cli-recipes) עבור exit-code-handling דפוסים וJSON output צורות. +אלה עושים את ה-CLI בטוח לsript: coding agent יכול branch על `4` להנושא אותך re-authenticate, או `5` to surface החסרה הרשאה. ראה [CLI recipes לagents](/he/cloud/cli-recipes) עבור exit-code-handling דפוסים וJSON output צורות. --- ## הצעדים הבאים -- **[CLI recipes לagents](/he/agenteye/cli-recipes)**: copy-paste query דפוסים, `jq` one-liners, `--fields` הקרנות, exit-code handling, וJSON output צורות, כתוב עבור agents coding driving ה-CLI. -- **[CLI agent skill](/he/agenteye/cli-skill)**: חבילה זה CLI כמו installable Claude Code / Codex *skill* כך agent coding drives Failproof AI Observability מ-plain-English בקשות. -- **[API keys](/he/agenteye/api-keys)**: דגם ההרשאה מאחוריי `keys create --add …`. -- **[AI assistant](/he/agenteye/assistant)**: enabling ה-assistant כי `agent ask` דברים ל. \ No newline at end of file +- **[CLI recipes לagents](/he/cloud/cli-recipes)**: copy-paste query דפוסים, `jq` one-liners, `--fields` הקרנות, exit-code handling, וJSON output צורות, כתוב עבור agents coding driving ה-CLI. +- **[CLI agent skill](/he/cloud/agent-skills)**: חבילה זה CLI כמו installable Claude Code / Codex *skill* כך agent coding drives FailproofAI Cloud מ-plain-English בקשות. +- **[API keys](/he/cloud/access)**: דגם ההרשאה מאחוריי `keys create --add …`. +- **[AI assistant](/he/cloud/assistant)**: enabling ה-assistant כי `agent ask` דברים ל. \ No newline at end of file diff --git a/docs/he/cloud/connect.mdx b/docs/he/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/he/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/he/agenteye/dashboards.mdx b/docs/he/cloud/dashboards.mdx similarity index 73% rename from docs/he/agenteye/dashboards.mdx rename to docs/he/cloud/dashboards.mdx index 3870ea5b..4d1b9558 100644 --- a/docs/he/agenteye/dashboards.mdx +++ b/docs/he/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "הפוך את נתוני הסוכן הלייב שלך לתמונה הפוך את נתוני הסוכן הלייב שלך לתמונה משותפת אחת שכל הצוות שלך משקיף עליה. הצמד את השאילתות החשובות ביותר כגרפים, וכולם יפתחו את אותם מספרים במבט אחד, ללא הרצה חוזרת של שאילתה אחת. -![לוח בקרה הבנוי משאילתות שמורות: קו אירועים לשעה, עמודות שגיאות לפי סוג, גרף שטח של השהיה, ואסימונים לפי מודל](/agenteye/images/dashboard-fleet.png) +![לוח בקרה הבנוי משאילתות שמורות: קו אירועים לשעה, עמודות שגיאות לפי סוג, גרף שטח של השהיה, ואסימונים לפי מודל](/cloud/images/dashboard-fleet.png) *לוח אחד, ארבע שאילתות שמורות: אירועים לשעה, שגיאות לפי סוג, השהיה, ואסימונים לפי מודל.* @@ -25,15 +25,15 @@ description: "הפוך את נתוני הסוכן הלייב שלך לתמונה ## הצמד את השאילתות שכבר שמרת -כל אריח מתחיל כשאילתה שמורה. בנה ושמור את השאילתה שחשובה לך בספריית [Queries](/he/agenteye/queries) (הגדרות מוגדרות מראש בנוסף לשלך, על האירועים וההערכות שלך), ואז הצמד אותה ללוח בקרה כגרף המתאים לנתונים: **שורה** לטרנדים לאורך זמן, **עמודות** להשוואת קטגוריות, **שטח** לנפח, או **עוגה** לפירוט חלקים. +כל אריח מתחיל כשאילתה שמורה. בנה ושמור את השאילתה שחשובה לך בספריית [Queries](/he/cloud/queries) (הגדרות מוגדרות מראש בנוסף לשלך, על האירועים וההערכות שלך), ואז הצמד אותה ללוח בקרה כגרף המתאים לנתונים: **שורה** לטרנדים לאורך זמן, **עמודות** להשוואת קטגוריות, **שטח** לנפח, או **עוגה** לפירוט חלקים. מכיוון שאריח הוא פשוט השאילתה השמורה שלך המוצגת כגרף, אין כלום שצריך להסנכרן ביד. עדכן את השאילתה פעם אחת וכל לוח בקרה שמשתמש בה יתעדכן גם כן. ## צפה באיכות, לא רק בנפח -נפח אומר לך שהסוכנים עסוקים. איכות אומרת לך שהם באמת עושים את העבודה. כוונן לוח בקרה ל[ניקוד ההערכות](/he/agenteye/evaluations) שלך ותקבל לוח שעוקב אחרי עד כמה טוב הרצות מתנהלות לאורך זמן, כך שרגרסיה באיכות תופיע כטבילה בגרף במקום הפתעה מלקוח. +נפח אומר לך שהסוכנים עסוקים. איכות אומרת לך שהם באמת עושים את העבודה. כוונן לוח בקרה ל[ניקוד ההערכות](/he/cloud/evaluations) שלך ותקבל לוח שעוקב אחרי עד כמה טוב הרצות מתנהלות לאורך זמן, כך שרגרסיה באיכות תופיע כטבילה בגרף במקום הפתעה מלקוח. -![לוח בקרה ממוקד איכות הבנוי משאילתות הערכה שמורות](/agenteye/images/dashboard-quality.png) +![לוח בקרה ממוקד איכות הבנוי משאילתות הערכה שמורות](/cloud/images/dashboard-quality.png) *לוח איכות שומר את ניקוד ההערכות שלך בחזית, ממש לצד המספרים התפעוליים.* @@ -41,6 +41,6 @@ description: "הפוך את נתוני הסוכן הלייב שלך לתמונה ## קשור -- [Queries](/he/agenteye/queries): בנה ושמור את השאילתות שהופכות לאריחים שלך. -- [Evaluations](/he/agenteye/evaluations): דרג את ההרצות שלך כך שתוכל לתרשים איכות לאורך זמן. -- [Alerts](/he/agenteye/alerts): הפוך סף בכל אחד מהמדדים הללו לעמוד. \ No newline at end of file +- [Queries](/he/cloud/queries): בנה ושמור את השאילתות שהופכות לאריחים שלך. +- [Evaluations](/he/cloud/evaluations): דרג את ההרצות שלך כך שתוכל לתרשים איכות לאורך זמן. +- [Alerts](/he/cloud/alerts): הפוך סף בכל אחד מהמדדים הללו לעמוד. \ No newline at end of file diff --git a/docs/he/agenteye/error-tracking.mdx b/docs/he/cloud/errors.mdx similarity index 67% rename from docs/he/agenteye/error-tracking.mdx rename to docs/he/cloud/errors.mdx index dd68a0fb..0799cf74 100644 --- a/docs/he/agenteye/error-tracking.mdx +++ b/docs/he/cloud/errors.mdx @@ -6,14 +6,14 @@ description: "ראה כל כשל שהסוכנים שלך מייצרים במקו ראה כל כשל שהסוכנים שלך מייצרים במקום אחד, מקובצים כך שפיצוץ רועם נקרא כבעיה אחת. אתה מקבל נתיב בלחיצה אחת מ"משהו אדום" לריצה המדויקת שהשתברה, ללא צורך בגלילה בזרם חי כדי למצוא אותה. -![עמוד השגיאות: היסטוגרמה של כשלים לאורך זמן מעל שורות שגיאה אדומות מקובצות, כל אחת עם כפתור "+התראה" בלחיצה אחת](/agenteye/images/errors.png) +![עמוד השגיאות: היסטוגרמה של כשלים לאורך זמן מעל שורות שגיאה אדומות מקובצות, כל אחת עם כפתור "+התראה" בלחיצה אחת](/cloud/images/errors.png) *עמוד השגיאות: היסטוגרמה של כשלים לאורך זמן, כשכשלים חוזרים מקופלים לשורה אחת לכל תקרית.* ## כל כשל, כבר אסוף עבורך כאשר סוכן משתבר, לא צריך לגלול בזרם אירועים חי בתקווה לתפוס את השורות האדומות לפני שהן גללו. עמוד **השגיאות** עושה את האיסוף בשבילך. הוא אוסף הכל שלוח המחוונים היה צובע באדום למשטח ניתוח אחד, כך שהדבר הראשון שאתה רואה הוא מה נכשל, לא היכן ללכת לחפש אותו. -וזה תופס יותר מהברורות. לצד אירועי `error` מפורשים, Failproof AI Observability משטח גם את הכשלים השקטים: כל `tool_result`, `hook_completed`, או `agent_end` שהמטען שלו נושא כשל מופיע כאן. כלי שהחזיר שגיאה, או hook שיצא בצורה גרועה, כבר לא מחמק אליך רק מכיוון שלא הטילו חריג חזק. +וזה תופס יותר מהברורות. לצד אירועי `error` מפורשים, FailproofAI Cloud משטח גם את הכשלים השקטים: כל `tool_result`, `hook_completed`, או `agent_end` שהמטען שלו נושא כשל מופיע כאן. כלי שהחזיר שגיאה, או hook שיצא בצורה גרועה, כבר לא מחמק אליך רק מכיוון שלא הטילו חריג חזק. על פני החלק העליון, היסטוגרמה מתווה שגיאות לאורך זמן. מבט אחד אומר לך האם זה זרימה עמוקה קבועה או דוקן שהתחיל לפני כמה דקות, כך שאתה יודע מיד האם להשליך מה שאתה עושה. @@ -23,19 +23,19 @@ description: "ראה כל כשל שהסוכנים שלך מייצרים במקו תלות אחת שבורה יכולה להדליק את אותה שגיאה מאות פעמים בדקה. נותרה גולמית, זו קיר של קווים כמעט זהים שקוברים את הדבר האחד שאתה בעצם צריך לראות. -Failproof AI Observability מקפל כשלים חוזרים השותפים לאותו סשן וסוג שגיאה לשורה אחת. פיצוץ נקרא כתקרית אחת. בסוף אתה סופר בעיות, לא שורות log, והאות שחשובה נשארת על גבי במקום להיות טבולה בנפחה שלה. +FailproofAI Cloud מקפל כשלים חוזרים השותפים לאותו סשן וסוג שגיאה לשורה אחת. פיצוץ נקרא כתקרית אחת. בסוף אתה סופר בעיות, לא שורות log, והאות שחשובה נשארת על גבי במקום להיות טבולה בנפחה שלה. ## מ"משהו אדום" לאירוע המדויק לחץ על כל שורה כדי להנחות ישר בתוך הסשן של הריצה הזו, ממוקם על האירוע המדויק שנכשל. אין העתקת מזהי סשן, אין גלילה כדי לחפש את הרגע שזה השתבר: אתה מגיע לזה, כשגרף הביצוע המלא במבט אחד כך שאתה יכול לראות מה הסוכן עשה בשניות לפני שזה השתבר. -אם יש לך `alerts:write`, כל שורה גם נושאת כפתור **+התראה**. לחץ עליו ו-Observability פותח כלל התראה חדש כבר מלא כדי לתפוס את אותו כשל שוב. התקרית שזה עתה ערכת ניתוח הופכת לזו שמעמודה אותך בפעם הבאה, במקום להפתיע אותך פעמיים. +אם יש לך `alerts:write`, כל שורה גם נושאת כפתור **+התראה**. לחץ עליו ו-FailproofAI Cloud פותח כלל התראה חדש כבר מלא כדי לתפוס את אותו כשל שוב. התקרית שזה עתה ערכת ניתוח הופכת לזו שמעמודה אותך בפעם הבאה, במקום להפתיע אותך פעמיים. **היכן למצוא זה:** עמוד **השגיאות** חי בסעיף הצפייה של לוח המחוונים, ב `//errors`. ## קשור -- [התראות](/he/agenteye/alerts): הפוך כל כשל לכלל עמודה. -- [תקריות](/he/agenteye/incidents): עקוב אחר התראה שנורה מפתיחה לפתרון. -- [סשנים](/he/agenteye/sessions): פתח את הריצה המלאה מאחורי כל שגיאה. -- [ביקורות](/he/agenteye/audits): תן ל-Observability למצוא דפוסי כשל על פני הריצות שלך בשבילך. \ No newline at end of file +- [התראות](/he/cloud/alerts): הפוך כל כשל לכלל עמודה. +- [תקריות](/he/cloud/incidents): עקוב אחר התראה שנורה מפתיחה לפתרון. +- [סשנים](/he/cloud/sessions): פתח את הריצה המלאה מאחורי כל שגיאה. +- [ביקורות](/he/cloud/audits): תן ל-FailproofAI Cloud למצוא דפוסי כשל על פני הריצות שלך בשבילך. \ No newline at end of file diff --git a/docs/he/agenteye/evaluations.mdx b/docs/he/cloud/evaluations.mdx similarity index 66% rename from docs/he/agenteye/evaluations.mdx rename to docs/he/cloud/evaluations.mdx index af3ed0b4..cd108457 100644 --- a/docs/he/agenteye/evaluations.mdx +++ b/docs/he/cloud/evaluations.mdx @@ -4,15 +4,15 @@ description: "בעיות איכות מוצאות אותך כעת, במקום ש --- -בעיות איכות מוצאות אותך כעת, במקום שתשמע עליהן בתלונת משתמש. חבר את שירות ההדירוג שלך פעם אחת ו-Failproof AI Observability מדרג כל הרצה שהושלמה באופן אוטומטי, כך שירידה בעזרתיות או עלייה בהלוצינציות מופיעה מעצמה, לפני שלקוח חש בכך. +בעיות איכות מוצאות אותך כעת, במקום שתשמע עליהן בתלונת משתמש. חבר את שירות ההדירוג שלך פעם אחת ו-FailproofAI Cloud מדרג כל הרצה שהושלמה באופן אוטומטי, כך שירידה בעזרתיות או עלייה בהלוצינציות מופיעה מעצמה, לפני שלקוח חש בכך. -![רשת ההפעלות עם עמודת ניקוד: כל הרצה נושאת תג סטטוס הערכה ותגי עזרתיות, עובדתיות וַיעילות כלים בקודים צבעים](/agenteye/images/sessions-list.png) +![רשת ההפעלות עם עמודת ניקוד: כל הרצה נושאת תג סטטוס הערכה ותגי עזרתיות, עובדתיות וַיעילות כלים בקודים צבעים](/cloud/images/sessions-list.png) *כל הרצה ברשת ההפעלות נושאת את הניקודים שלה; תגים אדומים, כתומים וירוקים הופכים את ההרצות החלשות לבולטות מבלי שתפתח אפילו תמלול אחד.* ## הפסק דגימה ידנית של הרצות -נהגת לבדוק כמה הרצות וקיווית שהשאר בסדר. כעת כל סשן שהושלם מקבל ניקוד ברגע שהוא מסתיים, בממדים שחשובים לך: עזרתיות, יעילות כלים, עובדתיות, בטיחות, כל מה שקובע את רמת האיכות שלך. אתה מגדיר את מפתחות הניקוד; Failproof AI Observability שומר, עוקב אחר מגמות ומציג כל מה שמעריך שלך חוזר חזור. אף הרצה לא מחליקה ללא ניקוד, והתה מפסיק ללמוד על נסיגה מכרטיס תמיכה. +נהגת לבדוק כמה הרצות וקיווית שהשאר בסדר. כעת כל סשן שהושלם מקבל ניקוד ברגע שהוא מסתיים, בממדים שחשובים לך: עזרתיות, יעילות כלים, עובדתיות, בטיחות, כל מה שקובע את רמת האיכות שלך. אתה מגדיר את מפתחות הניקוד; FailproofAI Cloud שומר, עוקב אחר מגמות ומציג כל מה שמעריך שלך חוזר חזור. אף הרצה לא מחליקה ללא ניקוד, והתה מפסיק ללמוד על נסיגה מכרטיס תמיכה. הניקודים נוסעים עם רשת ההפעלות ב-**`//sessions`** (סרגל צד → *observe* → *sessions*), אשכול תגים אחד לכל שורה. רוצה רק את ההרצות שירדו? סנן את הרשת לפי טווח ניקוד, נניח עזרתיות מתחת ל-0.5, וציין בדיוק את ההרצות שכדאי לקרוא. צפייה בניקודים דורשת את ההרשאה `evaluations:read`. @@ -20,7 +20,7 @@ description: "בעיות איכות מוצאות אותך כעת, במקום ש מספר אומר לך שהרצה הייתה חלשה; דף ההפעלה אומר לך למה. פתח כל הרצה והרגל הימני מתחיל עם סיכום הכותרת, ואז מציג עמודה לכל ממד עם הנימוק של המעריך שלך מתחתה, כך שתעבור מ"זה קיבל 0.4 בעובדתיות" לטעות המדויקת בשניות. -![הרגל הימני של הפעלה: סיכום ההערכה בחלקו העליון, ואז עמודות ניקוד לכל ממד כל אחת עם שורת נימוק, לצד ציר הזמן המלא של האירוע](/agenteye/images/session-detail.png) +![הרגל הימני של הפעלה: סיכום ההערכה בחלקו העליון, ואז עמודות ניקוד לכל ממד כל אחת עם שורת נימוק, לצד ציר הזמן המלא של האירוע](/cloud/images/session-detail.png) *תצוגת פרטי ההפעלה: סיכום, עמודות ניקוד לכל ממד, והנימוק מאחורי כל ניקוד, ממש לצד ציר הזמן של האירוע של ההרצה.* @@ -30,7 +30,7 @@ description: "בעיות איכות מוצאות אותך כעת, במקום ש הרצה אחת עם ניקוד נמוך היא רעש; קוהורטה שלמה שמחליקה היא סימן. לוחות בקרה שמורים הופכים את הניקודים שלך למגמה שאתה יכול לצפות בה במבט אחד: עזרתיות ממוצעת השבוע מול השבוע שעבר, לכל סוכן, לכל סביבה. -![לוח בקרה איכות: עמודות ניקוד ממוצע לכל ממד מעריך לצד מגמה לאורך זמן](/agenteye/images/dashboard-quality.png) +![לוח בקרה איכות: עמודות ניקוד ממוצע לכל ממד מעריך לצד מגמה לאורך זמן](/cloud/images/dashboard-quality.png) *לוח בקרה איכות שמור מעקב אחר מפתחות הניקוד שאתה מציג, כך שסחיפה איטית היא ברורה הרבה לפני שהוא הופך לתקרית.* @@ -38,14 +38,14 @@ description: "בעיות איכות מוצאות אותך כעת, במקום ש ## חבר מעריך פעם אחת -ניקוד הוא בחירה וnמשמר כיבוי לחלוטין עד שאתה מצביע את Failproof AI Observability על מתדרג. אתה מקים שירות HTTP קטן אחד (Observability משלח התייחסות עובדת שתוכל להעתיק), קובע שני ערכים בשרת שלך, וכל הרצה מעתה מתדרגת בשבילך. ההדרכה המלאה, חוזה הניקוד, וה-SDK חיים בהנחיה העמוקה. +ניקוד הוא בחירה וnמשמר כיבוי לחלוטין עד שאתה מצביע את FailproofAI Cloud על מתדרג. אתה מקים שירות HTTP קטן אחד (FailproofAI Cloud משלח התייחסות עובדת שתוכל להעתיק), קובע שני ערכים בשרת שלך, וכל הרצה מעתה מתדרגת בשבילך. ההדרכה המלאה, חוזה הניקוד, וה-SDK חיים בהנחיה העמוקה. -לא בטוח איזה ממדים כדאי לדרג בהתחלה? [כישורון סוכן המעריך](/he/agenteye/evaluator-skill) מאפשר לסוכן קידוד שלך לעבוד זאת כנגד ההפעלות שלך, ואז לבנות ולפרוס את השירות. +לא בטוח איזה ממדים כדאי לדרג בהתחלה? [כישורון סוכן המעריך](/he/cloud/agent-skills) מאפשר לסוכן קידוד שלך לעבוד זאת כנגד ההפעלות שלך, ואז לבנות ולפרוס את השירות. ## קשור -- [חבילת הערכה](/he/agenteye/evaluation-suite): חבר את המעריך שלך, חוזה הניקוד, וה-SDK. -- [כישורון סוכן מעריך](/he/agenteye/evaluator-skill): תן לסוכן קידוד לבחור את ממדי הניקוד שלך ובנה את המעריך. -- [הפעלות](/he/agenteye/sessions): רשת ההרצה-אחר-הרצה שבה ניקודים מופיעים. -- [לוחות בקרה](/he/agenteye/dashboards): שמור וחלוק מגמות איכות על פני הארגון שלך. -- [ביקורות](/he/agenteye/audits): תכונת האיכות האוטומטית האחרת של Observability, לחקירות חוצות-הפעלה. \ No newline at end of file +- [חבילת הערכה](/he/cloud/evaluators): חבר את המעריך שלך, חוזה הניקוד, וה-SDK. +- [כישורון סוכן מעריך](/he/cloud/agent-skills): תן לסוכן קידוד לבחור את ממדי הניקוד שלך ובנה את המעריך. +- [הפעלות](/he/cloud/sessions): רשת ההרצה-אחר-הרצה שבה ניקודים מופיעים. +- [לוחות בקרה](/he/cloud/dashboards): שמור וחלוק מגמות איכות על פני הארגון שלך. +- [ביקורות](/he/cloud/audits): תכונת האיכות האוטומטית האחרת של FailproofAI Cloud, לחקירות חוצות-הפעלה. \ No newline at end of file diff --git a/docs/he/agenteye/evaluation-suite.mdx b/docs/he/cloud/evaluators.mdx similarity index 83% rename from docs/he/agenteye/evaluation-suite.mdx rename to docs/he/cloud/evaluators.mdx index 18fe96fb..99b50eb0 100644 --- a/docs/he/agenteye/evaluation-suite.mdx +++ b/docs/he/cloud/evaluators.mdx @@ -1,19 +1,19 @@ --- title: "חבילת הערכה" -description: "Failproof AI Observability יכול לדרג באופן אוטומטי כל הרצה של סוכן שהסתיימה מבחינת איכות: אתה מספק שירות דירוג קטן, ו-Observability מטפל בשאר." +description: "FailproofAI Cloud יכול לדרג באופן אוטומטי כל הרצה של סוכן שהסתיימה מבחינת איכות: אתה מספק שירות דירוג קטן, ו-FailproofAI Cloud מטפל בשאר." --- -Failproof AI Observability יכול לדרג באופן אוטומטי כל הרצה של סוכן שהסתיימה מבחינת איכות: אתה מספק שירות דירוג קטן, ו-Observability מטפל בשאר. השתמש בו כדי לעקוב אחר הממדים שחשובים לך (עזרתיות, יעילות כלים, עובדתיות, בטיחות; אתה בוחר), לתפוס רגרסיות מוקדם, ולהשוות סוכנים או סביבות בהצצה. הדירוג הוא אופציונלי: הצינור לא עושה כלום עד שתגדיר את `EVALUATOR_ENDPOINT` בשרת. +FailproofAI Cloud יכול לדרג באופן אוטומטי כל הרצה של סוכן שהסתיימה מבחינת איכות: אתה מספק שירות דירוג קטן, ו-FailproofAI Cloud מטפל בשאר. השתמש בו כדי לעקוב אחר הממדים שחשובים לך (עזרתיות, יעילות כלים, עובדתיות, בטיחות; אתה בוחר), לתפוס רגרסיות מוקדם, ולהשוות סוכנים או סביבות בהצצה. הדירוג הוא אופציונלי: הצינור לא עושה כלום עד שתגדיר את `EVALUATOR_ENDPOINT` בשרת. -> **הערה:** אתה מגדיר את ממדי הציון. ההערכה שלך יכולה להחזיר כל מפתחות מספריים שהיא רוצה; Observability אחסן, טרנד ומציג כל מה שאתה שולח חזרה. +> **הערה:** אתה מגדיר את ממדי הציון. ההערכה שלך יכולה להחזיר כל מפתחות מספריים שהיא רוצה; FailproofAI Cloud אחסן, טרנד ומציג כל מה שאתה שולח חזרה. ## במבט חטוף -1. **כתוב מדרג.** הקם שירות HTTP קטן שקורא תמליל של סשן ומחזיר ציונים. Observability משלח התייחסות עובדת שאתה יכול להעתיק. ראה [כתיבת מעריך עם ה-SDK](#writing-an-evaluator-with-the-sdk). -2. **הצביע ל-Observability על זה.** קבע את `EVALUATOR_ENDPOINT` (ו-`EVALUATOR_TOKEN` משותף) בתהליך השרת. +1. **כתוב מדרג.** הקם שירות HTTP קטן שקורא תמליל של סשן ומחזיר ציונים. FailproofAI Cloud משלח התייחסות עובדת שאתה יכול להעתיק. ראה [כתיבת מעריך עם ה-SDK](#writing-an-evaluator-with-the-sdk). +2. **הצביע ל-FailproofAI Cloud על זה.** קבע את `EVALUATOR_ENDPOINT` (ו-`EVALUATOR_TOKEN` משותף) בתהליך השרת. 3. **צפה בציונים שנחתו.** כל סשן שהסתיים מדורג באופן אוטומטי; התוצאות מופיעות בעמוד פרטי הסשן, בגריד הסשנים ובלוחות שנשמרו. -![תצוגת פרטי סשן עם סיכום ההערכה, סרגלי ציון לממד, וטקסט נמקות בפס ימני](/agenteye/images/session-detail.png) +![תצוגת פרטי סשן עם סיכום ההערכה, סרגלי ציון לממד, וטקסט נמקות בפס ימני](/cloud/images/session-detail.png) *לאחר הגדרת מעריך, כל הרצה שהושלמה מדורגת והתוצאות מופיעות בפס הימני של הסשן: הסיכום בחלק העליון, ואחריו סרגלי ציון לממד עם נמקות.* @@ -23,7 +23,7 @@ Failproof AI Observability יכול לדרג באופן אוטומטי כל הר ```mermaid flowchart LR - ING["ingest /events
agent_end"] --> SRV["Observability server"] + ING["ingest /events
agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -31,14 +31,14 @@ flowchart LR SRV --> RES["evaluations
terminal results"] ``` -כאשר Failproof AI Observability SDK פולט אירוע `agent_end` לסשן, השרת מתכנן הערכה. לאחר מכן הוא עושה POST של תמליל האירוע המלא לשירות ההערכה שלך, שיכול: +כאשר FailproofAI Cloud SDK פולט אירוע `agent_end` לסשן, השרת מתכנן הערכה. לאחר מכן הוא עושה POST של תמליל האירוע המלא לשירות ההערכה שלך, שיכול: - **להחזיר את התוצאה בשורה** עם `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`. התוצאה מנוספת לציר הזמן של ההערכה של הסשן. `reasoning` ו-`summary` הם אופציונליים. -- **לדחות** עם `{"status":"pending", "job_id":"abc-123"}`. Observability ואז קורא `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` עד שההערכה שלך מחזירה `{"status":"done", ...}` או `{"status":"error", "error":"..."}`. +- **לדחות** עם `{"status":"pending", "job_id":"abc-123"}`. FailproofAI Cloud ואז קורא `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` עד שההערכה שלך מחזירה `{"status":"done", ...}` או `{"status":"error", "error":"..."}`. - קצב הסקר הוא לכל עבודה: תגובת `pending` עשויה לכלול `next_poll_secs` כדי לדרוג; אחרת Observability משתמש בערך `default_poll_interval_secs` מ-`GET /config`; אחרת השרת חוזר אל `EVALUATOR_POLLING_INTERVAL_SECS` (ברירת מחדל 10 שניות). כל הערכים מוגבלים ל-[1 שניה, 1 שעה]. + קצב הסקר הוא לכל עבודה: תגובת `pending` עשויה לכלול `next_poll_secs` כדי לדרוג; אחרת FailproofAI Cloud משתמש בערך `default_poll_interval_secs` מ-`GET /config`; אחרת השרת חוזר אל `EVALUATOR_POLLING_INTERVAL_SECS` (ברירת מחדל 10 שניות). כל הערכים מוגבלים ל-[1 שניה, 1 שעה]. -סשנים שלא פלטו `agent_end` (לדוגמה, תהליך סוכן שהתרסק) יכולים גם להיאסף: `GET /config` של ההערכה עשוי להחזיר `{"inactivity_timeout_secs": 1800}`, וה-Observability יעריך כל סשן שנשמר בחוסר פעילות לפי זמן זה. קבע את השדה ל-`null` או השמיט אותו כדי להשבית את הנופל החלופי. +סשנים שלא פלטו `agent_end` (לדוגמה, תהליך סוכן שהתרסק) יכולים גם להיאסף: `GET /config` של ההערכה עשוי להחזיר `{"inactivity_timeout_secs": 1800}`, וה-FailproofAI Cloud יעריך כל סשן שנשמר בחוסר פעילות לפי זמן זה. קבע את השדה ל-`null` או השמיט אותו כדי להשבית את הנופל החלופי. הצינור הוא כל ל-no-op כאשר `EVALUATOR_ENDPOINT` לא מוגדר. @@ -46,7 +46,7 @@ flowchart LR הנופל החלופי של חוסר פעילות מחדש בסשנים שנעתקו: אם אירועים חדשים מגיעים לאחר הערכה סוף קודמת וסשן ואז הולך ללא פעילות בעבר `inactivity_timeout_secs`, הערכה טרייה מתורה. -כשלים חולפים (5xx, 429, timeouts, שגיאות רשת) מנסים שוב עם backoff אקספוננציאלי עד `EVALUATOR_MAX_ATTEMPTS`; תגובות 4xx הן סופיות. Observability בטוח להריץ עם מספר מקבלות שרת במרובה; העבודה מחולקת כך שאותו סשן לעולם לא יישלח פעמיים במקביל. +כשלים חולפים (5xx, 429, timeouts, שגיאות רשת) מנסים שוב עם backoff אקספוננציאלי עד `EVALUATOR_MAX_ATTEMPTS`; תגובות 4xx הן סופיות. FailproofAI Cloud בטוח להריץ עם מספר מקבלות שרת במרובה; העבודה מחולקת כך שאותו סשן לעולם לא יישלח פעמיים במקביל. --- @@ -54,7 +54,7 @@ flowchart LR כל מסלול מאומת משתמש **ב-Bearer Token Auth**. אותו ערך חייב להיות מוגדר משני הצדדים: -- שרת Observability: משתנה env `EVALUATOR_TOKEN` +- שרת FailproofAI Cloud: משתנה env `EVALUATOR_TOKEN` - שירות Evaluator: מוגדר באותו אופן (ה-SDK `agenteye-evaluator` קורא `EVALUATOR_TOKEN` לפי מוסכמה) אם `EVALUATOR_TOKEN` לא מוגדר, השרת לא שולח כותרת `Authorization`; ההערכה עשויה לקבל בקשות אנונימיות, שזה בסדר לרשת פנימית בלבד אך מודחה באינטרנט הציבורי. @@ -125,7 +125,7 @@ flowchart LR אתה לא חייב ליישם את חוזה HTTP ביד. החבילה Python `agenteye-evaluator` נותנת לך ליפוף FastAPI מוקלד שמטפל בהתאמה, ניתוב וצורות בקשה/תגובה בשבילך. -Failproof AI Observability גם משלח **מעריך התייחסות עובד** שמדרג `helpfulness`, `tool_efficiency` ו-`factuality` מצורת התמליל. העתק אותו כנקודת התחלה וחליף בלוגיקה שלך: שופט LLM, מנוע כללים, כל מה שמתאים לסטנדרט האיכות שלך. +FailproofAI Cloud גם משלח **מעריך התייחסות עובד** שמדרג `helpfulness`, `tool_efficiency` ו-`factuality` מצורת התמליל. העתק אותו כנקודת התחלה וחליף בלוגיקה שלך: שופט LLM, מנוע כללים, כל מה שמתאים לסטנדרט האיכות שלך. מעריך ברור ברירת מחדל: @@ -148,7 +148,7 @@ def run(req: EvalRequest) -> EvalResponse: מופע ה-`app` פועל תחת כל שרת ASGI, כך שתחילת `uvicorn module:app`. -עבור הערכות שצריכות לדחות עבודה יקרה, החזור ב-`JobPending` בעוד רושם `@app.job_lookup` handler; שרת Observability סוקר `GET /evaluate/{job_id}` עד שתחזיר סטטוס סופי או עד שהמכסה `EVALUATOR_MAX_POLL_DURATION_SECS` (ברירת מחדל 1 שעה) חולפת. +עבור הערכות שצריכות לדחות עבודה יקרה, החזור ב-`JobPending` בעוד רושם `@app.job_lookup` handler; שרת FailproofAI Cloud סוקר `GET /evaluate/{job_id}` עד שתחזיר סטטוס סופי או עד שהמכסה `EVALUATOR_MAX_POLL_DURATION_SECS` (ברירת מחדל 1 שעה) חולפת. ה-API reference המלא, דפוס אסינכרוני וסכמת אירועים תועדו ב-README של SDK ה-`agenteye-evaluator`. @@ -156,7 +156,7 @@ def run(req: EvalRequest) -> EvalResponse: ## הרצת המעריך שלך -ההערכה היא **השירות שלך** — Failproof AI Observability לא משלח מעריך ברירת מחדל, כך שאתה בונה והרץ אותו במקום שבו אתה מריץ את השירותים שלך. הוא פועל תחת כל שרת ASGI (לדוגמה `uvicorn my_evaluator:app`); הגיש את נתיבי `/health`, `/config` ו-`/evaluate` מ-[חוזה HTTP](#http-contract), ואז הצביע את השרת אליו (ראה [הגדרת השרת](#configuring-the-server)). +ההערכה היא **השירות שלך** — FailproofAI Cloud לא משלח מעריך ברירת מחדל, כך שאתה בונה והרץ אותו במקום שבו אתה מריץ את השירותים שלך. הוא פועל תחת כל שרת ASGI (לדוגמה `uvicorn my_evaluator:app`); הגיש את נתיבי `/health`, `/config` ו-`/evaluate` מ-[חוזה HTTP](#http-contract), ואז הצביע את השרת אליו (ראה [הגדרת השרת](#configuring-the-server)). ברגע שההערכה ניתנת להשגה, `GET /health` מחזיר `{"status":"ok"}`. לאחר הרצה של סוכן מקצה לקצה, `GET /evaluations` בשרת מחזיר שורה עם `status: "done"` וציונים שההערכה שלך ייצרה. @@ -254,7 +254,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. - **`/sessions`**: גריד סשנים ניתן לסינון; עמודת הציון מציגה את סטטוס ההערכה וציונים של כל סשן בהצצה. - **`/dashboards`**: צפיות בריאות eval שמורה (ראה [לוחות](#dashboards) להלן). -![גריד הסשנים עם כלולי סטטוס הערכה לכל סשן ובתגים מדורגים בצבע (עזרתיות, עובדתיות, tool_efficiency, בטיחות, קוהרנטיות)](/agenteye/images/sessions-list.png) +![גריד הסשנים עם כלולי סטטוס הערכה לכל סשן ובתגים מדורגים בצבע (עזרתיות, עובדתיות, tool_efficiency, בטיחות, קוהרנטיות)](/cloud/images/sessions-list.png) *גריד הסשנים מציג את סטטוס ההערכה וציונים של כל הרצה בהצצה; תגים אדומים/כהים/ירוקים גורמים לציונים נמוכים לקפוץ החוצה.* @@ -271,7 +271,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. כל כרטיס מציג את מספר הסשנים התואמים, פירוט done/error/timeout, ממוצע של כל ציון בתכונה וטרנדלין ספארק קטן. פתיחת לוח מציגה את הפנלים במלוא הגודל; **"פתח בסשנים"** מושיב אותך לעמוד הסשנים מקדים מסונן לאותה פרוסה בדיוק. מדדים מחושבים בצד שרת על פני כל הסט התואם (דרך `GET /evaluations/aggregate`), כך המספרים מדויקים ולא דגומים. -![לוח בריאות eval עם סרגלי ציון ממוצע לממד evaluator, breakdown tool ok-vs-error, כלים למעלה וטרנד events-per-hour](/agenteye/images/dashboard-quality.png) +![לוח בריאות eval עם סרגלי ציון ממוצע לממד evaluator, breakdown tool ok-vs-error, כלים למעלה וטרנד events-per-hour](/cloud/images/dashboard-quality.png) **הרשאות:** צפייה צריכה הן `dashboards:read` והן `evaluations:read`; יצירה ועריכה צריכה `dashboards:write`; מחיקה צריכה `dashboards:delete`. ה-bootstrap admin מקבל את כל אלה באופן אוטומטי. @@ -293,7 +293,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. ## שלבים הבאים -- [מיומנות סוכן Evaluator](/he/agenteye/evaluator-skill): יש לסוכן קידוד עיצוב הממדים שלך כנגד סשנים אמיתיים וביצוע שירות זה בשבילך. -- [Python SDK](/he/agenteye/python-sdk): פלטו את אירועי `agent_end` שמפעילים דירוג. -- [API keys](/he/agenteye/api-keys): הרשאות `evaluations:read` ו-`evaluations:trigger`. -- [Audits](/he/agenteye/audits): תכונת בריאות אוטומטית נוספת של Observability, לבדיקה מבוססת מדיניות. \ No newline at end of file +- [מיומנות סוכן Evaluator](/he/cloud/agent-skills): יש לסוכן קידוד עיצוב הממדים שלך כנגד סשנים אמיתיים וביצוע שירות זה בשבילך. +- [Python SDK](/he/cloud/sdk): פלטו את אירועי `agent_end` שמפעילים דירוג. +- [API keys](/he/cloud/access): הרשאות `evaluations:read` ו-`evaluations:trigger`. +- [Audits](/he/cloud/audits): תכונת בריאות אוטומטית נוספת של FailproofAI Cloud, לבדיקה מבוססת מדיניות. \ No newline at end of file diff --git a/docs/he/agenteye/event-stream.mdx b/docs/he/cloud/event-stream.mdx similarity index 79% rename from docs/he/agenteye/event-stream.mdx rename to docs/he/cloud/event-stream.mdx index 38225d18..77f775f8 100644 --- a/docs/he/agenteye/event-stream.mdx +++ b/docs/he/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "ברגע שהエージェנט שלך עושה משהו, אתה ר ברגע שהエージェนט שלך עושה משהו, אתה רואה את זה. ה-Event Stream הוא הדופק החי שלך על כל agent בייצור: ללא המתנה, ללא חיפוש בלוגים, ללא ניחוש מה זה עתה קרה. -![ה-Event Stream החי: שורות אירוע בצבעים שונים המתעדכנות בזמן אמת, ניתנות לסינון לפי סביבה, agent, session, סוג אירוע וחיפוש חופשי](/agenteye/images/events-stream.png) +![ה-Event Stream החי: שורות אירוע בצבעים שונים המתעדכנות בזמן אמת, ניתנות לסינון לפי סביבה, agent, session, סוג אירוע וחיפוש חופשי](/cloud/images/events-stream.png) *כל אירוע מכל agent בארגון שלך, החדש ביותר קודם, מתעדכן כשזה קורה.* @@ -37,14 +37,14 @@ description: "ברגע שהエージェנט שלך עושה משהו, אתה ר ה-Event Stream הוא בית הארגון שלך. התחברות והוא הראשון בו אתה נוחת, ב-`//`, כך שהטריאז מתחיל ברגע שאתה מגיע. -מאחוריו, ה-agents שלך פולטים אירועים דרך ה-SDK, ה-collector משלח אותם לשרת Failproof AI Observability שלך, והזרם עוקב אחריהם כשהם מגיעים לתשתית שאתה שולט בה. כאשר אתה רוצה את התצוגה המצטברת במקום את השביל הגולמי, האירועים של כל run קורסים לשורה אחת ב-Sessions, קליק אחד משם. +מאחוריו, ה-agents שלך פולטים אירועים דרך ה-SDK, ה-collector משלח אותם לשרת FailproofAI Cloud שלך, והזרם עוקב אחריהם כשהם מגיעים לתשתית שאתה שולט בה. כאשר אתה רוצה את התצוגה המצטברת במקום את השביל הגולמי, האירועים של כל run קורסים לשורה אחת ב-Sessions, קליק אחד משם. זה האמת הגולמית שעליה כל משטח observe אחר בנוי, כך שכאשר מספר נראה לא נכון במקום אחר, הזרם הוא המקום בו אתה מאשר מה שבאמת קרה. ## קשור -- [Sessions](/he/agenteye/sessions): אותם אירועים מצטברים לשורה אחת לכל run, עם גרף ביצוע בסגנון git. -- [Telemetry](/he/agenteye/telemetry): מה שה-agents שלך שולחים וכיצד אירועים מגיעים לזרם. -- [Error tracking](/he/agenteye/error-tracking): משטח טריאז אחד לכל מה שנפל. -- [Alerts](/he/agenteye/alerts): הפוך כל סף לכלל paging. -- [CLI and agents](/he/agenteye/cli-and-agents): אותו שביל חי מהטרמינל שלך. \ No newline at end of file +- [Sessions](/he/cloud/sessions): אותם אירועים מצטברים לשורה אחת לכל run, עם גרף ביצוע בסגנון git. +- [Telemetry](/he/cloud/performance): מה שה-agents שלך שולחים וכיצד אירועים מגיעים לזרם. +- [Error tracking](/he/cloud/errors): משטח טריאז אחד לכל מה שנפל. +- [Alerts](/he/cloud/alerts): הפוך כל סף לכלל paging. +- [CLI and agents](/he/cloud/cli): אותו שביל חי מהטרמינל שלך. \ No newline at end of file diff --git a/docs/he/cloud/fleet.mdx b/docs/he/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/he/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/he/agenteye/incidents.mdx b/docs/he/cloud/incidents.mdx similarity index 85% rename from docs/he/agenteye/incidents.mdx rename to docs/he/cloud/incidents.mdx index 6989381b..c076d4f1 100644 --- a/docs/he/agenteye/incidents.mdx +++ b/docs/he/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "כאשר התראה משתלחת, כולם יכולים לראות כאשר התראה משתלחת, השאלה הראשונה היא תמיד "מי עוסק בזה?" תקריות עונות לזה: ברגע שמשהו חורץ, כולם יכולים לראות שהתקרית פתוחה, מי בעלות עליה, בדיוק מה קרה עד כה, עם רשומה נקייה ומיוחסת שאתה יכול להעביר ישירות לניתוח-פוסט-מורטם. -![תיבת הנכנסים של התקריות: כרטיסי תקרית המקושרים להתראה וכרטיסים שנפתחו ידנית, מקובצים לפי מצב, כל אחד עם תג חומרה ו-assignee](/agenteye/images/incidents.png) +![תיבת הנכנסים של התקריות: כרטיסי תקרית המקושרים להתראה וכרטיסים שנפתחו ידנית, מקובצים לפי מצב, כל אחד עם תג חומרה ו-assignee](/cloud/images/incidents.png) *תיבת הנכנסים מקבצת תקריות פתוחות לפי מצב ומסננת לפי חומרה ו-assignee, כך שאתה רואה מה זקוק לתשומת לב אנושית כעת.* ## דע מי בעלות, במבט אחד @@ -17,10 +17,10 @@ description: "כאשר התראה משתלחת, כולם יכולים לראות כשהתקרית מסתיימת, כבר יש לך את הכתיבה. פתח כל תקרית ותקבל את ראיות ההפרה, את ה-assignees והמנויים שלה, שרשור הערות לתיאום במקום, וציר זמן פעילות יחיד-כיווני. -![תצוגה פרטי תקרית: ההתראה ההורית וסיכום ההפרה, assignees ומנויים, ציר זמן פעילות מיוחס, ושרשור הערות](/agenteye/images/incident-detail.png) +![תצוגה פרטי תקרית: ההתראה ההורית וסיכום ההפרה, assignees ומנויים, ציר זמן פעילות מיוחס, ושרשור הערות](/cloud/images/incident-detail.png) *כל מה שקרה, בסדר, כל שורה חתומה על ידי מי שעשה זאת.* -כל פעולה (פתוח, אושר, פתור וכו') נכתבת לציר הזמן הזה ולעולם לא עורכה. כל ערך מיוחס: לאופרטור שלקח אותו, לפי דוא"ל, או ל**automated** עבור כל מה ש-Failproof AI Observability עשה בעצמו, כמו פתיחת התקרית בהפרה. שום דבר אינו אנונימי ושום דבר לא אבד, כך שניתוח-פוסט-מורטם כתוב לעצמו בערך. +כל פעולה (פתוח, אושר, פתור וכו') נכתבת לציר הזמן הזה ולעולם לא עורכה. כל ערך מיוחס: לאופרטור שלקח אותו, לפי דוא"ל, או ל**automated** עבור כל מה ש-FailproofAI Cloud עשה בעצמו, כמו פתיחת התקרית בהפרה. שום דבר אינו אנונימי ושום דבר לא אבד, כך שניתוח-פוסט-מורטם כתוב לעצמו בערך. ## איך תקרית זז @@ -45,6 +45,6 @@ stateDiagram-v2 ## קשור -- [Alerts](/he/agenteye/alerts): הכללים שפותחים תקריות אלה כאשר סף חורץ. -- [Error tracking](/he/agenteye/error-tracking): ראה כל כישלון במקום אחד והעלה אחד להתראה. -- [Audits](/he/agenteye/audits): האנליסט המתוכנן שמוצא את הכישלונות שלא היה שום כלל צפה בהם. \ No newline at end of file +- [Alerts](/he/cloud/alerts): הכללים שפותחים תקריות אלה כאשר סף חורץ. +- [Error tracking](/he/cloud/errors): ראה כל כישלון במקום אחד והעלה אחד להתראה. +- [Audits](/he/cloud/audits): האנליסט המתוכנן שמוצא את הכישלונות שלא היה שום כלל צפה בהם. \ No newline at end of file diff --git a/docs/he/cloud/managed-policies.mdx b/docs/he/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/he/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/he/agenteye/overview.mdx b/docs/he/cloud/overview.mdx similarity index 51% rename from docs/he/agenteye/overview.mdx rename to docs/he/cloud/overview.mdx index 62b248ca..665810c1 100644 --- a/docs/he/agenteye/overview.mdx +++ b/docs/he/cloud/overview.mdx @@ -1,16 +1,16 @@ --- --- title: "Failproof AI: צפו בסוכנים בחיפוש כשלים" -description: "Failproof AI Observability היא פלטפורמה מארוחסנת בעצמך לצפייה, הערכה וشיפור של סוכנים בבינה מלאכותית בייצור." +description: "FailproofAI Cloud היא פלטפורמה מארוחסנת בעצמך לצפייה, הערכה וشיפור של סוכנים בבינה מלאכותית בייצור." --- -Failproof AI Observability היא פלטפורמה מאורחסנת בעצמך לצפייה, הערכה ושיפור של סוכנים בבינה מלאכותית בייצור. היא משמרת הכל שהסוכנים שלכם עושים (כל קריאת כלי, בקשת מודל, hook ושגיאה), מדרגת את איכות כל הרצה, וחושפת את הכשלים שלא ידעתם שצריך לחפש, הכל בדוח בקרים שאתה מפעיל בתוך תשתית שלך. +FailproofAI Cloud היא פלטפורמה מאורחסנת בעצמך לצפייה, הערכה ושיפור של סוכנים בבינה מלאכותית בייצור. היא משמרת הכל שהסוכנים שלכם עושים (כל קריאת כלי, בקשת מודל, hook ושגיאה), מדרגת את איכות כל הרצה, וחושפת את הכשלים שלא ידעתם שצריך לחפש, הכל בדוח בקרים שאתה מפעיל בתוך תשתית שלך. -אם אתה משגר סוכנים בבינה מלאכותית ואתה עייף מ"ניחוש" למה הרצה השתבשה, זה הדף להתחיל ממנו. הוא מסביר מה Failproof AI Observability נותן לך וכיצד החלקים מתאימים יחד, לפני שתתקין כל דבר. +אם אתה משגר סוכנים בבינה מלאכותית ואתה עייף מ"ניחוש" למה הרצה השתבשה, זה הדף להתחיל ממנו. הוא מסביר מה FailproofAI Cloud נותן לך וכיצד החלקים מתאימים יחד, לפני שתתקין כל דבר. -> **Failproof AI Observability היא מוצר ארגוני מ-Failproof AI.** רוצה לראות את זה בפעולה? בקש הדגמה: שלח דוא"ל ל-[nikita@befailproof.ai](mailto:nikita@befailproof.ai). +> **FailproofAI Cloud היא מוצר ארגוני מ-Failproof AI.** רוצה לראות את זה בפעולה? בקש הדגמה: שלח דוא"ל ל-[nikita@befailproof.ai](mailto:nikita@befailproof.ai). -![הפעלת Failproof AI Observability מצויירת כגרף ביצוע בסגנון git לצד ציר הזמן של האירועים שלה, עם פירוט לכל הרצה של כלים, מודלים וואקות בפס הימני](/agenteye/images/session-detail.png) +![הפעלת FailproofAI Cloud מצויירת כגרף ביצוע בסגנון git לצד ציר הזמן של האירועים שלה, עם פירוט לכל הרצה של כלים, מודלים וואקות בפס הימני](/cloud/images/session-detail.png) *כל הרצה של סוכן מצויירת כגרף ביצוע בסגנון git (משמאל) לצד ציר הזמן של האירועים שלה. לכל תת-סוכן מקביל יש נתיב משלו; פס הימני מפרק את הכלים, המודלים, הקשרים וההוצאה לטוקנים עבור ההרצה.* @@ -30,52 +30,52 @@ Failproof AI Observability היא פלטפורמה מאורחסנת בעצמך
-*Failproof Audit: תן ל-Failproof AI Observability לחפור בתיעודים שלך בחסות סשנים ולהגיד לך מה לתקן.* +*Failproof Audit: תן ל-FailproofAI Cloud לחפור בתיעודים שלך בחסות סשנים ולהגיד לך מה לתקן.* --- ## למה צוותים משתמשים בזה - **ראה מה הסוכן שלך בעצם עשה.** כל הרצה הופכת לגרף ביצוע קריא בסגנון git: איזה כלים רצו במקביל, אילו תת-סוכנים התפצלו, איפה זה קפא, והוצאות מה. -- **תפס רגרסיות איכות באופן אוטומטי.** חבר שירות דירוג קטן וה-Failproof AI Observability ידרג כל הרצה מסיימת, כך שירידה בשימושיות או עלייה בהזיות תופיע בעצמה. +- **תפס רגרסיות איכות באופן אוטומטי.** חבר שירות דירוג קטן וה-FailproofAI Cloud ידרג כל הרצה מסיימת, כך שירידה בשימושיות או עלייה בהזיות תופיע בעצמה. - **מצא כשלים שלא כתבת כלל עבורם.** ביקורות חוזרות חופרות בתיעודים שלך בחסות סשנים לאשכולות שגיאות, חריגי זמן תגובה, ניקוד נמוך והרצות תקועות, ואז מעניקות לך ממצאים מדורגים ומבוססי ראיות. - **קבל עמוד כשזה משנה.** כללי סף כן על שיעור שגיאה, זמן תגובה, עלות או ניקוד מעריך ופתח תקלות שאתה יכול להשתמע, להקצות ולפתור. - **שאל שאלות באנגלית רגילה.** עוזר בינה מלאכותית בתוך הדוח משיב על האם איכות עוברת מגמה בייצור השבוע? על הנתונים שלך. כל שינוי שהיא עושה כפוף לאישור. -- **שמור על הנתונים שלך.** Failproof AI Observability מאורחסן בעצמך: אירועים, הנושאים והניתוחים נשארים בתשתית שאתה שולט בה. +- **שמור על הנתונים שלך.** FailproofAI Cloud מאורחסן בעצמך: אירועים, הנושאים והניתוחים נשארים בתשתית שאתה שולט בה. --- ## מה אתה מקבל -Failproof AI Observability מארגנה סביב שלוש רעיונות (**צפייה**, **ניתוח** ו**ניהול**), משתקפת בסרגל הצד השמאלי של הדוח. +FailproofAI Cloud מארגנה סביב שלוש רעיונות (**צפייה**, **ניתוח** ו**ניהול**), משתקפת בסרגל הצד השמאלי של הדוח. **צפייה** (האמת הגולמית של מה שקרה): -- **[ספר אירועים](/he/agenteye/event-stream)**: שביל חי לכל שלב של כל הרצה (קריאות כלים, קריאות מודל, קשרים, שגיאות). -- **[סשנים](/he/agenteye/sessions)**: אירועים אלה מצטברים לשורה אחת לכל הרצה, כל אחד מוכן להיות מדורג, עם גרף ביצוע בסגנון git. -- **[מטרי ביצוע](/he/agenteye/telemetry)**: מפות חום זמן תגובה לכל משטח וחיוני p50/p95/p99 עבור מודלים, כלים וקשרים, כך שקוצץ זנב בולט מהחציון. -- **[עקבוב שגיאות](/he/agenteye/error-tracking)**: משטח טריאז אחד לכל מה שהשתבש, קליק אחד מהתראה שנורתה. +- **[ספר אירועים](/he/cloud/event-stream)**: שביל חי לכל שלב של כל הרצה (קריאות כלים, קריאות מודל, קשרים, שגיאות). +- **[סשנים](/he/cloud/sessions)**: אירועים אלה מצטברים לשורה אחת לכל הרצה, כל אחד מוכן להיות מדורג, עם גרף ביצוע בסגנון git. +- **[מטרי ביצוע](/he/cloud/performance)**: מפות חום זמן תגובה לכל משטח וחיוני p50/p95/p99 עבור מודלים, כלים וקשרים, כך שקוצץ זנב בולט מהחציון. +- **[עקבוב שגיאות](/he/cloud/errors)**: משטח טריאז אחד לכל מה שהשתבש, קליק אחד מהתראה שנורתה. -![עמוד כלים של צפייה: מפת חום זמן תגובה, פס אחוז ובר התפלגות כלים על 24 פחי זמן](/agenteye/images/tools.png) +![עמוד כלים של צפייה: מפת חום זמן תגובה, פס אחוז ובר התפלגות כלים על 24 פחי זמן](/cloud/images/tools.png) *כל משטח צפייה משלב קו ניצנים וחיוני p50/p95/p99 עם מפת חום זמן תגובה ופס אחוז. מוצג כאן: כלים.* **ניתוח** (הפוך פעילות לתשובות): -- **[שאילתות](/he/agenteye/queries)** ו**[דוחות בקרים](/he/agenteye/dashboards)**: SQL שנשמר על אירועים והערכות שלך, תורשמו לדוחות בקרים משותפים בהיקף ארגוני. -- **[הערכות](/he/agenteye/evaluations)**: ניקוד איכות שמופקים משירות המעריך שלך, עם נימוק לכל ניקוד. -- **[ביקורות](/he/agenteye/audits)**: חקירות חוזרות המפיקות דפוסי כשל בחסות סשנים. -- **[התראות](/he/agenteye/alerts)** ו**[תקלות](/he/agenteye/incidents)**: כללי סף שעמודים לך, בתוספת זרימת עבודה תקלה לטריאז שלהם. +- **[שאילתות](/he/cloud/queries)** ו**[דוחות בקרים](/he/cloud/dashboards)**: SQL שנשמר על אירועים והערכות שלך, תורשמו לדוחות בקרים משותפים בהיקף ארגוני. +- **[הערכות](/he/cloud/evaluations)**: ניקוד איכות שמופקים משירות המעריך שלך, עם נימוק לכל ניקוד. +- **[ביקורות](/he/cloud/audits)**: חקירות חוזרות המפיקות דפוסי כשל בחסות סשנים. +- **[התראות](/he/cloud/alerts)** ו**[תקלות](/he/cloud/incidents)**: כללי סף שעמודים לך, בתוספת זרימת עבודה תקלה לטריאז שלהם. **ממשקים** (הגע לנתונים שלך בדרכך שלך): -- **[CLI](/he/agenteye/cli-and-agents)**: נהג בכל ההטמעה שלך מהטרמינל או סקריפט, והתן לסוכן קוד לעשות את זה עבורך באנגלית רגילה. -- **[עוזר בינה מלאכותית](/he/agenteye/assistant)**: שאל שאלות על הסוכנים שלך באנגלית רגילה, ממש בתוך הדוח. -- **REST API**: הכל שהדוח והקלי עושים מגובה על ידי REST API שאתה יכול להתקשר אליו ישירות עם [מפתח API](/he/agenteye/api-keys) בהיקף - ספוג אירועים, שאל סשנים והערכות, וניהל דוחות בקרים, התראות, ביקורות, משתמשים ומפתחות, כך שאתה יכול לחווט את Failproof AI Observability לתוך הכלים שלך. +- **[CLI](/he/cloud/cli)**: נהג בכל ההטמעה שלך מהטרמינל או סקריפט, והתן לסוכן קוד לעשות את זה עבורך באנגלית רגילה. +- **[עוזר בינה מלאכותית](/he/cloud/assistant)**: שאל שאלות על הסוכנים שלך באנגלית רגילה, ממש בתוך הדוח. +- **REST API**: הכל שהדוח והקלי עושים מגובה על ידי REST API שאתה יכול להתקשר אליו ישירות עם [מפתח API](/he/cloud/access) בהיקף - ספוג אירועים, שאל סשנים והערכות, וניהל דוחות בקרים, התראות, ביקורות, משתמשים ומפתחות, כך שאתה יכול לחווט את FailproofAI Cloud לתוך הכלים שלך. **ניהול** (הפעל את זה בשביל הצוות שלך): -- **[מפתחות API](/he/agenteye/api-keys)**: אסימונים בהיקף עבור הלקט, הדוח והעוזר. +- **[מפתחות API](/he/cloud/access)**: אסימונים בהיקף עבור הלקט, הדוח והעוזר. - **משתמשים**: כניסה ללא סיסמה מבוססת דוא"ל עם רשימת הרשאה. - **הגדרות**: תצורה לכל ארגון, כולל דריסות חלון הקשר של מודל. @@ -91,18 +91,18 @@ Failproof AI Observability מארגנה סביב שלוש רעיונות (**צפ - **דוח**: איפה אתה חוקר הכל. - **שירותים אופציונליים**: שירות דירוג (הערכות), ושירות עוזר בינה מלאכותית (הצ'אט בתוך הדוח). -עבור אוצר המילים בשימוש לאורך הדוקים (*אירוע, סשן, הערכה, ביקורת, ממצא, תקלה*), ראה [קונספטים](/he/agenteye/concepts). +עבור אוצר המילים בשימוש לאורך הדוקים (*אירוע, סשן, הערכה, ביקורת, ממצא, תקלה*), ראה [קונספטים](/he/concepts). --- -## קבלת Failproof AI Observability +## קבלת FailproofAI Cloud -Failproof AI Observability היא מוצר ארגוני מ-Failproof AI, והיא פועלת לצד Failproof AI Enforcement — המוצר של מדיניות ומגן — תחת המותג Failproof AI. היא פועלת כליל בסביבה שלך. אם אין לך גישה לחבילות עדיין, בקש הדגמה ואנחנו נקבע אותך: שלח דוא"ל ל-[nikita@befailproof.ai](mailto:nikita@befailproof.ai). +FailproofAI Cloud היא מוצר ארגוני מ-Failproof AI, והיא פועלת לצד FailproofAI guardrails — המוצר של מדיניות ומגן — תחת המותג Failproof AI. היא פועלת כליל בסביבה שלך. אם אין לך גישה לחבילות עדיין, בקש הדגמה ואנחנו נקבע אותך: שלח דוא"ל ל-[nikita@befailproof.ai](mailto:nikita@befailproof.ai). --- ## הצעדים הבאים -- [קונספטים](/he/agenteye/concepts): Failproof AI Observability אוצר מילים במקום אחד. -- [צפייה](/he/agenteye/observability): עקוב מה הסוכנים שלך עושים, הרצה אחר הרצה. -- [אבטחה](/he/agenteye/security): כיצד Failproof AI Observability שומר על הנתונים שלך מבודדים ובשליטתך. \ No newline at end of file +- [קונספטים](/he/concepts): FailproofAI Cloud אוצר מילים במקום אחד. +- [צפייה](/he/cloud/overview): עקוב מה הסוכנים שלך עושים, הרצה אחר הרצה. +- [אבטחה](/he/cloud/security): כיצד FailproofAI Cloud שומר על הנתונים שלך מבודדים ובשליטתך. \ No newline at end of file diff --git a/docs/he/agenteye/telemetry.mdx b/docs/he/cloud/performance.mdx similarity index 81% rename from docs/he/agenteye/telemetry.mdx rename to docs/he/cloud/performance.mdx index 071080c6..55bd9d0e 100644 --- a/docs/he/agenteye/telemetry.mdx +++ b/docs/he/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "ראה את הרגע בו המודלים, הכלים או ה-hooks ראה את הרגע בו המודלים, הכלים או ה-hooks מאטים או מרימים את החשבון, ותופסו עיכוב זנב לפני שהמשתמשים שלך ירגישו זאת. שלוש דפים ייעודיים הופכים תזמוני גולמיים ל-p50, p95, ו-p99 שתוכל לקרוא בחטף. -![דף Models המציג מפת חום של latency, קו אחוזון ומספרים לפי מודל של טוקנים, עלות וחלון context](/agenteye/images/models.png) +![דף Models המציג מפת חום של latency, קו אחוזון ומספרים לפי מודל של טוקנים, עלות וחלון context](/cloud/images/models.png) *דף Models: מפת חום של latency, קו אחוזון וטוקנים לפי מודל, עלות משוערת ומילוי חלון context.* ## הפסק להתיר לממוצעים להסתיר את ההרצות הגרועות שלך @@ -24,13 +24,13 @@ crosshair ריחוף משותף קושר את מפת החום והקו, כך ש דף Models (המוצג למעלה) עונה על שתי השאלות שכל חשבון מעלה: איזה מודל, וכמה. על גבי התצוגה latency המשותפת, הוא מוסיף **צריכת טוקנים לפי מודל**, **עלות משוערת** ו**מילוי חלון context**, כך שגדילה בלתי מבוקרת של prompt וcompaction קרוב יותר גלויים לפני שהם תופסים אותך בפתיעה. -Failproof AI Observability מזהה מזהי מודל נפוצים באופן אוטומטי. אם חלון נראה לא תקין, או שאתה מריץ מודל פרטי משלך, תקן אותו או הוסף אחד תחת **Settings**, ב**model context windows**, וקריאות המילוי עוקבות. +FailproofAI Cloud מזהה מזהי מודל נפוצים באופן אוטומטי. אם חלון נראה לא תקין, או שאתה מריץ מודל פרטי משלך, תקן אותו או הוסף אחד תחת **Settings**, ב**model context windows**, וקריאות המילוי עוקבות. ## Tools: הבחן בין האיטי לשבור קריאת tool יכולה להיות איטית, או שהיא יכולה להיכשל בשקט, ואתה רוצה לדעת איזה מהם בעוד שניות, לא אחרי שחפרת דרך יומנים. -![דף Tools המציג את מפת החום של latency המשותפת וקו האחוזון ליד פירוק הצלחה וכישלון וקו התפלגות כלי](/agenteye/images/tools.png) +![דף Tools המציג את מפת החום של latency המשותפת וקו האחוזון ליד פירוק הצלחה וכישלון וקו התפלגות כלי](/cloud/images/tools.png) *דף Tools: אותה מפת חום וקו אחוזון, בתוספת פירוק הצלחה וכישלון וקו התפלגות כלי.* לצד התצוגה latency המשותפת, דף Tools מוסיף **פירוק הצלחה וכישלון** ו**קו התפלגות כלי**, כך שתראה בחטף אילו כלים אתה מסתמך עליהם הכי הרבה ואילו אוכלים את תקציב השגיאות שלך. @@ -39,14 +39,14 @@ Failproof AI Observability מזהה מזהי מודל נפוצים באופן א כאשר lifecycle hook משך run, "hooks הם איטיים" אינו משהו שאתה יכול לפעול לפיו. דף Hooks מקבל אותך לזה שחשוב. -![דף Hooks המציג latency מפורק לפי שם hook ואירוע טריגר על מפת החום והקו האחוזון המשותפים](/agenteye/images/hooks.png) +![דף Hooks המציג latency מפורק לפי שם hook ואירוע טריגר על מפת החום והקו האחוזון המשותפים](/cloud/images/hooks.png) *דף Hooks: latency מפורק לפי שם hook ואירוע טריגר.* על אותה מפת חום של latency וקו אחוזון, דף Hooks מפרק את הפעילות לפי **שם hook** ו**אירוע טריגר**, כך שתנחת על ה-hook האחד ואירוע אחד שצריכים תשומת לב. ## קשור -- [Event stream](/he/agenteye/event-stream): השביל החי וקידוד הצבע של כל אירוע. -- [Sessions](/he/agenteye/sessions): צבור אירועים לשורה אחת לכל ריצה ופתח את גרף ההוצאה לפועל שלה. -- [Error tracking](/he/agenteye/error-tracking): משטח triage אחד לכל מה שהלוח הבקרה צובע אדום. -- [Dashboards](/he/agenteye/dashboards): צפייה rolled-up על פני הצי שלך. \ No newline at end of file +- [Event stream](/he/cloud/event-stream): השביל החי וקידוד הצבע של כל אירוע. +- [Sessions](/he/cloud/sessions): צבור אירועים לשורה אחת לכל ריצה ופתח את גרף ההוצאה לפועל שלה. +- [Error tracking](/he/cloud/errors): משטח triage אחד לכל מה שהלוח הבקרה צובע אדום. +- [Dashboards](/he/cloud/dashboards): צפייה rolled-up על פני הצי שלך. \ No newline at end of file diff --git a/docs/he/agenteye/queries.mdx b/docs/he/cloud/queries.mdx similarity index 79% rename from docs/he/agenteye/queries.mdx rename to docs/he/cloud/queries.mdx index fd852322..9a6a98d1 100644 --- a/docs/he/agenteye/queries.mdx +++ b/docs/he/cloud/queries.mdx @@ -4,9 +4,9 @@ description: "שאל כל שאלה על נתוני הסוכן שלך וקבל ת --- -שאל כל שאלה על נתוני הסוכן שלך וקבל תשובה תוך שניות. Failproof AI Observability מספק לך ספרייה של שאילתות שמורות וגמורות לשימוש על האירועים וההערכות שלך, כך שתוכל להתחיל מדוגמה עובדת במקום מעורך SQL ריק. +שאל כל שאלה על נתוני הסוכן שלך וקבל תשובה תוך שניות. FailproofAI Cloud מספק לך ספרייה של שאילתות שמורות וגמורות לשימוש על האירועים וההערכות שלך, כך שתוכל להתחיל מדוגמה עובדת במקום מעורך SQL ריק. -![ספרית השאילתות השמורות: רשת של שאילתות בנות שימוש חוזר, גם הפריסטים המובנים וגם אלה שכוללים משלך](/agenteye/images/queries.png) +![ספרית השאילתות השמורות: רשת של שאילתות בנות שימוש חוזר, גם הפריסטים המובנים וגם אלה שכוללים משלך](/cloud/images/queries.png) *ספרית השאילתות השמורות שלך ב-`//queries`: פריסטים מובנים לצד השאילתות שהצוות שלך שמר ושימ.* @@ -22,7 +22,7 @@ description: "שאל כל שאלה על נתוני הסוכן שלך וקבל ת פתח כל שאילתה והיא תנחת בספר ההרכב SQL, שם אתה יכול להתאים אותה ולראות את התשובה מיד: ללא ייצוא, ללא הליך הלוך וחזור, ללא המתנה למישהו אחר. -![ספר ההרכב של שאילתות SQL מריץ שאילתה שמורה, עם סרגל בחצי טוב ורשת תוצאות חי](/agenteye/images/query-lab.png) +![ספר ההרכב של שאילתות SQL מריץ שאילתה שמורה, עם סרגל בחצי טוב ורשת תוצאות חי](/cloud/images/query-lab.png) *ספר ההרכב של SQL: השאילתה שלך משמאל, סרגל בחצי טוב כדי שלעולם לא תנחש שם עמודה, ורשת תוצאות חי מתחת.* @@ -43,14 +43,14 @@ agenteye query list # אותן שאילתות שמורות, מה agenteye query run errs --arg prod # הרץ אחת והדפיס את השורות (הוסף --json כדי לצנור אותה) ``` - ראה [CLI וסוכנים](/he/agenteye/cli-and-agents) לסט הפקודה המלא. + ראה [CLI וסוכנים](/he/cloud/cli) לסט הפקודה המלא. -- **מהעוזר AI.** לא בטוח איך לנסח את SQL? שאל את [עוזר ה-AI](/he/agenteye/assistant) בתוך הדשבורד בעברית רגילה והוא יסיר את השאילתה וישמור אותה בספרייה שלך בשבילך. +- **מהעוזר AI.** לא בטוח איך לנסח את SQL? שאל את [עוזר ה-AI](/he/cloud/assistant) בתוך הדשבורד בעברית רגילה והוא יסיר את השאילתה וישמור אותה בספרייה שלך בשבילך. הרצת שאילתה שמורה מגובלת על ידי הרשאת `queries:run`, המופרדת מההרשאות ליצור או למחוק שאילתות, כך שאתה יכול להעניק גישת קריאה ללא רשות לכולם לכתוב מחדש את הספרייה. ## קשור -- [Dashboards](/he/agenteye/dashboards): צמיד תוצאות שאילתה לתרשימים משותפים בהיקף ארגון. -- [עוזר AI](/he/agenteye/assistant): שאל שאלות בעברית רגילה וקבל שאילתה חזרה. -- [CLI וסוכנים](/he/agenteye/cli-and-agents): הרץ ושמור את אותן שאילתות מהטרמינל שלך. \ No newline at end of file +- [Dashboards](/he/cloud/dashboards): צמיד תוצאות שאילתה לתרשימים משותפים בהיקף ארגון. +- [עוזר AI](/he/cloud/assistant): שאל שאלות בעברית רגילה וקבל שאילתה חזרה. +- [CLI וסוכנים](/he/cloud/cli): הרץ ושמור את אותן שאילתות מהטרמינל שלך. \ No newline at end of file diff --git a/docs/he/agenteye/python-sdk.mdx b/docs/he/cloud/sdk.mdx similarity index 92% rename from docs/he/agenteye/python-sdk.mdx rename to docs/he/cloud/sdk.mdx index 1ef405c8..9c0a974b 100644 --- a/docs/he/agenteye/python-sdk.mdx +++ b/docs/he/cloud/sdk.mdx @@ -4,11 +4,11 @@ description: "ראה בדיוק מה עשו הסוכנים AI שלך בייצו --- -ראה בדיוק מה עשו הסוכנים AI שלך בייצור: כל הרצת סוכן, קריאת כלי, בקשת מודל, hook והתערבות אנוש. ה-SDK של Failproof AI Observability Python מתעד את השביל הזה מתוך קוד הסוכן שלך כדי שתוכל לתקן, לתקן באופן הולם ולהעריך מה קרה. השתמש בו בכל פעם שתרצה ש-Failproof AI Observability תצפה בסוכנים שלך. +ראה בדיוק מה עשו הסוכנים AI שלך בייצור: כל הרצת סוכן, קריאת כלי, בקשת מודל, hook והתערבות אנוש. ה-SDK של FailproofAI Cloud Python מתעד את השביל הזה מתוך קוד הסוכן שלך כדי שתוכל לתקן, לתקן באופן הולם ולהעריך מה קרה. השתמש בו בכל פעם שתרצה ש-FailproofAI Cloud תצפה בסוכנים שלך. מתחת להנהלה, ה-SDK כותב אירועים מובנים לקבצי JSONL מקומיים, וה-daemon של הקלט אוסף אותם ומשלח אותם לפלטפורמה באופן אוטומטי. אתה לא מנהל את הקבצים הללו בעצמך. -> **Tip:** חדש ל-Failproof AI Observability? דף זה הוא ההפניה המלאה של אירועי SDK. +> **Tip:** חדש ל-FailproofAI Cloud? דף זה הוא ההפניה המלאה של אירועי SDK.
@@ -26,7 +26,7 @@ description: "ראה בדיוק מה עשו הסוכנים AI שלך בייצו python -c "import agenteye; print(agenteye.__version__)" ``` -מעדיף להניח לסוכן קידוד לבצע את כל השילוב? [Python SDK Agent Skill](/he/agenteye/python-sdk-skill) מכיר את נתיב ההתקנה, מתכנן את נקודות הכלים, כותב אותן ומאמת שהאירועים מגיעים. +מעדיף להניח לסוכן קידוד לבצע את כל השילוב? [Python SDK Agent Skill](/he/cloud/agent-skills) מכיר את נתיב ההתקנה, מתכנן את נקודות הכלים, כותב אותן ומאמת שהאירועים מגיעים. --- @@ -60,7 +60,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### הקנת קריאה אמיתית -בפועל אתה עוטף את קוד הסוכן הקיים שלך. קוצץ קריאת מודל עם `model_request` לפני ו-`model_response` אחרי, כך ששני האירועים משתרעים על הבקשה האמיתית ו-Failproof AI Observability יכולה לעשות זוג עם אותם: +בפועל אתה עוטף את קוד הסוכן הקיים שלך. קוצץ קריאת מודל עם `model_request` לפני ו-`model_response` אחרי, כך ששני האירועים משתרעים על הבקשה האמיתית ו-FailproofAI Cloud יכולה לעשות זוג עם אותם: ```python import anthropic @@ -99,7 +99,7 @@ agenteye.event.model_response( הנה איך נראים אירועים אלה לאחר שהם מגיעים לדashboard, מיוחסים בצבעים לפי סוג וניתנים לסינון לפי סביבה, סוכן וסשן: -![זרם האירועים החי, מקודד בצבעים לפי סוג אירוע וניתן לסינון לפי סביבה, סוכן וסשן](/agenteye/images/events-stream.png) +![זרם האירועים החי, מקודד בצבעים לפי סוג אירוע וניתן לסינון לפי סביבה, סוכן וסשן](/cloud/images/events-stream.png) --- @@ -156,9 +156,9 @@ export AGENTEYE_ENVIRONMENT=production ## הפניה אירוע -רוב האירועים מגיעים בצמדי התחלה/סיום השותפים מזהה קורלציה: `tool_use` ו-`tool_result` חולקים `tool_call_id`, `hook_triggered` ו-`hook_completed` חולקים `hook_id`, ו-`human_wait` ו-`human_input` חולקים `input_id`. פתוח את אירוע ההתחלה, בצע את העבודה, ואז פתוח את אירוע הסיום עם אותו מזהה. Failproof AI Observability תאם את הזוג ותחשב `duration_ms` עבורך, כך שאתה לא מעביר `duration_ms` בעצמך. +רוב האירועים מגיעים בצמדי התחלה/סיום השותפים מזהה קורלציה: `tool_use` ו-`tool_result` חולקים `tool_call_id`, `hook_triggered` ו-`hook_completed` חולקים `hook_id`, ו-`human_wait` ו-`human_input` חולקים `input_id`. פתוח את אירוע ההתחלה, בצע את העבודה, ואז פתוח את אירוע הסיום עם אותו מזהה. FailproofAI Cloud תאם את הזוג ותחשב `duration_ms` עבורך, כך שאתה לא מעביר `duration_ms` בעצמך. -![גרף ביצוע בסגנון git של סשן לצד ציר הזמן של האירוע שלו, שנבנה מחדש מהאירועים המזוווגים, עם פירוק כלי/מודל/חטיף](/agenteye/images/session-detail.png) +![גרף ביצוע בסגנון git של סשן לצד ציר הזמן של האירוע שלו, שנבנה מחדש מהאירועים המזוווגים, עם פירוק כלי/מודל/חטיף](/cloud/images/session-detail.png) כל שיטות אירוע דורשות שני שדות אלה: @@ -432,5 +432,5 @@ agenteye.event.tool_use( ## שלבים הבאים -- [Event stream](/he/agenteye/event-stream): צפה באירועים אלה מגיעים בחיים, מיוחסים בצבעים וניתנים לסינון לפי סביבה, סוכן וסשן. -- [Sessions](/he/agenteye/sessions): ראה כיצד האירועים המזוווגים משחזרים כל הרצת סוכן כגרף ביצוע וציר זמן. \ No newline at end of file +- [Event stream](/he/cloud/event-stream): צפה באירועים אלה מגיעים בחיים, מיוחסים בצבעים וניתנים לסינון לפי סביבה, סוכן וסשן. +- [Sessions](/he/cloud/sessions): ראה כיצד האירועים המזוווגים משחזרים כל הרצת סוכן כגרף ביצוע וציר זמן. \ No newline at end of file diff --git a/docs/he/cloud/security.mdx b/docs/he/cloud/security.mdx new file mode 100644 index 00000000..2e0913eb --- /dev/null +++ b/docs/he/cloud/security.mdx @@ -0,0 +1,67 @@ +--- +title: "אבטחה" +description: "FailproofAI Cloud בנוי כך שיעמוד קרוב לאגנטים הייצור שלך, מה שאומר שהוא רואה את ההנמקות שלך, קלטי הכלים, והפלטים שלהם." +--- + +FailproofAI Cloud בנוי כך שיעמוד קרוב לאגנטים הייצור שלך, מה שאומר שהוא רואה את ההנמקות שלך, קלטי הכלים, והפלטים שלהם. דף זה מסביר כיצד הוא משמר את הנתונים הללו בצורה מבודדת, מבוקרת, וברשותך. אם אתה בתהליך הערכה של FailproofAI Cloud לסקירת אבטחה, התחל כאן. + +--- + +## הנתונים שלך נשארים בסביבתך + +FailproofAI Cloud הוא self-hosted. אירועים, הנמקות, תגובות מודל, וניתוחים מאוחסנים בבסיסי הנתונים שלך, בסביבתך שלך. שום דבר לא נשלח ל-SaaS של צד שלישי לאחסון, והנתונים שלך נשארים בחשבון הענן שלך. + +--- + +## בידוד דיירים + +מופע אחד של FailproofAI Cloud יכול להנחות ארגונים רבים, וכל אחד מבודד בשכבת האחסון — מאופשר על ידי מסד הנתונים, לא רק על ידי ממשק המשתמש: + +- הנתונים התפעוליים של ארגון (משתמשים, מפתחות, לוחות מחוונים, שאילתות שמורות) מוגבלים לארגון זה, וקריאות חוצות-ארגוניות חסומות על ידי מסד הנתונים עצמו. +- כל אירוע שנקלט מוקלד עם הארגון שבעליו, כך שאירועים של ארגון אחד לעולם לא יוכלו להיקרא על ידי ארגון אחר. + +כל נתיב לוח מחוונים מוגבל תחת slug ארגוני (`//…`). + +--- + +## כניסה למערכת + +FailproofAI Cloud משתמש בכניסה ללא ססמה, מבוססת דוא״ל. אין ססמה שאפשר לתפוס או לדלוף. משתמש מבקש קוד חד-פעמי (או קישור קסום של לחיצה אחת), שנשלח להם בדוא״ל ותוקפו פוקע במהירות. הכניסה מוגדרת על ידי **רשימת אישור**: רק כתובות דוא״ל (או דומיינים) שאתה מאשר יכולות להתחקות. + +![מסך הכניסה של FailproofAI Cloud, המשדר קוד חד-פעמי לדוא״ל שלך](/cloud/images/login.png) + +--- + +## גישה מוגבלת עם מפתחות API + +כל לקוח מתחקה עם מפתח API שנושא הרשאות דקות, עם עקרון הפחות-הרשאות. קולקטור צריך רק `events:add`; מפתח לוח מחוונים או עוזר יכול להיות קריאה בלבד; פעולות הרסניות (מחיקה, יצירה מחדש) הן הנחות נפרדות שאתה בוחר להכליל. + +![דף מפתחות ה-API: הנחות ההרשאות של כל מפתח, בקודים צבע לפי היקף קריאה, כתיבה, והרסני](/cloud/images/api-keys.png) + +שמור על מפתח bootstrap המנהל להגדרה, והנפק מפתחות צרים לכל השאר. ראה [מפתחות API](/he/cloud/access). + +--- + +## עוזר קריאה-בלבד, בשער אישור + +[העוזר בלוח המחוונים](/he/cloud/assistant) משובץ מענה על שאלות על הנתונים שלך, אך הוא מוגבל בעיצוב: + +- הוא **קריאה-בלבד כברירת מחדל**: SQL שלו עובר דרך שומר שמותר רק `SELECT`/`WITH` שאילתות, הצהרה יחידה, עם מכסה שורות. +- כל דבר שהוא יוצר (שאילתה שמורה, לוח מחוונים) הוא **בשער אישור**: אתה סוקר ומאשר כל כתיבה לפני שזה קורה. +- הוא **לא יכול למחוק לעולם**. + +אז חברה יכולה לשאול "אילו אגנטים השגיאו הכי הרבה השבוע?" ולפעול על פי התשובה, ללא שהעוזר יכול לשנות או להסיר את הנתונים שלך בעצמו. + +--- + +## בדרך + +כל התעבורה עובדת על HTTPS. אתה מסיים TLS עם התעודות שלך, כך שתעבורת קולקטור-לשרת ודפדפן-לשרת מוצפנת בדרך. + +--- + +## הצעדים הבאים + +- [סקירה כללית](/he/cloud/overview): כיצד FailproofAI Cloud מתחברים ביחד. +- [מפתחות API](/he/cloud/access): הגבל גישה לקולקטור, לוח מחוונים, ועוזר. +- [FailproofAI Cloud](/he/cloud/overview): מה FailproofAI Cloud לוקח מהאגנטים שלך. \ No newline at end of file diff --git a/docs/he/agenteye/sessions.mdx b/docs/he/cloud/sessions.mdx similarity index 75% rename from docs/he/agenteye/sessions.mdx rename to docs/he/cloud/sessions.mdx index 086f62bc..4fff266d 100644 --- a/docs/he/agenteye/sessions.mdx +++ b/docs/he/cloud/sessions.mdx @@ -5,9 +5,9 @@ description: "כל event מ-run, מקופל לשורה אחת קריאה וממ --- -תוך כדי שאתה מנחש למה run נכשל. Failproof AI Observability מקפל כל event מ-run לשורה אחת קריאה, ואז מציירת את כל ה-run כתמונה בסגנון git שאתה יכול לקרוא בשניות, כך שאתה רואה בדיוק מה עשה ה-agent שלך, שלב אחר שלב. +תוך כדי שאתה מנחש למה run נכשל. FailproofAI Cloud מקפל כל event מ-run לשורה אחת קריאה, ואז מציירת את כל ה-run כתמונה בסגנון git שאתה יכול לקרוא בשניות, כך שאתה רואה בדיוק מה עשה ה-agent שלך, שלב אחר שלב. -![רשימת ה-Sessions: שורה אחת לכל run, על פני environments ו-agents, עם status pills ו-evaluation score badges](/agenteye/images/sessions-list.png) +![רשימת ה-Sessions: שורה אחת לכל run, על פני environments ו-agents, עם status pills ו-evaluation score badges](/cloud/images/sessions-list.png) *שורה אחת לכל run: ה-status pill אומר לך איך הסתיים ה-run במבט אחד, ותגי score רכובים לצד זה ברגע שמעריך מחובר.* @@ -31,7 +31,7 @@ description: "כל event מ-run, מקופל לשורה אחת קריאה וממ ## קרא את כל ה-run כתמונה -![גרף ביצוע בסגנון git של session לצד ציר הזמן של events שלו, עם פירוט של tool, model, ו-hook panel](/agenteye/images/session-detail.png) +![גרף ביצוע בסגנון git של session לצד ציר הזמן של events שלו, עם פירוט של tool, model, ו-hook panel](/cloud/images/session-detail.png) *גרף הביצוע (שמאל) יושב ליד ציר הזמן של events; ה-rail הימני מפרק את ה-tools, models, hooks, ו-token spend של ה-run.* @@ -39,7 +39,7 @@ description: "כל event מ-run, מקופל לשורה אחת קריאה וממ ה-rail הימני נותן לך את הפירוט per-run: אילו tools ו-models רצו, אילו hooks בעירו, ומה ה-run הוציא בתוקנים. זו התשובה ל-"למה ה-run הזה עלה כל כך הרבה?" או "איזה tool הוא ה-slow אחד?" יושבת ממש לצד הגרף שגרם לזה. -Events בודדים ניתנים לפנייה, כך שאתה יכול לתת למישהו קישור לרגע אחד ולא "ה-session, בערך שתיים שלישים למטה". העתק את הקישור מכל event, או עקוב אחרי אחד מ-[audit](/he/agenteye/audits) finding או שגיאה, והוא session נפתח עם אותו event נבחר וגלול אליו. זה מתקיים גם עבור runs ארוך מאוד: ציר הזמן טוען חלון מוגבל למען הדפדפן שלך, וקישור שמצביע מעבר לחלון זה עדיין מוצא את ה-event שלו ולא משליך אותך לתחילה. אם ה-event התיישן מחלון ה-retention שלך, הדף אומר לך את זה במקום לבחור בשקט כלום. +Events בודדים ניתנים לפנייה, כך שאתה יכול לתת למישהו קישור לרגע אחד ולא "ה-session, בערך שתיים שלישים למטה". העתק את הקישור מכל event, או עקוב אחרי אחד מ-[audit](/he/cloud/audits) finding או שגיאה, והוא session נפתח עם אותו event נבחר וגלול אליו. זה מתקיים גם עבור runs ארוך מאוד: ציר הזמן טוען חלון מוגבל למען הדפדפן שלך, וקישור שמצביע מעבר לחלון זה עדיין מוצא את ה-event שלו ולא משליך אותך לתחילה. אם ה-event התיישן מחלון ה-retention שלך, הדף אומר לך את זה במקום לבחור בשקט כלום. --- @@ -47,12 +47,12 @@ Events בודדים ניתנים לפנייה, כך שאתה יכול לתת ל כל דף dashboard מוגבל לארגון שלך (`//…`). Sessions חי תחת **Observe** בסרגל הצד השמאלי, ליד Events, עם טווח התאריכים, environment, agent, ו-session filters על פני החלק העליון של הרשימה. כל שורה היא קליק אחד מגרף הביצוע המלא שלה. -כדי להפעיל את תגי הציונים וסינון טווח ציונים, חבר מעריך: ראה [Evaluations](/he/agenteye/evaluations). +כדי להפעיל את תגי הציונים וסינון טווח ציונים, חבר מעריך: ראה [Evaluations](/he/cloud/evaluations). --- ## קשור -- [Event stream](/he/agenteye/event-stream): השביל הגולמי, per-step כל session מקופל ממנו. -- [Evaluations](/he/agenteye/evaluations): חבר מעריך כך שכל run יקבל תג ציון שאתה יכול לסנן לפיו. -- [Telemetry](/he/agenteye/telemetry): איך runs מגיעים מ-agent שלך אל sessions אלה. \ No newline at end of file +- [Event stream](/he/cloud/event-stream): השביל הגולמי, per-step כל session מקופל ממנו. +- [Evaluations](/he/cloud/evaluations): חבר מעריך כך שכל run יקבל תג ציון שאתה יכול לסנן לפיו. +- [Telemetry](/he/cloud/performance): איך runs מגיעים מ-agent שלך אל sessions אלה. \ No newline at end of file diff --git a/docs/he/concepts.mdx b/docs/he/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/he/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/he/daemon.mdx b/docs/he/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/he/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/he/dashboard.mdx b/docs/he/dashboard.mdx index 1d962dba..8cc38499 100644 --- a/docs/he/dashboard.mdx +++ b/docs/he/dashboard.mdx @@ -73,7 +73,7 @@ Hermes ו-OpenClaw הם במתחם משתמש ואין להם ספריית עב 5. **Come back better** — שתי כרטיסיות זה לצד זה. שמאל: קבע תזכורת (`3d` / `7d` / `14d` / `30d` בחיר קדנציה; נשמר דרך `/api/auth/reminder` לאחר אימות). ימין: בטל את הצפיפויות failproof — `invite a friend` פותח מודל שלוקח רשימה מופרדת בפסיקים/רווח/שורה חדשה של דוא״ל חברים (מקסימום 10 לשליחה), POSTs אותם ל-`/api/audit/invite`, אשר מעביר ל-`POST /v0/invite` של ה-api-server. ה-api-server שולח דוא״ל אחד לכל נמען מ-`invite@failproof.ai` עם Cc של השולח ו-`Reply-To` מוגדר, כך שהנמען רואה מי הזמין אותו והשולח מקבל עותק בתיבת הדואר שלו. משתמשים אנונימיים מנותבים דרך ה-`AuthDialog` תחילה כך שדוא״ל השולח ידוע לפני שהזמנות יוצאות. זכאות / מילוי הטבות הוא מעקב. -מונע על ידי ריצת `failproofai audit` — ראה [Audit CLI](/he/cli/audit) לעניין סריקת המנוע הבסיסי, הדגלים הנתמכים, ואי-שונות במטמון לכל תמליל. לוח הבקרה מטמן את התוצאה האחרונה ב-`~/.failproofai/audit-dashboard.json` (מצב `0600`, משבצת יחידה, הריצות החדשות משכתבות) כך שביקורים חוזרים מיידיים; **גם המטמון לכל-תמליל וכל-תוצאה דחויים בקריאה ברגע שהם מעבר לתגובת TTL 7 ימים** כך שלוח הבקרה לעולם לא משרת בשקט תוצאה בן שבוע — עבר ה-TTL `/audit` נופל דרך למצב הריק שלו ומעודד ריצה חדשה. לחיצה על `[ re-audit now ]` ליד החלק התחתון של הדוח POSTs `/api/audit/run` עם `noCache: true` — ביקורת חוזרת עוקפת את המטמון לכל-תמליל וסורקת מחדש כל תמליל מאפס ולא משרתת בשקט את התוצאה המטומנת — ולוח הבקרה סוקר `/api/audit/status` ב-1Hz עד שהריצה מסתיימת; רצועת התקדמות ורודה ודבוקה מנעוצה בחלק העליון של התצוגה במהלך הריצה עם טיימר שחלף, והתוצאה הטרייה מחליפה במקום בהצלחה (ללא טעינה מחדש של דף מלא; אי-ביקורת כושלת משאירה את הדוח הקודם שלם). בכשל הרצועה הופכת לאדום עם עותק מפתוח מהסוג `RerunError.kind` (`timeout` / `network` / `post_failed`). מצב ריק (אין מטמון או פג) ומצב אפס-הפעלות (המטמון קיים אך הסריקה לא מצאה תמליל) משטחים בנפרד. +מונע על ידי ריצת `failproofai audit` — ראה [Audit CLI](/he/audit) לעניין סריקת המנוע הבסיסי, הדגלים הנתמכים, ואי-שונות במטמון לכל תמליל. לוח הבקרה מטמן את התוצאה האחרונה ב-`~/.failproofai/audit-dashboard.json` (מצב `0600`, משבצת יחידה, הריצות החדשות משכתבות) כך שביקורים חוזרים מיידיים; **גם המטמון לכל-תמליל וכל-תוצאה דחויים בקריאה ברגע שהם מעבר לתגובת TTL 7 ימים** כך שלוח הבקרה לעולם לא משרת בשקט תוצאה בן שבוע — עבר ה-TTL `/audit` נופל דרך למצב הריק שלו ומעודד ריצה חדשה. לחיצה על `[ re-audit now ]` ליד החלק התחתון של הדוח POSTs `/api/audit/run` עם `noCache: true` — ביקורת חוזרת עוקפת את המטמון לכל-תמליל וסורקת מחדש כל תמליל מאפס ולא משרתת בשקט את התוצאה המטומנת — ולוח הבקרה סוקר `/api/audit/status` ב-1Hz עד שהריצה מסתיימת; רצועת התקדמות ורודה ודבוקה מנעוצה בחלק העליון של התצוגה במהלך הריצה עם טיימר שחלף, והתוצאה הטרייה מחליפה במקום בהצלחה (ללא טעינה מחדש של דף מלא; אי-ביקורת כושלת משאירה את הדוח הקודם שלם). בכשל הרצועה הופכת לאדום עם עותק מפתוח מהסוג `RerunError.kind` (`timeout` / `network` / `post_failed`). מצב ריק (אין מטמון או פג) ומצב אפס-הפעלות (המטמון קיים אך הסריקה לא מצאה תמליל) משטחים בנפרד. ### Policies diff --git a/docs/he/architecture.mdx b/docs/he/how-it-works.mdx similarity index 100% rename from docs/he/architecture.mdx rename to docs/he/how-it-works.mdx diff --git a/docs/he/introduction.mdx b/docs/he/introduction.mdx index b0239605..0e8e507e 100644 --- a/docs/he/introduction.mdx +++ b/docs/he/introduction.mdx @@ -55,4 +55,4 @@ failproofai policies --install # enable policies (or skip — `failproofai` wi failproofai # launch the dashboard ``` -ראה את [Starting](/he/getting-started) guide להסבר המלא. \ No newline at end of file +ראה את [Starting](/he/quickstart) guide להסבר המלא. \ No newline at end of file diff --git a/docs/he/policies.mdx b/docs/he/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/he/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/he/getting-started.mdx b/docs/he/quickstart.mdx similarity index 100% rename from docs/he/getting-started.mdx rename to docs/he/quickstart.mdx diff --git a/docs/he/reference/files.mdx b/docs/he/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/he/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/hi/agent-support.mdx b/docs/hi/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/hi/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/hi/agenteye/cli-and-agents.mdx b/docs/hi/agenteye/cli-and-agents.mdx deleted file mode 100644 index 9554133d..00000000 --- a/docs/hi/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "आपका पूरा Failproof AI Observability डिप्लॉयमेंट, एक कमांड की दूरी पर।" ---- - - -आपका पूरा Failproof AI Observability डिप्लॉयमेंट, एक कमांड की दूरी पर। प्रोडक्शन को चेक करें, API कुंजी जारी करें, या अपने टर्मिनल से बाहर निकले बिना किसी इंसिडेंट को स्वीकार करें, फिर इसे CI में स्क्रिप्ट करें, या एक कोडिंग एजेंट को सादे अंग्रेजी में करने दें। - -```bash -pipx install agenteye -agenteye login --email you@example.com # a 6-digit code lands in your inbox -agenteye --json sessions --since 24h # every agent run from the last day, newest first -``` - -*`agenteye` CLI आपके डैशबोर्ड से बात करता है। यह कलेक्टर से एक अलग टूल है, जो सर्वर को ईवेंट भेजता है।* - -## आपका पूरा डिप्लॉयमेंट, एक कमांड की दूरी पर - -एक त्वरित सवाल का जवाब देने के लिए टैब-हॉपिंग बंद करें। `agenteye` CLI आपके डेटा को पढ़ता है और एक ही बाइनरी से आपके संगठन का प्रबंधन करता है, इसलिए एक चेक जो पहले डैशबोर्ड के माध्यम से क्लिक करने का मतलब था, अब एक पंक्ति बन जाता है जिसे आप दोबारा चला सकते हैं, उपनाम दे सकते हैं, या एक रनबुक में पेस्ट कर सकते हैं। आपको चार सतहें मिलती हैं: - -- **अपना डेटा पढ़ें:** `sessions`, `events`, `evals`, और `errors`, समय, एजेंट और पर्यावरण द्वारा फ़िल्टर किए गए। -- **अपने संगठन का प्रबंधन करें:** `keys`, `users`, `settings`, `alerts`, और `incidents`। -- **विश्लेषण चलाएं:** सहेजे गए SQL के साथ-साथ आपके इवेंट डेटा पर एक ad-hoc `query` रनर। -- **असिस्टेंट से पूछें:** `agent ask` उसी read-only विश्लेषक तक पहुंचता है जिससे आप डैशबोर्ड में चैट करते हैं। - -इसे `pipx` के साथ एक बार इंस्टॉल करें, एक ईमेल की गई 6-अंकीय कोड से साइन इन करें, और आप तैयार हैं। सेशन लगभग एक दिन तक चलता है; जब यह समाप्त हो जाए तो `agenteye login` को दोबारा चलाएं। प्रोडक्शन को स्पॉट-चेक करने, एक कुंजी प्रदान करने, या एक फायरिंग इंसिडेंट को ट्रिएज करने के लिए इसका उपयोग करें, सब कुछ बिना ब्राउज़र खोले: - -```bash -agenteye errors --since 24h --aggregate # what is breaking, grouped by error type -agenteye incidents list --state firing # what is on fire right now -agenteye keys create ci --add events:add # a key that can only push events, secret shown once -``` - -एक आदत जानने के लिए: `--json` जैसे वैश्विक विकल्प कमांड से पहले जाते हैं। `agenteye --json sessions` सही है; `agenteye sessions --json` नहीं है। - -## इसे स्क्रिप्ट करें, इसे CI में वायर करें - -प्रत्येक कमांड `--json` लेता है, और यह सब कुछ बदल देता है। स्वच्छ JSON stdout पर जाता है जबकि मानव स्थिति और चेतावनियां stderr पर जाती हैं, इसलिए एक `--json` कैप्चर सीधे `jq` में पाइप करता है बिना किसी भटकाऊ पंक्ति को छीने। यही वह है जो CLI को आपके लिए एक प्रॉम्प्ट पर और एक कोडिंग एजेंट के आउटपुट को पार्स करने के लिए समान रूप से अच्छा बनाता है: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -यह बिना किसी निरीक्षण के चलाने के लिए बनाया गया है। पुष्टिकरण प्रॉम्प्ट स्वतः-स्किप हो जाते हैं जब कोई टर्मिनल संलग्न नहीं होता है, इसलिए पाइपलाइन में कुछ नहीं रुकता, और प्रत्येक कमांड एक सार्थक निकास कोड लौटाता है: `0` सफलता, `4` लॉगिन नहीं किया गया, `5` एक अनुमति गायब है (संदेश इसे नाम देता है, उदाहरण के लिए `alerts:write`), `3` डैशबोर्ड अप्राप्य। एक स्क्रिप्ट एक `4` पर पुनः-प्रमाणीकृत करने के लिए या एक `5` पर आपको बताने के लिए शाखा कर सकता है कि सही से क्या माँगना है, बजाय अंधे तरीके से विफल होने के। - -## एक कोडिंग एजेंट को सादे अंग्रेजी में इसे चलाने दें - -बेहतर अभी, आपको इन झंडों में से किसी को भी याद नहीं रखना चाहिए। **CLI कौशल** एक छोटा Agent Skill फ़ोल्डर है जिसका नाम `agenteye-cli` है जो Claude Code या Codex जैसे एक कोडिंग एजेंट को सादे-अंग्रेजी अनुरोधों से CLI चलाने के लिए सिखाता है। पूछें "क्या आज कुछ टूट गया है?" और एजेंट कमांड चुनता है, इसे आपके रूप में चलाता है, और गद्य में उत्तर देता है। - -Claude Code के लिए, `agenteye-cli` फ़ोल्डर को `~/.claude/skills/` में ड्रॉप करें और इसे स्वतः-खोजा जाता है। Failproof AI Observability फ़ोल्डर प्रदान करता है; इंस्टॉल करने के लिए कुछ अतिरिक्त नहीं है, क्योंकि यह केवल CLI को चलाता है जिसे आप पहले से ही इंस्टॉल कर चुके हैं। पहले स्वयं लॉगिन करें: कौशल ईमेल-कोड लॉगिन को आपके लिए पूरा नहीं कर सकता। - -क्योंकि एजेंट CLI को आपके रूप में चलाता है, यह सब कुछ कर सकता है जो आपकी लॉगिन अनुमति देता है, पढ़ता है और लिखता है: कुंजियां बनाएं, सेटिंग्स बदलें, इंसिडेंट्स को हल करें। CLI का "क्या आप निश्चित हैं?" प्रॉम्प्ट एजेंट के लिए फायर नहीं करता है, इसलिए कौशल को सटीक कमांड बताने और किसी भी परिवर्तन से पहले आपकी OK की प्रतीक्षा करने के लिए लिखा गया है। आप पुष्टिकरण चरण हैं। - -```text -you Why did session run-001 fail? - -agent Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -पढ़ता तुरंत रहता है, और हर लेखन आपके लिए रुकता है: - -```text -you Give CI a key that can only push events. - -agent This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you yes - -agent Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## संबंधित - -- [CLI संदर्भ](/hi/agenteye/cli): प्रत्येक कमांड, ध्वज, और JSON आकार। -- [एजेंट्स के लिए CLI व्यंजन](/hi/agenteye/cli-recipes): copy-paste `jq` पैटर्न और निकास-कोड हैंडलिंग। -- [CLI एजेंट कौशल](/hi/agenteye/cli-skill): `agenteye-cli` कौशल को इंस्टॉल और चलाएं। -- [AI सहायक](/hi/agenteye/assistant): डैशबोर्ड विश्लेषक जिससे `agent ask` बात करता है। \ No newline at end of file diff --git a/docs/hi/agenteye/cli-skill.mdx b/docs/hi/agenteye/cli-skill.mdx deleted file mode 100644 index 0d0adf88..00000000 --- a/docs/hi/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Failproof AI Observability CLI Agent Skill" -description: "अपने कोडिंग एजेंट से पूछें \"क्या आज कुछ टूटा है?\" और इसे अपने लाइव Failproof AI Observability डेटा से जवाब दें, कोई कमांड याद रखने की जरूरत नहीं।" ---- - - -अपने कोडिंग एजेंट से *"क्या आज कुछ टूटा है?"* पूछें और इसे अपने लाइव Failproof AI Observability डेटा से जवाब दें, कोई कमांड याद रखने की जरूरत नहीं। **Failproof AI Observability CLI स्किल** (`agenteye-cli`) एक *Agent Skill* है: निर्देशों का एक छोटा फोल्डर जिसे Claude Code या Codex जैसा कोडिंग एजेंट मांग पर लोड करता है। यह एजेंट को [`agenteye` CLI](/hi/agenteye/cli) के माध्यम से आपके Observability डिप्लॉयमेंट को संचालित करना सिखाता है साधारण अंग्रेजी अनुरोधों से जैसे *"CI को एक कुंजी दें जो केवल इवेंट पुश कर सके"* या *"फायरिंग इंसिडेंट को स्वीकृति दें और इसे मुझे असाइन करें।"* - -यह **नहीं** एक सेवा या अलग बाइनरी है; तैनात करने के लिए कुछ भी नहीं है। यह उस CLI के ऊपर काम करता है जिसे आप पहले से इंस्टॉल कर चुके हैं: एजेंट `agenteye --json …` को शेल करता है, स्वच्छ JSON को पार्स करता है, और आपको गद्य में जवाब देता है। यह जो कुछ भी कर सकता है, आप इसे स्वयं कर सकते हैं। - ---- - -## यह अन्य Failproof AI Observability इंटरफेस से कैसे संबंधित है - -Failproof AI Observability आपको समान डेटा और नियंत्रण तक पहुंचने के चार तरीके देता है। वे एक दूसरे की पूरक हैं: - -| इंटरफेस | यह क्या है | यह कहां चलता है | इसे कब चुनें | -|---|---|---|---| -| **[CLI](/hi/agenteye/cli)** | `agenteye` के लिए कमांड/फ्लैग संदर्भ | आपका टर्मिनल | जब आप एक विशिष्ट कमांड चलाना या स्क्रिप्ट करना चाहते हैं | -| **[CLI recipes](/hi/agenteye/cli-recipes)** | कॉपी-पेस्ट `jq`/पाइपलाइन पैटर्न | आपका टर्मिनल / स्क्रिप्ट | जब आप CLI को ऑटोमेशन में वायर कर रहे हैं | -| **CLI स्किल** (यह दस्तावेज़) | CLI पर एक प्राकृतिक भाषा का प्रवेश द्वार | आपका कोडिंग एजेंट, आपके वर्कस्टेशन पर | जब आप बस पूछना चाहते हैं और एजेंट को कमांड चुनने दें | -| **[Evaluator स्किल](/hi/agenteye/evaluator-skill)** | एक सहायक स्किल जो आपकी स्कोरिंग सेवा डिज़ाइन और बनाती है | आपका कोडिंग एजेंट, आपके वर्कस्टेशन पर | जब आप eval स्कोर पढ़ने के बजाय *उत्पन्न* करना चाहते हैं | -| **[Python SDK स्किल](/hi/agenteye/python-sdk-skill)** | एक सहायक स्किल जो आपके एजेंट को सभी टेलीमेट्री उत्सर्जित करने के लिए सक्षम करती है | आपका कोडिंग एजेंट, आपके वर्कस्टेशन पर | जब आप अपने एजेंट को यह स्किल जो इवेंट पढ़ती है उन्हें *उत्पन्न* करना चाहते हैं | -| **[In-dashboard AI सहायक](/hi/agenteye/assistant)** | डैशबोर्ड में एम्बेड किया गया एक चैट | सर्वर-साइड (डैशबोर्ड में) | जब आप अपने डेटा पर इन-डैशबोर्ड प्रश्नोत्तर चाहते हैं | - -स्किल के अपने कोई विशेषाधिकार नहीं हैं; यह केवल आपके शब्दों को CLI कॉल में बदलता है जो आपके रूप में चलते हैं: - -```mermaid -flowchart TD - YOU["आप: 'फायरिंग इंसिडेंट को स्वीकृति दें'"] --> AGENT["कोडिंग एजेंट (Claude Code / Codex)
agenteye-cli स्किल लोड करता है"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|आपका प्रमाणित CLI सेशन| API["Observability डैशबोर्ड API"] -``` - -### बनाम in-dashboard AI सहायक: एक महत्वपूर्ण अंतर - -ये दो बिल्कुल अलग उपकरण हैं जिनके अलग-अलग प्रभाव हैं: - -- **in-dashboard AI सहायक** ([AI सहायक](/hi/agenteye/assistant)) डैशबोर्ड में एम्बेड किया गया एक चैट है, जो एजेंट सेवा द्वारा समर्थित है। यह **केवल-पढ़ने योग्य और अनुमोदन-गेटेड लेखन** है: यह सहेजे गए क्वेरी और डैशबोर्ड का ड्राफ्ट कर सकता है, लेकिन प्रत्येक लिखने से आपकी स्पष्ट क्लिक-अनुमोदन के लिए रुकता है, और यह कभी हटाता नहीं है। यह `agent:use` अनुमति द्वारा गेट किया जाता है और केवल उस संगठन के लिए डेटा देखता है जिसे आप देख रहे हैं। -- **CLI स्किल** आपके वर्कस्टेशन पर आपके कोडिंग एजेंट के अंदर चलती है और `agenteye` CLI को **आपके रूप में** चलाती है। यह CLI की **पूर्ण सतह, म्यूटेशन सहित** कर सकती है (API कुंजी बनाएं/घुमाएं/अक्षम करें, संगठन सेटिंग्स बदलें, इंसिडेंट हल करें, सहेजे गए क्वेरी हटाएं), केवल आपकी CLI लॉगिन की अनुमतियों द्वारा सीमित। इसे बिल्कुल उसी तरह व्यवहार करें जैसे आप उन कमांडों को हाथ से चलाना चाहते हैं। - ---- - -## आवश्यकताएं - -1. **`agenteye` CLI इंस्टॉल** और `PATH` पर (देखें [CLI](/hi/agenteye/cli) संदर्भ: `pipx install agenteye`)। -2. आपका **डैशबोर्ड URL सेट** (`AGENTEYE_DASHBOARD_URL`, या एजेंट `--base-url` पास करता है)। -3. एक **लॉगिन किया गया सेशन**: पहले स्वयं `agenteye login` चलाएं। स्किल **नहीं** कर सकता ईमेल किए गए एकबारी-कोड लॉगिन को आपके लिए पूरा करना; यह आपको `agenteye login` चलाने के लिए कहेगा यदि सेशन अनुपस्थित या समाप्त है (CLI एक्सिट कोड `4`)। - ---- - -## इसे कहां से प्राप्त करें - -स्किल Failproof AI के सार्वजनिक स्किल संग्रह में प्रकाशित है: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -इसके बारे में कुछ भी गेटेड नहीं है — रिपॉजिटरी सार्वजनिक है और स्किल को अपने क्रेडेंशियल की आवश्यकता नहीं है, क्योंकि यह केवल **सार्वजनिक** `agenteye` CLI को आपके डैशबोर्ड के विरुद्ध चलाता है, सेशन का उपयोग करते हुए *आप* लॉगिन किए हैं। आपको किसी से इसके लिए पूछना नहीं है। - -नोट करें कि यह अपने स्वयं के फोल्डर के रूप में शिप करता है और `pipx install agenteye` पैकेज के अंदर **नहीं** है, इसलिए इसे वहां न ढूंढें। - -## स्किल स्थापित करना - -सबसे तेज़ रास्ता [`skills`](https://skills.sh) CLI है, जो फोल्डर लाता है और इसे वहां डालता है जहां आपका एजेंट देखता है: - -```bash -# Claude Code, केवल यह प्रोजेक्ट -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# हर प्रोजेक्ट (~/.claude/skills/ में इंस्टॉल करता है) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# इसके बजाय Codex -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -फिर इसे किसी अन्य स्किल की तरह प्रबंधित करें: - -```bash -npx skills list -a claude-code # क्या इंस्टॉल है -npx skills update agenteye-cli # नवीनतम संस्करण लाएं -npx skills remove agenteye-cli # इसे निकालें -``` - -हाथ से इंस्टॉल करना पसंद करते हैं? एक Agent Skill केवल एक फोल्डर है जिसमें एक `SKILL.md` है (साथ ही वैकल्पिक संदर्भ), इसलिए इसे कॉपी करना भी काम करता है: - -- **Claude Code**: `agenteye-cli/` फोल्डर को `~/.claude/skills/` (हर प्रोजेक्ट) या `/.claude/skills/` (केवल वह रेपो) में रखें। Claude Code इसे स्वचालित रूप से खोजता है — `/skills` सूची के साथ सत्यापित करें, या बस एक प्रश्न पूछें जो इसके विवरण से मेल खाता हो। -- **Codex (OpenAI)**: Codex समान `SKILL.md` को पढ़ता है। बंडल किया गया `agents/openai.yaml` `allow_implicit_invocation: true` सेट करता है, इसलिए Codex स्वचालित रूप से कार्य से मेल खाने पर स्किल चुनता है; अन्यथा इसे `$agenteye-cli` के रूप में स्पष्ट रूप से आह्वान करें। - ---- - -## सुरक्षा: म्यूटेशन जब एजेंट CLI चलाता है तो प्रॉम्प्ट नहीं करता - -> **चेतावनी:** एजेंट को परिवर्तन करने देने से पहले यह पढ़ें। - -`agenteye` CLI आमतौर पर विनाशकारी कार्य से पहले *"क्या आप सुनिश्चित हैं?"* पूछता है। यह **स्वचालित रूप से पुष्टि को छोड़ देता है जब यह टर्मिनल से जुड़ा नहीं होता है (जो बिल्कुल वैसे ही होता है जैसे एक कोडिंग एजेंट इसे चलाता है), और `--json` भी इसे छोड़ देता है।** तो सुरक्षा प्रॉम्प्ट एजेंट के लिए **नहीं** चलेगा। - -स्किल इसे मुआवजे के लिए लिखी गई है: इसे सटीक कमांड बताने के लिए निर्देश दिया जाता है जो यह चलाएगा और किसी भी स्थिति परिवर्तन से पहले आपकी स्पष्ट **OK** प्राप्त करना होगा। उस अनुशासन को रखें। जब आप Failproof AI Observability को एजेंट के माध्यम से चलाते हैं, *आप* पुष्टि चरण हैं। स्थिति-परिवर्तन कमांड देखने के लिए: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- लिखने वाली `incidents` उप-कमांड: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -**Observe** के अंतर्गत सब कुछ (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) केवल-पढ़ने योग्य है और कुछ नहीं बदलता। - -क्योंकि एजेंट **आपके रूप में** कार्य करता है, यह केवल वही कर सकता है जो आपकी लॉगिन को अनुमति दी गई है; अनुमतियां **प्रति संगठन** हल होती हैं (देखें [API कुंजी](/hi/agenteye/api-keys))। एक कमांड जिसके लिए आपके पास अनुमति नहीं है exit code `5` को सटीक अनुमति के साथ लौटाता है, इसलिए एजेंट आपको बता सकता है कि आप प्रशासक से क्या पूछ सकते हैं बजाय अस्पष्ट रूप से विफल होने के। - ---- - -## आप इससे क्या पूछ सकते हैं - -यहां एक वास्तविक राउंड-ट्रिप दिखता है, एक पढ़ना, फिर एक परिवर्तन जो आपकी OK की प्रतीक्षा करता है: - -```text -आप ▸ क्या पिछले दिन में कुछ टूटा है? - -एजेंट ▸ चल रहा है: agenteye --json errors --since 24h --aggregate - पिछले 24 घंटे में 3 त्रुटि प्रकार। सबसे बड़ा TimeoutError - (18 हिट, सभी checkout-agent पर), फिर RateLimited (5)। - क्या आप TimeoutError के लिए विफल सेशन चाहते हैं? - -आप ▸ CI को एक कुंजी दें जो केवल इवेंट पुश कर सके। - -एजेंट ▸ यह एक API कुंजी बनाता है। मैं चलाऊंगा: - agenteye keys create ci --add events:add - क्या आगे बढ़ना OK है? - -आप ▸ हां - -एजेंट ▸ हो गया। कुंजी "ci" केवल events:add के साथ बनाई गई। - गुप्त केवल एक बार दिखाया जाता है, इसलिए इसे अभी स्टोर करें। मैं इसे फिर से प्रिंट नहीं कर सकता। -``` - -स्किल प्रत्येक सादे अंग्रेजी इरादे को सही `agenteye` कमांड पर मैप करती है, पहले मान्य मान खोजती है (`list `, `whoami`) इसलिए यह अनुमान नहीं लगाता, और किसी भी परिवर्तन से पहले सटीक कमांड बताता है। अधिक उदाहरण: - -- *"क्या पिछले 24 घंटों में कुछ टूटा / विफल है?"* → `errors --since 24h --aggregate`, फिर एक विस्तृतीकरण। -- *"सेशन `run-001` क्यों विफल रहा?"* → `events --session-id run-001 --all` + `evals --session-id run-001`। -- *"इस हफ्ते गुणवत्ता कैसी है?"* → `evals --aggregate --since 7d`, फिर कम-स्कोरिंग रन में ड्रिल करें। -- *"CI को एक कुंजी दें जो केवल इवेंट पुश कर सके।"* → `keys create ci --add events:add` (यह कमांड बताता है, फिर इसे बनाता है और एकबारी गुप्त को कैप्चर करता है)। -- *"किसके पास पहुंच है? Dana को केवल-पढ़ने योग्य बनाएं।"* → `users list` → `users update dana@… --permission-set read-only` (आपकी पुष्टि के बाद)। -- *"फायरिंग इंसिडेंट को स्वीकृति दें और इसे मुझे असाइन करें।"* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`। - -सटीक कमांड, फ्लैग और JSON आकार के लिए, [CLI](/hi/agenteye/cli) संदर्भ और [एजेंट के लिए CLI recipes](/hi/agenteye/cli-recipes) देखें। - ---- - -## अगले कदम - -- **[CLI](/hi/agenteye/cli)**: `agenteye` के लिए पूर्ण कमांड और फ्लैग संदर्भ। -- **[एजेंट के लिए CLI recipes](/hi/agenteye/cli-recipes)**: कॉपी-पेस्ट `jq` पैटर्न और exit-code हैंडलिंग। -- **[Evaluator एजेंट स्किल](/hi/agenteye/evaluator-skill)**: सहायक स्किल, evaluator बनाने के लिए जिसके स्कोर `agenteye evals` पढ़ता है। -- **[Python SDK एजेंट स्किल](/hi/agenteye/python-sdk-skill)**: सहायक स्किल, एजेंट को सक्षम करने के लिए जो टेलीमेट्री उत्सर्जित करता है `agenteye` पढ़ता है। -- **[AI सहायक](/hi/agenteye/assistant)**: in-dashboard सहायक (इस टर्मिनल स्किल के साथ भ्रमित न करें)। -- **[API कुंजी](/hi/agenteye/api-keys)**: प्रति-संगठन अनुमति मॉडल जो स्किल को क्या कर सकता है इसे बांधता है। \ No newline at end of file diff --git a/docs/hi/agenteye/codex-capture.mdx b/docs/hi/agenteye/codex-capture.mdx deleted file mode 100644 index 575b58be..00000000 --- a/docs/hi/agenteye/codex-capture.mdx +++ /dev/null @@ -1,56 +0,0 @@ ---- ---- -title: "Codex सत्र कैप्चर" -description: "अपनी टीम के स्थानीय OpenAI Codex सत्रों को AgentEye में सामान्य सत्र और ईवेंट के रूप में कैप्चर करें — Codex चलाने के तरीके में कोई बदलाव नहीं।" ---- - -आपके इंजीनियर पहले से ही हर दिन OpenAI Codex चलाते हैं। Codex सत्र कैप्चर उन कोडिंग सत्रों को AgentEye में सामान्य सत्र और ईवेंट के रूप में लाता है, ताकि आप उन्हें खोज सकें, दोबारा चला सकें, और आप जो अन्य सब कुछ देखते हैं उसके साथ उनका मूल्यांकन कर सकें। यह [Python SDK](/hi/agenteye/python-sdk) को पूरक करता है: SDK आपके द्वारा लिखे गए एजेंटों को प्रस्तुत करता है, जबकि यह आपकी टीम द्वारा पहले से किए जा रहे Codex कार्य को कैप्चर करता है — इसे चलाने के तरीके में कोई बदलाव नहीं। - -एक छोटा बैकग्राउंड कलेक्टर Codex के स्थानीय सत्र प्रतिलेखन को पढ़ता है क्योंकि वे लिखे जाते हैं और उन्हें AgentEye को भेजता है। एक मशीन प्रति कलेक्टर एक बार में हर स्थानीय Codex सतह को कैप्चर करता है — प्रति-सतह सेटअप की कोई आवश्यकता नहीं है। - -एक ही कलेक्टर अन्य एजेंटों को भी कैप्चर करता है — [OpenClaw](/hi/agenteye/openclaw-capture) और [Hermes](/hi/agenteye/hermes-capture) देखें। आप जो भी चलाते हैं उसे सक्षम करें; एक एकल कलेक्टर एक साथ कई को कैप्चर कर सकता है। - ---- - -## यह क्या कैप्चर करता है - -हर Codex सतह जो **स्थानीय रूप से** चलती है, डिस्क पर समान सत्र प्रतिलेखन तैयार करती है, और कलेक्टर उन सभी को उठाता है: - -- Codex **CLI** और `codex exec` -- **VS Code / IDE एक्सटेंशन** -- **डेस्कटॉप ऐप**, जब यह स्थानीय रूप से एक सत्र चलाता है - -प्रत्येक Codex सत्र AgentEye [सत्र](/hi/agenteye/sessions) बन जाता है; इसके उपयोगकर्ता और सहायक संदेश, तर्क, उपकरण कॉल, उपकरण परिणाम, और टोकन उपयोग मिलान करने वाले [ईवेंट](/hi/agenteye/event-stream) बन जाते हैं। जिस सतह से प्रत्येक सत्र आया था (CLI, IDE, या डेस्कटॉप) रिकॉर्ड किया जाता है, ताकि आप उन्हें अलग बता सकें। - -> **क्लाउड सत्र कैप्चर नहीं किए जाते हैं।** डेस्कटॉप ऐप तेजी से Codex क्लाउड में सत्र चलाता है और मशीन पर केवल उनके मेटाडेटा को रखता है — पढ़ने के लिए कोई स्थानीय प्रतिलेखन नहीं है। केवल स्थानीय रूप से निष्पादित सत्र कैप्चर किए जाते हैं। - ---- - -## इसे चालू करें - -कैप्चर तब तक बंद रहता है जब तक आप इसे सक्षम न करें। `events:add` अनुमति वाली API कुंजी के साथ कलेक्टर को इंस्टॉल करें ([API कुंजियां](/hi/agenteye/api-keys) देखें), और Codex कैप्चर को चालू करें: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -यह कलेक्टर को इंस्टॉल करता है, इसे एक बैकग्राउंड सेवा के रूप में पंजीकृत करता है, और कैप्चर करना शुरू करता है। पुष्टि करें कि यह चल रहा है: - -```bash -agenteye-collector health -``` - -पहली बार चलने पर, आपके मौजूदा Codex सत्रों को एक बार भर दिया जाता है और नई गतिविधि फिर सेकंड के भीतर स्ट्रीम होती है। Codex की अपनी फाइलें केवल पढ़ी जाती हैं — कभी भी संशोधित, स्थानांतरित, या हटाई नहीं जाती — और प्रत्येक सत्र बिल्कुल एक बार भेजा जाता है, यहां तक कि पुनरारंभ भी। - ---- - -## यह कहाँ दिखाई देता है - -कैप्चर किए गए सत्र **सत्र** में दिखाई देते हैं, और उनके ईवेंट **ईवेंट** स्ट्रीम में, किसी अन्य एजेंट की तरह ही जिसे आप देखते हैं — इसलिए [सत्र पुनरावृत्ति](/hi/agenteye/sessions), [खोज](/hi/agenteye/queries), [मूल्यांकन](/hi/agenteye/evaluations), और [सतर्कताएं](/hi/agenteye/alerts) सभी उन पर काम करती हैं। Codex एजेंट के अनुसार फ़िल्टर करें उन्हें अपने आप से देखने के लिए। - ---- - -## गोपनीयता - -Codex प्रतिलेखन में पूरा सत्र होता है — कमांड आउटपुट, फाइल सामग्री, और कुछ भी Codex ने पढ़ा या लिखा सहित — और इसमें रहस्य हो सकते हैं। कैप्चर किए गए सत्र यथावत भेजे जाते हैं, इसलिए केवल उन मशीनों और टीमों पर कैप्चर सक्षम करें जहां उस सामग्री को AgentEye में केंद्रीकृत करना उपयुक्त है, और कलेक्टर को केवल `events:add` के लिए सीमित एक कुंजी दें। [सुरक्षा](/hi/agenteye/security) देखें कि आपके डेटा को कैसे अलग रखा जाता है। \ No newline at end of file diff --git a/docs/hi/agenteye/concepts.mdx b/docs/hi/agenteye/concepts.mdx deleted file mode 100644 index 88bc9b89..00000000 --- a/docs/hi/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "अवधारणाएं" -description: "Failproof AI Observability के पीछे की शब्दावली — events, sessions, evaluations, audits, findings, और incidents — एक जगह परिभाषित।" ---- - - -यह पेज Failproof AI Observability द्वारा उपयोग की जाने वाली शब्दावली को परिभाषित करता है। यदि किसी अन्य गाइड में कोई शब्द अपरिचित है, तो यह यहाँ परिभाषित है। आपको इसे अंत तक पढ़ने की आवश्यकता नहीं है: इसे स्किम करें, या जब आप कोई शब्द स्पष्ट करना चाहते हैं तो वापस जाएं। - ---- - -## डेटा मॉडल - -**Event** -डेटा की सबसे छोटी इकाई। एक event आपके agent द्वारा उठाया गया एक एकल कदम रिकॉर्ड करता है: एक `tool_use`, एक `model_request`, एक `hook_completed`, एक `error`, और इसी तरह। आपका agent [Python SDK](/hi/agenteye/python-sdk) के माध्यम से events को emit करता है; वे **Events** पेज पर लाइव दिखाई देते हैं। - -**Session** -एक agent run, जिसे एक `session_id` द्वारा चिन्हित किया जाता है। एक session वह सभी events हैं जो उस id को साझा करते हैं, **Sessions** पेज पर एक एकल row में rolled up हैं और इसके detail page पर एक execution graph के रूप में खींचे गए हैं। एक session आमतौर पर `agent_start` से शुरू होता है और `agent_end` के साथ समाप्त होता है। - -**Agent** -एक run के अंदर एक नामित actor, जिसे एक `agent_id` द्वारा चिन्हित किया जाता है। एक run में कई agents शामिल हो सकते हैं: एक planner जो एक summarizer sub-agent को spawn करता है, उदाहरण के लिए। Sub-agents एक `parent_id` रखते हैं, जो Failproof AI Observability को execution graph में उन्हें अपनी lanes पर खींचने देता है। - -**Environment** -एक label जहाँ run हुआ: `production`, `staging`, `dev`। आप इसे SDK कॉन्फ़िगर करते समय एक बार सेट करते हैं। लगभग हर dashboard पेज environment के द्वारा फ़िल्टर कर सकता है। - -**Context-window fill** -एक model के context window का वह प्रतिशत जो एक response ने consume किया। Failproof AI Observability इसे उन models के लिए `model_response` events पर stamp करता है जिन्हें यह पहचानता है, इसलिए prompt growth और impending compaction event stream में सही दिखाई देते हैं। - ---- - -## गुणवत्ता - -**Evaluation** -एक finished session के लिए एक गुणवत्ता score, जो आप चलाने वाली एक scoring service द्वारा produced। Evaluations opt-in हैं: जब तक आप एक evaluator को connect नहीं करते, sessions रिकॉर्ड किए जाते हैं लेकिन scored नहीं होते। प्रत्येक evaluation कई named scores ले सकता है (उदाहरण के लिए `helpfulness`, `factuality`, `tool_efficiency`), प्रत्येक एक संक्षिप्त reasoning note के साथ। [Evaluation suite](/hi/agenteye/evaluation-suite) देखें। - -**Score key** -एक dimension का नाम जो एक evaluator रिपोर्ट करता है, जैसे `helpfulness`। Alerts और audits समय के साथ एक specific score key को देख सकते हैं। - -**Evaluator** -आपकी scoring service। Failproof AI Observability एक finished run का transcript उसे POST करता है और यह जो scores return करता है उन्हें store करता है। यह एक default evaluator ship नहीं करता है; scoring logic आपका है। - ---- - -## failures को खोजना और fix करना - -**Hook** -एक guardrail या side-effect जो आपका agent framework एक step के चारों ओर चलाता है: एक content-safety check, PII redaction, एक budget guard। Hooks `hook_triggered` / `hook_completed` events को एक `outcome` (allow, deny, modify) के साथ emit करते हैं, और अपना स्वयं का observe page प्राप्त करते हैं। - -**Alert rule** -एक rule जो तब fires जब एक metric आपके द्वारा सेट की गई threshold को cross करता है: error rate, p95 latency, token cost, या एक evaluator score। जब एक rule fires, यह एक incident खोलता है और आपके चुने हुए channels (email, Slack, webhook, in-dashboard) को notify करता है। [Alerts](/hi/agenteye/alerts) देखें। - -**Incident** -एक open issue जो तब created होता है जब एक alert rule fires। Incidents के पास एक lifecycle (acknowledge, assign, resolve) है और एक activity timeline है जो हर action को रिकॉर्ड करता है। आप एक को manually भी खोल सकते हैं। - -**Audit** -एक recurring investigation (hourly to weekly) जो आपके logs को *across* sessions में mine करता है failure patterns के लिए जिनके लिए आपने एक rule नहीं लिखा है: error clusters, low scores, latency outliers, tool-call loops, और runs जो कभी finished नहीं हुए। जहाँ एक alert एक metric को देखता है जिसके बारे में आप पहले से जानते हैं, एक audit आपको बताता है कि आगे क्या देखना है। [Audits](/hi/agenteye/audits) देखें। - -**Finding** -एक audit run से एक ranked, evidence-backed result। एक finding एक pattern का नाम देता है, इसके पीछे के exact sessions को link करता है, और एक triage lifecycle (acknowledge, resolve, mute, dismiss) रखता है। Failproof AI Observability findings को run-over-run deduplicate करता है इसलिए एक known pattern update होता है बजाय इसके कि pile up हो। - -**The AI assistant** -in-dashboard chat जो आपके agents के बारे में plain English में, आपके स्वयं के data के ऊपर सवालों के जवाब देता है। यह default रूप से read-only है; कुछ भी जो यह create करता है (एक saved query, एक dashboard) approval-gated है, और यह कभी delete नहीं कर सकता। [AI assistant](/hi/agenteye/assistant) देखें। - ---- - -## इसे चलाना - -**Organization (tenant)** -एक isolated workspace। एक Failproof AI Observability instance कई organizations को host कर सकता है, प्रत्येक के साथ अपने स्वयं के users, keys, और data। हर dashboard URL आपके org slug (`//…`) के अंतर्गत scoped है। - -**Collector** -`agenteye-collector`, lightweight daemon जो प्रत्येक agent machine पर runs करता है, SDK द्वारा disk में लिखे गए events को batch करता है, और उन्हें server को ship करता है। - -**API key** -एक scoped token जो एक client को server के साथ authenticate करता है। Keys में granular permissions होते हैं (उदाहरण के लिए `events:add` collector के लिए, read-only scopes एक dashboard key के लिए)। [API keys](/hi/agenteye/api-keys) देखें। - -**Server** -ingest और API service। यह events को ingest करता है, operational state को आपके databases में store करता है, और dashboard और CLI को serve करता है। - -**Dashboard** -web UI। हर page एक organization के लिए scoped है और server के API के माध्यम से पढ़ता है। - ---- - -## अगले कदम - -- [Overview](/hi/agenteye/overview): ये pieces कैसे एक साथ fit होते हैं। -- [Observability](/hi/agenteye/observability): observe surfaces (Events, Sessions, Models, Tools, Hooks, Errors)। \ No newline at end of file diff --git a/docs/hi/agenteye/evaluator-skill.mdx b/docs/hi/agenteye/evaluator-skill.mdx deleted file mode 100644 index 8192b1d2..00000000 --- a/docs/hi/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,171 +0,0 @@ ---- ---- -title: "Failproof AI Observability Evaluator Agent Skill" -description: "Go from \"I think our agent is sometimes bad\" to a deployed scoring service, with your coding agent doing both the deciding and the building." ---- - -*"मुझे लगता है हमारा agent कभी-कभी खराब है"* से लेकर deployed scoring service तक जाएं, आपके coding agent के साथ both the deciding और building दोनों कर रहे हों। **Failproof AI Observability evaluator skill** (`agenteye-evaluator`) एक *Agent Skill* है: instructions का एक छोटा folder जो एक coding agent जैसे Claude Code या Codex on demand load करता है। यह agent को सिखाता है कि कौन से quality dimensions आपके *agent* के लिए tracking के लायक हैं, फिर [evaluator service](/hi/agenteye/evaluation-suite) को write, test, और deploy करते हैं जो उन्हें score करता है। - -यह एक **hosted scorer नहीं है**, न ही एक registry जहां आप upload करते हैं, न ही एक plugin system। आपका evaluator आपका अपना HTTP service रहता है आपके अपने infrastructure पर, बिल्कुल जैसा [Evaluation suite](/hi/agenteye/evaluation-suite) guide में described है। skill केवल आपके agent को इसे अच्छे तरीके से बनाना सिखाती है, इसलिए जो कुछ भी यह करता है, आप खुद कर सकते हैं same code लिखकर। - ---- - -## कठिन हिस्सा है कि क्या score करें यह तय करना - -SDK surface छोटा है — एक decorator और two models — और एक agent इसे [contract](/hi/agenteye/evaluation-suite#http-contract) से ही लिख सकता है। यहीं से evaluators fail नहीं होते। वे fail होते हैं क्योंकि वे गलत चीज़ को score करते हैं, और एक evaluator जो गलत चीज़ को score करता है वह कोई भी नहीं होने से बदतर है: यह एक dashboard produce करता है जिसे सब ignore करना सीख जाते हैं। - -तो skill का ज़्यादातर हिस्सा code exist करने से पहले का है। इसमें agent आपसे interview करता है (*"एक run describe करें जो अच्छा गया; अब एक जो बुरा गया"*), फिर आपके real sessions को [`agenteye` CLI](/hi/agenteye/cli) के through pull करता है और उन्हें end to end पढ़ता है। ये दोनों halves आमतौर पर disagree करते हैं, और gap ही वह point है: जो आप measure करना चाहते हैं बनाम जो आपके transcripts actually support कर सकते हैं। एक dimension तभी survive करता है जब वह events से **computable** हो और **discriminating** हो — अगर यह आपके good run और bad run दोनों पर 0.9 score करता है, तो यह कुछ नहीं सिखाता और cut हो जाता है। - -जो वापस आता है वह 2-4 dimensions का एक proposal है reasoning के साथ, जिसे आप कोई भी line लिखने से पहले sign off करते हैं। - -```mermaid -flowchart TD - YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent (Claude Code / Codex)
loads the agenteye-evaluator skill"] - AGENT -->|"interview: what does good vs bad look like?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["your real sessions
what actually happens"] - DATA --> DIMS["2-4 dimensions, you sign off"] - DIMS --> SVC["your evaluator service
agenteye-evaluator SDK"] - SVC --> SCORES["scores land in the dashboard
and agenteye evals"] -``` - ---- - -## यह दूसरे evaluation pieces से कैसे संबंधित है - -चार docs scoring को cover करते हैं, और वे order में एक-दूसरे को hand off करते हैं: - -| Page | यह क्या है | इसे तब use करें जब | -|---|---|---| -| **[Evaluations](/hi/agenteye/evaluations)** | Feature: sessions grid पर scores, dashboards, re-evaluate | आप जानना चाहते हैं कि automatic scoring आपको क्या देता है | -| **[Evaluation suite](/hi/agenteye/evaluation-suite)** | HTTP contract, SDK, server env vars | आप evaluator को खुद implement या debug कर रहे हैं | -| **Evaluator skill** (यह doc) | Scorer को design *और* build करने का एक natural-language front door | आप "I want evals" से एक running service तक जाना चाहते हैं | -| **[CLI skill](/hi/agenteye/cli-skill)** | `agenteye` CLI का एक natural-language front door | आप scores को पढ़ना चाहते हैं जो आपके पास पहले से हैं | -| **[Python SDK skill](/hi/agenteye/python-sdk-skill)** | अपने agent को instrument करने का एक natural-language front door | आपका agent sessions emit नहीं कर रहा है — score करने के लिए कुछ नहीं है | - -### CLI skill के मुकाबले: build बनाम read - -दोनों skills intentionally non-overlapping हैं, और दोनों को install करना normal setup है — agent यह तय करता है कि आप क्या पूछते हैं इसके आधार पर: - -- **`agenteye-evaluator`** (यह doc) उस चीज़ को build करता है जो scores *produce* करता है। इसका job तब खत्म होता है जब scores पहली बार land करते हैं। -- **[`agenteye-cli`](/hi/agenteye/cli-skill)** scores को पढ़ता है जो पहले से exist करते हैं (`agenteye evals`)। *"क्या quality इस हफ्ते drop हुई?"* इसका सवाल है, इस skill का नहीं। - ---- - -## Prerequisites - -1. **`agenteye` CLI installed और logged in** (`pipx install agenteye`, फिर `agenteye login`)। Skill इसे दो बार use करती है: real sessions को pull करने के लिए जिसके against यह design करती है, और यह confirm करने के लिए कि आपके scores end में land हुए। आपके login को `events:read` की ज़रूरत है, प्लस उस final check के लिए `evaluations:read`। CLI skill की तरह, यह **नहीं** कर सकता emailed one-time-code login को complete करना आपके लिए। -2. **Evaluator के लिए कहीं रहने के लिए जगह।** यह एक image में build हो जाता है और एक long-running service के रूप में run होता है, तो इसे एक real repo की ज़रूरत है, scratch file नहीं। Evaluators अक्सर अपने अपने repo में रहते हैं, agent से अलग जिसे scored किया जा रहा है — skill एक existing को look करती है और नए को scaffold करने से पहले पूछती है। -3. **`agenteye-evaluator` SDK wheel** — अपने agent के `pip` commands type करना शुरू करने से पहले अगला section पढ़ें। - ---- - -## इसे कहां से प्राप्त करें - -Skill Failproof AI के public skills collection में publish है: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -Repository public है और skill को अपने credential की ज़रूरत नहीं है — यह केवल `agenteye` CLI को drive करता है session के साथ *जिसे* आप logged in थे, और *आपके* repo में code लिखता है। ध्यान दें कि यह अपने folder के रूप में ship होता है और `pipx install agenteye` package के inside **नहीं** है, तो इसे वहां न ढूंढें। - -## Skill को install करना - -सबसे तेज़ path [`skills`](https://skills.sh) CLI है, जो folder को fetch करता है और वहां drop करता है जहां आपका agent look करता है: - -```bash -# Claude Code, यह project केवल -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# हर project (installs to ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codex इसकी जगह -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -फिर इसे किसी दूसरे skill की तरह manage करें: - -```bash -npx skills list -a claude-code # क्या installed है -npx skills update agenteye-evaluator # latest version pull करें -npx skills remove agenteye-evaluator # इसे हटाएं -``` - -हाथ से install करना पसंद हैं? एक Agent Skill सिर्फ एक folder है जिसमें `SKILL.md` है (plus optional references), तो इसे copy करना काम करता है: - -- **Claude Code**: `agenteye-evaluator/` folder को `~/.claude/skills/` में रखें (हर project) या `/.claude/skills/` में (केवल वह repo)। Claude Code इसे auto-discover करता है — `/skills` list से verify करें, या बस evals के लिए पूछें। -- **Codex (OpenAI)**: Codex same `SKILL.md` को read करता है। Bundled `agents/openai.yaml` `allow_implicit_invocation: true` set करता है, तो Codex skill को auto-select करता है जब task match हो; अन्यथा इसे explicitly invoke करें `$agenteye-evaluator` के रूप में। - ---- - -## SDK public PyPI पर नहीं है - -> **Warning:** Agent को SDK install करने देने से पहले यह पढ़ें। - -Skill public है; जिस SDK को यह drive करता है वह नहीं है। `agenteye-evaluator` केवल एक private release artifact के रूप में ship होता है, और `agenteye` के विपरीत, name public PyPI पर **unclaimed** है — तो एक bare `pip install agenteye-evaluator` एक stranger के package को pull कर सकता है उस service में जो आपके production transcripts को read करता है। यह एक supply-chain problem है, typo नहीं। - -Skill यह जानता है और इसकी जगह एक install ladder down करता है, पहले rung पर रुकता है जो apply होता है: monorepo source अगर आप AgentEye repo के inside हैं, अन्यथा GitHub Releases से private release wheel (needs access), और अगर न ही कोई reachable है तो यह **रुकता है और आपको अपने Failproof AI contact से wheel पूछने के लिए बताता है** improvising करने की बजाय। - -तो अगर आपका agent public PyPI से bare `pip install agenteye-evaluator` propose करता है, वह flag है कि skill कभी load नहीं हुई। वहीं रुकें और check करें कि यह installed है। - ---- - -## आप इससे क्या पूछ सकते हैं - -एक real round-trip एक vague ask से शुरू होता है और code के साथ नहीं, signed-off design के साथ end होता है: - -```text -you ▸ मुझे हमारे support bot के लिए evals चाहिए। मुझे लगता है यह कभी-कभी - खराब है। - -agent ▸ कुछ लिखने से पहले: एक run describe करें जो अच्छा गया, - और एक जो बुरा गया। - -you ▸ अच्छा: यह docs से answer करता है। बुरा: यह stuff बनाता है, - या हार मानता है और कहता है "contact support"। - -agent ▸ Running: agenteye --json sessions --status error --since 7d - 5 sessions पढ़ रहे हैं, दो सहित जिन्हें आप bad कहेंगे। - - आपके शब्द factuality और deflection कहते हैं। Transcripts - deflection पर सहमत हैं (5 में से 2 support hand-off के साथ - end होते हैं, और यह model_response.content में plain है), - लेकिन factuality को ground truth की ज़रूरत है जो आपके - events carry नहीं करते। - - तो मैं तीन चीज़ें score करूंगा: - - deflection rules, cheap, आपके दोनों runs को अलग करता है - - tool_efficiency rules, run-004 में retry loop को catch करता है - - groundedness LLM judge: क्या हर claim जो यह retrieved है - docs में है? - - "factuality" को ground truth की ज़रूरत होगी जो हमारे पास - नहीं है; groundedness सबसे करीबी चीज़ है जो आपके transcripts - actually support कर सकते हैं। ये तीनों names के साथ ठीक है? - बाद में rename करने से history split हो जाता है। - -you ▸ हां - -agent ▸ इस repo में कोई evaluator नहीं। क्या मुझे यहां एक scaffold - करना चाहिए, या क्या आपके पास कहीं और एक है? -``` - -वहां से यह पहले rules-based dimensions लिखता है (free, instant, deterministic), उन्हें एक real captured session के against test करता है जिसमें empty और never-finished ones भी शामिल हैं जो naive evaluators को crash करते हैं, और केवल subjective dimension पर एक LLM judge के लिए पहुंचता है। यह [dispatcher की limits](/hi/agenteye/evaluation-suite#configuring-the-server) को जानता है — 30s request timeout और 8 concurrent calls deployment-wide — तो अगर judge reliably fit नहीं होगा, तो यह `JobPending` के साथ async जाता है न कि आपके judge को cancelled और retried होने देता है पांच बार पांच बार लागत पर। - -फिर यह deploy करता है, दो server env vars set करता है, और `agenteye --json evals --session-id ` से confirm करता है कि scores actually land हुए। Scores landing ही एकमात्र proof है। - ---- - -## देखने के लिए क्या है - -- **Dimension names करीब-करीब permanent हैं।** Score keys arbitrary strings हैं और platform जो कुछ भी आप send करते हैं उसे trend करता है, जिसका मतलब है कि कोई भी downstream एक bad choice को correct नहीं करता। बाद में rename करें और history split हो जाता है: old sessions old key को keep करते हैं और trend break हो जाता है। यही है कि skill को code लिखने से पहले explicit sign-off क्यों मिलता है — वह prompt को seriously लें। -- **Fixtures real production transcripts हैं।** Real sessions के against design करने का मतलब है उन्हें disk पर pull करना, और उनमें customer data हो सकता है। Skill यह commit करने से पहले पूछती है कि क्या git में करें; अगर doubt हो तो `fixtures/` को repo के बाहर रखें और हर developer को अपने अपने pull करने दें। -- **Agent एक service write और deploy करता है जो हर transcript को read करता है।** यह आपके रूप में कार्य करता है, bounded by आपके CLI login की permissions, लेकिन evaluator को review करें जैसे कोई अन्य code जो production data को touch करता है। - ---- - -## अगले कदम - -- **[Evaluation suite](/hi/agenteye/evaluation-suite)**: HTTP contract, SDK, और server env vars जिन्हें skill configure करता है। -- **[Evaluations](/hi/agenteye/evaluations)**: जहां scores show up होते हैं एक बार जब वे land करते हैं। -- **[CLI skill](/hi/agenteye/cli-skill)**: sibling skill, scorer build करने की बजाय results read करने के लिए। -- **[CLI](/hi/agenteye/cli)**: command reference जिसके against skill session data design करती है। \ No newline at end of file diff --git a/docs/hi/agenteye/hermes-capture.mdx b/docs/hi/agenteye/hermes-capture.mdx deleted file mode 100644 index e41c4d7f..00000000 --- a/docs/hi/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Hermes session capture" -description: "अपनी टीम के Hermes gateway sessions — Slack, Telegram, CLI, और scheduled runs — को AgentEye में ordinary sessions और events के रूप में लाएं।" ---- - -[Hermes](https://hermes-agent.nousresearch.com) आपकी टीम को जहां भी वह काम करती है वहां से उत्तर देता है — Slack, Telegram, CLI, scheduled runs। Hermes session capture इन सभी को AgentEye में ordinary sessions और events के रूप में लाता है, ताकि आपकी टीम जिस assistant से हर दिन बात करती है वह उतना ही observable हो जितना कि आप जो agents लिखते हैं। - -एक छोटा सा background collector Hermes के local session store को जब भी लिखा जाता है तब पढ़ता है और sessions को AgentEye को भेजता है। यह [Codex](/hi/agenteye/codex-capture) और [OpenClaw](/hi/agenteye/openclaw-capture) capture के समान ही काम करता है, और एक collector कई को एक साथ capture कर सकता है। - ---- - -## यह क्या capture करता है - -मशीन पर हर Hermes session को capture किया जाता है, चाहे वह किसी भी channel से आया हो। प्रत्येक एक AgentEye [session](/hi/agenteye/sessions) बन जाता है; इसके user और assistant messages, tool calls, और tool results matching [events](/hi/agenteye/event-stream) बन जाते हैं। - -जिस channel से एक session शुरू हुआ — Slack, Telegram, CLI, या एक scheduled run — वह session पर रिकॉर्ड किया जाता है, ताकि आप उन्हें अलग बता सकें और एक बार में एक को filter कर सकें। इसके साथ session जिस model पर चला, जिस chat और person से शुरू किया गया, और जब एक session ने दूसरे को spawn किया, तो अपने parent की link भी आती है। - -Sessions तुरंत दिखाई देते हैं जब Hermes उन्हें शुरू करता है, चाहे कुछ भी कहा गया हो या नहीं, और एक turn का reply और उसके tool calls वास्तविक क्रम में रहते हैं। जब एक session समाप्त होता है तो आप यह भी जानते हैं कि यह क्यों समाप्त हुआ, इसकी लागत क्या थी, और इसने कितने tokens का उपयोग किया। - ---- - -## इसे चालू करें - -Capture तब तक बंद रहता है जब तक आप इसे enable न करें। एक API key के साथ collector install करें जिसके पास `events:add` permission हो (देखें [API keys](/hi/agenteye/api-keys)), और Hermes capture को चालू करें: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -यह collector को install करता है, इसे एक background service के रूप में register करता है, और capturing शुरू करता है। पुष्टि करें कि यह चल रहा है: - -```bash -agenteye-collector health -``` - -एक ही मशीन पर एक से अधिक agent को capture कर रहे हैं? एक ही command में प्रत्येक का flag जोड़ें — उदाहरण के लिए `--hermes-enabled --codex-enabled`। - -पहली बार चलाने पर, आपके मौजूदा Hermes sessions को एक बार backfill किया जाता है और नई activity फिर कुछ सेकंड के भीतर stream होती है। Hermes के अपने data को केवल पढ़ा जाता है — कभी भी संशोधित या deleted नहीं किया जाता है — और प्रत्येक message एक बार भेजा जाता है, restarts के बीच भी। - -`health` यह भी बताता है कि क्या collector ने जो कुछ भी capture किया वह वास्तव में AgentEye तक पहुंचा है। यदि कोई batch deliver नहीं किया जा सका तो उसे रखा जाता है और फिर से प्रयास किया जाता है न कि discarded किया जाता है, और check तब तक unhealthy रिपोर्ट करता है जब तक कुछ भी outstanding हो — तो "healthy" का अर्थ है आपका data पहुंचा, केवल यह नहीं कि process alive है। - ---- - -## यह कहां दिखाई देता है - -Captured sessions **Sessions** में दिखाई देते हैं, और उनके events **Events** stream में, किसी भी अन्य agent के समान जिसे आप observe करते हैं — तो [session replay](/hi/agenteye/sessions), [search](/hi/agenteye/queries), [evaluations](/hi/agenteye/evaluations), और [alerts](/hi/agenteye/alerts) सभी उन पर काम करते हैं। उन्हें अपने आप पर देखने के लिए Hermes agent द्वारा filter करें। - ---- - -## गोपनीयता - -Hermes sessions में पूरी transcript होती है — command output, file contents, और कुछ भी जो agent ने पढ़ा या लिखा था सहित — और इसमें secrets हो सकते हैं। Captured sessions को जैसे-तैसे भेजा जाता है, तो capture को केवल वहां enable करें जहां उस content को AgentEye में centralize करना appropriate हो, और collector को एक ऐसी key दें जो केवल `events:add` तक scoped हो। देखें [Security](/hi/agenteye/security) कि आपके data को कैसे अलग रखा जाता है। \ No newline at end of file diff --git a/docs/hi/agenteye/observability.mdx b/docs/hi/agenteye/observability.mdx deleted file mode 100644 index ba4c8bd6..00000000 --- a/docs/hi/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Observe" -description: "Observe सर्फेस वह जगह हैं जहां आप अपने एजेंटों को अभी क्या कर रहे हैं यह देख सकते हैं और किसी भी एक रन में ड्रिल डाउन कर सकते हैं।" ---- - - -Observe सर्फेस वह जगह हैं जहां आप अपने एजेंटों को अभी क्या कर रहे हैं यह देख सकते हैं और किसी भी एक रन में ड्रिल डाउन कर सकते हैं। यहां सब कुछ लाइव है, आपके संगठन के स्कोप में है, और तारीख की रेंज, वातावरण, एजेंट, और सेशन के आधार पर फ़िल्टर किया जा सकता है, ताकि आप "कुछ गलत महसूस हो रहा है" से सटीक रन तक सेकंडों में पहुंच सकें। - -![लाइव इवेंट स्ट्रीम, प्रकार के अनुसार रंग-कोडित और वातावरण, एजेंट, और सेशन के आधार पर फ़िल्टर किया जा सकता है](/agenteye/images/events-stream.png) - -चार सर्फेस, प्रत्येक के साथ अपना-अपना पेज: - -- **[Event stream](/hi/agenteye/event-stream)**: हर एजेंट भर में हर रन की लाइव, प्रति-चरण ट्रेल, सबसे नया पहले। आपका संगठन होम और ट्राइएज के लिए पहला स्टॉप। -- **[Sessions and execution graph](/hi/agenteye/sessions)**: वे इवेंट प्रति रन एक पंक्ति में रोल अप किए गए, साथ ही एक git-शैली की तस्वीर कि हर रन कैसे सामने आया। -- **[Performance metrics](/hi/agenteye/telemetry)**: विलंबता हीट-मैप और आपके मॉडल, टूल्स, और हुक के लिए p50/p95/p99 महत्वपूर्ण संकेत, ताकि एक टेल स्पाइक माध्यिका से अलग दिखाई दे। -- **[Error tracking](/hi/agenteye/error-tracking)**: सब कुछ के लिए एक ट्राइएज सर्फेस जो गलत हुआ, एक फायरिंग अलर्ट से रन तक एक क्लिक दूर जो टूट गया। - -## संबंधित - -- [Evaluations](/hi/agenteye/evaluations): हर रन को गुणवत्ता के लिए स्कोर करें। -- [Alerts](/hi/agenteye/alerts): किसी भी थ्रेशहोल्ड को एक पेजिंग नियम में बदलें। -- [Audits](/hi/agenteye/audits): Failproof AI Observability को सेशन भर में विफलता पैटर्न खोजने दें। -- [CLI and agents](/hi/agenteye/cli-and-agents): आपके टर्मिनल से समान अवलोकनशीलता। \ No newline at end of file diff --git a/docs/hi/agenteye/openclaw-capture.mdx b/docs/hi/agenteye/openclaw-capture.mdx deleted file mode 100644 index d65632cb..00000000 --- a/docs/hi/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,50 +0,0 @@ ---- ---- -title: "OpenClaw सत्र कैप्चर" -description: "अपनी टीम के स्थानीय OpenClaw सत्रों को AgentEye में साधारण सत्रों और घटनाओं के रूप में प्राप्त करें — OpenClaw चलाने के तरीके में कोई परिवर्तन नहीं।" ---- - -यदि आपकी टीम [OpenClaw](https://docs.openclaw.ai) चलाती है, तो OpenClaw सत्र कैप्चर उन सत्रों को AgentEye में साधारण सत्रों और घटनाओं के रूप में लाता है, ताकि आप उन्हें खोज सकें, पुनः चला सकें और उन्हें आपके द्वारा देखी गई किसी भी अन्य चीज़ के साथ-साथ मूल्यांकन कर सकें। यह [Python SDK](/hi/agenteye/python-sdk) की पूरक है: SDK आपके द्वारा लिखे गए एजेंटों को साधन देता है, जबकि यह आपकी टीम द्वारा पहले से किए जा रहे OpenClaw कार्य को कैप्चर करता है — इसे चलाने के तरीके में कोई परिवर्तन नहीं। - -एक छोटा पृष्ठभूमि कलेक्टर OpenClaw के स्थानीय सत्र प्रतिलेखों को पढ़ता है क्योंकि वे लिखे जाते हैं और उन्हें AgentEye को भेजता है। यह [Codex कैप्चर](/hi/agenteye/codex-capture) के समान तरीके से काम करता है, और एक कलेक्टर एक साथ दोनों को कैप्चर कर सकता है। - ---- - -## यह क्या कैप्चर करता है - -किसी मशीन के OpenClaw सेटअप में कॉन्फ़िगर किया गया प्रत्येक एजेंट उस मशीन के कलेक्टर द्वारा कैप्चर किया जाता है — कोई प्रति-एजेंट सेटअप नहीं है। - -प्रत्येक OpenClaw सत्र एक AgentEye [सत्र](/hi/agenteye/sessions) बन जाता है; इसके उपयोगकर्ता और सहायक संदेश, उपकरण कॉल और उपकरण परिणाम मिलान वाली [घटनाओं](/hi/agenteye/event-stream) बन जाते हैं। - ---- - -## इसे चालू करें - -कैप्चर तब तक बंद है जब तक आप इसे सक्षम नहीं करते। `events:add` अनुमति वाली API कुंजी के साथ कलेक्टर को इंस्टॉल करें (देखें [API कुंजियाँ](/hi/agenteye/api-keys)), और OpenClaw कैप्चर चालू करें: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -यह कलेक्टर को इंस्टॉल करता है, इसे एक पृष्ठभूमि सेवा के रूप में पंजीकृत करता है, और कैप्चरिंग शुरू करता है। पुष्टि करें कि यह चल रहा है: - -```bash -agenteye-collector health -``` - -एक ही मशीन पर एक से अधिक एजेंटों को कैप्चर कर रहे हैं? प्रत्येक का ध्वज एक ही कमांड में जोड़ें — उदाहरण के लिए `--openclaw-enabled --codex-enabled`। - -पहली बार चलने पर, आपके मौजूदा OpenClaw सत्रों को एक बार बैकफिल किया जाता है और नई गतिविधि फिर कुछ सेकंड के भीतर स्ट्रीम होती है। OpenClaw की अपनी फाइलें केवल पढ़ी जाती हैं — कभी भी संशोधित, स्थानांतरित या हटाई नहीं जाती हैं — और प्रत्येक सत्र पुनः प्रारंभ के भीतर भी बिल्कुल एक बार भेजा जाता है। - ---- - -## यह कहाँ दिखाई देता है - -कैप्चर किए गए सत्र **Sessions** में दिखाई देते हैं, और उनकी घटनाएं **Events** स्ट्रीम में, किसी भी अन्य एजेंट के समान जिसे आप देखते हैं — इसलिए [सत्र पुनः चलाना](/hi/agenteye/sessions), [खोज](/hi/agenteye/queries), [मूल्यांकन](/hi/agenteye/evaluations), और [अलर्ट](/hi/agenteye/alerts) सभी उन पर काम करते हैं। उन्हें अपने आप से देखने के लिए OpenClaw एजेंट द्वारा फ़िल्टर करें। - ---- - -## गोपनीयता - -OpenClaw प्रतिलेख में पूर्ण सत्र होता है — जिसमें कमांड आउटपुट, फाइल सामग्री और कुछ भी शामिल है जो एजेंट ने पढ़ा या लिखा — और इसमें गोपनीय जानकारी हो सकती है। कैप्चर किए गए सत्रों को जैसा है वैसा भेजा जाता है, इसलिए केवल उन मशीनों और टीमों के लिए कैप्चर सक्षम करें जहाँ उस सामग्री को AgentEye में केंद्रीकृत करना उपयुक्त है, और कलेक्टर को केवल `events:add` के लिए निर्धारित कुंजी दें। [सुरक्षा](/hi/agenteye/security) के लिए देखें कि आपका डेटा कैसे अलग रखा जाता है। \ No newline at end of file diff --git a/docs/hi/agenteye/python-sdk-skill.mdx b/docs/hi/agenteye/python-sdk-skill.mdx deleted file mode 100644 index e877d8a5..00000000 --- a/docs/hi/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,133 +0,0 @@ ---- -title: "Failproof AI Observability Python SDK Agent Skill" -description: "बिना instrumented agent से शुरू करके ऐसे events तक पहुंचें जिन्हें आप देख सकें, आपके coding agent के साथ instrumentation points खोजते हुए, उन्हें लिखते हुए, और यह साबित करते हुए कि वे काम कर रहे हैं।" ---- - -अपने coding agent को बताएं *"इस agent में Failproof AI Observability जोड़ें"* और इसे अपना loop पढ़ने दें, काम करें कि instrumentation कहां जाना चाहिए, इसे लिखें, और events को verify करें इससे पहले कि वह काम पूरा करे। - -**Python SDK skill** (`agenteye-python-sdk`) एक *Agent Skill* है: निर्देशों का एक folder जिसे coding agent जैसे Claude Code या Codex demand पर load करता है जब कोई task उससे match करे। यह agent को [Python SDK](/hi/agenteye/python-sdk) का उपयोग करना सिखाता है — यह एक library नहीं है, और यह SDK के काम करने के तरीके में कुछ नहीं बदलता। - -## Instrumentation लिखना आसान है और आसानी से गलत हो सकता है - -SDK छोटा है: तेरह event methods, सभी keyword-only। एक coding agent [Python SDK](/hi/agenteye/python-sdk) reference को पढ़ सकता है और एक मिनट में plausible instrumentation बना सकता है। - -समस्या यह है कि यह SDK गलत होने पर raise नहीं करता, और गलत instrumentation बिल्कुल सही instrumentation जैसी दिखती है जब तक कोई dashboard नहीं खोलता और इसे खाली नहीं पाता। असली समय खर्च करने वाली गलतियां सभी silence हैं: - -| गलती | आप क्या देखते हैं | -|---|---| -| No `agent_start` | हर event land होता है। Zero sessions। | -| Environment कभी set नहीं होता | सब कुछ काम करता है, `dev` के तहत filed। | -| `outcome="failure"` | Run green दिखता है — केवल `failed`, `error`, `timeout`, `rejected` count होते हैं। | -| Typo'd field name | Accepted होता है और एक नए field के रूप में stored। | -| Thread pool से emitted events | Silently dropped। | - -इनमें से कोई भी raise नहीं करता। कोई भी tests में दिखाई नहीं देता। हर एक skill में है, एक contract के रूप में stated जिसमें check है जो इसे catch करता है। - -## यह क्या करता है, क्रम में - -Skill उन्हीं तीन steps को चलाता है जो एक सावधान engineer करेगा: - -1. **Plan.** यह आपके agent loop को पढ़ता है और दो सवाल पूछता है जिनका जवाब केवल आप दे सकते हैं: क्या एक run के लिए गिना जाए (`session_id`), और अलग-अलग actors कौन हैं (`agent_id`)। यह code लिखने से पहले उन पर सहमति प्राप्त करता है, क्योंकि बाद में उन्हें बदलने से आपका history split होता है और trends टूट जाते हैं। -2. **Write.** यह identity को एक बार per run bind करता है बजाय हर call site के माध्यम से thread करने के, और एक concurrency-safe shape चुनता है — एक विवरण जो मायने रखता है, क्योंकि स्पष्ट shortcut silently दो overlapping runs को एक session में mix कर सकता है। -3. **Verify.** यह आपके agent को चलाता है और resulting event files को पढ़ता है, यह check करते हुए कि `agent_start` present है, environment सही है, और एक run ने एक session बनाया है। - -वह तीसरा step है जिसे लोग skip करते हैं। SDK events को local files में लिखता है, तो एक complete integration को एक laptop पर server, API key, या network के बिना proved किया जा सकता है — जो बिल्कुल वही कारण है कि skill इसे करने पर настаивает। - -## यह अन्य skills से कैसे संबंधित है - -तीन skills, एक स्पष्ट split: - -| Skill | इसे तब प्राप्त करें जब | यह क्या छूता है | -|---|---|---| -| **Python SDK skill** (यह पृष्ठ) | आप चाहते हैं कि आपका agent *emit* करे telemetry — "observability जोड़ें", "मेरा agent क्यों दिखाई नहीं दे रहा?" | आपके agent के repo में code लिखता है। कुछ नहीं पढ़ता। | -| **[Evaluator skill](/hi/agenteye/evaluator-skill)** | आप *score* करना चाहते हैं runs — "हमें क्या मापना चाहिए?" | आपके repo में code लिखता है; telemetry पढ़ता है | -| **[CLI skill](/hi/agenteye/cli-skill)** | आप *read* करना चाहते हैं कि क्या हुआ, या अपनी deployment operate करना चाहते हैं | CLI को as you drive करता है, changes सहित | - -वे उसी order में hand off करते हैं: यह skill events को flowing करता है, evaluator उन्हें score करता है, CLI उन्हें वापस पढ़ता है। जब तक आपका agent sessions emit नहीं करता तब तक evaluate करने के लिए कुछ नहीं है और read करने के लिए कुछ नहीं है, तो यदि आप scratch से शुरू कर रहे हैं, तो यहां से शुरू करें। - -## Prerequisites - -1. **Python 3.10+** और agent codebase जिसे आप instrument करना चाहते हैं। -2. **The SDK.** यह customers को एक private wheel के रूप में distributed किया जाता है एक public index से नहीं — आपके onboarding में यह शामिल है कि इसे कैसे प्राप्त करें और install करें। Skill install path को जानता है और यदि इसे नहीं मिल सकता तो आपसे पूछेगा बजाय अनुमान लगाने के। -3. **कुछ नहीं।** कोई dashboard login नहीं, कोई API key नहीं, कोई network नहीं। Skill SDK द्वारा लिखी जाने वाली event files के विरुद्ध verify करता है, तो यह offline काम पूरा कर सकता है और साबित कर सकता है। - -## इसे कहां प्राप्त करें - -Skill public [`FailproofAI/skills`](https://github.com/FailproofAI/skills) collection में रहता है: - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -हर project के लिए install करने के लिए `-g` जोड़ें बजाय सिर्फ current के, और `--copy` जोड़ें यदि आपका environment symlinks को follow नहीं करता है। Codex के लिए, `-a codex` pass करें। - -## इसे manually install करना - -Agent Skills `SKILL.md` और references वाले folders हैं। यदि आप installer का उपयोग नहीं करना चाहते: - -- **Claude Code**: `agenteye-python-sdk/` folder को `~/.claude/skills/` (हर project) या `/.claude/skills/` (सिर्फ वह repo) में copy करें। Claude Code इसे automatically discover करता है — `/skills` list check करें, या बस कुछ ऐसा पूछें जो इससे match करे। -- **Codex**: Codex एक ही `SKILL.md` पढ़ता है। Bundled `agents/openai.yaml` `allow_implicit_invocation: true` set करता है, तो यह auto-selected होता है जब कोई task match करे; अन्यथा इसे `$agenteye-python-sdk` के रूप में invoke करें। - -अपने agent को **उस repository में** run करें जिसमें आप instrumented करना चाहते हैं — skill code लिखने से पहले आपके agent loop को पढ़ता है। - -## एक session कैसा दिखता है - -```text -आप ▸ इस agent में Failproof AI Observability जोड़ें। - -agent ▸ आपके loop को read करें — tools.py में एक tool dispatcher, llm.py - में एक Anthropic wrapper। कुछ भी लिखने से पहले दो सवाल। - - क्या एक run के लिए गिना जाए? आप requests को concurrently - handle करते हैं, तो मैं ctx.request_id में request id का उपयोग - करूंगा — तब यहां एक session और आपके logs में एक request - एक ही string हैं। - - और actors कौन हैं? मैं एक planner path और एक researcher path - देखता हूं। उन्हें अलग से label करें, या यह आपके लिए एक agent है? - -आप ▸ Request id सही है। वे एक agent हैं, इसे "assistant" कहें। - -agent ▸ पूरा हुआ। Dispatcher और LLM client को wrap किया; - agent_start और agent_end handle_request को bracket करते हैं, - error path सहित। Identity के लिए एक context-local का उपयोग - किया — आपका dispatcher tools को एक thread pool में चलाता है, - और सरल approach concurrent runs को एक session में mix कर - सकता था बिना complain किए। - - एक real run के विरुद्ध verify किया: 4 files में 11 events — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success)। environment=production, हर request - के लिए एक session, durations paired। - - एक चीज जो मैंने नहीं की: आपके batch.py में worker pool - सीधे executor में submit करता है, तो वहां से events - drop होते। क्या आप चाहते हैं कि मैं उन्हें भी fix करूं? -``` - -ध्यान देने योग्य pattern: यह code लिखने से पहले code को पढ़ा, केवल वे सवाल पूछे जिनका जवाब आप दे सकते हैं, एक id को reuse किया जो आप पहले से had करते हैं, concurrency-safe shape को चुना *क्योंकि* इसने एक thread pool देखा, और **actual events को पढ़कर verify किया** बजाय सफलता की घोषणा करने के — फिर उस एक जगह को flag किया जहां यह जानता था कि silently fail होगा। - -## आप इससे क्या पूछ सकते हैं - -- *"मेरा agent dashboard पर क्यों नहीं दिख रहा है?"* → ladder को walk करता है: क्या events write हो रहे हैं, क्या `agent_start` है, क्या environment सही है, क्या collector एक ही जगह से read कर रहा है। -- *"सब कुछ dev के तहत land हो रहा है।"* → environment कभी set नहीं हुआ, या एक later call द्वारा reset हुआ। -- *"Token tracking जोड़ें।"* → आपके LLM wrapper को खोजता है और model, stop reason, और usage record करता है। -- *"Sub-agents को भी instrument करें।"* → एक session, distinct agent labels, अपने parent के तहत nested। -- *"Instrumentation के लिए tests लिखें।"* → SDK को एक temporary directory की ओर point करता है और इसके द्वारा लिखी गई events पर assert करता है। - -## इस पर ध्यान दें - -**इसे verify करने दें।** वह step जो इस skill को उपयोग करने के लायक बनाता है वह आखिरी है — आपके agent को चलाना और events को वापस पढ़ना। एक agent जो instrumentation लिखता है और रुकता है आसान आधा किया है, और आधा जो silently fail होता है वह दूसरा है। - -**Names पर code से पहले सहमति प्राप्त करें।** `session_id` और `agent_id` वह axes हैं जिन पर हर surface group करता है। उन्हें बाद में rename करने से history split होता है: पुरानी runs पुरानी labels रखती हैं और आपके trends टूट जाते हैं। Skill पूछेगा; answer एक मिनट के विचार के लायक है। - -**यदि आपका agent SDK को एक public index से install करने का प्रस्ताव देता है, तो skill load नहीं हुई।** SDK privately distributed है। वह प्रस्ताव एक reliable tell है कि आपका coding agent skill को follow करने के बजाय अनुमान लगा रहा है — इसे वहीं रोकें और check करें कि skill install है। - -इसके आगे इसका blast radius छोटा है: यह आपकी working directory में code लिखता है और event files जहां आप कहते हैं। यह अपनी deployment से कुछ नहीं पढ़ता और इसके बारे में कुछ नहीं बदलता। - -## अगले कदम - -- **[Python SDK](/hi/agenteye/python-sdk)**: complete event reference — हर event type और field — जो यह skill automate करता है। -- **[Sessions](/hi/agenteye/sessions)**: आपका instrumentation क्या produce करता है एक बार events land हो जाएं। -- **[Evaluator Agent Skill](/hi/agenteye/evaluator-skill)**: अगला step एक बार runs land होने लगें — उन्हें score करना। -- **[CLI Agent Skill](/hi/agenteye/cli-skill)**: आपकी telemetry को वापस read करना। \ No newline at end of file diff --git a/docs/hi/agenteye/security.mdx b/docs/hi/agenteye/security.mdx deleted file mode 100644 index d9906939..00000000 --- a/docs/hi/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "सुरक्षा" -description: "Failproof AI Observability आपके उत्पादन एजेंटों के पास रखने के लिए बनाया गया है, जिसका अर्थ है कि यह आपके prompts, tool inputs, और outputs को देखता है।" ---- - - -Failproof AI Observability आपके उत्पादन एजेंटों के पास रखने के लिए बनाया गया है, जिसका अर्थ है कि यह आपके prompts, tool inputs, और outputs को देखता है। यह पृष्ठ बताता है कि यह उस डेटा को कैसे अलग-थलग, नियंत्रित, और आपके हाथों में रखता है। यदि आप सुरक्षा समीक्षा के लिए Failproof AI Observability का मूल्यांकन कर रहे हैं, तो यहाँ से शुरू करें। - ---- - -## आपका डेटा आपके परिवेश में रहता है - -Failproof AI Observability self-hosted है। Events, prompts, मॉडल responses, और analytics आपके अपने डेटाबेस में, आपके अपने परिवेश में संग्रहीत हैं। कोई भी डेटा storage के लिए किसी third-party SaaS को नहीं भेजा जाता है, और आपका डेटा आपके अपने cloud account में रहता है। - ---- - -## टेनेंट isolation - -एक Failproof AI Observability instance कई संगठनों को host कर सकता है, और प्रत्येक को storage layer पर अलग किया जाता है — सिर्फ UI द्वारा नहीं, बल्कि डेटाबेस द्वारा लागू किया जाता है: - -- किसी संगठन का operational data (users, keys, dashboards, saved queries) उस org तक सीमित है, और cross-org reads को डेटाबेस द्वारा ही block किया जाता है। -- प्रत्येक ingested event को अपने owning org के साथ stamp किया जाता है, इसलिए एक संगठन की events को कभी भी दूसरे द्वारा नहीं पढ़ा जा सकता। - -प्रत्येक dashboard route एक org slug (`//…`) के अंतर्गत scoped है। - ---- - -## Sign-in - -Failproof AI Observability passwordless, email-based sign-in का उपयोग करता है। phish या leak करने के लिए कोई password नहीं है। एक उपयोगकर्ता एक one-time code (या एक one-click magic link) का अनुरोध करता है, जो उन्हें email किया जाता है और जल्दी expire हो जाता है। Sign-in को एक **allowlist** द्वारा gate किया जाता है: केवल email addresses (या domains) जिन्हें आप permit करते हैं, authenticate कर सकते हैं। - -![Failproof AI Observability sign-in screen, जो आपके email को एक single-use code भेजता है](/agenteye/images/login.png) - ---- - -## API keys के साथ scoped access - -प्रत्येक client एक API key के साथ authenticate करता है जो granular, least-privilege permissions रखता है। एक collector को केवल `events:add` की जरूरत है; एक dashboard या assistant key read-only हो सकता है; destructive actions (delete, regenerate) अलग grants हैं जिन्हें आप शामिल करना चुनते हैं। - -![API keys page: प्रत्येक key की permission grants, read, write, और destructive scope द्वारा colour-coded](/agenteye/images/api-keys.png) - -Admin bootstrap key को setup के लिए रखें, और बाकी सब कुछ के लिए narrow keys जारी करें। [API keys](/hi/agenteye/api-keys) देखें। - ---- - -## एक read-only, approval-gated assistant - -Dashboard में [AI assistant](/hi/agenteye/assistant) आपके डेटा पर प्रश्नों का उत्तर देता है, लेकिन यह design द्वारा constrained है: - -- यह **डिफ़ॉल्ट रूप से read-only है**: इसका SQL एक guard के माध्यम से चलता है जो केवल `SELECT`/`WITH` queries को permit करता है, single-statement, एक row cap के साथ। -- जो कुछ भी यह creates करता है (एक saved query, एक dashboard) **approval-gated है**: आप प्रत्येक write से पहले review और approve करते हैं। -- यह **कभी delete नहीं कर सकता**। - -इसलिए एक teammate यह पूछ सकता है "इस सप्ताह किन agents में सबसे अधिक errors थीं?" और answer पर कार्रवाई कर सकता है, बिना इसके कि assistant अपने आप पर आपके डेटा को change या remove कर सके। - ---- - -## Transit में - -सभी traffic HTTPS के माध्यम से चलता है। आप अपने अपने certificates के साथ TLS को terminate करते हैं, इसलिए collector-to-server और browser-to-server traffic transit में encrypted है। - ---- - -## अगले कदम - -- [Overview](/hi/agenteye/overview): Failproof AI Observability कैसे एक साथ आता है। -- [API keys](/hi/agenteye/api-keys): collector, dashboard, और assistant के लिए access scope करें। -- [Observability](/hi/agenteye/observability): Failproof AI Observability आपके agents से क्या captures करता है। \ No newline at end of file diff --git a/docs/hi/cli/audit.mdx b/docs/hi/audit.mdx similarity index 100% rename from docs/hi/cli/audit.mdx rename to docs/hi/audit.mdx diff --git a/docs/hi/cli/backfill.mdx b/docs/hi/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/hi/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/hi/cli/config.mdx b/docs/hi/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/hi/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/hi/cli/flush.mdx b/docs/hi/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/hi/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/hi/cli/harness.mdx b/docs/hi/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/hi/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/hi/cli/migrate.mdx b/docs/hi/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/hi/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/hi/cli/uninstall.mdx b/docs/hi/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/hi/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/hi/cli/update.mdx b/docs/hi/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/hi/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/hi/agenteye/api-keys.mdx b/docs/hi/cloud/access.mdx similarity index 95% rename from docs/hi/agenteye/api-keys.mdx rename to docs/hi/cloud/access.mdx index 0ff497ad..5dd1bd38 100644 --- a/docs/hi/agenteye/api-keys.mdx +++ b/docs/hi/cloud/access.mdx @@ -1,9 +1,9 @@ --- title: "API कुंजियाँ" -description: "API कुंजियाँ नियंत्रित करती हैं कि कौन और क्या आपके Failproof AI Observability सर्वर तक पहुँच सकता है, जिससे एक कलेक्टर कभी भी पढ़ने या व्यवस्थापक शक्तियों को प्राप्त किए बिना ईवेंट भेज सकता है।" +description: "API कुंजियाँ नियंत्रित करती हैं कि कौन और क्या आपके FailproofAI Cloud सर्वर तक पहुँच सकता है, जिससे एक कलेक्टर कभी भी पढ़ने या व्यवस्थापक शक्तियों को प्राप्त किए बिना ईवेंट भेज सकता है।" --- -API कुंजियाँ नियंत्रित करती हैं कि कौन और क्या आपके Failproof AI Observability सर्वर तक पहुँच सकता है, जिससे एक कलेक्टर कभी भी पढ़ने या व्यवस्थापक शक्तियों को प्राप्त किए बिना ईवेंट भेज सकता है। प्रत्येक कुंजी एक या अधिक अनुमतियाँ रखती है, और प्रत्येक अनुमति विशिष्ट सर्वर रूट को नियंत्रित करती है; आप केवल वह अनुमतियाँ देते हैं जो एक कार्य को चाहिए। अधिकांश परिनियोजन केवल तीन प्रकार की कुंजियाँ बनाते हैं। +API कुंजियाँ नियंत्रित करती हैं कि कौन और क्या आपके FailproofAI Cloud सर्वर तक पहुँच सकता है, जिससे एक कलेक्टर कभी भी पढ़ने या व्यवस्थापक शक्तियों को प्राप्त किए बिना ईवेंट भेज सकता है। प्रत्येक कुंजी एक या अधिक अनुमतियाँ रखती है, और प्रत्येक अनुमति विशिष्ट सर्वर रूट को नियंत्रित करती है; आप केवल वह अनुमतियाँ देते हैं जो एक कार्य को चाहिए। अधिकांश परिनियोजन केवल तीन प्रकार की कुंजियाँ बनाते हैं। ## 3 कुंजियाँ जो अधिकांश परिनियोजन को चाहिए @@ -81,7 +81,7 @@ API कुंजियाँ नियंत्रित करती हैं ये अनुमतियाँ डैशबोर्ड के **उपयोगकर्ता** पृष्ठ को समर्थन देती हैं, जहाँ प्रत्येक सदस्य के दिए गए दायरे चिप्स के रूप में दिखाए जाते हैं: -![उपयोगकर्ता पृष्ठ: प्रत्येक डैशबोर्ड उपयोगकर्ता के लिए एक कार्ड उनके ईमेल, दी गई अनुमतियों, और संपादन/अक्षम नियंत्रण के साथ](/agenteye/images/users.png) +![उपयोगकर्ता पृष्ठ: प्रत्येक डैशबोर्ड उपयोगकर्ता के लिए एक कार्ड उनके ईमेल, दी गई अनुमतियों, और संपादन/अक्षम नियंत्रण के साथ](/cloud/images/users.png) ### परिचालन सेटिंग्स @@ -90,7 +90,7 @@ API कुंजियाँ नियंत्रित करती हैं | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | डैशबोर्ड-प्रबंधित परिचालन सेटिंग्स और उनके मेटाडेटा को देखें; प्रति-मॉडल संदर्भ-विंडो ओवरराइड की सूची बनाएँ; और एक मॉडल के लिए प्रभावी विंडो को हल करें। | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | परिचालन सेटिंग्स को संपादित करें और प्रति-मॉडल संदर्भ-विंडो ओवरराइड को जोड़ें, बदलें, या हटाएँ। परिवर्तन सर्वर को पुनः आरंभ किए बिना नई ईवेंट को प्रभावित करते हैं। | -![सेटिंग्स पृष्ठ: डैशबोर्ड-प्रबंधित परिचालन सेटिंग्स जैसे अनुमति दी गई साइन-इन और सत्र/OTP जीवनकाल, पुनः आरंभ के बिना संपादन योग्य](/agenteye/images/settings.png) +![सेटिंग्स पृष्ठ: डैशबोर्ड-प्रबंधित परिचालन सेटिंग्स जैसे अनुमति दी गई साइन-इन और सत्र/OTP जीवनकाल, पुनः आरंभ के बिना संपादन योग्य](/cloud/images/settings.png) ### अलर्ट और घटनाएँ @@ -246,7 +246,7 @@ curl -s -X POST http://your-server/keys//regenerate \ जब आप डैशबोर्ड से एक कुंजी बनाते हैं तो आप गोपनीयता की आपूर्ति नहीं करते हैं; डैशबोर्ड आपके लिए एक मजबूत गोपनीयता उत्पन्न करता है और इसे **एक बार** निर्माण पर प्रदर्शित करता है। इसे तुरंत कॉपी करें और सुरक्षित रूप से स्टोर करें; यह कभी फिर से दिखाया नहीं जाता है, एक पुन: उत्पन्न के समान ही। आप अभी भी कुंजी की अनुमतियों को सीधे चुन सकते हैं, या एक अनुमति सेट से उन्हें बीजित कर सकते हैं (नीचे देखें)। -![API कुंजियाँ पृष्ठ: प्रत्येक कुंजी के लिए एक कार्ड इसके नाम, दी गई अनुमतियों, और निर्माण समय के साथ, पुन: उत्पन्न और अक्षम कार्य; `admin` जैसी संरक्षित कुंजियाँ चिह्नित हैं](/agenteye/images/api-keys.png) +![API कुंजियाँ पृष्ठ: प्रत्येक कुंजी के लिए एक कार्ड इसके नाम, दी गई अनुमतियों, और निर्माण समय के साथ, पुन: उत्पन्न और अक्षम कार्य; `admin` जैसी संरक्षित कुंजियाँ चिह्नित हैं](/cloud/images/api-keys.png) --- @@ -275,5 +275,5 @@ curl -s -X POST http://your-server/keys//regenerate \ ## अगले कदम -- [Python SDK](/hi/agenteye/python-sdk): कैसे आपका एजेंट कोड प्रमाणीकृत होता है जब ईवेंट भेज रहा हो। -- [सुरक्षा](/hi/agenteye/security): साइन-इन, एक्सेस नियंत्रण, और प्रति-संगठन डेटा अलगाव कैसे काम करता है। \ No newline at end of file +- [Python SDK](/hi/cloud/sdk): कैसे आपका एजेंट कोड प्रमाणीकृत होता है जब ईवेंट भेज रहा हो। +- [सुरक्षा](/hi/cloud/security): साइन-इन, एक्सेस नियंत्रण, और प्रति-संगठन डेटा अलगाव कैसे काम करता है। \ No newline at end of file diff --git a/docs/hi/cloud/agent-skills.mdx b/docs/hi/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/hi/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/hi/agenteye/alerts.mdx b/docs/hi/cloud/alerts.mdx similarity index 79% rename from docs/hi/agenteye/alerts.mdx rename to docs/hi/cloud/alerts.mdx index 1f5028bf..98a35089 100644 --- a/docs/hi/agenteye/alerts.mdx +++ b/docs/hi/cloud/alerts.mdx @@ -3,9 +3,9 @@ title: "सतर्कताएं" description: "उसी क्षण जानें जब कोई चीज़ आपकी सीमा को पार करे, उसी चैनल पर जो आपकी टीम पहले से देखती है, बजाय इसके कि किसी ग्राहक से सुनें।" --- -उसी क्षण जानें जब कोई चीज़ आपकी सीमा को पार करे, उसी चैनल पर जो आपकी टीम पहले से देखती है, बजाय इसके कि किसी ग्राहक से सुनें। एक बार नियम सेट करें और Failproof AI Observability इसे निर्धारित अनुसूची पर जांचता है, फिर ईमेल, Slack, webhook, या सीधे डैशबोर्ड में आपको सूचित करता है। +उसी क्षण जानें जब कोई चीज़ आपकी सीमा को पार करे, उसी चैनल पर जो आपकी टीम पहले से देखती है, बजाय इसके कि किसी ग्राहक से सुनें। एक बार नियम सेट करें और FailproofAI Cloud इसे निर्धारित अनुसूची पर जांचता है, फिर ईमेल, Slack, webhook, या सीधे डैशबोर्ड में आपको सूचित करता है। -![सतर्कता पृष्ठ: सतर्कता-नियम कार्डों का एक ग्रिड, प्रत्येक अपने ट्रिगर, मूल्यांकन विंडो, चैनल, और एक सूचना, चेतावनी, या महत्वपूर्ण गंभीरता बैज दिखा रहा है](/agenteye/images/alerts.png) +![सतर्कता पृष्ठ: सतर्कता-नियम कार्डों का एक ग्रिड, प्रत्येक अपने ट्रिगर, मूल्यांकन विंडो, चैनल, और एक सूचना, चेतावनी, या महत्वपूर्ण गंभीरता बैज दिखा रहा है](/cloud/images/alerts.png) *एक नज़र में हर सतर्कता नियम: यह क्या देखता है, कितनी बार, कहां सूचित करता है, और कितना जरूरी है।* ## अपने उपयोगकर्ताओं से पहले समस्याओं के बारे में जानें @@ -21,9 +21,9 @@ description: "उसी क्षण जानें जब कोई चीज ## फॉर्म में नियम बनाएं, JSON में नहीं -आप एक फॉर्म में बताते हैं कि "टूटा हुआ" का अर्थ क्या है, और Failproof AI Observability आपके लिए अंतर्निहित नियम लिखता है। JSON spec केवल वह है जो वह फॉर्म हुड के नीचे बनाता है, इसलिए आप इसे एक नियम को समझने के लिए पढ़ सकते हैं लेकिन आप शायद ही कभी इसे टाइप करते हैं। +आप एक फॉर्म में बताते हैं कि "टूटा हुआ" का अर्थ क्या है, और FailproofAI Cloud आपके लिए अंतर्निहित नियम लिखता है। JSON spec केवल वह है जो वह फॉर्म हुड के नीचे बनाता है, इसलिए आप इसे एक नियम को समझने के लिए पढ़ सकते हैं लेकिन आप शायद ही कभी इसे टाइप करते हैं। -![नई-सतर्कता फॉर्म: नाम और विवरण, एक सक्षम टॉगल, और एक ट्रिगर पिकर जो मेट्रिक थ्रेसहोल्ड, कस्टम SQL, मूल्यांकन स्कोर, यौगिक मूल्यांकन, और प्रति-ईवेंट शर्तें प्रदान करता है](/agenteye/images/alert-new.png) +![नई-सतर्कता फॉर्म: नाम और विवरण, एक सक्षम टॉगल, और एक ट्रिगर पिकर जो मेट्रिक थ्रेसहोल्ड, कस्टम SQL, मूल्यांकन स्कोर, यौगिक मूल्यांकन, और प्रति-ईवेंट शर्तें प्रदान करता है](/cloud/images/alert-new.png) *एक ट्रिगर चुनें और फॉर्म सही फील्ड में स्वैप करता है; सहेजें नियम लिखता है।* खुशियों की राह तेज़ है: इसका नाम दें, एक **ट्रिगर** चुनें (क्या देखना है), **थ्रेसहोल्ड और विंडो** सेट करें (कितना बुरा, कितने समय में), कम से कम एक **चैनल** संलग्न करें, फिर **सहेजें** और **परीक्षण** दबाएं एक कृत्रिम सूचना भेजने के लिए और पुष्टि करें कि हर गंतव्य सेट अप है। हुड के नीचे यह एक छोटा spec बनाता है जैसे: @@ -42,7 +42,7 @@ description: "उसी क्षण जानें जब कोई चीज | **यौगिक मूल्यांकन** | कई स्कोर जांचें any, all, या कम से कम-N तर्क के साथ संयोजित होते हैं, एक प्रतिगमन को पकड़ने के लिए जो केवल स्कोर में दिखाई देता है | | **प्रति ईवेंट** | एक एकल मिलान वाली ईवेंट आती है: एक विशिष्ट agent, एक विशिष्ट त्रुटि प्रकार, या एक संदेश सबस्ट्रिंग | -पहले से ही [त्रुटि पृष्ठ](/hi/agenteye/error-tracking) पर एक विफलता को देख रहे हैं? वहां हर पंक्ति में एक **+ alert** बटन है जो इसी फॉर्म को खोलता है उस सटीक विफलता को पकड़ने के लिए पूर्वनिर्धारित, इसलिए घटना जिसे आपने अभी ट्रियेज किया वह वह है जो अगली बार आपको सूचित करती है। +पहले से ही [त्रुटि पृष्ठ](/hi/cloud/errors) पर एक विफलता को देख रहे हैं? वहां हर पंक्ति में एक **+ alert** बटन है जो इसी फॉर्म को खोलता है उस सटीक विफलता को पकड़ने के लिए पूर्वनिर्धारित, इसलिए घटना जिसे आपने अभी ट्रियेज किया वह वह है जो अगली बार आपको सूचित करती है। **इसे कहां खोजें:** Alerts `//alerts` पर रहते हैं। नियम बनाना, संपादन, हटाना, और परीक्षण करना **`alerts:write`** की आवश्यकता है; `alerts:read` देखने के लिए पर्याप्त है। प्राप्तकर्ता पिकर आपके org के सदस्यों को नाम के अनुसार सूचीबद्ध करता है, इसलिए आप फॉर्म छोड़े बिना एक व्यक्ति को सूचित कर सकते हैं। @@ -52,11 +52,11 @@ description: "उसी क्षण जानें जब कोई चीज ## जब कोई सतर्कता फायर होती है तो क्या होता है -एक उल्लंघन एक **घटना** खोलता है और आपके चैनलों को एक बार सूचित करता है। वहां से आपकी टीम इसे स्वीकार करती है, एक मालिक निर्दिष्ट करती है, इसके माध्यम से बात करती है, और इसे हल करती है, सब कुछ एक स्वच्छ, जिम्मेदार रिकॉर्ड के विरुद्ध। वह ट्रियेज वर्कफ़्लो का अपना घर है: [घटनाएं](/hi/agenteye/incidents) देखें। +एक उल्लंघन एक **घटना** खोलता है और आपके चैनलों को एक बार सूचित करता है। वहां से आपकी टीम इसे स्वीकार करती है, एक मालिक निर्दिष्ट करती है, इसके माध्यम से बात करती है, और इसे हल करती है, सब कुछ एक स्वच्छ, जिम्मेदार रिकॉर्ड के विरुद्ध। वह ट्रियेज वर्कफ़्लो का अपना घर है: [घटनाएं](/hi/cloud/incidents) देखें। ## संबंधित -- [घटनाएं](/hi/agenteye/incidents): एक फायर की हुई सतर्कता को खुले से स्वीकृत से हल तक ट्रैक करें। -- [त्रुटि ट्रैकिंग](/hi/agenteye/error-tracking): agent विफलताओं को समूहीकृत करें और एक क्लिक में एक को सतर्कता में प्रचार करें। -- [डैशबोर्ड](/hi/agenteye/dashboards): साझा बोर्ड देखें जिन थ्रेसहोल्ड पर आप सतर्क होते हैं वे कहां से आते हैं। -- [CLI और agents](/hi/agenteye/cli-and-agents): अपने टर्मिनल से सतर्कता बनाएं और घटनाओं को स्वीकार करें, या उन्हें CI में स्क्रिप्ट करें। \ No newline at end of file +- [घटनाएं](/hi/cloud/incidents): एक फायर की हुई सतर्कता को खुले से स्वीकृत से हल तक ट्रैक करें। +- [त्रुटि ट्रैकिंग](/hi/cloud/errors): agent विफलताओं को समूहीकृत करें और एक क्लिक में एक को सतर्कता में प्रचार करें। +- [डैशबोर्ड](/hi/cloud/dashboards): साझा बोर्ड देखें जिन थ्रेसहोल्ड पर आप सतर्क होते हैं वे कहां से आते हैं। +- [CLI और agents](/hi/cloud/cli): अपने टर्मिनल से सतर्कता बनाएं और घटनाओं को स्वीकार करें, या उन्हें CI में स्क्रिप्ट करें। \ No newline at end of file diff --git a/docs/hi/agenteye/assistant.mdx b/docs/hi/cloud/assistant.mdx similarity index 86% rename from docs/hi/agenteye/assistant.mdx rename to docs/hi/cloud/assistant.mdx index 9ca76e7d..78f58270 100644 --- a/docs/hi/agenteye/assistant.mdx +++ b/docs/hi/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "अपने एजेंट डेटा से सामान --- -अपने एजेंट डेटा से सामान्य अंग्रेजी में प्रश्न पूछें और ऐसा उत्तर प्राप्त करें जो सीधे साक्ष्य से जुड़ा हो। कोई SQL लिखने की आवश्यकता नहीं, डैशबोर्ड को खोदने की जरूरत नहीं — **Failproof AI Observability** सहायक आपकी टीम के किसी भी सदस्य के लिए एजेंट्स के बारे में उत्तर पाने का सबसे तेज तरीका है। +अपने एजेंट डेटा से सामान्य अंग्रेजी में प्रश्न पूछें और ऐसा उत्तर प्राप्त करें जो सीधे साक्ष्य से जुड़ा हो। कोई SQL लिखने की आवश्यकता नहीं, डैशबोर्ड को खोदने की जरूरत नहीं — **FailproofAI Cloud** सहायक आपकी टीम के किसी भी सदस्य के लिए एजेंट्स के बारे में उत्तर पाने का सबसे तेज तरीका है। -![Failproof AI Observability सहायक डैशबोर्ड के अंदर एक सामान्य-अंग्रेजी प्रश्न का उत्तर दे रहा है, जो एक लाइव एजेंट एक्टिविटी टेबल, प्रति-एजेंट मॉडल-उपयोग विभाजन और लिखित निष्कर्ष दिखा रहा है, जिसमें यह दिखाए गए क्वेरीज इनलाइन हैं](/agenteye/images/assistant.png) +![FailproofAI Cloud सहायक डैशबोर्ड के अंदर एक सामान्य-अंग्रेजी प्रश्न का उत्तर दे रहा है, जो एक लाइव एजेंट एक्टिविटी टेबल, प्रति-एजेंट मॉडल-उपयोग विभाजन और लिखित निष्कर्ष दिखा रहा है, जिसमें यह दिखाए गए क्वेरीज इनलाइन हैं](/cloud/images/assistant.png) *सामान्य अंग्रेजी में पूछें और अपने स्वयं के डेटा से बनाया गया उत्तर प्राप्त करें। यहां यह दिखाता है कि कौन से एजेंट सबसे व्यस्त हैं और वे किन मॉडल का उपयोग करते हैं, और यह दिखाता है कि यह किन क्वेरीज को चलाता है ताकि आप प्रत्येक संख्या को सत्यापित कर सकें।* सीखने के लिए कुछ नहीं है। चैट खोलें, टाइप करें कि आप क्या जानना चाहते हैं, और इसके द्वारा दिए गए लिंक का पालन करें: @@ -36,7 +36,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a **Queries** पेज पर यह एक कदम आगे जाता है और एक SQL लेखक बन जाता है: उस क्वेरी का वर्णन करें जो आप चाहते हैं ("पिछले 7 दिनों के लिए एजेंट द्वारा त्रुटि दर दिखाएं") और यह SQL को सीधे संपादक में स्ट्रीम करता है, एक अंतर दृश्य खोलता है ताकि आप **स्वीकार करें** या **अस्वीकार करें** परिवर्तन से पहले यह चेक कर सकें। -![Observability Queries पृष्ठ और इसका SQL संपादक](/agenteye/images/query-lab.png) +![FailproofAI Cloud Queries पृष्ठ और इसका SQL संपादक](/cloud/images/query-lab.png) *Queries पृष्ठ: यह संपादक वह स्थान है जहां सहायक एक मसौदा, केवल-पठन योग्य क्वेरी स्ट्रीम करता है ताकि आप स्वीकार या अस्वीकार कर सकें।* यहां SQL लेखन करना `queries:run` अनुमति का उपयोग करता है, जो संपादक के **Run** बटन के पीछे भी है। अन्य जगह चैट करने के लिए `agent:use` की आवश्यकता है। @@ -49,7 +49,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a - **हर लेखन आपके लिए प्रतीक्षा करता है।** सहेजी गई क्वेरीज और डैशबोर्ड केवल आपकी स्पष्ट अनुमोदन क्लिक के बाद बनाए जाते हैं, और कोई सेटिंग नहीं है जो उस गेट को बंद करता है। - **यह कभी भी कुछ नहीं हटा सकता।** कोई हटाने का उपकरण नहीं है और सहायक के पास कोई हटाने की अनुमति नहीं है। हटाने डैशबोर्ड में आपके हाथों में रहते हैं। - **यह आपके संगठन के अंदर रहता है।** सहायक केवल उस संगठन को देखता है जिसे आप वर्तमान में देख रहे हैं। -- **आपके प्रश्न आपके हैं।** संकेत और उत्तर आपके स्वयं के Observability डेटाबेस में रहते हैं; उत्पाद विश्लेषण केवल उपयोग मेटाडेटा रिकॉर्ड करता है, कभी भी आपके संकेत पाठ को नहीं। +- **आपके प्रश्न आपके हैं।** संकेत और उत्तर आपके स्वयं के FailproofAI Cloud डेटाबेस में रहते हैं; उत्पाद विश्लेषण केवल उपयोग मेटाडेटा रिकॉर्ड करता है, कभी भी आपके संकेत पाठ को नहीं। ## इसे कहां खोजें @@ -57,7 +57,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a ## संबंधित -- [CLI और agents](/hi/agenteye/cli-and-agents) -- [Queries](/hi/agenteye/queries) -- [Dashboards](/hi/agenteye/dashboards) -- [Evaluation suite](/hi/agenteye/evaluation-suite) \ No newline at end of file +- [CLI और agents](/hi/cloud/cli) +- [Queries](/hi/cloud/queries) +- [Dashboards](/hi/cloud/dashboards) +- [Evaluation suite](/hi/cloud/evaluators) \ No newline at end of file diff --git a/docs/hi/agenteye/audits.mdx b/docs/hi/cloud/audits.mdx similarity index 82% rename from docs/hi/agenteye/audits.mdx rename to docs/hi/cloud/audits.mdx index ea0b96fb..e27c23e9 100644 --- a/docs/hi/agenteye/audits.mdx +++ b/docs/hi/cloud/audits.mdx @@ -1,9 +1,9 @@ --- title: "ऑडिट: आपका स्वचालित विश्वसनीयता विश्लेषक" -description: "Failproof AI Observability उन विफलताओं को खोजता है जिनके लिए आपने कोई नियम नहीं लिखा था और आपको ठीक करने के लिए आवश्यक चीजों की एक रैंक की गई, साक्ष्य-समर्थित सूची देता है।" +description: "FailproofAI Cloud उन विफलताओं को खोजता है जिनके लिए आपने कोई नियम नहीं लिखा था और आपको ठीक करने के लिए आवश्यक चीजों की एक रैंक की गई, साक्ष्य-समर्थित सूची देता है।" --- -Failproof AI Observability उन विफलताओं को खोजता है जिनके लिए आपने कोई नियम नहीं लिखा था और आपको ठीक करने के लिए आवश्यक चीजों की एक रैंक की गई, साक्ष्य-समर्थित सूची देता है। यह ऐसा है जैसे कोई विश्लेषक हर रात आपके लॉग को देखे, और फिर सुबह तक छोटी सूची आपकी डेस्क पर छोड़ दे। +FailproofAI Cloud उन विफलताओं को खोजता है जिनके लिए आपने कोई नियम नहीं लिखा था और आपको ठीक करने के लिए आवश्यक चीजों की एक रैंक की गई, साक्ष्य-समर्थित सूची देता है। यह ऐसा है जैसे कोई विश्लेषक हर रात आपके लॉग को देखे, और फिर सुबह तक छोटी सूची आपकी डेस्क पर छोड़ दे।
@@ -11,7 +11,7 @@ Failproof AI Observability उन विफलताओं को खोजत *दो मिनट का दौरा: एक निर्धारित रन से लेकर एक ऐसे फिक्स तक जिस पर आप कार्य कर सकते हैं।* -![ऑडिट पृष्ठ: आवर्ती कार्य जो आपके सत्रों को विफलता पैटर्न के लिए स्कैन करते हैं, प्रत्येक के साथ एक शेड्यूल और संवेदनशीलता](/agenteye/images/audits.png) +![ऑडिट पृष्ठ: आवर्ती कार्य जो आपके सत्रों को विफलता पैटर्न के लिए स्कैन करते हैं, प्रत्येक के साथ एक शेड्यूल और संवेदनशीलता](/cloud/images/audits.png) *प्रत्येक ऑडिट एक आवर्ती कार्य है जो आपके सत्रों को माइन करता है और रैंक की गई, साक्ष्य-समर्थित सिफारिशें लिखता है।* ## अनुमान लगाना बंद करें कि आगे क्या ठीक करना है @@ -47,7 +47,7 @@ Failproof AI Observability उन विफलताओं को खोजत ## संबंधित -- [अलर्ट](/hi/agenteye/alerts): जिस पल एक थ्रेसहोल्ड को पार किया जाता है उस पल एक पेज प्राप्त करें। -- [मूल्यांकन](/hi/agenteye/evaluations): हर रन को स्कोर करें ताकि गुणवत्ता प्रतिगमन अपने आप सामने आएं। -- [त्रुटि ट्रैकिंग](/hi/agenteye/error-tracking): एजेंटों द्वारा फेंकी जाने वाली त्रुटियों को समूहित और अनुसरण करें। -- [घटनाएं](/hi/agenteye/incidents): एक ऑडिट के माध्यम से एक समस्या को ट्रैक करें जो यह इसके फिक्स के माध्यम से बदल देता है। \ No newline at end of file +- [अलर्ट](/hi/cloud/alerts): जिस पल एक थ्रेसहोल्ड को पार किया जाता है उस पल एक पेज प्राप्त करें। +- [मूल्यांकन](/hi/cloud/evaluations): हर रन को स्कोर करें ताकि गुणवत्ता प्रतिगमन अपने आप सामने आएं। +- [त्रुटि ट्रैकिंग](/hi/cloud/errors): एजेंटों द्वारा फेंकी जाने वाली त्रुटियों को समूहित और अनुसरण करें। +- [घटनाएं](/hi/cloud/incidents): एक ऑडिट के माध्यम से एक समस्या को ट्रैक करें जो यह इसके फिक्स के माध्यम से बदल देता है। \ No newline at end of file diff --git a/docs/hi/cloud/capture.mdx b/docs/hi/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/hi/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/hi/agenteye/cli-recipes.mdx b/docs/hi/cloud/cli-recipes.mdx similarity index 89% rename from docs/hi/agenteye/cli-recipes.mdx rename to docs/hi/cloud/cli-recipes.mdx index 148cbede..e7618ada 100644 --- a/docs/hi/agenteye/cli-recipes.mdx +++ b/docs/hi/cloud/cli-recipes.mdx @@ -3,9 +3,9 @@ title: "एजेंटों के लिए CLI रेसिपीज़" description: "कॉपी-पेस्ट क्वेरी पैटर्न और jq रेसिपीज़ जो सेशन, इवेंट और मूल्यांकन डेटा को ऐसी चीज़ में बदल देते हैं जिसे एक स्क्रिप्ट या कोडिंग एजेंट स्वचालित कर सकता है।" --- -एक स्क्रिप्ट या कोडिंग एजेंट से सीधे सेशन, इवेंट और मूल्यांकन डेटा खींचें (और पुनः-मूल्यांकन ट्रिगर करें), स्टडआउट पर स्वच्छ JSON के साथ जो सीधे `jq` में पाइप होता है। ये रेसिपीज़ Failproof AI Observability के डेटा को ऐसी चीज़ में बदल देते हैं जिसे एक टर्मिनल उपयोगकर्ता या एक AI कोडिंग एजेंट (Claude Code, Cursor) क्वेरी और स्वचालित कर सकता है, डैशबोर्ड के माध्यम से क्लिक किए बिना। +एक स्क्रिप्ट या कोडिंग एजेंट से सीधे सेशन, इवेंट और मूल्यांकन डेटा खींचें (और पुनः-मूल्यांकन ट्रिगर करें), स्टडआउट पर स्वच्छ JSON के साथ जो सीधे `jq` में पाइप होता है। ये रेसिपीज़ FailproofAI Cloud के डेटा को ऐसी चीज़ में बदल देते हैं जिसे एक टर्मिनल उपयोगकर्ता या एक AI कोडिंग एजेंट (Claude Code, Cursor) क्वेरी और स्वचालित कर सकता है, डैशबोर्ड के माध्यम से क्लिक किए बिना। -नीचे दिए गए पैटर्न Failproof AI Observability CLI (`agenteye`) के लिए कॉपी-पेस्ट के लिए तैयार हैं। इंस्टॉलेशन, प्रमाणीकरण और पूर्ण विकल्प सूची के लिए [CLI](/hi/agenteye/cli) देखें; अंतर्निहित सहायता के लिए `agenteye -h` या `agenteye -h` चलाएं। +नीचे दिए गए पैटर्न FailproofAI Cloud CLI (`agenteye`) के लिए कॉपी-पेस्ट के लिए तैयार हैं। इंस्टॉलेशन, प्रमाणीकरण और पूर्ण विकल्प सूची के लिए [CLI](/hi/cloud/cli) देखें; अंतर्निहित सहायता के लिए `agenteye -h` या `agenteye -h` चलाएं। ## मुख्य नियम @@ -172,7 +172,7 @@ esac ## अगले चरण -- [CLI](/hi/agenteye/cli): इंस्टॉलेशन, प्रमाणीकरण और प्रत्येक कमांड के लिए पूर्ण विकल्प संदर्भ। -- [CLI agent skill](/hi/agenteye/cli-skill): इन रेसिपीज़ को एक कौशल के रूप में पैकेज करें जो आपका कोडिंग एजेंट लोड कर सकता है। -- [API keys](/hi/agenteye/api-keys): कुंजीज़ बनाएं और स्कोप करें जो CLI, SDK और कलेक्टर प्रमाणीकरण करते हैं। -- [Python SDK](/hi/agenteye/python-sdk): Failproof AI Observability में इवेंट भेजें ताकि इन रेसिपीज़ के लिए क्वेरी करने के लिए डेटा हो। \ No newline at end of file +- [CLI](/hi/cloud/cli): इंस्टॉलेशन, प्रमाणीकरण और प्रत्येक कमांड के लिए पूर्ण विकल्प संदर्भ। +- [CLI agent skill](/hi/cloud/agent-skills): इन रेसिपीज़ को एक कौशल के रूप में पैकेज करें जो आपका कोडिंग एजेंट लोड कर सकता है। +- [API keys](/hi/cloud/access): कुंजीज़ बनाएं और स्कोप करें जो CLI, SDK और कलेक्टर प्रमाणीकरण करते हैं। +- [Python SDK](/hi/cloud/sdk): FailproofAI Cloud में इवेंट भेजें ताकि इन रेसिपीज़ के लिए क्वेरी करने के लिए डेटा हो। \ No newline at end of file diff --git a/docs/hi/agenteye/cli.mdx b/docs/hi/cloud/cli.mdx similarity index 90% rename from docs/hi/agenteye/cli.mdx rename to docs/hi/cloud/cli.mdx index efe81e2b..eaa93372 100644 --- a/docs/hi/agenteye/cli.mdx +++ b/docs/hi/cloud/cli.mdx @@ -1,10 +1,10 @@ --- title: "CLI" -description: "Failproof AI Observability को टर्मिनल या स्क्रिप्ट से चलाएँ: कोई डैशबोर्ड राउंड-ट्रिप नहीं।" +description: "FailproofAI Cloud को टर्मिनल या स्क्रिप्ट से चलाएँ: कोई डैशबोर्ड राउंड-ट्रिप नहीं।" --- -Failproof AI Observability को टर्मिनल या स्क्रिप्ट से पूरी तरह चलाएँ: कोई डैशबोर्ड राउंड-ट्रिप नहीं। `agenteye` CLI आपके डेटा (सेशन, इवेंट लॉग, मूल्यांकन) को क्वेरी करता है और आपके संगठन (API कुंजियाँ, उपयोगकर्ता, सेटिंग्स, अलर्ट, घटनाएँ, सहेजी गई क्वेरी) का प्रबंधन करता है, इसलिए जब आप किसी जाँच को स्वचालित करना चाहते हैं, CI में Observability को जोड़ना चाहते हैं, या कोई कोडिंग एजेंट प्रोडक्शन का निरीक्षण करे, तो इसका उपयोग करें। प्रत्येक कमांड `--json` फ़्लैग को सपोर्ट करता है, इसलिए यह प्रॉम्प्ट पर आपके लिए समान रूप से अच्छी तरह काम करता है या कोई कोडिंग एजेंट (Claude Code, Cursor) शेल आउट करके परिणाम पार्स कर सकता है। +FailproofAI Cloud को टर्मिनल या स्क्रिप्ट से पूरी तरह चलाएँ: कोई डैशबोर्ड राउंड-ट्रिप नहीं। `agenteye` CLI आपके डेटा (सेशन, इवेंट लॉग, मूल्यांकन) को क्वेरी करता है और आपके संगठन (API कुंजियाँ, उपयोगकर्ता, सेटिंग्स, अलर्ट, घटनाएँ, सहेजी गई क्वेरी) का प्रबंधन करता है, इसलिए जब आप किसी जाँच को स्वचालित करना चाहते हैं, CI में FailproofAI Cloud को जोड़ना चाहते हैं, या कोई कोडिंग एजेंट प्रोडक्शन का निरीक्षण करे, तो इसका उपयोग करें। प्रत्येक कमांड `--json` फ़्लैग को सपोर्ट करता है, इसलिए यह प्रॉम्प्ट पर आपके लिए समान रूप से अच्छी तरह काम करता है या कोई कोडिंग एजेंट (Claude Code, Cursor) शेल आउट करके परिणाम पार्स कर सकता है। एक ही बाइनरी के साथ आप कर सकते हैं: @@ -69,7 +69,7 @@ agenteye --version agenteye --help ``` -> **नोट:** Failproof AI Observability Python SDK भी `agenteye` वितरण नाम का उपयोग करता है। `pipx` या `uv tool` के साथ CLI को स्थापित करना (साझा virtualenv में `pip install` के बजाय) दोनों को टकराने से रोकता है। एक सादा `pip install agenteye` तभी ठीक है यदि SDK उसी वातावरण में स्थापित नहीं है। +> **नोट:** FailproofAI Cloud Python SDK भी `agenteye` वितरण नाम का उपयोग करता है। `pipx` या `uv tool` के साथ CLI को स्थापित करना (साझा virtualenv में `pip install` के बजाय) दोनों को टकराने से रोकता है। एक सादा `pip install agenteye` तभी ठीक है यदि SDK उसी वातावरण में स्थापित नहीं है। --- @@ -91,7 +91,7 @@ agenteye logout # revoke the session and clear the stored token `whoami` कभी भी लापता या समाप्त सेशन पर त्रुटि नहीं करता; बजाय इसके `logged_in: false` की रिपोर्ट करता है, इसलिए एक स्क्रिप्ट या एजेंट सुरक्षित रूप से प्रमाणन स्थिति की जाँच कर सकता है (यदि कोई आधार URL सेट नहीं है या डैशबोर्ड अप्राप्य है तो यह अभी भी गैर-शून्य बाहर निकल सकता है)। -**आवश्यकताएँ:** आपके ईमेल को डैशबोर्ड में साइन इन करने की अनुमति दी जानी चाहिए (अपने Failproof AI Observability व्यवस्थापक से पूछें), और डैशबोर्ड को इसके आधार URL पर पहुँचने योग्य होना चाहिए (देखें [कॉन्फ़िगरेशन](#configuration))। यदि आप कोड का अनुरोध करते हैं और कोई भी नहीं आता है, तो आपका ईमेल शायद अभी तक डैशबोर्ड पहुँच के लिए सक्षम नहीं है। +**आवश्यकताएँ:** आपके ईमेल को डैशबोर्ड में साइन इन करने की अनुमति दी जानी चाहिए (अपने FailproofAI Cloud व्यवस्थापक से पूछें), और डैशबोर्ड को इसके आधार URL पर पहुँचने योग्य होना चाहिए (देखें [कॉन्फ़िगरेशन](#configuration))। यदि आप कोड का अनुरोध करते हैं और कोई भी नहीं आता है, तो आपका ईमेल शायद अभी तक डैशबोर्ड पहुँच के लिए सक्षम नहीं है। --- @@ -338,13 +338,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | प्रमाणित, लेकिन आपके खाते में आवश्यक अनुमति नहीं है (संदेश इसका नाम देता है) | | 6 | अनुरोधित संसाधन नहीं मिला (जैसे अज्ञात सेशन या घटना id) | -ये CLI को स्क्रिप्ट के लिए सुरक्षित बनाते हैं: एक कोडिंग एजेंट एक `4` पर शाखा बना सकता है आपको फिर से प्रमाणित करने का संकेत देने के लिए, या एक `5` लापता अनुमति की सतह के लिए। CLI रेसिपी देखें [एजेंट के लिए](/hi/agenteye/cli-recipes) निकास-कोड-संभालने के पैटर्न और JSON आउटपुट आकार के लिए। +ये CLI को स्क्रिप्ट के लिए सुरक्षित बनाते हैं: एक कोडिंग एजेंट एक `4` पर शाखा बना सकता है आपको फिर से प्रमाणित करने का संकेत देने के लिए, या एक `5` लापता अनुमति की सतह के लिए। CLI रेसिपी देखें [एजेंट के लिए](/hi/cloud/cli-recipes) निकास-कोड-संभालने के पैटर्न और JSON आउटपुट आकार के लिए। --- ## अगले कदम -- **[एजेंट के लिए CLI रेसिपी](/hi/agenteye/cli-recipes)**: कॉपी-पेस्ट क्वेरी पैटर्न, `jq` एक-लाइनर, `--fields` प्रक्षेपण, निकास-कोड संभालना, और JSON आउटपुट आकार, कोडिंग एजेंट के लिए लिखा हुआ CLI चला रहे हैं। -- **[CLI एजेंट कौशल](/hi/agenteye/cli-skill)**: इस CLI को स्थापन योग्य Claude Code / Codex *कौशल* के रूप में पैकेज करें ताकि एक कोडिंग एजेंट सादे-अंग्रेजी अनुरोध से Failproof AI Observability चला सके। -- **[API कुंजियाँ](/hi/agenteye/api-keys)**: `keys create --add …` के पीछे अनुमति मॉडल। -- **[AI सहायक](/hi/agenteye/assistant)**: सहायक को सक्षम करना जिससे `agent ask` बात करता है। \ No newline at end of file +- **[एजेंट के लिए CLI रेसिपी](/hi/cloud/cli-recipes)**: कॉपी-पेस्ट क्वेरी पैटर्न, `jq` एक-लाइनर, `--fields` प्रक्षेपण, निकास-कोड संभालना, और JSON आउटपुट आकार, कोडिंग एजेंट के लिए लिखा हुआ CLI चला रहे हैं। +- **[CLI एजेंट कौशल](/hi/cloud/agent-skills)**: इस CLI को स्थापन योग्य Claude Code / Codex *कौशल* के रूप में पैकेज करें ताकि एक कोडिंग एजेंट सादे-अंग्रेजी अनुरोध से FailproofAI Cloud चला सके। +- **[API कुंजियाँ](/hi/cloud/access)**: `keys create --add …` के पीछे अनुमति मॉडल। +- **[AI सहायक](/hi/cloud/assistant)**: सहायक को सक्षम करना जिससे `agent ask` बात करता है। \ No newline at end of file diff --git a/docs/hi/cloud/connect.mdx b/docs/hi/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/hi/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/hi/agenteye/dashboards.mdx b/docs/hi/cloud/dashboards.mdx similarity index 73% rename from docs/hi/agenteye/dashboards.mdx rename to docs/hi/cloud/dashboards.mdx index b06f4446..46bd1957 100644 --- a/docs/hi/agenteye/dashboards.mdx +++ b/docs/hi/cloud/dashboards.mdx @@ -7,7 +7,7 @@ description: "अपने लाइव एजेंट डेटा को ए अपने लाइव एजेंट डेटा को एक साझा चित्र में बदलें जिसे आपकी पूरी टीम देखती है। जो क्वेरीज़ महत्वपूर्ण हैं उन्हें चार्ट के रूप में पिन करें, और हर कोई एक नज़र में एक जैसे नंबर देखता है, बिना एक भी क्वेरी को फिर से चलाए। -![एक डैशबोर्ड सहेजी गई क्वेरीज़ से बना है: एक घंटे में ईवेंट्स की लाइन, प्रकार के अनुसार त्रुटियों की बार, लेटेंसी एरिया चार्ट, और मॉडल के अनुसार टोकन](/agenteye/images/dashboard-fleet.png) +![एक डैशबोर्ड सहेजी गई क्वेरीज़ से बना है: एक घंटे में ईवेंट्स की लाइन, प्रकार के अनुसार त्रुटियों की बार, लेटेंसी एरिया चार्ट, और मॉडल के अनुसार टोकन](/cloud/images/dashboard-fleet.png) *एक बोर्ड, चार सहेजी गई क्वेरीज़: प्रति घंटा ईवेंट्स, प्रकार के अनुसार त्रुटियां, लेटेंसी, और मॉडल के अनुसार टोकन।* @@ -26,15 +26,15 @@ description: "अपने लाइव एजेंट डेटा को ए ## उन क्वेरीज़ को पिन करें जिन्हें आपने पहले से सहेज रखा है -प्रत्येक टाइल एक सहेजी गई क्वेरी से शुरू होता है। उस क्वेरी को बनाएं और सहेजें जिसकी आपको परवाह है [Queries](/hi/agenteye/queries) लाइब्रेरी में (निर्मित प्रीसेट्स प्लस आपकी अपनी, आपके ईवेंट्स और मूल्यांकन के ऊपर), फिर इसे डैशबोर्ड पर उस चार्ट के रूप में पिन करें जो डेटा के अनुरूप हो: एक **line** समय के साथ ट्रेंड्स के लिए, एक **bar** श्रेणियों की तुलना के लिए, एक **area** वॉल्यूम के लिए, या एक **pie** शेयर विभाजन के लिए। +प्रत्येक टाइल एक सहेजी गई क्वेरी से शुरू होता है। उस क्वेरी को बनाएं और सहेजें जिसकी आपको परवाह है [Queries](/hi/cloud/queries) लाइब्रेरी में (निर्मित प्रीसेट्स प्लस आपकी अपनी, आपके ईवेंट्स और मूल्यांकन के ऊपर), फिर इसे डैशबोर्ड पर उस चार्ट के रूप में पिन करें जो डेटा के अनुरूप हो: एक **line** समय के साथ ट्रेंड्स के लिए, एक **bar** श्रेणियों की तुलना के लिए, एक **area** वॉल्यूम के लिए, या एक **pie** शेयर विभाजन के लिए। क्योंकि एक टाइल केवल आपकी सहेजी गई क्वेरी है जिसे चार्ट के रूप में प्रदर्शित किया गया है, हाथ से सिंक रखने के लिए कुछ भी नहीं है। क्वेरी को एक बार अपडेट करें और हर डैशबोर्ड जो इसका उपयोग करता है वह भी अपडेट हो जाता है। ## वॉल्यूम नहीं, गुणवत्ता देखें -वॉल्यूम आपको बताता है कि एजेंट व्यस्त हैं। गुणवत्ता आपको बताती है कि वे वास्तव में काम कर रहे हैं। अपने डैशबोर्ड को अपने [evaluation scores](/hi/agenteye/evaluations) की ओर इशारा करें और आपको एक बोर्ड मिलता है जो समय के साथ ट्रैक करता है कि रन कितनी अच्छी तरह चल रहे हैं, इसलिए गुणवत्ता में गिरावट एक चार्ट पर एक डिप के रूप में दिखाई देती है, न कि ग्राहक से एक आश्चर्य के रूप में। +वॉल्यूम आपको बताता है कि एजेंट व्यस्त हैं। गुणवत्ता आपको बताती है कि वे वास्तव में काम कर रहे हैं। अपने डैशबोर्ड को अपने [evaluation scores](/hi/cloud/evaluations) की ओर इशारा करें और आपको एक बोर्ड मिलता है जो समय के साथ ट्रैक करता है कि रन कितनी अच्छी तरह चल रहे हैं, इसलिए गुणवत्ता में गिरावट एक चार्ट पर एक डिप के रूप में दिखाई देती है, न कि ग्राहक से एक आश्चर्य के रूप में। -![सहेजी गई मूल्यांकन क्वेरीज़ से बना एक गुणवत्ता-केंद्रित डैशबोर्ड](/agenteye/images/dashboard-quality.png) +![सहेजी गई मूल्यांकन क्वेरीज़ से बना एक गुणवत्ता-केंद्रित डैशबोर्ड](/cloud/images/dashboard-quality.png) *एक गुणवत्ता बोर्ड आपके मूल्यांकन स्कोर को सामने और केंद्र में रखता है, संचालन संख्याओं के ठीक बगल में।* @@ -42,6 +42,6 @@ description: "अपने लाइव एजेंट डेटा को ए ## संबंधित -- [Queries](/hi/agenteye/queries): उन क्वेरीज़ को बनाएं और सहेजें जो आपके टाइल्स बन जाती हैं। -- [Evaluations](/hi/agenteye/evaluations): अपने रन को स्कोर करें ताकि आप समय के साथ गुणवत्ता को चार्ट कर सकें। -- [Alerts](/hi/agenteye/alerts): इन मेट्रिक्स में से किसी भी थ्रेसहोल्ड को एक पेज में बदलें। \ No newline at end of file +- [Queries](/hi/cloud/queries): उन क्वेरीज़ को बनाएं और सहेजें जो आपके टाइल्स बन जाती हैं। +- [Evaluations](/hi/cloud/evaluations): अपने रन को स्कोर करें ताकि आप समय के साथ गुणवत्ता को चार्ट कर सकें। +- [Alerts](/hi/cloud/alerts): इन मेट्रिक्स में से किसी भी थ्रेसहोल्ड को एक पेज में बदलें। \ No newline at end of file diff --git a/docs/hi/agenteye/error-tracking.mdx b/docs/hi/cloud/errors.mdx similarity index 68% rename from docs/hi/agenteye/error-tracking.mdx rename to docs/hi/cloud/errors.mdx index 36adc98e..c900e419 100644 --- a/docs/hi/agenteye/error-tracking.mdx +++ b/docs/hi/cloud/errors.mdx @@ -7,14 +7,14 @@ description: "अपने एजेंटों द्वारा उत्प अपने एजेंटों द्वारा उत्पन्न सभी विफलताओं को एक जगह देखें, समूहित ताकि शोर भरा विस्फोट एक एकल समस्या के रूप में दिखे। आपको "कुछ लाल है" से लेकर उस सटीक रन तक एक-क्लिक पथ मिलता है जो टूटा है, लाइव फीड को स्क्रॉल किए बिना। -![Errors पृष्ठ: समय के साथ विफलताओं का एक हिस्टोग्राम ऊपर समूहित लाल त्रुटि पंक्तियों के साथ, प्रत्येक में एक-क्लिक "+ alert" बटन है](/agenteye/images/errors.png) +![Errors पृष्ठ: समय के साथ विफलताओं का एक हिस्टोग्राम ऊपर समूहित लाल त्रुटि पंक्तियों के साथ, प्रत्येक में एक-क्लिक "+ alert" बटन है](/cloud/images/errors.png) *Errors पृष्ठ: समय के साथ विफलताओं का हिस्टोग्राम, दोहराई गई विफलताओं को प्रति घटना एक पंक्ति में संपीड़ित किया गया है।* ## हर विफलता, पहले से ही आपके लिए एकत्र की गई जब कोई एजेंट विफल होता है, तो आपको यह आशा नहीं करनी चाहिए कि एक लाइव इवेंट स्ट्रीम को स्क्रॉल करें और लाल पंक्तियों को देखते रहें। **Errors** पृष्ठ आपके लिए एकत्रण करता है। यह डैशबोर्ड को लाल रंग में दिखाए जाने वाली सभी चीजों को एक ट्रिएज सतह में लाता है, ताकि आप जो पहली चीज देखें वह है क्या विफल हो रहा है, न कि इसे कहां खोजने के लिए जाएं। -और यह स्पष्ट लोगों से अधिक कैच करता है। स्पष्ट `error` इवेंट्स के साथ-साथ, Failproof AI Observability शांत विफलताओं को भी सतह पर लाता है: कोई भी `tool_result`, `hook_completed`, या `agent_end` जिसका पेलोड विफलता ले जाता है वह यहां दिखाई देता है। एक उपकरण जो त्रुटि लौटाता है, या एक हुक जो बुरी तरह बाहर निकलता है, अब आपसे छिप नहीं जाता क्योंकि कुछ भी जोर से अपवाद नहीं फेंकता। +और यह स्पष्ट लोगों से अधिक कैच करता है। स्पष्ट `error` इवेंट्स के साथ-साथ, FailproofAI Cloud शांत विफलताओं को भी सतह पर लाता है: कोई भी `tool_result`, `hook_completed`, या `agent_end` जिसका पेलोड विफलता ले जाता है वह यहां दिखाई देता है। एक उपकरण जो त्रुटि लौटाता है, या एक हुक जो बुरी तरह बाहर निकलता है, अब आपसे छिप नहीं जाता क्योंकि कुछ भी जोर से अपवाद नहीं फेंकता। शीर्ष में, एक हिस्टोग्राम समय के साथ त्रुटियों को प्लॉट करता है। एक नजर आपको बताता है कि यह एक स्थिर पृष्ठभूमि ट्रिकल है या एक स्पाइक जो कुछ मिनट पहले शुरू हुई, इसलिए आप तुरंत जानते हैं कि आप क्या कर रहे हैं। @@ -24,19 +24,19 @@ description: "अपने एजेंटों द्वारा उत्प एक टूटी हुई निर्भरता प्रति मिनट सैकड़ों बार एक ही त्रुटि को फायर कर सकती है। कच्चे रूप में छोड़ दिया, यह लगभग समान लाइनों की एक दीवार है जो एक चीज को दफन कर देती है जिसे आप वास्तव में देखना चाहते हैं। -Failproof AI Observability एक ही सेशन और त्रुटि प्रकार साझा करने वाली विफलताओं को दोहराते हुए एक एकल पंक्ति में संपीड़ित करता है। एक विस्फोट एक घटना के रूप में पढ़ता है। आप समस्याओं को गिनते हैं, लॉग लाइनों को नहीं, और महत्वपूर्ण सिग्नल शीर्ष पर रहता है अपनी स्वयं की मात्रा में डूबने के बजाय। +FailproofAI Cloud एक ही सेशन और त्रुटि प्रकार साझा करने वाली विफलताओं को दोहराते हुए एक एकल पंक्ति में संपीड़ित करता है। एक विस्फोट एक घटना के रूप में पढ़ता है। आप समस्याओं को गिनते हैं, लॉग लाइनों को नहीं, और महत्वपूर्ण सिग्नल शीर्ष पर रहता है अपनी स्वयं की मात्रा में डूबने के बजाय। ## "कुछ लाल है" से सटीक इवेंट तक किसी भी पंक्ति पर क्लिक करें उस रन के सेशन के अंदर सीधे उतरने के लिए, जो विफल हुए सटीक इवेंट पर स्थित है। कोई सेशन ID की नकल नहीं, इसे गलत होने के क्षण को खोजने के लिए स्क्रॉल नहीं करना: आप सीधे इस पर पहुंचते हैं, पूर्ण निष्पादन ग्राफ के साथ एक नज़र दूर ताकि आप देख सकें कि एजेंट ने उसके टूटने से पहले के क्षणों में क्या किया। -यदि आपके पास `alerts:write` है, तो हर पंक्ति में एक **+ alert** बटन भी है। इस पर क्लिक करें और Observability एक नया अलर्ट नियम खोलता है जो पहले से ही उसी विफलता को पकड़ने के लिए भरा हुआ है। जिस घटना का आपने अभी ट्रिएज किया है वह अगली बार आपको पेज करने वाली होगी, इसके बजाय दूसरी बार आपको आश्चर्यचकित करने के बजाय। +यदि आपके पास `alerts:write` है, तो हर पंक्ति में एक **+ alert** बटन भी है। इस पर क्लिक करें और FailproofAI Cloud एक नया अलर्ट नियम खोलता है जो पहले से ही उसी विफलता को पकड़ने के लिए भरा हुआ है। जिस घटना का आपने अभी ट्रिएज किया है वह अगली बार आपको पेज करने वाली होगी, इसके बजाय दूसरी बार आपको आश्चर्यचकित करने के बजाय। **इसे कहां खोजें:** **Errors** पृष्ठ डैशबोर्ड के अवलोकन अनुभाग में रहता है, `//errors` में। ## संबंधित -- [Alerts](/hi/agenteye/alerts): किसी भी विफलता को एक पेजिंग नियम में बदलें। -- [Incidents](/hi/agenteye/incidents): खुले से समाधान तक एक फायरिंग अलर्ट को ट्रैक करें। -- [Sessions](/hi/agenteye/sessions): किसी भी त्रुटि के पीछे पूरा रन खोलें। -- [Audits](/hi/agenteye/audits): Observability को अपने रन के पार विफलता पैटर्न खोजने दें। \ No newline at end of file +- [Alerts](/hi/cloud/alerts): किसी भी विफलता को एक पेजिंग नियम में बदलें। +- [Incidents](/hi/cloud/incidents): खुले से समाधान तक एक फायरिंग अलर्ट को ट्रैक करें। +- [Sessions](/hi/cloud/sessions): किसी भी त्रुटि के पीछे पूरा रन खोलें। +- [Audits](/hi/cloud/audits): FailproofAI Cloud को अपने रन के पार विफलता पैटर्न खोजने दें। \ No newline at end of file diff --git a/docs/hi/agenteye/evaluations.mdx b/docs/hi/cloud/evaluations.mdx similarity index 70% rename from docs/hi/agenteye/evaluations.mdx rename to docs/hi/cloud/evaluations.mdx index 43a02d05..b02f8b57 100644 --- a/docs/hi/agenteye/evaluations.mdx +++ b/docs/hi/cloud/evaluations.mdx @@ -4,15 +4,15 @@ description: "गुणवत्ता की समस्याएं अब --- -गुणवत्ता की समस्याएं अब आपको मिलती हैं, इसके बजाय कि आप किसी उपयोगकर्ता की शिकायत में इनके बारे में सुनें। अपनी स्कोरिंग सेवा को एक बार कनेक्ट करें और Failproof AI Observability हर पूरी हुई रन को स्वचालित रूप से ग्रेड करता है, इसलिए सहायकता में गिरावट या मतिभ्रम में वृद्धि अपने आप दिखाई देती है, इससे पहले कि कोई ग्राहक इसे महसूस करे। +गुणवत्ता की समस्याएं अब आपको मिलती हैं, इसके बजाय कि आप किसी उपयोगकर्ता की शिकायत में इनके बारे में सुनें। अपनी स्कोरिंग सेवा को एक बार कनेक्ट करें और FailproofAI Cloud हर पूरी हुई रन को स्वचालित रूप से ग्रेड करता है, इसलिए सहायकता में गिरावट या मतिभ्रम में वृद्धि अपने आप दिखाई देती है, इससे पहले कि कोई ग्राहक इसे महसूस करे। -![सत्र ग्रिड एक स्कोर कॉलम के साथ: प्रत्येक रन एक मूल्यांकन स्थिति पिल और रंग-कोडित सहायकता, तथ्यात्मकता, और उपकरण-दक्षता बैज ले जाता है](/agenteye/images/sessions-list.png) +![सत्र ग्रिड एक स्कोर कॉलम के साथ: प्रत्येक रन एक मूल्यांकन स्थिति पिल और रंग-कोडित सहायकता, तथ्यात्मकता, और उपकरण-दक्षता बैज ले जाता है](/cloud/images/sessions-list.png) *सत्र ग्रिड पर हर रन अपने स्कोर ले जाता है; लाल, नारंगी, और हरे बैज कमजोर रनों को एक भी प्रतिलेख खोले बिना ही सामने ला देते हैं।* ## रनों को हाथ से नमूना लेना बंद करें -आप कुछ रनों को देखा-भाली के आधार पर जांचते थे और बाकी सब ठीक हों यह आशा करते थे। अब हर पूरा किया गया सत्र समाप्त होते ही स्कोर किया जाता है, उन आयामों पर जिनकी आपको परवाह है: सहायकता, उपकरण दक्षता, तथ्यात्मकता, सुरक्षा, जो कुछ भी आपकी गुणवत्ता की मानक है। आप स्कोर कुंजियों को परिभाषित करते हैं; Failproof AI Observability जो कुछ भी आपका मूल्यांकनकर्ता वापस भेजता है उसे संग्रहीत, प्रवृत्ति और प्रदर्शित करता है। कोई भी रन बिना स्कोर किए नहीं छूटता है, और आप किसी प्रतिगमन के बारे में सहायता टिकट से सीखना बंद कर देते हैं। +आप कुछ रनों को देखा-भाली के आधार पर जांचते थे और बाकी सब ठीक हों यह आशा करते थे। अब हर पूरा किया गया सत्र समाप्त होते ही स्कोर किया जाता है, उन आयामों पर जिनकी आपको परवाह है: सहायकता, उपकरण दक्षता, तथ्यात्मकता, सुरक्षा, जो कुछ भी आपकी गुणवत्ता की मानक है। आप स्कोर कुंजियों को परिभाषित करते हैं; FailproofAI Cloud जो कुछ भी आपका मूल्यांकनकर्ता वापस भेजता है उसे संग्रहीत, प्रवृत्ति और प्रदर्शित करता है। कोई भी रन बिना स्कोर किए नहीं छूटता है, और आप किसी प्रतिगमन के बारे में सहायता टिकट से सीखना बंद कर देते हैं। स्कोर सत्र ग्रिड पर **`//sessions`** (साइडबार → *observe* → *sessions*) पर सवार होते हैं, प्रति पंक्ति एक बैज क्लस्टर। केवल वे रन चाहते हैं जो कम हो गईं? स्कोर रेंज के आधार पर ग्रिड को फ़िल्टर करें, कहें 0.5 से नीचे सहायकता, और बिल्कुल पढ़ने योग्य रन निकालें। स्कोर देखने के लिए `evaluations:read` अनुमति की आवश्यकता है। @@ -20,7 +20,7 @@ description: "गुणवत्ता की समस्याएं अब एक संख्या आपको बताती है कि एक रन कमजोर था; सत्र पृष्ठ आपको बताता है कि क्यों। कोई भी रन खोलें और दाईं ओर की रेल सुर्खी सारांश के साथ शुरू होती है, फिर प्रत्येक आयाम के लिए एक बार दिखाती है और आपके मूल्यांकनकर्ता का अपना तर्क प्रत्येक के तहत दिखाती है, इसलिए आप "इसे तथ्यात्मकता पर 0.4 मिला" से सेकंड में उस सटीक दावे तक जाते हैं जो यह गलत हो गया। -![एक सत्र की दाईं ओर की रेल: शीर्ष पर मूल्यांकन सारांश, फिर प्रति-आयाम स्कोर बार प्रत्येक के साथ तर्क की एक पंक्ति, पूरी घटना समयरेखा के बगल में](/agenteye/images/session-detail.png) +![एक सत्र की दाईं ओर की रेल: शीर्ष पर मूल्यांकन सारांश, फिर प्रति-आयाम स्कोर बार प्रत्येक के साथ तर्क की एक पंक्ति, पूरी घटना समयरेखा के बगल में](/cloud/images/session-detail.png) *सत्र विस्तार दृश्य: सारांश, प्रति-आयाम स्कोर बार, और प्रत्येक स्कोर के पीछे तर्क, रन की घटना समयरेखा के बगल में।* @@ -30,7 +30,7 @@ description: "गुणवत्ता की समस्याएं अब एक रन कम स्कोरिंग शोर है; पूरे समूह का स्लाइड करना एक संकेत है। सहेजे गए डैशबोर्ड आपके स्कोर को एक प्रवृत्ति में बदलते हैं जो आप एक नज़र में देख सकते हैं: इस हफ्ते की औसत सहायकता पिछले हफ्ते के विरुद्ध, प्रति एजेंट, प्रति वातावरण। -![एक गुणवत्ता डैशबोर्ड: मूल्यांकनकर्ता आयाम प्रति औसत-स्कोर बार समय के साथ एक प्रवृत्ति के साथ](/agenteye/images/dashboard-quality.png) +![एक गुणवत्ता डैशबोर्ड: मूल्यांकनकर्ता आयाम प्रति औसत-स्कोर बार समय के साथ एक प्रवृत्ति के साथ](/cloud/images/dashboard-quality.png) *एक सहेजा गया गुणवत्ता डैशबोर्ड स्कोर कुंजियों को प्रवृत्ति देता है जिन्हें आप प्रदर्शित करते हैं, इसलिए एक धीमी बहाव स्पष्ट है यह एक घटना बनने से बहुत पहले।* @@ -38,14 +38,14 @@ description: "गुणवत्ता की समस्याएं अब ## एक बार एक मूल्यांकनकर्ता कनेक्ट करें -स्कोरिंग ऑप्ट-इन है और तब तक बिल्कुल बंद रहती है जब तक आप Failproof AI Observability को एक स्कोरर की ओर इंगित नहीं करते। आप एक छोटी सी HTTP सेवा खड़ी करते हैं (Observability एक कार्यशील संदर्भ भेजता है जिसे आप कॉपी कर सकते हैं), अपने सर्वर पर दो मान सेट करते हैं, और तब से हर रन आपके लिए स्कोर किया जाता है। पूरी मार्गदर्शिका, स्कोरिंग अनुबंध, और SDK गहन गाइड में रहते हैं। +स्कोरिंग ऑप्ट-इन है और तब तक बिल्कुल बंद रहती है जब तक आप FailproofAI Cloud को एक स्कोरर की ओर इंगित नहीं करते। आप एक छोटी सी HTTP सेवा खड़ी करते हैं (FailproofAI Cloud एक कार्यशील संदर्भ भेजता है जिसे आप कॉपी कर सकते हैं), अपने सर्वर पर दो मान सेट करते हैं, और तब से हर रन आपके लिए स्कोर किया जाता है। पूरी मार्गदर्शिका, स्कोरिंग अनुबंध, और SDK गहन गाइड में रहते हैं। -यह सुनिश्चित नहीं हैं कि कौन से आयाम पहली जगह में स्कोर करने योग्य हैं? [evaluator agent skill](/hi/agenteye/evaluator-skill) में आपके कोडिंग एजेंट को अपने स्वयं के सत्रों के विरुद्ध इसे काम करना पड़ता है, फिर सेवा बनाएं और तैनात करें। +यह सुनिश्चित नहीं हैं कि कौन से आयाम पहली जगह में स्कोर करने योग्य हैं? [evaluator agent skill](/hi/cloud/agent-skills) में आपके कोडिंग एजेंट को अपने स्वयं के सत्रों के विरुद्ध इसे काम करना पड़ता है, फिर सेवा बनाएं और तैनात करें। ## संबंधित -- [Evaluation suite](/hi/agenteye/evaluation-suite): अपने मूल्यांकनकर्ता को कनेक्ट करें, स्कोरिंग अनुबंध, और SDK। -- [Evaluator agent skill](/hi/agenteye/evaluator-skill): एक कोडिंग एजेंट को अपने स्कोर आयाम चुनने और मूल्यांकनकर्ता बनाने दें। -- [Sessions](/hi/agenteye/sessions): रन-दर-रन ग्रिड जहां स्कोर दिखाई देते हैं। -- [Dashboards](/hi/agenteye/dashboards): अपने संगठन में गुणवत्ता प्रवृत्ति को सहेजें और साझा करें। -- [Audits](/hi/agenteye/audits): Observability की अन्य स्वचालित गुणवत्ता सुविधा, क्रॉस-सत्र जांचों के लिए। \ No newline at end of file +- [Evaluation suite](/hi/cloud/evaluators): अपने मूल्यांकनकर्ता को कनेक्ट करें, स्कोरिंग अनुबंध, और SDK। +- [Evaluator agent skill](/hi/cloud/agent-skills): एक कोडिंग एजेंट को अपने स्कोर आयाम चुनने और मूल्यांकनकर्ता बनाने दें। +- [Sessions](/hi/cloud/sessions): रन-दर-रन ग्रिड जहां स्कोर दिखाई देते हैं। +- [Dashboards](/hi/cloud/dashboards): अपने संगठन में गुणवत्ता प्रवृत्ति को सहेजें और साझा करें। +- [Audits](/hi/cloud/audits): FailproofAI Cloud की अन्य स्वचालित गुणवत्ता सुविधा, क्रॉस-सत्र जांचों के लिए। \ No newline at end of file diff --git a/docs/hi/agenteye/evaluation-suite.mdx b/docs/hi/cloud/evaluators.mdx similarity index 81% rename from docs/hi/agenteye/evaluation-suite.mdx rename to docs/hi/cloud/evaluators.mdx index d839b36c..6d30ab0f 100644 --- a/docs/hi/agenteye/evaluation-suite.mdx +++ b/docs/hi/cloud/evaluators.mdx @@ -1,19 +1,19 @@ --- title: "मूल्यांकन सूट" -description: "Failproof AI Observability प्रत्येक पूर्ण agent run को गुणवत्ता के लिए स्वचालित रूप से स्कोर कर सकता है: आप एक छोटी स्कोरिंग सेवा प्रदान करते हैं, और Observability बाकी को संभालता है।" +description: "FailproofAI Cloud प्रत्येक पूर्ण agent run को गुणवत्ता के लिए स्वचालित रूप से स्कोर कर सकता है: आप एक छोटी स्कोरिंग सेवा प्रदान करते हैं, और FailproofAI Cloud बाकी को संभालता है।" --- -Failproof AI Observability प्रत्येक पूर्ण agent run को गुणवत्ता के लिए स्वचालित रूप से स्कोर कर सकता है: आप एक छोटी स्कोरिंग सेवा प्रदान करते हैं, और Observability बाकी को संभालता है। इसका उपयोग उन आयामों को ट्रैक करने के लिए करें जिनकी आपको परवाह है (सहायकता, tool efficiency, तथ्यात्मकता, सुरक्षा; आप चुनते हैं), regression को जल्दी पकड़ें, और agents या environments की तुलना एक नज़र में करें। स्कोरिंग opt-in है: pipeline तब तक कुछ नहीं करता जब तक आप server पर `EVALUATOR_ENDPOINT` सेट नहीं करते। +FailproofAI Cloud प्रत्येक पूर्ण agent run को गुणवत्ता के लिए स्वचालित रूप से स्कोर कर सकता है: आप एक छोटी स्कोरिंग सेवा प्रदान करते हैं, और FailproofAI Cloud बाकी को संभालता है। इसका उपयोग उन आयामों को ट्रैक करने के लिए करें जिनकी आपको परवाह है (सहायकता, tool efficiency, तथ्यात्मकता, सुरक्षा; आप चुनते हैं), regression को जल्दी पकड़ें, और agents या environments की तुलना एक नज़र में करें। स्कोरिंग opt-in है: pipeline तब तक कुछ नहीं करता जब तक आप server पर `EVALUATOR_ENDPOINT` सेट नहीं करते। -> **नोट:** आप स्कोर आयाम परिभाषित करते हैं। आपका evaluator किसी भी संख्यात्मक keys को return कर सकता है; Observability जो भी आप भेजते हैं उसे store, trend, और display करता है। +> **नोट:** आप स्कोर आयाम परिभाषित करते हैं। आपका evaluator किसी भी संख्यात्मक keys को return कर सकता है; FailproofAI Cloud जो भी आप भेजते हैं उसे store, trend, और display करता है। ## एक नज़र में -1. **एक scorer लिखें।** एक छोटी HTTP सेवा स्थापित करें जो एक session transcript पढ़ता है और scores return करता है। Observability एक कार्यशील reference ships करता है जिसे आप copy कर सकते हैं। [SDK के साथ एक evaluator लिखना](#writing-an-evaluator-with-the-sdk) देखें। -2. **Observability को इसकी ओर निर्देशित करें।** Server process पर `EVALUATOR_ENDPOINT` (और एक साझा `EVALUATOR_TOKEN`) सेट करें। +1. **एक scorer लिखें।** एक छोटी HTTP सेवा स्थापित करें जो एक session transcript पढ़ता है और scores return करता है। FailproofAI Cloud एक कार्यशील reference ships करता है जिसे आप copy कर सकते हैं। [SDK के साथ एक evaluator लिखना](#writing-an-evaluator-with-the-sdk) देखें। +2. **FailproofAI Cloud को इसकी ओर निर्देशित करें।** Server process पर `EVALUATOR_ENDPOINT` (और एक साझा `EVALUATOR_TOKEN`) सेट करें। 3. **Scores को उतरते देखें।** प्रत्येक पूर्ण session स्वचालित रूप से स्कोर किया जाता है; results session detail page, sessions grid, और saved dashboards पर दिखाई देते हैं। -![एक session detail view जिसमें evaluation summary, per-dimension score bars, और right rail में reasoning text है](/agenteye/images/session-detail.png) +![एक session detail view जिसमें evaluation summary, per-dimension score bars, और right rail में reasoning text है](/cloud/images/session-detail.png) *एक बार evaluator configure हो जाने के बाद, प्रत्येक पूर्ण run को स्कोर किया जाता है और results session के right rail में दिखाई देते हैं: शीर्ष पर summary, फिर reasoning के साथ per-dimension score bars।* @@ -23,7 +23,7 @@ Failproof AI Observability प्रत्येक पूर्ण agent run ```mermaid flowchart LR - ING["ingest /events
agent_end"] --> SRV["Observability server"] + ING["ingest /events
agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -31,14 +31,14 @@ flowchart LR SRV --> RES["evaluations
terminal results"] ``` -जब Observability SDK एक session के लिए `agent_end` event emit करता है, server एक evaluation को schedule करता है। फिर यह full event transcript को आपकी evaluator सेवा में POST करता है, जो निम्नलिखित में से कर सकता है: +जब FailproofAI Cloud SDK एक session के लिए `agent_end` event emit करता है, server एक evaluation को schedule करता है। फिर यह full event transcript को आपकी evaluator सेवा में POST करता है, जो निम्नलिखित में से कर सकता है: - **Inline result return करें** `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}` के साथ। Result को session के evaluation timeline में append किया जाता है। `reasoning` और `summary` optional हैं। -- **Defer करें** `{"status":"pending", "job_id":"abc-123"}` के साथ। Observability फिर `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` को तब तक call करता है जब तक आपका evaluator `{"status":"done", ...}` या `{"status":"error", "error":"..."}` return नहीं करता। +- **Defer करें** `{"status":"pending", "job_id":"abc-123"}` के साथ। FailproofAI Cloud फिर `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` को तब तक call करता है जब तक आपका evaluator `{"status":"done", ...}` या `{"status":"error", "error":"..."}` return नहीं करता। - Polling cadence per-job है: एक `pending` response में `next_poll_secs` शामिल हो सकता है को override करने के लिए; अन्यथा Observability `GET /config` से `default_poll_interval_secs` value का उपयोग करता है; अन्यथा server `EVALUATOR_POLLING_INTERVAL_SECS` (default 10s) पर fallback करता है। सभी values को [1s, 1h] में clamp किया जाता है। + Polling cadence per-job है: एक `pending` response में `next_poll_secs` शामिल हो सकता है को override करने के लिए; अन्यथा FailproofAI Cloud `GET /config` से `default_poll_interval_secs` value का उपयोग करता है; अन्यथा server `EVALUATOR_POLLING_INTERVAL_SECS` (default 10s) पर fallback करता है। सभी values को [1s, 1h] में clamp किया जाता है। -जो sessions कभी `agent_end` emit नहीं करते (उदाहरण के लिए, एक crashed agent process) को भी pick up किया जा सकता है: evaluator का `GET /config` `{"inactivity_timeout_secs": 1800}` return कर सकता है, और Observability किसी भी session को evaluate करेगा जो उतने समय के लिए idle गया हो। इस fallback को disable करने के लिए field को `null` सेट करें या इसे omit करें। +जो sessions कभी `agent_end` emit नहीं करते (उदाहरण के लिए, एक crashed agent process) को भी pick up किया जा सकता है: evaluator का `GET /config` `{"inactivity_timeout_secs": 1800}` return कर सकता है, और FailproofAI Cloud किसी भी session को evaluate करेगा जो उतने समय के लिए idle गया हो। इस fallback को disable करने के लिए field को `null` सेट करें या इसे omit करें। `EVALUATOR_ENDPOINT` unset होने पर pipeline पूरी तरह no-op है। @@ -46,7 +46,7 @@ flowchart LR Inactivity fallback भी resumed sessions पर re-engages करता है: यदि नए events पहले के terminal evaluation के बाद आते हैं और session फिर `inactivity_timeout_secs` के पिछले idle जाता है, तो एक fresh evaluation को enqueue किया जाता है। -Transient failures (5xx, 429, timeouts, network errors) को `EVALUATOR_MAX_ATTEMPTS` तक exponential backoff के साथ retry किया जाता है; 4xx responses terminal होते हैं। Observability multiple horizontally-scaled server instances के साथ चलाने के लिए safe है; work को partition किया जाता है इसलिए एक ही session को कभी concurrently दो बार dispatch नहीं किया जाता। +Transient failures (5xx, 429, timeouts, network errors) को `EVALUATOR_MAX_ATTEMPTS` तक exponential backoff के साथ retry किया जाता है; 4xx responses terminal होते हैं। FailproofAI Cloud multiple horizontally-scaled server instances के साथ चलाने के लिए safe है; work को partition किया जाता है इसलिए एक ही session को कभी concurrently दो बार dispatch नहीं किया जाता। --- @@ -54,7 +54,7 @@ Transient failures (5xx, 429, timeouts, network errors) को `EVALUATOR_MAX_AT प्रत्येक authenticated route **bearer token auth** का उपयोग करता है। एक ही value दोनों sides पर configure की जानी चाहिए: -- Observability server: env var `EVALUATOR_TOKEN` +- FailproofAI Cloud server: env var `EVALUATOR_TOKEN` - Evaluator service: एक ही तरीके से configure किया गया (the `agenteye-evaluator` SDK convention के अनुसार `EVALUATOR_TOKEN` को read करता है) यदि `EVALUATOR_TOKEN` unset है, तो server कोई `Authorization` header नहीं भेजता है; evaluator फिर anonymous requests को accept कर सकता है, जो internal-only network के लिए ठीक है लेकिन public internet पर discouraged है। @@ -125,7 +125,7 @@ Server किसी अन्य 2xx body को protocol error के रूप आपको HTTP contract को manually implement नहीं करना है। `agenteye-evaluator` Python package आपको एक typed FastAPI wrapper देता है जो auth, routing, और request/response shapes को आपके लिए handle करता है। -Failproof AI Observability एक **कार्यशील reference evaluator** भी ships करता है जो transcript के shape से `helpfulness`, `tool_efficiency`, और `factuality` को score करता है। इसे starting point के रूप में copy करें और अपने स्वयं के logic को swap करें: एक LLM judge, एक rule engine, कुछ भी जो आपकी quality bar को fit करता है। +FailproofAI Cloud एक **कार्यशील reference evaluator** भी ships करता है जो transcript के shape से `helpfulness`, `tool_efficiency`, और `factuality` को score करता है। इसे starting point के रूप में copy करें और अपने स्वयं के logic को swap करें: एक LLM judge, एक rule engine, कुछ भी जो आपकी quality bar को fit करता है। Minimum viable evaluator: @@ -148,7 +148,7 @@ def run(req: EvalRequest) -> EvalResponse: `app` instance किसी भी ASGI server के अंतर्गत चलता है, इसलिए `uvicorn module:app` इसे start करता है। -उन evaluators के लिए जिन्हें expensive work को defer करने की आवश्यकता है, `JobPending` को instead return करें और एक `@app.job_lookup` handler को register करें; Observability server `GET /evaluate/{job_id}` को तब तक poll करता है जब तक आप एक terminal status return नहीं करते या `EVALUATOR_MAX_POLL_DURATION_SECS` cap (default 1 h) elapse न हो। +उन evaluators के लिए जिन्हें expensive work को defer करने की आवश्यकता है, `JobPending` को instead return करें और एक `@app.job_lookup` handler को register करें; FailproofAI Cloud server `GET /evaluate/{job_id}` को तब तक poll करता है जब तक आप एक terminal status return नहीं करते या `EVALUATOR_MAX_POLL_DURATION_SECS` cap (default 1 h) elapse न हो। Full API reference, async pattern, और event schema को `agenteye-evaluator` SDK के README में document किया गया है। @@ -156,7 +156,7 @@ Full API reference, async pattern, और event schema को `agenteye-evaluato ## अपने evaluator को चलाना -Evaluator **आपकी सेवा** है — Failproof AI Observability एक default evaluator ship नहीं करता है, इसलिए आप इसे जहां अपनी सेवाओं को चलाते हैं वहां build और run करते हैं। यह किसी भी ASGI server के अंतर्गत चलता है (उदाहरण के लिए `uvicorn my_evaluator:app`); [HTTP contract](#http-contract) से `/health`, `/config`, और `/evaluate` routes को serve करें, फिर server को इसकी ओर निर्देशित करें (देखें [Server को configure करना](#configuring-the-server))। +Evaluator **आपकी सेवा** है — FailproofAI Cloud एक default evaluator ship नहीं करता है, इसलिए आप इसे जहां अपनी सेवाओं को चलाते हैं वहां build और run करते हैं। यह किसी भी ASGI server के अंतर्गत चलता है (उदाहरण के लिए `uvicorn my_evaluator:app`); [HTTP contract](#http-contract) से `/health`, `/config`, और `/evaluate` routes को serve करें, फिर server को इसकी ओर निर्देशित करें (देखें [Server को configure करना](#configuring-the-server))। एक बार evaluator reachable हो जाने के बाद, `GET /health` `{"status":"ok"}` return करता है। एक agent को end-to-end चलाने के बाद, server पर `GET /evaluations` एक row return करता है `status: "done"` के साथ और scores जो आपका evaluator produce किया। @@ -254,7 +254,7 @@ Bootstrap admin (`ADMIN_KEY`, `ADMIN_EMAIL`) स्वचालित रूप - **`/sessions`**: filterable session grid; score column प्रत्येक session की evaluation status और scores को एक नज़र में दिखाता है। - **`/dashboards`**: saved eval-health views (देखें [Dashboards](#dashboards) नीचे)। -![Sessions grid per-session evaluation status pills और colour-coded score badges (helpfulness, factuality, tool_efficiency, safety, coherence) के साथ](/agenteye/images/sessions-list.png) +![Sessions grid per-session evaluation status pills और colour-coded score badges (helpfulness, factuality, tool_efficiency, safety, coherence) के साथ](/cloud/images/sessions-list.png) *Sessions grid प्रत्येक run की evaluation status और scores को एक नज़र में दिखाता है; red/amber/green badges low scores को jump out करते हैं।* @@ -271,7 +271,7 @@ Bootstrap admin (`ADMIN_KEY`, `ADMIN_EMAIL`) स्वचालित रूप प्रत्येक card matching sessions की संख्या दिखाता है, एक done/error/timeout breakdown, प्रत्येक featured score का average, और एक छोटा trend sparkline। एक dashboard को open करने से full-size panels दिखते हैं; **"open in sessions"** आपको sessions page में drop करता है उसी slice के लिए pre-filtered। Metrics को server-side पर पूरे matching set पर compute किया जाता है (`GET /evaluations/aggregate` के माध्यम से), इसलिए numbers exact हैं rather than sampled। -![एक eval-health dashboard जिसमें evaluator dimension per average-score bars, एक tool ok-vs-error breakdown, top tools, और एक events-per-hour trend है](/agenteye/images/dashboard-quality.png) +![एक eval-health dashboard जिसमें evaluator dimension per average-score bars, एक tool ok-vs-error breakdown, top tools, और एक events-per-hour trend है](/cloud/images/dashboard-quality.png) **Permissions:** viewing के लिए `dashboards:read` और `evaluations:read` दोनों चाहिए; creating और editing के लिए `dashboards:write` चाहिए; deleting के लिए `dashboards:delete` चाहिए। Bootstrap admin को automatically ये सभी मिलते हैं। @@ -293,7 +293,7 @@ Bootstrap admin (`ADMIN_KEY`, `ADMIN_EMAIL`) स्वचालित रूप ## अगले कदम -- [Evaluator agent skill](/hi/agenteye/evaluator-skill): एक coding agent को real sessions के विरुद्ध आपके dimensions को design करने और यह सेवा build करने दें। -- [Python SDK](/hi/agenteye/python-sdk): `agent_end` events emit करें जो scoring को trigger करते हैं। -- [API keys](/hi/agenteye/api-keys): the `evaluations:read` और `evaluations:trigger` permissions। -- [Audits](/hi/agenteye/audits): Observability का अन्य automated quality feature, policy-based review के लिए। \ No newline at end of file +- [Evaluator agent skill](/hi/cloud/agent-skills): एक coding agent को real sessions के विरुद्ध आपके dimensions को design करने और यह सेवा build करने दें। +- [Python SDK](/hi/cloud/sdk): `agent_end` events emit करें जो scoring को trigger करते हैं। +- [API keys](/hi/cloud/access): the `evaluations:read` और `evaluations:trigger` permissions। +- [Audits](/hi/cloud/audits): FailproofAI Cloud का अन्य automated quality feature, policy-based review के लिए। \ No newline at end of file diff --git a/docs/hi/agenteye/event-stream.mdx b/docs/hi/cloud/event-stream.mdx similarity index 81% rename from docs/hi/agenteye/event-stream.mdx rename to docs/hi/cloud/event-stream.mdx index c7748586..2700d9aa 100644 --- a/docs/hi/agenteye/event-stream.mdx +++ b/docs/hi/cloud/event-stream.mdx @@ -5,7 +5,7 @@ description: "जिस पल आपका agent कुछ करता है, जिस पल आपका agent कुछ करता है, आप उसे देखते हैं। Event Stream production में हर agent की live pulse है: कोई इंतज़ार नहीं, logs को grep करने की ज़रूरत नहीं, कोई अनुमान नहीं कि अभी क्या हुआ। -![Live Event Stream: color-coded event rows जो real time में tail करते हैं, environment, agent, session, event type, और free text से filterable](/agenteye/images/events-stream.png) +![Live Event Stream: color-coded event rows जो real time में tail करते हैं, environment, agent, session, event type, और free text से filterable](/cloud/images/events-stream.png) *आपके org के हर agent से हर event, सबसे नया पहले, जैसे-जैसे यह होता है अपडेट होता है।* @@ -36,14 +36,14 @@ Free-text search सीधे एक message, एक tool name, या एक id Event Stream आपका org home है। Sign in करें और यह पहली surface है जहाँ आप land करते हैं, `//` पर, ताकि triage दूसरे पल से शुरू हो जाए जब आप पहुँचते हैं। -इसके पीछे, आपके agents SDK के through events emit करते हैं, collector उन्हें आपके Failproof AI Observability server को ship करता है, और stream उन्हें tail करता है जैसे वे infrastructure में arrive करते हैं जो आप control करते हैं। जब आप raw trail की बजाय rolled-up view चाहते हैं, तो हर run के events Sessions पर एक single row में collapse हो जाते हैं, एक click दूर। +इसके पीछे, आपके agents SDK के through events emit करते हैं, collector उन्हें आपके FailproofAI Cloud server को ship करता है, और stream उन्हें tail करता है जैसे वे infrastructure में arrive करते हैं जो आप control करते हैं। जब आप raw trail की बजाय rolled-up view चाहते हैं, तो हर run के events Sessions पर एक single row में collapse हो जाते हैं, एक click दूर। यह raw source of truth है जिस पर हर दूसरी observe surface build होती है, इसलिए जब एक number कहीं और गलत दिखता है, तो stream वह जगह है जहाँ आप confirm करते हैं कि वास्तव में क्या हुआ। ## संबंधित -- [Sessions](/hi/agenteye/sessions): वही events हर run के लिए एक row में rolled up, एक git-style execution graph के साथ। -- [Telemetry](/hi/agenteye/telemetry): आपके agents क्या send करते हैं और कैसे events stream तक पहुँचते हैं। -- [Error tracking](/hi/agenteye/error-tracking): एक triage surface सब कुछ के लिए जो गलत हुआ। -- [Alerts](/hi/agenteye/alerts): किसी भी threshold को paging rule में बदलें। -- [CLI and agents](/hi/agenteye/cli-and-agents): आपके terminal से एक ही live trail। \ No newline at end of file +- [Sessions](/hi/cloud/sessions): वही events हर run के लिए एक row में rolled up, एक git-style execution graph के साथ। +- [Telemetry](/hi/cloud/performance): आपके agents क्या send करते हैं और कैसे events stream तक पहुँचते हैं। +- [Error tracking](/hi/cloud/errors): एक triage surface सब कुछ के लिए जो गलत हुआ। +- [Alerts](/hi/cloud/alerts): किसी भी threshold को paging rule में बदलें। +- [CLI and agents](/hi/cloud/cli): आपके terminal से एक ही live trail। \ No newline at end of file diff --git a/docs/hi/cloud/fleet.mdx b/docs/hi/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/hi/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/hi/agenteye/incidents.mdx b/docs/hi/cloud/incidents.mdx similarity index 86% rename from docs/hi/agenteye/incidents.mdx rename to docs/hi/cloud/incidents.mdx index df3738c1..f1a6a4f9 100644 --- a/docs/hi/agenteye/incidents.mdx +++ b/docs/hi/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "जब कोई alert trigger होता है, तो सभ जब कोई alert trigger होता है, तो पहला सवाल हमेशा यही होता है "इस पर कौन काम कर रहा है?" Incidents इसका जवाब देते हैं: जिस पल कोई breach होता है, सभी को दिखता है कि incident खुला है, इसका मालिक कौन है, और अब तक बिल्कुल क्या हुआ है, साथ ही एक स्वच्छ, attributed record जिसे आप सीधे post-mortem को दे सकते हैं। -![The Incidents inbox: alert-linked और manually opened incident cards, state के अनुसार grouped, हर एक के साथ severity badge और assignee](/agenteye/images/incidents.png) +![The Incidents inbox: alert-linked और manually opened incident cards, state के अनुसार grouped, हर एक के साथ severity badge और assignee](/cloud/images/incidents.png) *The inbox open incidents को state के अनुसार grouped करता है और severity और assignee के अनुसार filter करता है, इसलिए आप वह देखते हैं जिसे अभी किसी की ज़रूरत है।* ## एक नज़र में जानें कि किसके पास है @@ -17,10 +17,10 @@ description: "जब कोई alert trigger होता है, तो सभ जब incident ख़त्म हो जाता है, तो आपके पास पहले से ही write-up होता है। कोई भी incident खोलें और आपको breach का सबूत, इसके assignees और subscribers, coordinating के लिए एक comment thread, और एक append-only activity timeline मिलता है। -![An incident detail view: parent alert और breach summary, assignees और subscribers, एक attributed activity timeline, और एक comment thread](/agenteye/images/incident-detail.png) +![An incident detail view: parent alert और breach summary, assignees और subscribers, एक attributed activity timeline, और एक comment thread](/cloud/images/incident-detail.png) *सब कुछ जो हुआ, क्रम में, हर पंक्ति इस पर हस्ताक्षर की गई है कि किसने इसे किया।* -हर action (opened, acknowledged, resolved, आदि) उस timeline पर लिखा जाता है और कभी संपादित नहीं किया जाता। हर entry को attribute किया जाता है: उस operator को जिसने इसे किया, email से, या **automated** को उन चीज़ों के लिए जो Failproof AI Observability ने अपने आप की हैं, जैसे breach पर incident को खोलना। कुछ भी anonymous नहीं है और कुछ भी नष्ट नहीं होता, इसलिए post-mortem कम या ज़्यादा अपने आप लिख जाता है। +हर action (opened, acknowledged, resolved, आदि) उस timeline पर लिखा जाता है और कभी संपादित नहीं किया जाता। हर entry को attribute किया जाता है: उस operator को जिसने इसे किया, email से, या **automated** को उन चीज़ों के लिए जो FailproofAI Cloud ने अपने आप की हैं, जैसे breach पर incident को खोलना। कुछ भी anonymous नहीं है और कुछ भी नष्ट नहीं होता, इसलिए post-mortem कम या ज़्यादा अपने आप लिख जाता है। ## एक incident कैसे आगे बढ़ता है @@ -45,6 +45,6 @@ Incidents `//incidents` पर रहते हैं। Viewing क ## संबंधित -- [Alerts](/hi/agenteye/alerts): वह नियम जो threshold breach होने पर ये incidents खोलते हैं। -- [Error tracking](/hi/agenteye/error-tracking): हर failure को एक जगह देखें और एक को एक alert में promote करें। -- [Audits](/hi/agenteye/audits): scheduled analyst जो उन failures को खोजता है जिन पर कोई rule नज़र नहीं रख रहा था। \ No newline at end of file +- [Alerts](/hi/cloud/alerts): वह नियम जो threshold breach होने पर ये incidents खोलते हैं। +- [Error tracking](/hi/cloud/errors): हर failure को एक जगह देखें और एक को एक alert में promote करें। +- [Audits](/hi/cloud/audits): scheduled analyst जो उन failures को खोजता है जिन पर कोई rule नज़र नहीं रख रहा था। \ No newline at end of file diff --git a/docs/hi/cloud/managed-policies.mdx b/docs/hi/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/hi/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/hi/agenteye/overview.mdx b/docs/hi/cloud/overview.mdx similarity index 50% rename from docs/hi/agenteye/overview.mdx rename to docs/hi/cloud/overview.mdx index 4ea47378..438db472 100644 --- a/docs/hi/agenteye/overview.mdx +++ b/docs/hi/cloud/overview.mdx @@ -1,15 +1,15 @@ --- title: "Failproof AI: एजेंट्स की विफलताओं का अवलोकन" -description: "Failproof AI Observability एक स्व-होस्टेड प्लेटफॉर्म है जो आपके AI एजेंट्स को प्रोडक्शन में देखने, मूल्यांकन करने और सुधारने के लिए है।" +description: "FailproofAI Cloud एक स्व-होस्टेड प्लेटफॉर्म है जो आपके AI एजेंट्स को प्रोडक्शन में देखने, मूल्यांकन करने और सुधारने के लिए है।" --- -Failproof AI Observability एक स्व-होस्टेड प्लेटफॉर्म है जो आपके AI एजेंट्स को प्रोडक्शन में देखने, मूल्यांकन करने और सुधारने के लिए है। यह आपके एजेंट्स द्वारा किए गए सभी काम को रिकॉर्ड करता है (प्रत्येक टूल कॉल, मॉडल अनुरोध, हुक और त्रुटि), प्रत्येक रन की गुणवत्ता को स्कोर करता है, और उन विफलताओं को सामने लाता है जिन्हें आप खोजने के लिए नहीं जानते थे, सभी एक डैशबोर्ड में जो आप अपने बुनियादी ढांचे के अंदर चलाते हैं। +FailproofAI Cloud एक स्व-होस्टेड प्लेटफॉर्म है जो आपके AI एजेंट्स को प्रोडक्शन में देखने, मूल्यांकन करने और सुधारने के लिए है। यह आपके एजेंट्स द्वारा किए गए सभी काम को रिकॉर्ड करता है (प्रत्येक टूल कॉल, मॉडल अनुरोध, हुक और त्रुटि), प्रत्येक रन की गुणवत्ता को स्कोर करता है, और उन विफलताओं को सामने लाता है जिन्हें आप खोजने के लिए नहीं जानते थे, सभी एक डैशबोर्ड में जो आप अपने बुनियादी ढांचे के अंदर चलाते हैं। -यदि आप AI एजेंट्स शिप करते हैं और यह अनुमान लगाने से थक गए हैं कि एक रन गलत क्यों हुआ, तो यह शुरू करने के लिए सही पृष्ठ है। यह समझाता है कि Failproof AI Observability आपको क्या देता है और कैसे चीजें एक साथ फिट होती हैं, इससे पहले कि आप कुछ भी इंस्टॉल करें। +यदि आप AI एजेंट्स शिप करते हैं और यह अनुमान लगाने से थक गए हैं कि एक रन गलत क्यों हुआ, तो यह शुरू करने के लिए सही पृष्ठ है। यह समझाता है कि FailproofAI Cloud आपको क्या देता है और कैसे चीजें एक साथ फिट होती हैं, इससे पहले कि आप कुछ भी इंस्टॉल करें। -> **Failproof AI Observability एक enterprise उत्पाद है Failproof AI से।** इसे कार्य में देखना चाहते हैं? एक डेमो का अनुरोध करें: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) को ईमेल करें। +> **FailproofAI Cloud एक enterprise उत्पाद है Failproof AI से।** इसे कार्य में देखना चाहते हैं? एक डेमो का अनुरोध करें: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) को ईमेल करें। -![एक Failproof AI Observability सेशन को git-शैली के execution ग्राफ़ के रूप में खींचा गया है, जिसके साथ इसकी event timeline है, जिसमें दाईं ओर प्रति-रन tools, मॉडल्स और hooks का विवरण है](/agenteye/images/session-detail.png) +![एक FailproofAI Cloud सेशन को git-शैली के execution ग्राफ़ के रूप में खींचा गया है, जिसके साथ इसकी event timeline है, जिसमें दाईं ओर प्रति-रन tools, मॉडल्स और hooks का विवरण है](/cloud/images/session-detail.png) *हर एजेंट रन को git-शैली के execution ग्राफ़ (बाएं) के रूप में खींचा जाता है, इसके event timeline के बगल में। समानांतर sub-agents को प्रत्येक को अपनी लेन मिलती है; दाईं ओर की पट्टी रन के लिए tools, मॉडल्स, hooks और token spend को विभाजित करती है।* @@ -29,52 +29,52 @@ Failproof AI Observability एक स्व-होस्टेड प्ले
-*Failproof Audit: Failproof AI Observability को अपने लॉग्स को सेशन्स के पार खोदने और आपको बताने दें कि क्या ठीक करना है।* +*Failproof Audit: FailproofAI Cloud को अपने लॉग्स को सेशन्स के पार खोदने और आपको बताने दें कि क्या ठीक करना है।* --- ## टीमें इसका उपयोग क्यों करती हैं - **देखें कि आपका एजेंट वास्तव में क्या करता है।** हर रन एक पठनीय, git-शैली के execution ग्राफ़ में बदल जाता है: कौन से tools समानांतर में चले, कौन से sub-agents शाखा बंद हो गए, यह कहां रुका और इसने क्या खर्च किया। -- **गुणवत्ता रिग्रेशन को स्वचालित रूप से पकड़ें।** एक छोटी स्कोरिंग सेवा को कनेक्ट करें और Failproof AI Observability हर समाप्त रन को स्कोर करता है, इसलिए सहायकता में गिरावट या hallucinations में स्पाइक अपने आप दिखाई देता है। +- **गुणवत्ता रिग्रेशन को स्वचालित रूप से पकड़ें।** एक छोटी स्कोरिंग सेवा को कनेक्ट करें और FailproofAI Cloud हर समाप्त रन को स्कोर करता है, इसलिए सहायकता में गिरावट या hallucinations में स्पाइक अपने आप दिखाई देता है। - **उन विफलताओं को खोजें जिनके लिए आपने कोई नियम नहीं लिखा है।** पुनरावर्ती audits आपके लॉग्स को सेशन्स के पार खोदते हैं और त्रुटि क्लस्टर, latency आउटलायर्स, कम स्कोर और फंसे हुए runs को खोजते हैं, फिर आपको ranked, evidence-backed खोजें देते हैं। - **जब यह महत्वपूर्ण हो तो पेज प्राप्त करें।** Threshold नियम त्रुटि दर, latency, cost या evaluator स्कोर पर फायर करते हैं और incidents खोलते हैं जिन्हें आप स्वीकार कर सकते हैं, assign कर सकते हैं और resolve कर सकते हैं। - **सादे अंग्रेजी में सवाल पूछें।** एक in-dashboard AI सहायक आपके अपने डेटा पर यह जवाब देता है कि इस सप्ताह prod में गुणवत्ता कैसी चल रही है। यह जो भी परिवर्तन करता है वह approval-gated है। -- **अपना डेटा रखें।** Failproof AI Observability स्व-होस्टेड है: events, prompts और analytics उस बुनियादी ढांचे में रहते हैं जिसे आप नियंत्रित करते हैं। +- **अपना डेटा रखें।** FailproofAI Cloud स्व-होस्टेड है: events, prompts और analytics उस बुनियादी ढांचे में रहते हैं जिसे आप नियंत्रित करते हैं। --- ## आप क्या प्राप्त करते हैं -Failproof AI Observability तीन विचारों के चारों ओर संगठित है (**observe**, **analyze**, और **admin**), जो डैशबोर्ड के बाएं sidebar में प्रतिबिंबित हैं। +FailproofAI Cloud तीन विचारों के चारों ओर संगठित है (**observe**, **analyze**, और **admin**), जो डैशबोर्ड के बाएं sidebar में प्रतिबिंबित हैं। **Observe** (जो हुआ उसकी कच्ची सच्चाई): -- **[Event stream](/hi/agenteye/event-stream)**: हर रन की live, per-step trail (tool calls, model calls, hooks, errors)। -- **[Sessions](/hi/agenteye/sessions)**: वे events रन के प्रति एक पंक्ति में रोल अप किए गए, प्रत्येक को स्कोर करने के लिए तैयार, एक git-शैली के execution ग्राफ़ के साथ। -- **[Performance metrics](/hi/agenteye/telemetry)**: per-surface latency heat-maps और p50/p95/p99 vitals models, tools और hooks के लिए, इसलिए एक tail spike माध्य से अलग होकर दिखता है। -- **[Error tracking](/hi/agenteye/error-tracking)**: सभी गलत चीजों के लिए एक triage surface, एक firing alert से एक क्लिक दूर। +- **[Event stream](/hi/cloud/event-stream)**: हर रन की live, per-step trail (tool calls, model calls, hooks, errors)। +- **[Sessions](/hi/cloud/sessions)**: वे events रन के प्रति एक पंक्ति में रोल अप किए गए, प्रत्येक को स्कोर करने के लिए तैयार, एक git-शैली के execution ग्राफ़ के साथ। +- **[Performance metrics](/hi/cloud/performance)**: per-surface latency heat-maps और p50/p95/p99 vitals models, tools और hooks के लिए, इसलिए एक tail spike माध्य से अलग होकर दिखता है। +- **[Error tracking](/hi/cloud/errors)**: सभी गलत चीजों के लिए एक triage surface, एक firing alert से एक क्लिक दूर। -![Tools observe पृष्ठ: एक latency heat-map, एक percentile band और 24 समय bins पर एक tool-distribution bar](/agenteye/images/tools.png) +![Tools observe पृष्ठ: एक latency heat-map, एक percentile band और 24 समय bins पर एक tool-distribution bar](/cloud/images/tools.png) *प्रत्येक observe surface एक sparkline और p50/p95/p99 vitals को एक latency heat-map और एक percentile band के साथ जोड़ता है। यहां दिखाया गया है: Tools।* **Analyze** (activity को जवाबों में बदलें): -- **[Queries](/hi/agenteye/queries)** और **[dashboards](/hi/agenteye/dashboards)**: आपकी events और evaluations पर saved SQL, साझा, org-scoped dashboards में चार्ट किए गए। -- **[Evaluations](/hi/agenteye/evaluations)**: आपकी अपनी evaluator सेवा द्वारा उत्पादित गुणवत्ता स्कोर, per-score reasoning के साथ। -- **[Audits](/hi/agenteye/audits)**: पुनरावर्ती investigations जो sessions के पार विफलता पैटर्न को सामने लाते हैं। -- **[Alerts](/hi/agenteye/alerts)** और **[incidents](/hi/agenteye/incidents)**: threshold नियम जो आपको पेज करते हैं, साथ ही एक incident workflow उन्हें triage करने के लिए। +- **[Queries](/hi/cloud/queries)** और **[dashboards](/hi/cloud/dashboards)**: आपकी events और evaluations पर saved SQL, साझा, org-scoped dashboards में चार्ट किए गए। +- **[Evaluations](/hi/cloud/evaluations)**: आपकी अपनी evaluator सेवा द्वारा उत्पादित गुणवत्ता स्कोर, per-score reasoning के साथ। +- **[Audits](/hi/cloud/audits)**: पुनरावर्ती investigations जो sessions के पार विफलता पैटर्न को सामने लाते हैं। +- **[Alerts](/hi/cloud/alerts)** और **[incidents](/hi/cloud/incidents)**: threshold नियम जो आपको पेज करते हैं, साथ ही एक incident workflow उन्हें triage करने के लिए। **Interfaces** (अपने डेटा तक अपने तरीके से पहुंचें): -- **[CLI](/hi/agenteye/cli-and-agents)**: terminal या script से अपनी पूरी deployment चलाएं, और एक coding agent को इसे सादे अंग्रेजी में करने दें। -- **[AI assistant](/hi/agenteye/assistant)**: डैशबोर्ड के अंदर सादे अंग्रेजी में अपने एजेंट्स के बारे में सवाल पूछें। -- **REST API**: डैशबोर्ड और CLI जो करते हैं सब कुछ एक REST API द्वारा समर्थित है जिसे आप सीधे एक scoped [API key](/hi/agenteye/api-keys) के साथ कॉल कर सकते हैं — events ingest करें, sessions और evaluations query करें, और dashboards, alerts, audits, users और keys को manage करें, इसलिए आप Failproof AI Observability को अपने स्वयं के tooling में wire कर सकते हैं। +- **[CLI](/hi/cloud/cli)**: terminal या script से अपनी पूरी deployment चलाएं, और एक coding agent को इसे सादे अंग्रेजी में करने दें। +- **[AI assistant](/hi/cloud/assistant)**: डैशबोर्ड के अंदर सादे अंग्रेजी में अपने एजेंट्स के बारे में सवाल पूछें। +- **REST API**: डैशबोर्ड और CLI जो करते हैं सब कुछ एक REST API द्वारा समर्थित है जिसे आप सीधे एक scoped [API key](/hi/cloud/access) के साथ कॉल कर सकते हैं — events ingest करें, sessions और evaluations query करें, और dashboards, alerts, audits, users और keys को manage करें, इसलिए आप FailproofAI Cloud को अपने स्वयं के tooling में wire कर सकते हैं। **Admin** (अपनी टीम के लिए इसे चलाएं): -- **[API keys](/hi/agenteye/api-keys)**: collector, dashboard और assistant के लिए scoped tokens। +- **[API keys](/hi/cloud/access)**: collector, dashboard और assistant के लिए scoped tokens। - **Users**: passwordless, email-based sign-in एक allowlist के साथ। - **Settings**: per-org configuration, including model context-window overrides के साथ। @@ -90,18 +90,18 @@ Failproof AI Observability तीन विचारों के चारो - **Dashboard**: जहां आप सबकुछ explore करते हैं। - **Optional services**: एक स्कोरिंग सेवा (evaluations), और एक AI assistant सेवा (in-dashboard chat)। -docs में उपयोग की गई vocabulary के लिए (*event, session, evaluation, audit, finding, incident*), [Concepts](/hi/agenteye/concepts) देखें। +docs में उपयोग की गई vocabulary के लिए (*event, session, evaluation, audit, finding, incident*), [Concepts](/hi/concepts) देखें। --- -## Failproof AI Observability प्राप्त करना +## FailproofAI Cloud प्राप्त करना -Failproof AI Observability एक enterprise उत्पाद है Failproof AI से, और यह Failproof AI Enforcement — policy और guardrail उत्पाद — के साथ काम करता है, Failproof AI ब्रांड के तहत। यह पूरी तरह से अपने स्वयं के environment में चलता है। यदि आपको packages तक access नहीं है अभी भी, एक डेमो का अनुरोध करें और हम आपको set up करेंगे: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) को ईमेल करें। +FailproofAI Cloud एक enterprise उत्पाद है Failproof AI से, और यह FailproofAI guardrails — policy और guardrail उत्पाद — के साथ काम करता है, Failproof AI ब्रांड के तहत। यह पूरी तरह से अपने स्वयं के environment में चलता है। यदि आपको packages तक access नहीं है अभी भी, एक डेमो का अनुरोध करें और हम आपको set up करेंगे: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) को ईमेल करें। --- ## अगले कदम -- [Concepts](/hi/agenteye/concepts): Failproof AI Observability vocabulary एक जगह पर। -- [Observability](/hi/agenteye/observability): अपने एजेंट्स को जो करते हैं उसे follow करें, रन दर रन। -- [Security](/hi/agenteye/security): कैसे Failproof AI Observability आपके डेटा को isolated रखता है और आपके नियंत्रण में। \ No newline at end of file +- [Concepts](/hi/concepts): FailproofAI Cloud vocabulary एक जगह पर। +- [FailproofAI Cloud](/hi/cloud/overview): अपने एजेंट्स को जो करते हैं उसे follow करें, रन दर रन। +- [Security](/hi/cloud/security): कैसे FailproofAI Cloud आपके डेटा को isolated रखता है और आपके नियंत्रण में। \ No newline at end of file diff --git a/docs/hi/agenteye/telemetry.mdx b/docs/hi/cloud/performance.mdx similarity index 85% rename from docs/hi/agenteye/telemetry.mdx rename to docs/hi/cloud/performance.mdx index e741ba23..fa59b4b5 100644 --- a/docs/hi/agenteye/telemetry.mdx +++ b/docs/hi/cloud/performance.mdx @@ -5,7 +5,7 @@ description: "तुरंत देखें कि आपके मॉडल, तुरंत देखें कि आपके मॉडल, टूल या हुक कब धीमे हो रहे हैं या खर्च बढ़ रहा है, और अपने उपयोगकर्ताओं को महसूस होने से पहले टेल-लेटेंसी स्पाइक को पकड़ें। तीन समर्पित पृष्ठ कच्चे समय को p50, p95, और p99 में बदलते हैं जिन्हें आप एक नज़र में पढ़ सकते हैं। -![मॉडल पृष्ठ लेटेंसी हीट-मैप, प्रतिशतक बैंड, और प्रति-मॉडल टोकन, लागत और संदर्भ-विंडो आंकड़े दिखाता है](/agenteye/images/models.png) +![मॉडल पृष्ठ लेटेंसी हीट-मैप, प्रतिशतक बैंड, और प्रति-मॉडल टोकन, लागत और संदर्भ-विंडो आंकड़े दिखाता है](/cloud/images/models.png) *मॉडल पृष्ठ: लेटेंसी हीट-मैप, प्रतिशतक बैंड, और प्रति-मॉडल टोकन, अनुमानित लागत, और संदर्भ-विंडो भरण।* ## औसत को अपने सबसे बुरे रन को छिपाने दें @@ -23,13 +23,13 @@ description: "तुरंत देखें कि आपके मॉडल, मॉडल पृष्ठ (ऊपर दिखाया गया है) दो सवालों का जवाब देता है जो एक बिल हमेशा उठाता है: कौन सा मॉडल, और कितना। साझा लेटेंसी दृश्य के शीर्ष पर, यह **प्रति-मॉडल टोकन खपत**, **अनुमानित लागत**, और **संदर्भ-विंडो भरण** जोड़ता है, इसलिए भागते हुए प्रॉम्प्ट वृद्धि और आसन्न संपीड़न आपको आश्चर्य करने से पहले दिखाई देते हैं। -Failproof AI Observability सामान्य मॉडल ID को स्वचालित रूप से पहचानता है। यदि कोई विंडो गलत दिखता है, या आप अपना निजी मॉडल चलाते हैं, तो इसे **Settings** के तहत, **model context windows** में सही करें या जोड़ें, और भरण पठन अनुसरण करते हैं। +FailproofAI Cloud सामान्य मॉडल ID को स्वचालित रूप से पहचानता है। यदि कोई विंडो गलत दिखता है, या आप अपना निजी मॉडल चलाते हैं, तो इसे **Settings** के तहत, **model context windows** में सही करें या जोड़ें, और भरण पठन अनुसरण करते हैं। ## टूल: धीमे को टूटे हुए से अलग करें एक टूल कॉल धीमा हो सकता है, या यह शांति से विफल हो सकता है, और आप इसे सेकंड में जानना चाहते हैं, लॉग के माध्यम से खोदने के बाद नहीं। -![टूल पृष्ठ साझा लेटेंसी हीट-मैप और प्रतिशतक बैंड को सफलता और विफलता विभाजन और टूल-वितरण बार के बगल में दिखाता है](/agenteye/images/tools.png) +![टूल पृष्ठ साझा लेटेंसी हीट-मैप और प्रतिशतक बैंड को सफलता और विफलता विभाजन और टूल-वितरण बार के बगल में दिखाता है](/cloud/images/tools.png) *टूल पृष्ठ: समान हीट-मैप और प्रतिशतक बैंड, प्लस सफलता और विफलता विभाजन और टूल-वितरण बार।* साझा लेटेंसी दृश्य के साथ, टूल पृष्ठ एक **सफलता और विफलता विभाजन** और एक **टूल-वितरण बार** जोड़ता है, इसलिए आप एक नज़र में देखते हैं कि आप कौन से टूल पर सबसे अधिक निर्भर हैं और कौन सी आपकी त्रुटि बजट को खा रही हैं। @@ -38,14 +38,14 @@ Failproof AI Observability सामान्य मॉडल ID को स् जब एक लाइफसाइकल हुक एक रन को खींचता है, तो "हुक धीमे हैं" कुछ ऐसा नहीं है जिस पर आप कार्य कर सकते हैं। हुक पृष्ठ आपको वह लाता है जो महत्वपूर्ण है। -![हुक पृष्ठ साझा हीट-मैप और प्रतिशतक बैंड पर हुक नाम और ट्रिगर ईवेंट द्वारा विभाजित लेटेंसी दिखाता है](/agenteye/images/hooks.png) +![हुक पृष्ठ साझा हीट-मैप और प्रतिशतक बैंड पर हुक नाम और ट्रिगर ईवेंट द्वारा विभाजित लेटेंसी दिखाता है](/cloud/images/hooks.png) *हुक पृष्ठ: हुक नाम और ट्रिगर ईवेंट द्वारा विभाजित लेटेंसी।* समान लेटेंसी हीट-मैप और प्रतिशतक बैंड के ऊपर, हुक पृष्ठ गतिविधि को **हुक नाम** और **ट्रिगर ईवेंट** द्वारा विभाजित करता है, इसलिए आप एकल हुक और एकल ईवेंट पर उतरते हैं जिन्हें ध्यान देने की आवश्यकता है। ## संबंधित -- [Event stream](/hi/agenteye/event-stream): हर ईवेंट का लाइव, रंग-कोडित ट्रेल। -- [Sessions](/hi/agenteye/sessions): ईवेंट को एक पंक्ति प्रति रन में रोल करें और इसके निष्पादन ग्राफ को खोलें। -- [Error tracking](/hi/agenteye/error-tracking): डैशबोर्ड को लाल रंग में पेंट करने वाली हर चीज़ के लिए एक ट्रिएज सतह। -- [Dashboards](/hi/agenteye/dashboards): अपने फ्लीट में रोल-अप दृश्य। \ No newline at end of file +- [Event stream](/hi/cloud/event-stream): हर ईवेंट का लाइव, रंग-कोडित ट्रेल। +- [Sessions](/hi/cloud/sessions): ईवेंट को एक पंक्ति प्रति रन में रोल करें और इसके निष्पादन ग्राफ को खोलें। +- [Error tracking](/hi/cloud/errors): डैशबोर्ड को लाल रंग में पेंट करने वाली हर चीज़ के लिए एक ट्रिएज सतह। +- [Dashboards](/hi/cloud/dashboards): अपने फ्लीट में रोल-अप दृश्य। \ No newline at end of file diff --git a/docs/hi/agenteye/queries.mdx b/docs/hi/cloud/queries.mdx similarity index 82% rename from docs/hi/agenteye/queries.mdx rename to docs/hi/cloud/queries.mdx index 526f8748..57589779 100644 --- a/docs/hi/agenteye/queries.mdx +++ b/docs/hi/cloud/queries.mdx @@ -5,9 +5,9 @@ description: "अपने एजेंट डेटा से कोई भी --- -अपने एजेंट डेटा से कोई भी सवाल पूछें और सेकंड में जवाब पाएं। Failproof AI Observability आपको आपकी इवेंट्स और evaluations पर सहेजे गए, चलने के लिए तैयार queries की एक लाइब्रेरी देता है, ताकि आप खाली SQL एडिटर के बजाय एक काम करने वाले उदाहरण से शुरुआत कर सकें। +अपने एजेंट डेटा से कोई भी सवाल पूछें और सेकंड में जवाब पाएं। FailproofAI Cloud आपको आपकी इवेंट्स और evaluations पर सहेजे गए, चलने के लिए तैयार queries की एक लाइब्रेरी देता है, ताकि आप खाली SQL एडिटर के बजाय एक काम करने वाले उदाहरण से शुरुआत कर सकें। -![सहेजे गए-queries की लाइब्रेरी: पुनः उपयोग योग्य queries का एक ग्रिड, दोनों built-in presets और कस्टम](/agenteye/images/queries.png) +![सहेजे गए-queries की लाइब्रेरी: पुनः उपयोग योग्य queries का एक ग्रिड, दोनों built-in presets और कस्टम](/cloud/images/queries.png) *आपकी सहेजी गई-queries लाइब्रेरी `//queries` पर: built-in presets आपकी टीम द्वारा सहेजे गए queries के साथ बैठे हुए।* @@ -23,7 +23,7 @@ description: "अपने एजेंट डेटा से कोई भी कोई भी query खोलें और यह SQL composer में उतरता है, जहां आप इसे समायोजित कर सकते हैं और तुरंत जवाब देख सकते हैं: कोई export नहीं, कोई round-trip नहीं, किसी और के इंतजार में नहीं। -![SQL query composer एक सहेजे गए query को चला रहा है, एक schema sidebar और एक live result grid के साथ](/agenteye/images/query-lab.png) +![SQL query composer एक सहेजे गए query को चला रहा है, एक schema sidebar और एक live result grid के साथ](/cloud/images/query-lab.png) *SQL composer: आपका query बाईं ओर, एक schema sidebar ताकि आप कभी column name का अनुमान न लगाएं, और नीचे एक live result grid।* @@ -44,14 +44,14 @@ agenteye query list # वही सहेजे गए queries, agenteye query run errs --arg prod # एक को चलाएं और rows print करें (pipes के लिए --json जोड़ें) ``` - पूरे command set के लिए [CLI और agents](/hi/agenteye/cli-and-agents) देखें। + पूरे command set के लिए [CLI और agents](/hi/cloud/cli) देखें। -- **AI assistant से।** निश्चित नहीं कि SQL को कैसे phrase करें? in-dashboard [AI assistant](/hi/agenteye/assistant) से plain English में पूछें और यह query को draft करेगा और इसे आपकी लाइब्रेरी में सहेज देगा। +- **AI assistant से।** निश्चित नहीं कि SQL को कैसे phrase करें? in-dashboard [AI assistant](/hi/cloud/assistant) से plain English में पूछें और यह query को draft करेगा और इसे आपकी लाइब्रेरी में सहेज देगा। एक सहेजे गए query को चलाना `queries:run` permission द्वारा gated है, queries को create या delete करने की permissions से अलग रखा गया है, ताकि आप read access grant कर सकें बिना हर किसी को लाइब्रेरी को rewrite करने दिए। ## संबंधित -- [Dashboards](/hi/agenteye/dashboards): query results को shared, org-wide charts में pin करें। -- [AI assistant](/hi/agenteye/assistant): plain English में सवाल पूछें और एक query वापस पाएं। -- [CLI और agents](/hi/agenteye/cli-and-agents): आपके terminal से वही queries को चलाएं और सहेजें। \ No newline at end of file +- [Dashboards](/hi/cloud/dashboards): query results को shared, org-wide charts में pin करें। +- [AI assistant](/hi/cloud/assistant): plain English में सवाल पूछें और एक query वापस पाएं। +- [CLI और agents](/hi/cloud/cli): आपके terminal से वही queries को चलाएं और सहेजें। \ No newline at end of file diff --git a/docs/hi/agenteye/python-sdk.mdx b/docs/hi/cloud/sdk.mdx similarity index 92% rename from docs/hi/agenteye/python-sdk.mdx rename to docs/hi/cloud/sdk.mdx index bb8a4269..b788430f 100644 --- a/docs/hi/agenteye/python-sdk.mdx +++ b/docs/hi/cloud/sdk.mdx @@ -3,11 +3,11 @@ title: "Python SDK" description: "अपने AI एजेंट्स को प्रोडक्शन में बिल्कुल देखें: हर एजेंट रन, टूल कॉल, मॉडल रिक्वेस्ट, हुक, और मानव हस्तक्षेप।" --- -अपने AI एजेंट्स को प्रोडक्शन में बिल्कुल देखें: हर एजेंट रन, टूल कॉल, मॉडल रिक्वेस्ट, हुक, और मानव हस्तक्षेप। Failproof AI Observability Python SDK आपके एजेंट कोड के अंदर से उस ट्रेल को रिकॉर्ड करता है ताकि आप डीबग, ऑडिट, और मूल्यांकन कर सकें कि क्या हुआ। जब भी आप Failproof AI Observability को अपने एजेंट्स को देखना चाहते हैं, तब इसका उपयोग करें। +अपने AI एजेंट्स को प्रोडक्शन में बिल्कुल देखें: हर एजेंट रन, टूल कॉल, मॉडल रिक्वेस्ट, हुक, और मानव हस्तक्षेप। FailproofAI Cloud Python SDK आपके एजेंट कोड के अंदर से उस ट्रेल को रिकॉर्ड करता है ताकि आप डीबग, ऑडिट, और मूल्यांकन कर सकें कि क्या हुआ। जब भी आप FailproofAI Cloud को अपने एजेंट्स को देखना चाहते हैं, तब इसका उपयोग करें। हुड के नीचे, SDK स्ट्रक्चर्ड ईवेंट्स को लोकल JSONL फाइलों में लिखता है, और कलेक्टर डेमन उन्हें चुनता है और स्वचालित रूप से प्लेटफॉर्म को भेज देता है। आप इन फाइलों को स्वयं प्रबंधित नहीं करते हैं। -> **सुझाव:** Failproof AI Observability के लिए नए हैं? यह पृष्ठ संपूर्ण SDK ईवेंट संदर्भ है। +> **सुझाव:** FailproofAI Cloud के लिए नए हैं? यह पृष्ठ संपूर्ण SDK ईवेंट संदर्भ है।
@@ -25,7 +25,7 @@ SDK को ग्राहकों को एक प्राइवेट व python -c "import agenteye; print(agenteye.__version__)" ``` -क्या कोडिंग एजेंट को पूरा इंटीग्रेशन करने देना पसंद करते हैं? [Python SDK Agent Skill](/hi/agenteye/python-sdk-skill) इंस्टॉल पाथ को जानता है, इंस्ट्रूमेंटेशन पॉइंट्स की योजना बनाता है, उन्हें लिखता है, और ईवेंट्स के आने की पुष्टि करता है। +क्या कोडिंग एजेंट को पूरा इंटीग्रेशन करने देना पसंद करते हैं? [Python SDK Agent Skill](/hi/cloud/agent-skills) इंस्टॉल पाथ को जानता है, इंस्ट्रूमेंटेशन पॉइंट्स की योजना बनाता है, उन्हें लिखता है, और ईवेंट्स के आने की पुष्टि करता है। --- @@ -59,7 +59,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### असली कॉल को इंस्ट्रूमेंट करना -व्यवहार में आप अपने मौजूदा एजेंट कोड को लपेटते हैं। एक मॉडल कॉल को `model_request` से पहले और `model_response` के बाद ब्रैकेट करें, ताकि दोनों ईवेंट्स असली रिक्वेस्ट को स्पैन करें और Failproof AI Observability उन्हें जोड़ सकें: +व्यवहार में आप अपने मौजूदा एजेंट कोड को लपेटते हैं। एक मॉडल कॉल को `model_request` से पहले और `model_response` के बाद ब्रैकेट करें, ताकि दोनों ईवेंट्स असली रिक्वेस्ट को स्पैन करें और FailproofAI Cloud उन्हें जोड़ सकें: ```python import anthropic @@ -98,7 +98,7 @@ agenteye.event.model_response( यहाँ देखें कि वे ईवेंट्स डैशबोर्ड पर कैसे दिखते हैं, प्रकार के अनुसार रंग-कोडित और पर्यावरण, एजेंट, और सेशन के अनुसार फ़िल्टर योग्य: -![लाइव ईवेंट्स स्ट्रीम, ईवेंट प्रकार के अनुसार रंग-कोडित और पर्यावरण, एजेंट, और सेशन के अनुसार फ़िल्टर योग्य](/agenteye/images/events-stream.png) +![लाइव ईवेंट्स स्ट्रीम, ईवेंट प्रकार के अनुसार रंग-कोडित और पर्यावरण, एजेंट, और सेशन के अनुसार फ़िल्टर योग्य](/cloud/images/events-stream.png) --- @@ -154,9 +154,9 @@ SDK केवल उन फील्ड्स को रिकॉर्ड क ## ईवेंट संदर्भ -अधिकांश ईवेंट्स स्टार्ट/एंड पेयर्स में आते हैं जो एक कोरिलेशन ID साझा करते हैं: `tool_use` और `tool_result` एक `tool_call_id` साझा करते हैं, `hook_triggered` और `hook_completed` एक `hook_id` साझा करते हैं, और `human_wait` और `human_input` एक `input_id` साझा करते हैं। स्टार्ट ईवेंट उत्सर्जित करें, काम करें, फिर एंड ईवेंट को समान ID के साथ उत्सर्जित करें। Failproof AI Observability पेयर को मेल करता है और आपके लिए `duration_ms` की गणना करता है, इसलिए आप कभी `duration_ms` स्वयं पास नहीं करते हैं। +अधिकांश ईवेंट्स स्टार्ट/एंड पेयर्स में आते हैं जो एक कोरिलेशन ID साझा करते हैं: `tool_use` और `tool_result` एक `tool_call_id` साझा करते हैं, `hook_triggered` और `hook_completed` एक `hook_id` साझा करते हैं, और `human_wait` और `human_input` एक `input_id` साझा करते हैं। स्टार्ट ईवेंट उत्सर्जित करें, काम करें, फिर एंड ईवेंट को समान ID के साथ उत्सर्जित करें। FailproofAI Cloud पेयर को मेल करता है और आपके लिए `duration_ms` की गणना करता है, इसलिए आप कभी `duration_ms` स्वयं पास नहीं करते हैं। -![एक सेशन का git-शैली एक्सीक्यूशन ग्राफ इसकी ईवेंट टाइमलाइन के साथ, पेयर्ड ईवेंट्स से पुनर्निर्मित, टूल/मॉडल/हुक ब्रेकडाउन पैनल के साथ](/agenteye/images/session-detail.png) +![एक सेशन का git-शैली एक्सीक्यूशन ग्राफ इसकी ईवेंट टाइमलाइन के साथ, पेयर्ड ईवेंट्स से पुनर्निर्मित, टूल/मॉडल/हुक ब्रेकडाउन पैनल के साथ](/cloud/images/session-detail.png) सभी ईवेंट मेथड्स को ये दो फील्ड्स आवश्यक हैं: @@ -430,5 +430,5 @@ agenteye.event.tool_use( ## अगले कदम -- [ईवेंट स्ट्रीम](/hi/agenteye/event-stream): ये ईवेंट्स लाइव में आने देखें, रंग-कोडित और पर्यावरण, एजेंट, और सेशन के अनुसार फ़िल्टर योग्य। -- [सेशन्स](/hi/agenteye/sessions): देखें कि पेयर्ड ईवेंट्स प्रत्येक एजेंट रन को एक्सीक्यूशन ग्राफ और टाइमलाइन के रूप में कैसे पुनर्निर्मित करते हैं। \ No newline at end of file +- [ईवेंट स्ट्रीम](/hi/cloud/event-stream): ये ईवेंट्स लाइव में आने देखें, रंग-कोडित और पर्यावरण, एजेंट, और सेशन के अनुसार फ़िल्टर योग्य। +- [सेशन्स](/hi/cloud/sessions): देखें कि पेयर्ड ईवेंट्स प्रत्येक एजेंट रन को एक्सीक्यूशन ग्राफ और टाइमलाइन के रूप में कैसे पुनर्निर्मित करते हैं। \ No newline at end of file diff --git a/docs/hi/cloud/security.mdx b/docs/hi/cloud/security.mdx new file mode 100644 index 00000000..6fe57716 --- /dev/null +++ b/docs/hi/cloud/security.mdx @@ -0,0 +1,68 @@ +--- +title: "सुरक्षा" +description: "FailproofAI Cloud आपके उत्पादन एजेंटों के पास रखने के लिए बनाया गया है, जिसका अर्थ है कि यह आपके prompts, tool inputs, और outputs को देखता है।" +--- + + +FailproofAI Cloud आपके उत्पादन एजेंटों के पास रखने के लिए बनाया गया है, जिसका अर्थ है कि यह आपके prompts, tool inputs, और outputs को देखता है। यह पृष्ठ बताता है कि यह उस डेटा को कैसे अलग-थलग, नियंत्रित, और आपके हाथों में रखता है। यदि आप सुरक्षा समीक्षा के लिए FailproofAI Cloud का मूल्यांकन कर रहे हैं, तो यहाँ से शुरू करें। + +--- + +## आपका डेटा आपके परिवेश में रहता है + +FailproofAI Cloud self-hosted है। Events, prompts, मॉडल responses, और analytics आपके अपने डेटाबेस में, आपके अपने परिवेश में संग्रहीत हैं। कोई भी डेटा storage के लिए किसी third-party SaaS को नहीं भेजा जाता है, और आपका डेटा आपके अपने cloud account में रहता है। + +--- + +## टेनेंट isolation + +एक FailproofAI Cloud instance कई संगठनों को host कर सकता है, और प्रत्येक को storage layer पर अलग किया जाता है — सिर्फ UI द्वारा नहीं, बल्कि डेटाबेस द्वारा लागू किया जाता है: + +- किसी संगठन का operational data (users, keys, dashboards, saved queries) उस org तक सीमित है, और cross-org reads को डेटाबेस द्वारा ही block किया जाता है। +- प्रत्येक ingested event को अपने owning org के साथ stamp किया जाता है, इसलिए एक संगठन की events को कभी भी दूसरे द्वारा नहीं पढ़ा जा सकता। + +प्रत्येक dashboard route एक org slug (`//…`) के अंतर्गत scoped है। + +--- + +## Sign-in + +FailproofAI Cloud passwordless, email-based sign-in का उपयोग करता है। phish या leak करने के लिए कोई password नहीं है। एक उपयोगकर्ता एक one-time code (या एक one-click magic link) का अनुरोध करता है, जो उन्हें email किया जाता है और जल्दी expire हो जाता है। Sign-in को एक **allowlist** द्वारा gate किया जाता है: केवल email addresses (या domains) जिन्हें आप permit करते हैं, authenticate कर सकते हैं। + +![FailproofAI Cloud sign-in screen, जो आपके email को एक single-use code भेजता है](/cloud/images/login.png) + +--- + +## API keys के साथ scoped access + +प्रत्येक client एक API key के साथ authenticate करता है जो granular, least-privilege permissions रखता है। एक collector को केवल `events:add` की जरूरत है; एक dashboard या assistant key read-only हो सकता है; destructive actions (delete, regenerate) अलग grants हैं जिन्हें आप शामिल करना चुनते हैं। + +![API keys page: प्रत्येक key की permission grants, read, write, और destructive scope द्वारा colour-coded](/cloud/images/api-keys.png) + +Admin bootstrap key को setup के लिए रखें, और बाकी सब कुछ के लिए narrow keys जारी करें। [API keys](/hi/cloud/access) देखें। + +--- + +## एक read-only, approval-gated assistant + +Dashboard में [AI assistant](/hi/cloud/assistant) आपके डेटा पर प्रश्नों का उत्तर देता है, लेकिन यह design द्वारा constrained है: + +- यह **डिफ़ॉल्ट रूप से read-only है**: इसका SQL एक guard के माध्यम से चलता है जो केवल `SELECT`/`WITH` queries को permit करता है, single-statement, एक row cap के साथ। +- जो कुछ भी यह creates करता है (एक saved query, एक dashboard) **approval-gated है**: आप प्रत्येक write से पहले review और approve करते हैं। +- यह **कभी delete नहीं कर सकता**। + +इसलिए एक teammate यह पूछ सकता है "इस सप्ताह किन agents में सबसे अधिक errors थीं?" और answer पर कार्रवाई कर सकता है, बिना इसके कि assistant अपने आप पर आपके डेटा को change या remove कर सके। + +--- + +## Transit में + +सभी traffic HTTPS के माध्यम से चलता है। आप अपने अपने certificates के साथ TLS को terminate करते हैं, इसलिए collector-to-server और browser-to-server traffic transit में encrypted है। + +--- + +## अगले कदम + +- [Overview](/hi/cloud/overview): FailproofAI Cloud कैसे एक साथ आता है। +- [API keys](/hi/cloud/access): collector, dashboard, और assistant के लिए access scope करें। +- [FailproofAI Cloud](/hi/cloud/overview): FailproofAI Cloud आपके agents से क्या captures करता है। \ No newline at end of file diff --git a/docs/hi/agenteye/sessions.mdx b/docs/hi/cloud/sessions.mdx similarity index 76% rename from docs/hi/agenteye/sessions.mdx rename to docs/hi/cloud/sessions.mdx index a0e9c55f..dbaca1b4 100644 --- a/docs/hi/agenteye/sessions.mdx +++ b/docs/hi/cloud/sessions.mdx @@ -4,9 +4,9 @@ description: "किसी भी रन से हर ईवेंट, एक --- -यह अनुमान लगाना बंद करें कि कोई रन क्यों विफल हुआ। Failproof AI Observability किसी रन के हर ईवेंट को एक पठनीय पंक्ति में रखता है, फिर पूरे रन को गिट-स्टाइल चित्र के रूप में खींचता है जिसे आप सेकंड में समझ सकते हैं, इसलिए आप देखते हैं कि आपके एजेंट ने क्या किया, चरण दर चरण। +यह अनुमान लगाना बंद करें कि कोई रन क्यों विफल हुआ। FailproofAI Cloud किसी रन के हर ईवेंट को एक पठनीय पंक्ति में रखता है, फिर पूरे रन को गिट-स्टाइल चित्र के रूप में खींचता है जिसे आप सेकंड में समझ सकते हैं, इसलिए आप देखते हैं कि आपके एजेंट ने क्या किया, चरण दर चरण। -![सेशन की सूची: प्रति रन एक पंक्ति, सभी वातावरण और एजेंट्स के साथ, स्टेटस पिल्स और मूल्यांकन स्कोर बैजेज के साथ](/agenteye/images/sessions-list.png) +![सेशन की सूची: प्रति रन एक पंक्ति, सभी वातावरण और एजेंट्स के साथ, स्टेटस पिल्स और मूल्यांकन स्कोर बैजेज के साथ](/cloud/images/sessions-list.png) *प्रति रन एक पंक्ति: स्टेटस पिल आपको एक नज़र में बताता है कि रन कैसे समाप्त हुआ, और एक स्कोर बैज एक बार एक मूल्यांकनकर्ता जुड़ जाता है।* @@ -30,7 +30,7 @@ description: "किसी भी रन से हर ईवेंट, एक ## पूरे रन को चित्र के रूप में पढ़ें -![एक सेशन के गिट-स्टाइल एक्सीक्यूशन ग्राफ के बगल में इसका ईवेंट टाइमलाइन, टूल, मॉडल, और हुक ब्रेकडाउन पैनल के साथ](/agenteye/images/session-detail.png) +![एक सेशन के गिट-स्टाइल एक्सीक्यूशन ग्राफ के बगल में इसका ईवेंट टाइमलाइन, टूल, मॉडल, और हुक ब्रेकडाउन पैनल के साथ](/cloud/images/session-detail.png) *एक्सीक्यूशन ग्राफ (बाएं) ईवेंट टाइमलाइन के बगल में बैठता है; दाहिनी रेल रन के लिए टूल्स, मॉडल्स, हुक्स, और टोकन खर्च को विभाजित करता है।* @@ -38,7 +38,7 @@ description: "किसी भी रन से हर ईवेंट, एक दाहिनी रेल आपको प्रति-रन ब्रेकडाउन देता है: कौन से टूल्स और मॉडल्स चले, कौन से हुक्स फायर हुए, और रन ने टोकन में क्या खर्च किया। यह "इस रन की लागत इतनी अधिक क्यों थी?" या "कौन सा टूल धीमा है?" का उत्तर है, ठीक इसके बगल में ग्राफ बैठा है जो इसका कारण बना। -व्यक्तिगत ईवेंट्स एड्रेसेबल हैं, इसलिए आप किसी को "सेशन, लगभग दो तिहाई नीचे" के बजाय एक ही पल के लिए एक लिंक दे सकते हैं। किसी भी ईवेंट से लिंक कॉपी करें, या [ऑडिट](/hi/agenteye/audits) ढूंढ से या कोई त्रुटि से एक लिंक फॉलो करें, और सेशन उस ईवेंट को चुना हुआ और स्क्रॉल किए गए के साथ खुलता है। यह बहुत लंबे रन्स के लिए भी होता है: टाइमलाइन आपके ब्राउज़र की खातिर एक सीमित खिड़की लोड करता है, और एक लिंक जो उस खिड़की के बाहर इंगित करता है फिर भी अपना ईवेंट पाता है, न कि शुरुआत में आपको छोड़ देता है। अगर ईवेंट आपकी रिटेंशन विंडो से बाहर हो गया है, तो पेज आपको बताता है कि इसके बजाय शांति से कुछ नहीं चुनता। +व्यक्तिगत ईवेंट्स एड्रेसेबल हैं, इसलिए आप किसी को "सेशन, लगभग दो तिहाई नीचे" के बजाय एक ही पल के लिए एक लिंक दे सकते हैं। किसी भी ईवेंट से लिंक कॉपी करें, या [ऑडिट](/hi/cloud/audits) ढूंढ से या कोई त्रुटि से एक लिंक फॉलो करें, और सेशन उस ईवेंट को चुना हुआ और स्क्रॉल किए गए के साथ खुलता है। यह बहुत लंबे रन्स के लिए भी होता है: टाइमलाइन आपके ब्राउज़र की खातिर एक सीमित खिड़की लोड करता है, और एक लिंक जो उस खिड़की के बाहर इंगित करता है फिर भी अपना ईवेंट पाता है, न कि शुरुआत में आपको छोड़ देता है। अगर ईवेंट आपकी रिटेंशन विंडो से बाहर हो गया है, तो पेज आपको बताता है कि इसके बजाय शांति से कुछ नहीं चुनता। --- @@ -46,12 +46,12 @@ description: "किसी भी रन से हर ईवेंट, एक हर डैशबोर्ड पेज आपके संगठन (`//…`) के लिए स्कॉप किया गया है। सेशन **Observe** के अंतर्गत बाईं साइडबार में रहता है, ईवेंट्स के बगल में, सूची के शीर्ष में तारीख की रेंज, वातावरण, एजेंट, और सेशन फ़िल्टर के साथ। हर पंक्ति इसके पूर्ण एक्सीक्यूशन ग्राफ से एक क्लिक दूर है। -स्कोर बैजेज़ और स्कोर-रेंज फ़िल्टरिंग को चालू करने के लिए, एक मूल्यांकनकर्ता को कनेक्ट करें: [Evaluations](/hi/agenteye/evaluations) देखें। +स्कोर बैजेज़ और स्कोर-रेंज फ़िल्टरिंग को चालू करने के लिए, एक मूल्यांकनकर्ता को कनेक्ट करें: [Evaluations](/hi/cloud/evaluations) देखें। --- ## संबंधित -- [Event stream](/hi/agenteye/event-stream): कच्चा, प्रति-चरण ट्रेल जिससे हर सेशन रोल किया जाता है। -- [Evaluations](/hi/agenteye/evaluations): एक मूल्यांकनकर्ता को कनेक्ट करें, इसलिए हर रन को एक स्कोर बैज मिलता है जिससे आप फ़िल्टर कर सकते हैं। -- [Telemetry](/hi/agenteye/telemetry): रन्स अपने एजेंट से इन सेशन्स में कैसे जाते हैं। \ No newline at end of file +- [Event stream](/hi/cloud/event-stream): कच्चा, प्रति-चरण ट्रेल जिससे हर सेशन रोल किया जाता है। +- [Evaluations](/hi/cloud/evaluations): एक मूल्यांकनकर्ता को कनेक्ट करें, इसलिए हर रन को एक स्कोर बैज मिलता है जिससे आप फ़िल्टर कर सकते हैं। +- [Telemetry](/hi/cloud/performance): रन्स अपने एजेंट से इन सेशन्स में कैसे जाते हैं। \ No newline at end of file diff --git a/docs/hi/concepts.mdx b/docs/hi/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/hi/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/hi/daemon.mdx b/docs/hi/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/hi/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/hi/dashboard.mdx b/docs/hi/dashboard.mdx index 54815c48..812066e0 100644 --- a/docs/hi/dashboard.mdx +++ b/docs/hi/dashboard.mdx @@ -70,7 +70,7 @@ Session viewer स्वायत्त agents के लिए मुख्य 4. **How to improve** — calm row list, प्रत्येक prescribed policy के लिए एक: white में policy name, one-line description, install command + दाईं ओर copy button। Section header `enable all N → projected · ` को पढ़ता है (वह score जो आप हर fix को लागू करने के साथ प्राप्त करेंगे), और इसका `[install all]` button प्रत्येक prescribed policy के लिए संयुक्त `failproofai policy add a b c …` command को copy करता है। 5. **Come back better** — दो side-by-side cards। Left: एक reminder सेट करें (`3d` / `7d` / `14d` / `30d` cadence picker; authed होने के बाद `/api/auth/reminder` के माध्यम से persist); Right: failproof perks unlock करें — `invite a friend` एक modal खोलता है जो comma/space/newline-separated friend emails की एक सूची लेता है (प्रति send max 10), उन्हें `/api/audit/invite` पर POST करता है, जो api-server के `POST /v0/invite` को forward करता है। Api-server `invite@failproof.ai` से प्रत्येक recipient को एक email भेजता है sender Cc के साथ और `Reply-To` सेट, इसलिए recipient देखता है कि किसने उन्हें invite किया और sender को अपने inbox में एक copy मिलता है। Anonymous users पहले `AuthDialog` के माध्यम से routed होते हैं ताकि invites के आउट जाने से पहले sender का email जाना जाए। Entitlement / perks fulfillment एक follow-up है। -`failproofai audit` runtime द्वारा driven — अंतर्निहित scan engine, supported flags, और per-transcript cache invariants के लिए [Audit CLI](/hi/cli/audit) देखें। Dashboard सबसे हाल का result को `~/.failproofai/audit-dashboard.json` पर cache करता है (mode `0600`, single slot, new runs overwrite) ताकि revisits instant हों; **दोनों per-transcript और whole-result caches को read पर reject किया जाता है एक बार जब वे 7 days से पुरानी हों** ताकि dashboard कभी भी silently एक week-old result serve न करे — TTL के पास `/audit` अपनी empty state में गिरता है और एक fresh run के लिए prompt करता है। Report के निचले हिस्से के पास `[ re-audit now ]` को क्लिक करने से `/api/audit/run` पर `noCache: true` के साथ POST होता है — re-audit per-transcript cache को bypass करता है और silently cached result return करने के बजाय scratch से हर transcript को फिर से स्कैन करता है — और dashboard `/api/audit/status` को 1Hz पर poll करता है जब तक run समाप्त न हो जाए; एक sticky pink progress strip run के दौरान viewport के शीर्ष को pin करता है एक elapsed timer के साथ, और fresh result success पर जगह में swap होता है (कोई full-page reload नहीं; एक failed re-audit prior report को intact छोड़ता है)। Failure पर strip `RerunError.kind` (`timeout` / `network` / `post_failed`) से keyed copy के साथ red हो जाता है। Empty state (कोई cache नहीं या expired) और zero-sessions state (cache exists लेकिन scan को कोई transcripts नहीं मिले) को अलग से surface किया जाता है। +`failproofai audit` runtime द्वारा driven — अंतर्निहित scan engine, supported flags, और per-transcript cache invariants के लिए [Audit CLI](/hi/audit) देखें। Dashboard सबसे हाल का result को `~/.failproofai/audit-dashboard.json` पर cache करता है (mode `0600`, single slot, new runs overwrite) ताकि revisits instant हों; **दोनों per-transcript और whole-result caches को read पर reject किया जाता है एक बार जब वे 7 days से पुरानी हों** ताकि dashboard कभी भी silently एक week-old result serve न करे — TTL के पास `/audit` अपनी empty state में गिरता है और एक fresh run के लिए prompt करता है। Report के निचले हिस्से के पास `[ re-audit now ]` को क्लिक करने से `/api/audit/run` पर `noCache: true` के साथ POST होता है — re-audit per-transcript cache को bypass करता है और silently cached result return करने के बजाय scratch से हर transcript को फिर से स्कैन करता है — और dashboard `/api/audit/status` को 1Hz पर poll करता है जब तक run समाप्त न हो जाए; एक sticky pink progress strip run के दौरान viewport के शीर्ष को pin करता है एक elapsed timer के साथ, और fresh result success पर जगह में swap होता है (कोई full-page reload नहीं; एक failed re-audit prior report को intact छोड़ता है)। Failure पर strip `RerunError.kind` (`timeout` / `network` / `post_failed`) से keyed copy के साथ red हो जाता है। Empty state (कोई cache नहीं या expired) और zero-sessions state (cache exists लेकिन scan को कोई transcripts नहीं मिले) को अलग से surface किया जाता है। ### Policies diff --git a/docs/hi/architecture.mdx b/docs/hi/how-it-works.mdx similarity index 100% rename from docs/hi/architecture.mdx rename to docs/hi/how-it-works.mdx diff --git a/docs/hi/introduction.mdx b/docs/hi/introduction.mdx index 2cda61ac..f5d62922 100644 --- a/docs/hi/introduction.mdx +++ b/docs/hi/introduction.mdx @@ -55,4 +55,4 @@ failproofai policies --install # policies enable करें (या skip क failproofai # dashboard launch करें ``` -पूर्ण walkthrough के लिए [Getting started](/hi/getting-started) guide देखें। \ No newline at end of file +पूर्ण walkthrough के लिए [Getting started](/hi/quickstart) guide देखें। \ No newline at end of file diff --git a/docs/hi/policies.mdx b/docs/hi/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/hi/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/hi/getting-started.mdx b/docs/hi/quickstart.mdx similarity index 100% rename from docs/hi/getting-started.mdx rename to docs/hi/quickstart.mdx diff --git a/docs/hi/reference/files.mdx b/docs/hi/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/hi/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/how-it-works.mdx b/docs/how-it-works.mdx new file mode 100644 index 00000000..a7e9f111 --- /dev/null +++ b/docs/how-it-works.mdx @@ -0,0 +1,313 @@ +--- +title: How it works +description: "The whole path — from an agent's tool call, through a policy decision, to your dashboard." +icon: sitemap +--- + +You can use FailproofAI without reading this page. Read it when you want to know *why* a +decision came back the way it did, what happens when something in the chain is down, or +what exactly is on the wire between a machine and the cloud. + +--- + +## The one-paragraph version + +Every supported agent CLI can run an external command at fixed points in its loop — +before a tool call, after it, when the turn ends. FailproofAI installs itself at those +points. When the agent tries to do something, your machine evaluates the active policies +against that exact tool call and answers **allow**, **deny**, or **instruct** in a shape +the agent understands. The decision is recorded locally. If the machine is connected to +FailproofAI Cloud, the decision and the session go up, and centrally-managed policy comes +down. + +```mermaid +flowchart LR + A["Agent CLI
(Claude Code, Codex, …)"] -->|"tool call"| B["failproofai hook"] + B --> C{"daemon
configured?"} + C -->|yes| D["failproofaid
(background service)"] + C -->|no| E["in-process
evaluation"] + D --> F["policy engine"] + E --> F + F -->|"allow / deny / instruct"| A + F --> G["local activity log"] + G --> H["local dashboard"] + D <-->|"policy down · activity up"| I["FailproofAI Cloud"] +``` + +--- + +## Step 1 — The agent hands over the tool call + +Each CLI has its own hook contract, and FailproofAI speaks all of them. Three shapes exist +in the wild: + +| Shape | CLIs | How FailproofAI attaches | +|---|---|---| +| **External command** | Claude Code, Codex, Copilot, Cursor, Hermes, Factory Droid, Devin, Antigravity, Goose | A hook entry in the CLI's settings file runs `failproofai --hook --cli ` and passes the event as JSON on stdin. | +| **In-process plugin** | OpenCode, OpenClaw | A small generated plugin the CLI loads at startup; it calls the FailproofAI binary and translates the answer back into the plugin's own return shape. | +| **Extension package** | Pi | A bundled extension the CLI loads at startup, which shells out per event. | + +The payload carries the session id, the working directory, the tool name, and the tool's +input: + +```json +{ + "session_id": "abc123", + "transcript_path": "/home/you/.claude/projects/myproject/sessions/abc123.jsonl", + "cwd": "/home/you/myproject", + "hook_event_name": "PreToolUse", + "tool_name": "Bash", + "tool_input": { "command": "sudo apt install nodejs" } +} +``` + +Not every CLI spells those fields the same way. Copilot sends `path` where Claude sends +`file_path`; Antigravity sends camelCase protojson; Goose sends `working_dir` instead of +`cwd`. FailproofAI **canonicalizes all of it** — event names, tool names, and tool-input +keys — before a single policy runs. That is what lets one policy set work identically +across 12 CLIs, and why a rule you write for Claude Code also fires on Cursor. + +Payloads are capped at 1 MB. Anything larger is discarded and every policy implicitly +allows, rather than stalling the agent on a pathological input. + +--- + +## Step 2 — The machine decides who evaluates + +There are two evaluation paths, and which one runs is decided by a single flag on the +machine: whether setup configured the daemon. + +### On a machine that completed setup: the daemon evaluates + +`failproofaid` is a background service that stays warm. The hook connects to it over a +Unix socket in `~/.failproofai/run/`, hands over the event, and gets the decision back. +Two separate budgets apply, and the split matters: + +- **~150 ms to connect.** This is the "is anything listening?" probe. A dead daemon must + never add latency to a tool call. +- **30 s for the answer** once connected. A policy is allowed to do real work — shell out + to `git`, call an API, ask an LLM — and a correct-but-slow evaluation must not be + mistaken for a dead service. + +**If the daemon cannot answer, the tool call is denied.** Not allowed — denied. There is +no in-process fallback on this path, and that is the entire point: a second policy engine +reachable by stopping the first is not a guarantee, and a machine where killing a service +silently disables every guardrail is not a guarded machine. A protocol-version mismatch +denies too, with a message naming the version and telling you to run `failproofai config`, +because the fix is different from "the daemon is down." + +[More on the daemon, including how it is supervised →](/daemon) + +### On a machine that has not: in-process evaluation + +Exactly two situations reach this path, and neither is a configured user machine: + +1. **The machine is not set up yet.** No hooks are installed either, so nothing is + evaluating anything. +2. **This repository's own development configs.** FailproofAI's contributors run the + policy engine in-process against the package they are editing, deliberately. + +--- + +## Step 3 — Policies run, in order + +The engine loads and merges configuration for the working directory, registers every +enabled policy, and evaluates them in a fixed order: + + + + In definition order, with each policy's parameters resolved from your config merged + over its schema defaults. + + + Whatever your organization deployed to this machine. Each artifact's SHA-256 is + verified immediately before it is loaded, so a modified file is refused rather than + executed. Policies deployed in `observe` mode are evaluated exactly like any other, + then have their verdict discarded — that is how you measure a rollout against real + traffic before it can block anyone's work. + + + Files you named with `--custom`, in configured order. + + + `*policies.{js,mjs,ts}` from the project's `.failproofai/policies/`, then from + `~/.failproofai/policies/`. Alphabetical within each directory. + + + +Three rules govern the result: + +- **The first `deny` short-circuits.** Nothing after it runs, and its reason is the answer. +- **`instruct` messages accumulate.** All of them are delivered together. +- **`allow` messages accumulate too.** A policy can permit an action *and* tell the model + something useful ("all CI checks passed on this branch"). + +Custom policies are **fail-open by design**: a syntax error, a missing file, a thrown +exception, or a function that runs longer than 10 seconds is logged and treated as allow. +Your own broken rule never takes the built-ins down with it. + +--- + +## Step 4 — The decision goes back in the CLI's own dialect + +Each CLI honors a different channel, and getting this wrong is the difference between a +real block and a warning nobody reads. FailproofAI emits the right one per CLI and per event: + +| Channel | Used by | +|---|---| +| `{hookSpecificOutput:{permissionDecision:"deny"}}` JSON | Claude Code, Copilot, Codex | +| `{decision:"block", reason}` JSON on stdout | Devin, Goose, Hermes, Factory Droid (turn-end only) | +| Exit code 2 + stderr | Factory Droid (tool events), Claude Code `Stop` | +| `{decision:"deny"}` / `{decision:"continue"}` | Antigravity | +| `{followup_message}` | Cursor turn-end | +| Plugin return values (`{block:true}`, `{action:"revise"}`, thrown errors) | OpenCode, OpenClaw, Pi | + +The one that surprises people is the **turn-end gate**. Policies like +`require-commit-before-stop` do not block a tool — they refuse to let the agent *finish*. +Where a CLI supports it, a deny at turn end forces another turn with the reason as the +prompt, so the agent goes back and finishes the job. Where a CLI has no turn-end event +(Hermes, Goose), those policies simply do not apply — [the support +matrix](/agent-support) says exactly which is which, per CLI, rather than leaving you to +find out from a rule that quietly never fired. + +--- + +## Step 5 — Everything is recorded + +After each event, one line per non-allow decision is appended to +`~/.failproofai/hook-activity/`: + +```json +{ + "timestamp": "2026-08-12T12:34:56.789Z", + "sessionId": "abc123", + "eventType": "PreToolUse", + "toolName": "Bash", + "policyName": "block-sudo", + "decision": "deny", + "reason": "sudo command blocked by failproofai", + "durationMs": 12 +} +``` + +Plain allows are not logged — that keeps the file small enough to stay honest. This log is +what the [local dashboard's activity view](/dashboard) reads, and what the daemon ships to +the cloud when a machine is connected. + +The agent CLIs write their own session transcripts, in their own formats and locations. +FailproofAI reads them — never modifies, moves, or deletes them — to render session +replays, to run the [audit](/audit), and, on a connected machine, to give the cloud a full +picture of the run. + +--- + +## Step 6 — On a connected machine, two streams open + +Connecting is one command with one URL and one key, and it configures **two independent +capabilities**: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down
policies:pull"| D + D -->|"activity + sessions up
events:add"| S +``` + +- **Policy down.** The daemon polls for this machine's desired state, downloads any + policy artifacts it does not have, verifies each one's digest, and writes a manifest the + hook path reads. A machine that goes offline keeps enforcing the last deployment it + successfully fetched. +- **Activity up.** Hook decisions and — unless you passed `--no-transcripts` — session + transcripts are spooled to disk and uploaded in batches. If the network is down, the + spool grows and drains later; nothing is dropped on the floor. + +The two are verified and reported **separately**, because a key can carry one permission +and not the other. That is a real, supported state, not a broken setup: connecting with a +`policies:pull`-only key gives you enforcement with an empty dashboard, and the CLI says +exactly that instead of letting you discover it a week later. + +[Connecting a machine, in detail →](/cloud/connect) + +--- + +## Configuration, and how it merges + +Three files, evaluated in priority order: + +```text +[1] {cwd}/.failproofai/policies-config.json ← project (committed) +[2] {cwd}/.failproofai/policies-config.local.json ← local (gitignored) +[3] ~/.failproofai/policies-config.json ← global +``` + +- `enabledPolicies` — the **deduplicated union** of all three. A policy switched on + anywhere is on. +- `policyParams` — **first file that defines a policy's params wins entirely.** There is + no deep merge inside a policy's parameter block. +- `customPoliciesPaths`, `llm` — first file that defines it wins. +- `disabledCustomPolicies` — union across all scopes. + +Changes take effect on the next hook event. Nothing to restart. + +[Full configuration reference →](/configuration) + +--- + +## Failure modes, in one table + +The useful thing about a guardrail is knowing what it does when *it* breaks. + +| What breaks | What happens | +|---|---| +| The daemon is down, on a configured machine | **Tool call denied.** Fail-closed by design. | +| Daemon and CLI versions disagree on the protocol | **Denied**, with a message naming the version and pointing at `failproofai config`. | +| A custom policy throws, times out, or fails to parse | Logged; treated as **allow**. Built-ins are unaffected. | +| The payload exceeds 1 MB | Discarded; every policy implicitly allows. | +| A cloud-managed artifact fails its digest check | The deployment is **refused** rather than partially applied. | +| The cloud is unreachable | Enforcement continues on the last deployment. Activity spools locally and uploads when the network returns. | +| The agent CLI has no turn-end event | `require-*-before-stop` policies do not apply there. Stated per CLI in [the matrix](/agent-support). | + +--- + +## Performance + +The hook sits on the critical path of every tool call, so its cost is a product decision, +not an implementation detail: + +- The daemon **pre-warms its evaluation worker** as soon as it starts, off the hook path, + so the first real tool call of a session never pays a cold start. +- Typical evaluations with no external calls complete in well under 100 ms. +- Policies that shell out (`git`, `gh`) or call an LLM cost what those calls cost — which + is why the response budget is 30 seconds and the connect probe is 150 ms. +- Pattern matching inside policies runs against **parsed command tokens**, not raw + strings. An allowlist entry for `sudo systemctl status *` cannot be bypassed by + appending `; rm -rf /`. + +--- + +## Next + + + + + What `failproofaid` is, how it is supervised, and how it reaches your machine. + + + + Every CLI, every event, and what can actually be blocked where. + + + + Every term in these docs, defined once. + + + + The context object, the decision helpers, and the loading rules. + + + diff --git a/docs/images/local-session-viewer.png b/docs/images/local-session-viewer.png new file mode 100644 index 0000000000000000000000000000000000000000..f4d8a3f90acb0652bead53ed6483845c86267ed7 GIT binary patch literal 198779 zcmeFY_di>I8$Yh?yHn9=79B=Wv}WzLipH#6yS29(5k%-hYgX)7wKp|`NVHX(n28ai zW)Km3#K`CLzCYu>KfiqcfbS=dM;_vwoOALz=XG7L>p8A3FSXSeF0o&tqoZR0y?Cxi zM@RpJj_&-03+I3>74uJS0)I|<>#04XE63hirlY$}2YUY0z%Ole(%~J)0{Y}w!-4>v z23L@x-5#>K+Sur8GG&EnJ38_2q}6z51Z!(wx|kKp^Yz*7D*=+p)^AU_M*d`g%P4;ah$r4Qv@1!c--{8Rva;)kTc*Y;oVD-Qc+c9I}Kd@KVR?e{E+Ww_|GkW zKfhbMOZR_oeBE*T|84t!_kiyIfAiowqtBWDjF8)Qc2Vq0d~|fCH<`I^{rw>O^sM85 ze&g?F<#RW`o%)ZB=VGg>Y|c!6dfNRk@@tD0pS}_@m*QjI7UwkfF6*{{fZMlJI=ZEE zr~Cio7&=y;Um@vXsV_zVqYj)Bo>nA4A`;0{7|8 z`<;L2=t9N+uN(64AAjh1(pBI`@&EHJ?0#GblJWms9QdSt&cq}qK0(s?Tk0t~$$uS2NB5-pHl0FzvSyX6D_nD`nL*&NXKZ}y zsOyv)bAN2F{g8K`B5&BFKxh9rn0wi{`J_yfZ|I5ie`Lvr9Xz~wAb3|-G{ajB;N%Yt zuDs)VR!-X=`}W4W85*(Mv|eY+dZz#GZHcg!tj;*TE>|fHI3uU5#1G;BD=qWWZ#|Kw zo80Rev!Z9O+gcvEsPv!D9k?y=^SRG?`S{ZMw>v>!(4%+hlrP@=hU&Dg?69tU0hi~B zsi-P@!|K&wemr;%Vx!;xy(z(kx==fywCaoEq4@HG_~JnDd9Q}RmeA8J59c3^Q@(#E zxSbj2`mek4QO#Zz5w!o2@&bOcm$X-0Q`%%UWKw37UTvGT*qd6iU;h}SXJ?q8NOk-; zMSGn-;SR~HW&82x8C@es4__R<+}{}VWeTNYE|TObm8!`XH{p)NKHhRQG&0)JFw2lV zRyRpw)T(za@>SSU9VM!mx{g;JabTMHn|8xa9J#cAEF@WF(+;O;{GIEe=~{d#^9)ZZ z(?}~8bDsv$D+;4HJbooV)gx|b<}1$vr~G5v@n}pO+f^ZupmQPDpG3Nk8a_VpckiYN z3JBzIJ~{82#2Ps&^6#L0TlM`_Pgf^a5Bu4bmzU?hh=?lOz|bP6YV(rYG)CN(*?M~^ z77^U$L|t#$kxdS+Z%$>Pdko6W&!=)Z5N*LibZ$t(_Av#fUnC%Ks%36&w&v)Aw;M~9vEP2jHf8cMb2 z(7a-U0@Y(Y0*~+Cx_+HNV~{-sJRlBfclIUw*H;>#M@3ZZ`~f`o&=*#yL_YL+XZFu! z#-W0Rl~9iI1f2E2Vz1fG?>HJ4o=kUCX<_})8U@4-@cPkd-4?mrxb!P8JUaZ+VcG&w z)bQ!kEJdncoRr7d3Wp+O_p(;H7$_<#>Tif2uT*J>Zii$d-3eL3W_2L=B341AA&y^A zQc@+!qS3rhac}d9)b%Gyz_2XZfGNr^7sDSIV*8Qdit+?}yAsn*%#GpnI<<815s3YI zTw&MZIk)@5tV@gExT9{Rlz9Yim4BUEA#rREWzr}YdzGXi@KR=qmc~%NjzX4y`epKu zPN7GSo+=%0?D2wv>uFaxMeeV7A+Aq~h>H!zCnB;N8w1FM;BW0iHtS1KdsWD~^Sep^ z8t7$(x}_K32T$hRo#E{`uU}i9k`gc6`#fvAzuu4*)W7;=!@0anez}YA9jJY10Jt#*;-jaC+0(~hiFUSA<7&bgOSbh8+`7UEmoVej< zU^TttK=#XLyU!Sg-r~w&P>#~#hu9fLwBAkafV4n*KAMrh-89hC6WsG$ z8wxvJ&>{k^iee4T%Ga)oovA7BKTGTFSd!z5a*!`QoOR%sH-(LjMq!McdQWF!uOnzrT+A>nAu8h-m_sHe z5$BAn@@59XF+FmH3ees2kui~x{n=-yeubms~=W>p=vfP6FI$BzouaBDwMdR<4+0 zMrLP<2E`G0`>;(o#Yxg`^d7GXhhtqSyXeX)Qv&TIdslwX2)Q^zsY>FCRp9#`?lD=Z zhYnnxw9tlP?aSIt(2LkE}V!c^Hqc3dru9M+&&xCVITm2bcLCA2Pr;VvGE71%P-D?;{vlP9G_t%*#WVXKRXh#2UV zwk+88q0|w*8fx4p0~Qvz2WOPeVqTU^x*HoAg;t4)`z#*$hsVSidg*1aVMxsAb!#Hp zkjXxx&$19jY1Cq4W5c{LG{glgPWO?Df{#%{F5?0Mozyl3$kMidfnZmKvv1JJ8w==h zFIuYi^?DT4U0>bHYjgt)-l4XN29LxDuUK7gi@_OJ)IWdpXaJro>7s-;6Pl(!JYo~i zCpCGq2;5Pek}kzyo{|*JD{wA~vI5oDrVn-}!(*7HrjHUIFg29h16A%Z!xV0?UBgt$yuRiOt{Jc5@~3usXMs&nil z{$%3-R}oymVil05wgr(*r@qD!s>UCPfv88-Wu`TLDLgHy!BRzJZb@mds83+i8j50> z?%(DFZ-R)LJ|NF**e7Z1;vafjSbW4P(C$h~`35=E3@r~ImKb%}*Q$%vYz*~*_Y|gA zhU|X)*?m66jP~;@xZLtG9s1g1pDrN#+xq7d%1AR4t8Z5+n=EC*JHYM)1)&V-1X1Z_ zR()}}k-aYYFfTc};Qw6@yqj?t2Oeumt{Xv_#Wxu)r)ytrLTFxUTvs+Xk8F$N;*HCl zg%1XmFf(&<7FF7)3P?#A@qr9+IO^t)r#Sp>9|iILm3!D^xj@79Gog}_(j|3umS;m$ zER5LX^LD8EkHaoqGBh>i@|uWT)(;#p{N?1^Py;4JYiC}+em!r)ru_wcm%BqB4DKuU zPXGLV%ATq|6WWVuuj{zL8_Av5X)J8KYvNtAiRUgUs)8DZ924g>KO6t5zbd;UAN4ZX*1a;Q@hlPCa|tL06bv+sOZLp(@OFs+*{L)k_7 zsKa}VBJ>J$RtC1=230nh6K{&a_#_{$6;O)QQ;n1L>nUzVoX3G-UB2Wb@kf^TSTf5~ zteQ=k!z|2b!w0qNgQB~|ofTEyw(9U}ZP72u1J?{WT`B29Zy}t~8eNUd0JY|g%@9cF=?@Iem_HsR8@HW`=4h*m~+c82}l zsi~<6Nl6t=O@Xkg4e`I_CN?b?QEF6Dj#<&;lr~9Hgl^_;$z5|&#cP3(7V-RrFI&9 z_+MuoeB=mkS0=_9kHkFc3M(r?3kwUt>GTZ@aFKX~^B&8iD5=ei~hd;8PYmG>oiO|%R2ZQ3I7laR^$ z!;|&TD=VS9m0}MKfY`q$BJ#Adatf6ItTI!u!@xjO&y$1y+I7Vs^;jFYis=OM0^(*{FuxlIDeE zO9fF=)a8l>ncc^wZYJQ~oarAkNeN$?H8oaNwn;j*v|6fZCC8qMsHqO{DCnaIpF#I- z=P{kk8J`N4yFOR*$9P(QWgCP5kJOH@mnd zx^Ie^Js78sR*#<`Szs;VeaU&P%`{Oh)Pd0W;UG$^@xscX+40Be8c}1trgFb~KA)?_ zNx>W6i??DZTQX#PMVD)}Ghy0lxUN~A#DO_X83CVI1(!{@hz^!J7!@`M9v$px?H%9y zL#bpD#xE;j^i@bm$2kvfk_pru^Nbm#mCCgr=1V)&nqDW<{6O8A`(k9Rx^pt_t_3W)+xV`%?J=U6Ps@(6fQ$@CM#jWK=ke!cO}3o?Xb0j63C0~WZ(% zr^95ypD@_V$-dTb1@`(YDpoXkCuhecDyU`j_b88@>hI#FhNosi(Sq2;;lb@n_NzAw z0+1TqNw)^;y0AJw9``V($BI3J*HqR#(aK;J*5a<{^z8KF;3Aiaj2yOO4B`r){N00T z1BgtqXHHm0$$AUfSH@W};Y^)&J{vX@4}E;8$=aD!=*?vH)J6!=%-6OQtIn1e`Posj z(QMn!t-*-9o3!ioLeJ=5JBolEY@t4d7$t9@j}G%*SWlz`FVF9ix^nG< zgR|j4eP2_gAD@=?E#E$AmpoZ zXigz~bUHNP#+6&r?lofK;y2i4&`$Z;+3F!GDrjoUwrvs|o_lewUEt0gOJM5R+1(sR zJ5&@G7t5uK$O$QXdR7*f7vW~E7|Qk7SM%v?X5=aWzj_`4RJ66JlE+ZZ1k$keBbSx<&aR%GukCZX1F<;(PAYIUD&f;) z6#`K=yFWU+I7*j!^No?|Z53 znp0xus+Ec;Y4Fw_F!QKb@fokG@+v!itK8w4Zur%51CKftEUFaj z)G=cL0)dR&+yFAG+QW?Ga;Bm8@~5^teeiq47_O3S9*cfa(lzxaDhb-fD*H+cEy|BEm?fr@q_jY>TqAh9p@kG;jl2E z7)h-nW@g71%^B_w5RV5lawqUxJRYSNSuG*C&qFw*T($?r6Xd~z-!DDUNqeY$-K>wB zJXKvmm?6f;+olJvS<9qPamWE+cX)7i_vS(Pm+;lpb9+@lp2*7&_*fbov`P^a+FP4O zAQ0PO=eY)|>@;jEUrD&&FQX6F4xbJc6yzOnRRjcs`?Ykl{#6DP)t>gTvB_}`7#*q` zS~dtUEl$%$AyUl@EOoC|b?v{N^O2Rbex6ud;GC)Bxp}0Tp+!f0kCRwx#Rx#^*j1WJ zLh+Xu>MYNgHowV1q#l+-ypV)47OOz5I6keHde+&%#Ae`lhsAciG!1BUl`1JNDX#Gd z&M+@8CnTmPWcBoDj;$Kua5y1R(dKRv1|FZ1QXaYOBa_bfmpyx88I*MJzOHKw2v8uS zt7#V~tHp2)0hB~zAo^tf?Wm;l@P*d4wk~HNwlLbfym_)-w)?k+?y#O(G75cjUrI{V zzFGy~syu@dQW;ua1_U6wy3sRriHXbrPdHtVK=eb3r6eTIp;gW5M`)Eid3kvPckezk zDp3I%7&bI`Ub;tJI*yNo* zUlE8|RJ~%Ba*}AqL;fhB<_`>PqEz;}GqCattWtYBJ9p8RFc@s~hl-|VB#;9D9&~p0 zbcq>c_r%WLZ(0dp*gA6?8ymlF0Z>2Ok%28YkMDeBM0B*%Jr$g)sjj0lQfV{g%&euQbw|gGEtLOu=G36h&tF+9p#ud=CHD$e=4Jzr zkADyw_u{v1cz9GGjwd?@OTr*4?PgO998a{rtF81ESO|g|7h4>5;sod=CGRNwkx^SI z+3D5t@L|u7{{CvG+R1^mnI<3x*?*8EjUxruHCH649RAYwc$5MoUm#{gRV+{DSou=Z zE<_kAW!Ak)BF>*9>kA@QcBUN@+!}7q&MrH% z5f#qecu}hJqrh2f8=pfr>5yAU0I+Rui;McWC~Ai(Cs_r>E@EYP9S?mQOlww-ai(LP zXVa#f!zSH_j76GX6=2gjhoi^GWzp-iLKfvC@$o#9UPC)w9WR(j^=Q)kD z(g8?lXlz+-a;MSW%@egV1?Bc>RCr+7^s=A_k>y5!EO`V63pxfq|1^X|A8ib@*Fvn8 zV1BBq2zS)ZIjjDf-jn0=+rKm_Letu&zqltkqy-f-i#!&0^ix;&>Nvn`VFl3*n`jY& zV~VaIi; zfh^ABw-n6_Su6uuN3hUf+Ph2i%IB-ch<}-W1>-@}vtqow0{7|C5wi?=iSeH*^jJ7J z%FV{Szt2hEHft zOVgYToC5=QIACKzdV9Ixr!?w&`4!8BOv4E)8M>4rB^v4mR4*Mtr`_g<&`#S zLO_u%p06_iAop;`A-+JVZGHU3dt?&_+hg*VO^xZi9Kn<^`H zg{Cd@0N&}c%=FcJRX#XlQ6>a-ZeuAfDs_8MqoP~h?)Mbk|MUXfyt|i}o?cW^(PTcx zDQJV5bca=W##sj0sf@0sykRwci(hWspIxY;!tlFfnjXz>c3GR=b|*=No=7^{N;+$9 z3dnPF@tr41))0(A&cwqiEk?XMg9;TUgFhA6|RqoO0 zx_QnO!15j$3=9kyRQv4g?Dk*;ZTIYyQR%B9N)S!$=~m3=&;9(QkuK{;ND-78F(bGV zk;={+*LjFBdjJPHyZd&!S8Ip?)X2`T5-p$LjxvNB7+IEwL?oVtIRVtN8rRWB<_E4! z*85>k7c*saGJ`hXkQ#lu0qS`?Ei?EP@d4`FA+8d-WHwF3Was4A!tWSsM3gd@NMCmr zD=I27VGP^rzQ-%0BHFxPnw@>ujy2R0>J}2Tp2K~be&g0Ca3jQc>CEk5NTi(Mn678Q<_~*Lw^Cev@^g!1 zNf29XCAHfrY2_DAC$v1sm9;T=*URcG5sk|mHvwx77!@q?*a;3?Ejn*6I--y@xh#QWiS%A1$j11D5Z%1fpN?KE8`vav#XLl9172 z7$*e^uw83q=T3b9oIYnZ@TQbOw+(Aoc;wJg{8iIrDRm~SNqMSN(=d8RT+>9@$O77% zASf&}6w1J1ha^b)Z$9Vc=Xb)UnG_e7<4GR>(Dt3YTSm%p!va&+ebl0F{Z+2guot4( z#MK`S3^ZvUzD_YPG=k%Qo$wb^Xeds zvD>-U8K4@}K_I+8G25w&woCUHRez>T? z)E7Huft`Df>|R+xyj!-hwgz$+Z<os_tJIMrEwp0dwB|cMrq;BXBs9fiYoRp`rJ@l60E=@>$0vaEG-4p-M`!ekZFc16&d$0&r6BFOxvgg zIF-%%Nw4%LV0>YZ0Z32TWPj>z-fnSmD8El(zoYP{IqtMutK*3R<`lG@T(iL#7E3?3 zUAL*=AFi&Yx3H%s5)w+G{>)9EslE;@tYzi+(*2uiLt4naq zm3hI|!D+iNws9e!1g3qA*>z#e=XWNpY>zabW%;rknFqGwbWVly=(tq0aYl$U?0^?U zC3!)}sUu0C>P8=95AU?}T_aseccQ5(!lsHa?H*-;#vaYr&10?L0D##+9dh9%jjgWU z2$}-MmvOL1KXutu$x-P0_=c>$vvU>3b8lq9M|ODDm-N~a!_J->AD3RVzyH`Gpwmkx zJ^CBz8QXGEBXO6ZZ7!Hp=062ix>S)1z24zx3U#yQal17nA%&sG1s=z=u$Ck#dWCW zQCdDH1*6^oE*t`HI{PC$=4J^XKme4fy1M3Y@}nFukrTOM+D@r2VAZd;OjTQTF&O4-K*uRpIb;nAU%(N(iv%|z<)5Atn3KG{DWw2z6QZPahY z7w-_Sa^755T%D{oR%@xzfREo@R#A&q7dyxYf0B7*Q<-d6nS7ksp8fIs7Roh=r{QQy zDPruJuUr+O5g7%Z$X~s)X1S~dR3mSMQVvVX%M-G8oe5WCvqtj9Tp3?w-0_LFf6@!W z;;YXimmSQew+wkt$XGCSt0-wHayYgzXj}8xX<8Lm;}i6)?mnyr8$f)->EcQ|9V8Wu z%E$;#xW*aV=sxVD@SVS&fKge_fz1%7$>s&ktB1oQ(f>d{odmA5B2tozGBr(d4b_-@ zR=+%l&&T@AtnhyfmFr=4YhE{-G-Yxqh}o!}BV{&~*0z!}Thw-gvJdU8zR_3l`Hn}o zF53hGXI*`j(;PH*qTKB)wrnh(v5?*xc}p5^h7JbH5keQR_apFKt`wL zz;JoguErl{IRY;wLe zik7r58PJoP(+>$GkL5aod;jl?b6|W*0cmp2C~WoV9cf6m=+P0CL#TG;cvpl1tdf88 zlk;%l)yC~X`a^j-3tf|<>50Q9KfcSkH+QnwuSFN>^Brc;R$PR(f3ECu+yhWuIhuN@ zH$_~7nC4xlNZpV7dVUT)5G@cy+7XSRDWf1rW4qwP^gS>@h&tg*+0x5LfRZadc8z>` zEmKFK1YkeUMrKMlHLV>{g@JV5G&rOW-gfjv5~R&FQxi$_N=m!?N;Tfbhd}`ILTckE z#{s3D4|Z0C`sX7UW>o9AHerHU-)heASN!wONU8_Ag!}l9+`p5zA*F(0scvI70y{EW z&9c-Ho6|@+QSx=+T1xL*__D?wFo#|(BWFqPCb_A`a20_`iuedor17XftW|vy++N16 z5Rl6(EU!<&>^<$^VzBQ?&aaW6gbWj{{8YGM6G3PGeZ7-E1PhGfp*fJm|LosPiWdOr zuh`hDm^BioFA4x&;QDWcLryNvB!i(|!4K#%GD52`;}#(_1Xb>xea@>v&IWJ0wki&}=NLThR~WQTtyze3KJY!-va{<`Qi;O^yw4UNtl3+D_-qf6H|wA6 zT??#Al^B4E0L~Z_LLGT&pa$@qP?zTZkiuJ@2B*gOqUndzk1*xao?IW?UVk__y?&hm!@;C@ku<2^+ z8pEDvf!f<{xwdWYVXz$iB^qA?Jik9rEDMOnVbsRWq~zo!dh-mFYo)dvINAvIUCQ-J zWA9sY<0Xu+S@Q*sAoros(ZXihoB-gMz5nQ^*+O$^Q`N|?ipfVX({eAbCTZ4%1ZGU68cV zte6EmY*IOO?`~S-!XimBXyO&1uv$=>s@2tw6y;dzFc~-eI01oCb;Jsi#}~<60N6BK zj`km6{Hdw(P9LUTRq9d7Hthwj&CY?_Ii-aZgbn{%bcBWNoxu^vz<%jw&oF?P z^jUgK+wuG0`|+@_n7ctC!!!hb^EXT_iI}Z5S6ocQf&3&pne%Xr1G}5wr}pv=EA}O$59E0{mG3g z0-0S#%nZ7_%7pM;jC6wMe)yIB_s0Qr8Y#hex*vjpA>r4qOM2PXZEl_4d%C=hN4=uI zbm`2mKKS0<-`y3MEE#Y6PUlL;qy0+i(N-C3Fq*B51n!%HotjbYJmjQO!;z9M_(<&$ z7jsvrbTkTbHZyq?4b2SN&s0jX)ze5`^9w~$sR8JdHg!O`J`W6yvip7Im`TEhdIP8? z?CkSN>odO;o!aXwZFoUIoqIlg7y!IDmx|^*mx_jL9ZL>$iL_!8x-$4No1}!q3T|Zj z5-WaUb+8LEHDm#q5Cp2;?QMgb%;&fXk=#VIOlf^C&+VfJ{z^6#&6uk0KGy%4W)kA# z1%Wgp(AkOkAOi3Y7*FNaCEa_xaq&sgH6J4R?}~{%yQen)0>0{7qB?IYBrB+%syJPo zn27bb5%X47(tWk?+sw`m18l0+%UQjOog+2SFew#*h(`?eS~5d*gWbg$Q5O1M{9S8r zm!3@hR9wae^=zsDBKV7fL87)X*2H_>6IFX$(FcJlZUqGvP%7^!8ZB2BbTG4JWk;C- zwi)V_t5>t^q({RMXJGJ0#iLHx@dV>@h#&udB05(Yp@eLIN*+WEkI= z?=F^+Z)#N2(b?M-G_usixok-QjGvZ<;YOvkzxH5S>-5R)C(Fo~I3BIe^itrK>cO{Z zK^rcTl1Ss!vXT;|ET?lhbzB)b>CkXr%2dITa#%w0*Ka1(`JKx?ONbJY5sx+4!*9l} z5v?3bf+CEo?*Uo+S#!hQ&l8s;y2g1py#%<6$I!sw>`nYSM69f&EuRlyDjr1ONRjN({*CreObE-w zI&l%wsN;jm;3dLzlfFPFh>Dk9>p#PEEKs;_uBlmr*@=<^1gG4=K{18h{p;x6e;0@5 zy}f|Oqx-3wH$}6`q|EDa6GYWdo(nf@&~04{+VmyZ)%qK!6DF^{eK``_GP6E;)eZ2M zZT@wFwa%8^aIG~pw3-fLX=sU=c&3*_@`p;r|p z%uNgz_T-ff4FPiFA1yvZV_fIM$nOcUgkj$$)ja;mL}_W%jjJ zwXS?+TRj#?$QU!jn!zE^b@viI>t{`uo5>CNeA`Ub!1HRDbf0bnWTeb;-q6M~wF}PM z{{*v9%0%J_z`$IMs_x1Dj(?vaQ_nlUj+W##&G%G~x}XqKdcotqy&BlFL+-k-QGm|t zWGy~Bwck4?bh6d8oKWXynG+8RD~!rYKm$^0qrSGP0mKaz!zGl85Q1ZyJ9R3)#d@H; z7$euYU}Zwxp1*~LO4LUn&wc2d^eV0AncpMF-cy4eu8k&-UeQ;-SOy7X4lQc)uK(5{ z9rOFs@2A|qH8j_a#uUu2$aa_-8&~k397F;(;7jy!6;Fma$dC1I1X#Vaw9Ep8G?{el zcM^_v{@4CRn_Ac*_8U<``uZc(an|>i*{Du81^YjY_5mzy&-vJG*~!ocYGC z);c@7;(ukgn=C9PdA$43Y1$_M4BB=$*<_~~mh-dq+czp_5A7QN#{H||_32*{BkDr&Hiuj=aE2?;48slDSREIV>T(}bQH zrv`S(L1?8gAT$b11r~IF4bVy77(#sv#U&X5;y*3hCKWX;_oqIB`TP;N*rT6C2b1j< z4EqWF>GOGf>zBHu;aTIlk{%8O(Vel(v?3?dePmxF(ed}JzdAy~XimhmR9A!O(i^je zi;W^QW^{snjUk`_9ABMHnMC?5kI6y@QUW}1h<87RJ8icT`w1O@f4w;CPQBXTT)O>3 zS+DuRRjJfnveGx?^uZ75z;^6p<8qq|KtALFN~>c#cJ~>57=5^e=dV}7CQVPIQVq|Z zRRg46_Ns!xI72g?+Nh}Xutz%iVDS2%JCn3OnLPd#$`(5~jlXiJu7l1PN$_VD?k7Eaj!cvBoO^8`%mB#6O#}wza`xi&HV3%kujh4VfyA1Q@#a5 zTy+-5S*4}sb8UblMYR!lvigKN*qx+lNhD@Jx(rwvfg}T(K3PovONt_!{T-fMkmchW zmoBfVd0_j5{rDrDPYK`xPk%HGEeHjyb;o;?CG;YXN2E^{%_!>@x1~}vPn{Zhduq(C zJ8@;^d@$vZpkz*>z0&)<))9l$Ti@8Q;sPYP1Bdh+YJkVvifX#I5mqR>68OY z{yU16+`-%LD^SThB09zZuwGv`dPr1 zQl)@S01qzT!yjDr5H&(f3YKxUCH*xi!uNN3y9>T&*+O1=J3zsw@Cm?Q{H5&vdQ1Ux z!Sd^qJDhSxrN6T80iJ$(8O{%5u2Ob(cJ~DZ#r{ac@-OjQs2M{nb^kB*# zot>YB$(~E4QdKz>0($lyE0#xGJomn%@PZW`GN-G!%l+&(`iN*}qKF66JKt2TloU30 zY2O;_?3|haa(nvjIa3ppJRK#%Es3V;UoX&i01gN+J>><>PzML`g7X3(!AsuWt{x(S!DwWY8tAziZO_$AtHQn&$XlUf(+O=7qp9dPSmXZ>e8XKZJ?0Z0Zv77(~BIdV%ldg3;WW4!Fplf*1*1{M# zIACW7DFI}GF;}A~Hic(|<_oZo18=R2FO&fs5r@*T>$lXadTws%75feV(F(ZJ!Zp;l ze|!wSRJ=0ntEC3V%91YkCO@h@mM!aq3Kxr5{McXnq${^(aMW_2Q`z+7y?*uBgX)H0$SoU1l?Eq1w)!VDUxZPH3#O!K=44JA7v)-aN8s*F? zs5TWoMbXgt#abDpwEE!9X+O8|L;`LW9^8-48{@3eriyK^_PKim(~gm?(ZLW`o#(bo zm%z*=l7BSDnP=%ajwx}2r(t@sK5K(YqAnk>-H3<%y^rs>vo(6p2>}W~2;xRk_F1Q# z+S_h59)X_9WtNY(L%1f9x>^}iWwxiblTySBoECfjs6U>Zd30G}cZA;uP;vx?#FgU! zDto?@EUgF>l%zNvMI4G!##sOHIvHr+`P&V^EFxd6CLMNaW_$5p8t4-2-!_)(`lhD& zfcrv4MYS-mwQO{hl~Y#U2a61l=Ys8Tw3TN*{%l~3bHtr@X^AF9Gbd>n2K5OG?D!_h z2Ticr+1VF}@$h%r ziY<>;TI2AzJu0~BxT1+5v{f{kN;wSPl7>C_K+a9r{}QX;L3p%EwXC^=f~jwHLvLCMLp3 z-Mp#2`5hhWXtV0_D_z)hlpOl0=WRE_Zb$}n<|b$mJYt^J;O!tJBLD(Ony056gV~my zx?bpzc1aI_HMLvxv9#+3a%QC?)`mKl6GLj+!VUm%4lu9)98LD{@C~g@DV6R%t9+ab zei=71wX!o(4yE`EH)21{h+5JT^HpvwdJ8Flh<{bpi4kQg$Vh? zc<#Kj5V#XjAR#JawxzG?tmhiu*2*I&E~Z^{b)XSCE8h$18XQp!nt81B<;Rcl2Je`P zaP7zEx3*fMg+xSf=w|54x%QX=fB*zaqO6)41E5_E=&~~EP81#hnBkpuZf<(Hn3+|e z<7n8#A^lEFdyZD7o6*y!Z=Q^NV+^xkvG6;gtGlAC>j4wd@u9u*b!)#c6ZAB>(f`Iz zMWCsX?r~7Nx_UxfTwd-xPlK5$&`e@rEt%JV9T8I;gnVg27LP!$*;`v z_J3mMj9UkHta=Ytr6 zai0>cs;f%~HYYx)540DYs8u{@dFgXo)G9b|yCVGaTR|ZqWuP0b3yU4@Ng2Ayb~US- zMfhb<(EB>q$thBWg@r{<7`J~9ka7W}Xl%Sk)2hDkoK7|}wCy@YPe5vM9|dI-Y#5B@ z=l)fTnqvx?Gr>%m@~b9a7Y}{Y@$ho9o$M6LEq0F#T2-iA$7*={1@GD17Qyw)8zRl2 z5p{j44*W!Fj#)&VuWN;DaRH;dT$cZ3=Fr@oytAehMwWQtEtf$p*gS4yu{?L_~T_pCS`Rf`*iz1vVv$TIB<6 zAx?l-76I^ZfP)eGtHN9COLR2;=wOQnB-*?giifcVdhrViR*~p@blihDB+PD8TQ-73 zpimlJQBfzM%-cNqB=q{V>j4rbp3rsK3zLT$L!|K$a6 z)L=SoDw2eRuoQ^bw`&v(5#kKInZggY=WEYb*M%o=>CiqYLx0V@{To_pF*jYd7Jtz1 zi92W-0!9}=NdbJ(jgrk77zZ@qiZ)PIp8HzzCwXsIwzp}{nKzO%>nc5QWgVd-2Mf}G z<(MK63!kGBg=O{1$A5&sf0QzH>3!5qTsoG2u7)jK6EqynZ)F5NiGAnkc@O<_s!?(2 zoP2yDRkc<^s`Dy`FjO-h;j(hNjTl6}X4jjVx7WN^5_IuNL0?Ap+2*5i#tK5Nq|%XF zzaP{m9ywV?Gz_)Rr(pG|%1B7$leD3cc)9@2r{W%7-ikg~$hzQ>w z)iXe6z&5_P;-p0<6f6n2*VBWit^v%M69A1)Wi5RAddSYJx2mGsq>!~oekW`N2LYyRg_O0rX?FO$kL<@bxqIVb z;N8?5ph!L3BRbJe$XL)7V3pdNhH6|bAx78I7FJd>==__SWHAZFHsIjBu2`P;2C5*KXjl{sC@ znL*p!$u0Kq_HO46%_=A>zTBidJ-6(i)TPz?UQQaKqpf)$92ac(J6dOh>FnNo_wbx_ zUqg3?}^bNLUva|36@z=x4{+C;jFD8%0xYzX3X-d#NP6wO`k?BZ0>W%%q=(J{;2J9pci*St~}%JlE|N$=04MM zx#tJa56&?~QS$3W7bp-XyqWeg4*fAY6y+b6g}C7>o7mJS2C^B&zGpKL%fWTMIKZR* z^JbSp0wl=9WU6r?{karAdLu#7tSR~nY*hn(G}|w(XJVS6LB#gdgjBfh)=7;VI?cr#cE*0bV@lb^&`?>wp;7Ot zjg4}}ma7JSwXaa|*;~-&022+=B_z1{<0BI>#UX=GOlEy#y>Dbft6J#Z3Z_AvcE_L< zmpP2$KynnGNMtl9j?WjCu}X@VXv}ni4mDA?CO*%gn-lM;mBVN|;Ol|CN&OgXnC?C4 zW}DlD=o5@6a(CwqwC3&1FP`)ENDD~k=c_Fvs4TDft6?4kR13hX{CbX^9icbf|MgLh zT3%kQBtYFdIj%wr=gQv9e)5!|78Zfv%X>=kfPu#K?Fj&e3try0{d=ZW7OUj|Ol~d@ zid^S?Ddbt=UiGide0QiB&>oklus5Lq3izk7?xstzlBju&A9~AWCcAaav;88Qw0l?& z%xy|_q*kV4>`8_=A>x@CG^6<_|JG&$lU;L6TJ->(5Bx#kDqvTqe!aE6u{XaFd~7qa zg-Z9)45i3Fii&Wlw#J59&5dyT-Gw%hIA!L&iNp+w`PW}7(}LYD#a@5-`!wJ^!GFF8 zk5+3{|A{TA+Num&_@K#_iFw9k9S6@hqUDuyfY7;FZQIw`EM;c?61E$ppIbRxz$~(IWcHY zN~#-fb3v#YuqoLt^>;OqO7*?R7Dn)Gl?;{;^S?f|49ykkl(K7ThYY*6ssK zPr&Qy>I{qx7t$b^DLhW@?pVMOJy_eb8&JA5`Ss4tzoQv81VN%!idWLWi%uy4`&kF= zR!1(PDK&2G7LdtQ^zOvH1v@*LjZZe-fV2V8GHa+3DKV0C0LHsoM=EdNSBN;v08DXZ zCWXhK&0qAQ_9sW!fg7Q|^rDcOip=V9&w~HK-g`$ioqcbk%sA>;kg+m~z$hRp0xC^P zh$Ewj2q;y$NE1R)AYcdq%Lq1#L24)>2vS225JGJ95_+gX1ri`60YVa5l6x}W-?aDM z`|rEncdc*M%EXDLozFSDJbUlwp%`Lk8VKN>AX?*L$kD&1+4A+1S`hqP&~)2c z{Zd<@lw--Y<2k7GYfLG}C*_r+alc{C=JbQbpzHOmm%9$m-UY^gHjmN`56yqYIrG9C zTgT{2&EV5P@ox6+;kLHHaa=<}aN+BEul)hWq5MFYl$u9Sq=YPmV@Cqlxg$j_B)fe; z#lA;w*@4o02E5F~i|6XZ-I9xcJ|rX82tvg_O7e0s&!MV`KZ-h1Y8O3Ct z*0yM#PfIO-re4XL54!X5_bOS<4Ch*opC9oWPQBSIIqg%k=@lQb)tLAo@T0HhGJxaQ%6eQ)6Rp5Oat938;>acG%J%t$hJY3npz}^4gTb(^ zynPo-e^>07IG=Gfquw4I=|TGC>WT36<+`!K{)V+NTeW?1e6^og?GI}7gS39ax|d~2 z#6r)=<)w$3gOHe=C#wrha=dE=`uXF)uu456S+1N!LHY(smw*tkbUINbzf%Q&^$HS+ zbi$B{UsDPTqdbIu6VJ&(pEFN~hvmvN)Gb^zaLq`mS~Pv4Yc`T!_=cqd9&f*{$|^ep z;!f|}s}yC{xy7LlL{;GLb>qRK+S!KDZN{pk*^nnl3mhbFouJSZBb}Zf1*a&18)-=T zQl9rr_bI!2c`SC#qSTKP7fRV zjpgvqF`J!@k+Q#L+e$iRYU3zmTDh5kcM*Kx`+2rNO^x{lRWV>e?>MSma}Lq5I_`J$tWj&3jXK5=-ISp+eQ! z(Hn^1YoMSD-~4*iZuBgRvkGB;Rbfsx`1?}=yKGZH+$Z0f9zszo6(4UCOgjseB6#sE zPu_Um6#991W`Yv^Y}iU^$wWtlFlt9d*lIOdn;JdX+l$8d*UEVk*I&%zCiGXi`pCCg zSw}!0kt5>SdM>m-+m72iGSU&|ZM{qH=0w`vwN}LF8N*yK>#%?S^g&&ZtDs<3;cXQt zgPd>uFj5>1CQI9#h0{sM^kCqkr(5K9(L*;n{rSE++aFiraI{pIf7e#%LdJYq^Q$nr ze?#iDlC|4Psf(jk9IFnwhGE`NzEOIb2VJ-mqwnn4u9s4N z_1T8k(;1u_J$POkyQ^e@9j}(rH9uToaeQAn*zF!fa9mdwCzmn;yV@F)9JE`M8KZq9 zy`;xGB;7y$z9&M#X}U{$M%0zr?J&fw#odQm`{)!@v+J$Et zp-`^D4V6gW5yn2Y-r57#b?(OU-s8d>!GeSce40MB~U2K+)|5|7maHI6jr-{XT zfqU*V4FcJ9kp1h6oIH8R!)naFTPnR{_AcWSjkr17=Z`wEnqOZNUN+-?Y&emWljGS< z$W(+O5UV=tHaUt+&4Q9DZDG)mF8i@zN&7}_*TS&Kc4D@s#mOUz?_SKXYq(R$iK_`< zs;%~&h;k1bDBbA$>N&lx=0}2=z)zSgai3Z&bq-`rZcXawER2?t3H3F0IK!I_3v0_7 zMy(8mvG4C?^UNmF69*(Qz^aGw-s5N;M##K|Ds5;o!gGQ?LaN37_w8H)a$AFa|q zyeCA)c9JNep=h`37ATn;<%#?7IWMko>-m$XqftBm~ z4cMMY*@Tz@s~*(|&TJ(%e8Ms@ahi>kdAZFhec+7eDGZUaR5Z|Ec~W+LXmyq0FHCkX z^w(zqLyaH}v#$28ZobEa5ri|#-JfEoKktoc2UVY?#n|xUNlxKb-ms)R5GJm(aAsZF zc(vboALk`2f@aH|$UWmqfz%w}OaTc3jlh~8L(s~d$biTc^ z3DTD_KO+x@teIzu%d+kgzMhf|o#{!tvr%?XJK#5cZjZ$DH{+6)XZzDxtvjl|TF$L+r>$0v)xVmD-9Um~qn8`~Ev@U6&HkGK}Oh_;YU?M3AxCYLbBD<=S zNsi%gEaP2CNmqVe#;cf^n4o>z5i=hQ>n@2OS5+nDD`qBNFH{O4A>fm&0Y+y%g55Vr z$yh2c2eHrW>L*#1xw{lbZS$)%f{t6=5i|W&OA37O;K8$Qm-RQYcr=NKdU+)qiHp_3 za3l@vXfa&Md8Z zG;jJ<{6t1;d`P=i19!TF7tuTHDqQX}rF_wG5G4TZ$_|yHAl_JHi{54Kj{}~gVzdNb z#M=Pb0{&80ezD^q_LqfJhGOf($NU|I5W(jTbuBGV-idhjpuw)I=*r4UkHHF(zxm3& z|M5bg**nptFk&ZbX6CYiEAVL?hYXY-_fit0XsBACFW-t*Xs^%+%s{=%2iYE<7hdJHZ=%cciJVK%k&ug4(UEmNxuj`bU?YY zPl&Vv2@2K3yFD>=71Q2V`c89e*mBxcz4`m1yL@YbyV6l>r?NLH zA>(S0zC36R<*a|CE1aXD;MkPaGE=K4XGiH}Z}Ofk zEggot(T*#`LIrLD8cz~#g6B1CW#^lPsc$`8>K~)`ZNcGgV--O_Z78mxGuM{{Y)U+c zrw)gV^*~Mn7?H0$jATL7f;*Dagdf-elv>ZctgJI2hh-eU)0`gS8IC*ptC`*zH88Dj z4#Jsuxm=k;0MTbI#IvaZdPIoNu61&{hB zscUM2$>pQs;?cmpTror%9>Q3^E%Eb(69%g@{KsOVm*=E!j~A{bg85bwB!twK06(#B zyO+1mRk(=>y6fgW_v!AE)k}aj>q1s(UAg_*eDoOTG@QH(qvuLkbk}sn(UB}d$T5?- zQ>+PX^-H!kqt+!3DZAx#h)>9%BN2HGR~vi&f}!CN;Ncu4n`2_Q05;a{>@4gZ8$)xs z)&2cJ6!}7||7AGb3Y>2XkeeTwo_1``B4ncoCqwv)mkvbu`q&LrVIN;OB66k~C@xrl z@mP2Fkx|`pw?%YivAz%xbs*I%u|HWF8XCsB{^HONjXjl~bjF|{VBILQrJYlC(9Zg0 zdZA&EmFdQ!MPIcyVnuS!o6WZ6QB`?ReNo<_Yr1NgsG;??uKs93VPQ2{Q;Ui*^BKoj zj*l+(obd0&hfow%2bE21jl8%rXH=}F_s<^Rw?sG?lG9Mk3tU*O8*w;IZwT`xx(0sy z9XG|a&`%&^taFQU)c4wKjTzO^St?faXK2*amq^hdnW{{kS$PJrnAD=vAx~h zaYqbJT9oO0wzhH3e6CVg2a;Gd&VGNpB?s}Ah7u?0X$LJJ_7&)C0}tP zqoW|KO5!2-(ocd<-6%aSI`tHOiCW;sUH^8Ms0NK4k}6k?z-}G!YPQ^U5DlD~L+J1T zD)Kv7#PZ_dz~$YDcKhJYtrQ`po|hJqThgHaR^jczGq>*9JWln@MCw;~636H|)t=(f z(Y5R@d|+FvKk36(MkX$dE`?Z4u<08dOa(n8o=J|!rbW(XD&BPLODGuFN+~R?n6v3K-r`-qpgTHE60DCk96jq^7P(`kVq00O-J5>L{XFSoCQ>^fEB$J0^@N%O90# z;ZqKai<{V2@*nuDbF?TUEQG&~U%B(iBmReN9 z$t<`s0Jp{#%7|A0zN8k^(Na06k?SXTW3qjEu+uSFSNKga|7YuT7u{&%tn9eD+8w== zW>yHQc6RT?SkCHt`ErV6AUltko|SEw>pk|aQd9k{fYXTK%gsC|Q(j7939g{Mq!%7S zeM@Iv<P(VFZ>=@xj(EvPB0iU9A)43Qw(!1+&YB}X+rKK6R zwdEy-3SWG!00FXK4Fer3a`%L+JAe)00I;qM7clM$qOx|)pW2kUu2oG09wZLQLjD2_ z^_79{tbG65H~e4qzAoQzYs*n)fQ7^r*{U9%4Pa1@uzeU+FPY_d+=q1qa;L5om$oC82r zP7gR zSLCjZ*A4O?o~tYfy>BO=T3ef=RrSJr8$!P52DF(XgCJ1s-HQU|EIx4kAw{WnSB^7r z<+hWr!%Gt5jKTp2I6F6L~e}APyE>aOh=kc)i!?_Qzr09w@PCAQcM?9#+kc zv~)LL9SJV-u}x^F>8CsEYG6b;i})5 z%zAdlDMvd@yqV4!JhPlV*LB7y(Ae0cD@=#%6k=hse_Bv8zXU74-SbV9dk?!rN!LB> z_kT6&M^rLS9HjPxY#}{(q|9)0Z<14}HrWB`>XcH!DGZ^210(We0M$OBYnm2!!r~a# zx49iV)gK$iyGutp*bMXeqxrVeD_C z7`lIr0ysk2v(k!gQ!Tr-r@nMuIR*l*T5g?`D5UjQ7fYAtjq`5K(dPmOZS&kWYVJCI zGOYp-NUNeJwV=>{XhyPWVuLH8%_Fed`ael@00oi24p zM8Mqwwe-ZVgG>OEgNR#8f6{tL^O2{qQ^kXGoIe~5;JNfygRSjp_GbVyR z{#uw=+*06FkSvkd)L7-^Fcxw<5NU3YKh?*II8JNeIp)7RPCA&EnI=Ll0b*!!fP4Lf z06;yWCtqb5=svP!OP=cRkKjentvpU?8YiCWM4J@dClKr!!Y(GNvV_RG>}x}QN=qdU zwzwX#J0bpwu2nw6OFm&Q#xttPu8mEc7AfCNpENca$;1I#4u(g?@5uErGa#5%*(# ze6B)zo8*{&##Kl6SpR@lPIWaaWG0g9FL-!SYK%g)1Nggu7NOb!fN-#-%D{V*o+ZGg z_(YGU7kO%~O??M7eQsRZi8ZpdwY9UeGspW}w6lM{9=Of^l31OD`*`TjK_e?xRXI61 z%@-$UJu@8r*G7m)ed00o@zG`GB`ZtrkLtojUlywZg4q*B>5dhS=W)iO&_xanZ=9zW z$(A_|`1xb%B^ai2{UNErEdzt%QrJ_NK0w&4qL2WDUukmv#M8pt`t@%=DN0 z6AnTvj}?yCSCr--QGVPBHRQ=^m(W?OCj9lIfs-4wZ)T?}|Lg@2{l8j%=x@h%Nhii! zID%C_MJi8hfbJ~8#gLs_qOrudrv_1;%>^!CIKwI5Ib+1+M)M(K`$HlGXmo0NZq#lm zK9AL9mf4^iUl8T_TVwY=bbnPz8!Q@*88dQ60)oO+0B0^?CIm-5MT)oA+;#088hUl{ zdEzpDsU=n7GiZeesLDM%d%NU1$WT>qvk#J1RyB3_4kYp9a`f}wiHmQ4x&@OmMV4D zW4XoB%0}_=1uzR+UO5>3LQpf=g*@v!LP$A%H>{wrusp!sF;T^FP~)Rv1Tk4(dvaC} z++Rqmoy@)zeFbI@JD}n)MtdSb_XbD&qNMZ7LTv4ky^1kr#>U16lOievxVMC59FGu(H?`$u1(SU4! zug6)gsGaI99K>sF7UOI1s;cB%SrsHF#6p%~=+y~-BC1Xh&Zr_!lm`2n6#M$3g4O4Y z0Z?+tX}rGc2`Fbl&bo|(p_;*P_g!QwXXnhdH51jb7EF>@5YadEH(!I@L(J8o3t89d zKV1^{@W-Hw9m;$yxV~ae#0$`Z7yiobQU&3-uG*329C5qaTcfkn1Z8@-UQ)HXhZp0E zwe^jjk+l8A_WC{@@1A9?Hib7U7*REgT6QQU`I)OeK}0L5VatRG2xP@gE5`#xhsLs0 zbkH5XMaH*UqC0yTuZDXiDuMa-@*Y|KX`|ZP0!i^#uBvL+96|=U0!!(+Vk+ie0j_)s zG~(<1BA6FrNqPGPO^xEVc2#2C;=y66c>j8UF;>E7tP5Wxt(*@ot^1K zUR9Aw@ELy!eRWkeJRv0Qo=*()tM3U_C1WcqcTk)}zBrc>L@^9y)(+RYjzAEc0rA+b z?P5}Wy}hYhWy9nRWf1o$p)C(iDD&^gL~OLaN=TxW;$+hJ+)-Vl0O2Ny{ruaT-b)z( zyQ5?SV8el|&X|}BM5PHhd2ldsf4hyF$-TN_g53V;^n^!ADh%z8u7{gf=}pd>plsa5z<6I>lJf2TfXohm>I`f|rdIS%&r z_I|a#cFa1@P=MH*2Lo&#jAf?0l#5g}N(6Xb@a=U1 zm3`YC=`%C8&Ft-w4|PQ={>Z;K2Pbcos3M(wUHGgq`OIW|(Kt(k zXpoeUPbVhprD;CTupM_sHj8_Y|y4^!2_L) z@nU3$rdwdOjk$UG@CT~E+iP4i5_a3)-wcUdzeEE>cSbPi=-9;Ss!3>|Q4UhTHhs`TCj|8aDq@Ez<7!pIc~N;bDN(X@C7}Vc|qU=mZ4?{jJL3|AbNeO+!q~ z-c&4A-Uyrmklk*!Ef{59wR4K3fX=C&o6otbEIVy8nYKXx%aOp!CO7Mw0cwPuMZjo z=3|g?J&QeYfyHh}DL5iIINVx==Te6a#qKNr{T<$GzRk&TOaJ9g>l_V<31KX2Jwl@! z9#LR41bbdp>^=3k^!%Tn416M8xqDO?$HFM@tpCa%E_dAwV&&K00|Ar&+(vVlA1EK2 z+Kv*RZacS0Y+3%V-}?SF*fhg8TM!0EzGzvTueo;Ir&my)!j%?#P5fJer22?!v+2+C z$95KU<}Yt}w2Aq*F?Rrgj8Ub(m#O+7>;eIE;P-H~%@4F1CLWT+-g8U9=J4plcmBE1 z@1=^c*OoCSeyV5cH>7aI#csa-TPNlG^&y#$??*rBe=p{f-}~X8$|~n0w*6Cv<>J2- zP>vt^#|0KU-}BSIRzFhguT>BeyQcp4ZvxDaZ;t%64q%=B|F1dt%m04l|LF^RUEizM zehdB|@-SljH-8@i(Zz3r!6YSurT*toh#uB1{!>QipXdC)*Bq?C|LNEKzvuTKp6S0k z_J7}woqjY9IN0gB)a2}?muzUY~HykNehjm}ZGNn98dghZ040K>uxF^~TphtnUO8nN^iyr;J}_{_ z#UDJm^Bq>$m`U3a!5xsf+W0%lJw#BH%-`whns5|TwS~j zDV5^``n#U~>I3x5{&RazwfOGPv&#Sp3*W#^wBH47N%EjDHP(^Jng0b>kf$9;rRG7` z5#-78@?KD}KVo?J?mbh;bdG%MgRRU=Rn#JTbv>fyW@)O?OVA(U8qlT9x`GB%Zc{{{ ztKtyx$pm0}R{-hrAEme1=b0pX3GL$mz5=C8!CBe$6;F__o@r5@2o7VdL%XoxMXziu zrxI*L-8egg^ z5RlP{cF$4*He61)Dl0^zxD+LeV`g1h{t)?Cy|tm6>LC~ao**8L|3~Nwe6Rb;);dGt zVwxrO^{0djJm~CpC@B*Pbzb>qQ(hT7{0k608Vh7Ck0DunBb`E9*|6^K9*rG+))>Cd zjF-2q_OXp!2n+!kT`e_Ls!m3BYU3|awtE)l-yN#H3;Rg=Bk%QGS!Fw~c=tTcB;Q+0 zvDIeQxT57jgxSOQVweGE?_MG(KD`>riFP~~@l-W(y zOpRkEBWGmS-mfF|w8B5Gln^CASu|`C=FEf-=Vc-^K|z?kg&fiHVJmeie&VHL$gnnO zjxu$0L^*-A2pNYv=S~REgRG=FOmQu%y!0RQZ#Z3n|&Ky zvi<}B&t9P030Loq1K^sDlFgtyrG6CvD~`4F5+uM>da;@@hcr%t&hRUsvG4s0S*Sk1 z2v6k1MXp8GZ!9U*@?-v}a7~oLrQuWkhePJipiG_Z@TZ8Y3?a7=GVa@|ecjeJ1uFbT zL(4x;I@QS%&?H1HmtkMby4sIK9ZvrG$uOs(M5V$5lGKi^*CFYzhY?5AO?b!V{LeF|VvnPNq7vqAIXn~TrcDJ0Yh&>C`ZDv_h^))F^{KzdG(1K2my47EmD zOXKa0SnCN}+AC{<4x{TrTPnSH%Q_P=x~3fyf(86=NG4&;0ccEE2n4bbFF_-I;3LK; zz^_-kaDtOjvf;Ng*`UdT>M6YQVG9{f?2y`qYphYAEwF99s!574y zMRFq_^@Hb5K{r0&)Ig`4cvlDu#iOl5R~EO7>R+R;4H=S1p8EVelnT_dYb?>Ia1f|0w&rFhZ$XG(NOvG6@ z9BzaL%NZubPjqZ+mc?IO*Af-1o5)}Sp;(qb%S7Bhhs3i_2SWOZpKizw5xfXB&U!o~O1x_|%Ap-?gDH`yE*SJfFUBllIK( zO=$U$i9ly6iG_;LUwssKVX`cY;Cus*FUxti(?PU~;na9hfYDtW>%1P& zAV`{+hdX*^AnZWlAYNY0Q$L-!hGBU$&i%wX1UjfZ!vzx1)|sB{KoAa1hX~ot9l+mO zJCa$#J1#2qGrck-aUjvD&HkG@<;*O_)hokBKR87(EweX2cTT^Zl`)?|;~Lq4e5s=YR8Qro0cIe@uFI z82|A2GjXIg_u=%7pYWGiKlUQU(|?y&JCefX@7{J3ozm)NuDMZCbpz(XtpKqIRs2CC z#9z!6q#WvJckkXEI=fmn7dPO9wgjq0i>k!jSI16VS=HU=u($daX0XUn5jQG5f4%V4 zPqKM|UHTQ@Y8!6>k-~z|sUVxt>S6^WC7|#ErvEK`W8>GkFXgol@8Z)Q7+PJsc4z1= zp75J%#-Th{yoVB)HvuFU-aUGbKtd2w)OD+DX8FS=1tXOtV_wX?*XAodemMGU8L7~! zFmY}yKN}stH(@jzpKFS_KFJH8Rgip#q%4mR*u;3x~Cj80^SH7Mv}SWJ7PH+GXJg-zreshjSp&y`*m*U zj4=?-;Ekc8R0vZS_{(h7HP^PvPr>UP5qsk@aUvUtx5~xF=%3m+mB1R~&%yjpD&}Fz zbt)}L8QKF4&4b1ER(b*!UepUCV$vfwk$!K@*AxcC)JMXt*L3Z92V66TM`hc>NIim~fhqEUN^{Z}0 zeXaH#?LQ|rUTp$%3}FB!uHIgAKnrjj%L3G_#;}4{Ky;+n**MV}Nu1~I;p-#HRn7?= zQaU}{-K(%{u^_|>_g9Dpb(=fU4s5jcxNW@T3%YO4f+-3RB@YFkn8`lh;xRWxJN(SxBEGCRY4@H zv477Vb6*qi!h~1zSj~;F_gT6IhQHfThBXuvYQKT zIN<9;gU5_Vs*oY)`96)B#kP2rWVz6)P>$U=f{cnH9FX3dUE)}w;WMLf#n@zEl_oS^ z+_o4x^Hnj7Ykc+UwJQPWj~`ENO^c657oMAZ-Cf%Ktl2b*H=b;Kg$tW(>eQ#b%;}E% zNqlAk`)&29;<9CXVhTFPwt_kXDskLGGuOQbWULp5zTUZ43<(OMgGEzK_yE{0Ig5&W zms|5P3yp&nIdkwgp*$TUWxE_>^ierEPWL0ZDQ*o1yn~I7uv_CL;BQM~ju#NDF@wcQ zGB!M!h_zK4Ko-0iC@3E4CT7+J&lcHM(7ZWw8bICz=mk4sQS01Q4&~vn(v3N9lNtvK zQD!{j(bwDYt(NX4|F0df^}E}zU!G7_9uU_WFY#=SvBvo|pFDQ#lGa2J!LMO@m+$Z# zYU^Or``G{>jvsa-aC=o}T8(MbI3h08|X0HW4@K$}4ndsZzq?`FT@x z4xUx-$fl`UhKK)J1=+Ie+{H~>)65IC!H>%GY$m+^3aAzV+ukd{2aaf(jT-P7M(3Td z$n*M0bM~TjIdXdGx8L*sN;yRPBgTmuDI<2?eOHu$FogW0@_~kh)<1gzZY<=<1GTT~ zlg^>0pzBOr&z7<8z=63{$z~fpC~D#LN&2O+ki1Kjh>(jb!$1&-l(BJz2$KG}@R>tT zo;|yUsV+uI9-R@a!SelN!(&0)sEW}uz+`aHU<(q`?5CO^;5ZOejN%YkL4euk_P5Wn zme;On-x;q*sENp)*B1D=fkN^&C(rF3*zE6k|0%=#&!#@7Q?^OH`?B!6t+;^0vn{qo*FR z1CZ>T9H0Q9vATta?71*%1kk1H1-+Eki}hwKr@F6eFnJ^1n1+J^w#GUbIl0|aRvZ{c z&KgEI5J-x7DbHz{&E-K?YDU&n8#e5)4P1|FB?KT(8XX!?B%g0vp#G}7?9%VJ6Kqv{77O(l2A%EW!CH?xrr4FE@#zZp7X|-$xBa_se znS{m3S7`Lu`}oaW(*gN_N*ldTQr3QUC9cn`@Sa|1hZ>+Ea!UB>8Byby@yP6I?)^~9 z#_3?uG2BnoUs>1-CW6gHs=*fHclhvM@z$sS?eVRdD>cqkg9c>H_yfLRMG3HL-o1Ml z_?h_OdH(2N@7s1I87<$D%HBm_gHAkweY=K0jH>&NTqOuUDAs|#`=H@&Sf#shhP+5` z2lx+IoYRBbMX)zI;;8@C{CwYiUXB)2SFrO}MTKMP3vLgdKKaqc#|N>1aQ@J6rCH5h zM$QNbMc9?zd`WgXVElwUxw?64NT|u>Cpd zkiw&bn8dFBGRF#7fA$%`*(2mXd#-ygjF&-xYsSlq6DO@v!g0wp0baFV0UuAc%j>gU zyeAa!6Jli7WL$5(d8CyiIa=*Yj+j7UzafB!jee5^Jw#7kut80z{>zAR9*L>n6cg_J z6`(F&8^)xAXMcd)TlfX25E=E>aimU4Yhl`iH-O=J_M-%Eq|g^*?G)=jTAh81Hv{bG z{sRYq3|6MTV!m-w*Ie)03UbE4aLwHcN;nN=G_AX10?A5u!rIvE8ZXn5|BO7pkqI`k zO4O$#Msp)?D=ruq6c#gH_7#6y1Vf&%bMBo!BbBRP;K(4bKJ#UcCuRIQ+Y5ujz%Pj+ z82X1U&y*-dI_8S**2Q+#Ix>6qywN9^nP*ZI^N~DP0{7&}Y6L{4!|6l+4f1cUgnw;9 zu{mJ&-v6@e_t$wQm?K8sJx3i?8y0*nCJ+6%W#>$uz290$BijhNVVUpIM}6c%?lYCT z;fw9tBEEdkx_a0rRXea+{)E1YDQ-N#AOSu+oUXo7X`zt7(u2Ub4p>wnyQewRtFg@r zo#Se!P8RXdU&|Z;PAsHxa$Y4JR;dpnhR$<$h=6{$z0+y}s}{M_noJC-q13@KiKlGs zoce!-=O4zy>CyX15rS3R6^&rzQ;)8b5wRk|w6Kx;Cy|Yx zf1^|ItZpI}e?{29JX!0=yrKcj9*?5tkZlNgbC>ku`g*W$0ZPKaRMv8_rr^$<)5y&h zu8w=7IpKen^ZwWk?_`^dOuvuoo9*Qb5x<$~apOXH`qvL0R1O*|bJPf-Y(&ZwU-0hR zC(=d)PW0gZ{1{tX5a8H0!umWrdY5M_PNoCJP14*;i-vX<(2&Rj6i_JMxWq_VU>>03 zv%`7kSCI4#JTo|QdlP+!A)hfTfD`2?!(NeXq1t5;8 z3ue5j=sN7Md!NAXn>jbZNNQ)H@Fmbb>8wYn|@umX7%2?)B}E4uUmQ|WGW&fE4xueSbKD=rdT%mFpr1|+72R8!XGG!eE=h!iJSg0Nhbuu{~p%0(7C)Y>&_Vt`l*$j~(I z+=RM^Fy)Pa1L*T7nw9i9728$yD@ZzDn}t7L{oG+J)?#4QlhzY~KiZ64>@?){fv&Tv8jFEbsN1FJUEh58~T>ejG`< z^EfgxG#y5%-&{v)*lGz{6Q>t5BQ~r7hu6s=oS8w_IqZl&*%A4+r2bogQSV3rBgg_wH5Eka{wJRZ?OZ5K0b)Iz`+l zwvG<{=8_)tZBZWvl*IGChENI#NOocI!PBoZnM9V#fpMAGzXxDXcmn!od{DaGevUdQ?fmq^=B&#tw8VH8z~??*GvC!V{cE-^|iab zWONRkPKM2t(hC@HpwBh;aYq}msldMQLp|@Gi!1h`anHenoI8K4uAWS+z@hR60Moo? zuFEHEEyvb>EBB5~diCmdbBrUPwkdH!aZCto=GK!+ftL2d^{Er(qsIN^R>puJ4hRi< zRDiR?WhnJ4W}0?`e)Sw+at9bSCFmUIi1eapjHb=7f9D81RmV3HjoveGO4;hb^Xbw_ zz(x^DXYkaStBV0T=>uIL+I=fJE`nZj)N+sKz@V~I{cnmq##=zj1M;922KV;Y%;yEg z64mv-qZ+PFry3g@0jbX=7Z;baDOaNgz;&Px`ggxN(YGIH@_BUj?%NY^`Z5EY#Smr~ z1j0q2v=*auP_j|L*{!W6O;q~YXY7gJ1Aq^QS%~jQJ)q#<$iEaFZl{SEhY{g)&D(R8 zDG}JGl~-W5=!3qYJVNa9@(|6kJR+hB1%x)j1=Cu9pc2qAD}iyawqc>ea?+*PgN3c} zd4mWnwG-pjoxiDo5Cdj#=legil!OgiTupwDjP#zFPP&X<+Od6kz-GeQ9tl$4x&Hj@ zoH?@hX%^&PSIO07>us@UJC93|sp=*#huwNJAjqolPmy+v+`|VCW>IHnuUvkS)IMuY zsb;jbj~@8^H1^byBS*YPF+oqlFWbr^wE!>ri1bB6{A&fnA3UQsbs># z{IZg#O`-n|GJcT#yE)^}9}wx{H}fy+E&7Z1&j0L+_|HH6dCULDi+>e6^yeG>&*OcU ze|1~@@$Ai?|ND0TXde2{N=5#6?f%)8^WTjkT7mx_vj6)|#SN)YU@?30?QyPfAF>Ki zU5aaG^l%xY>d3CJVn=*Qmf^o5CNcF}xj8w zsu4&%TszJTO*?RICTA~l%@O%=5wh)kw(0WhxN);|5~IokxNE!8FQm}aAOC)P&{$*w z{Kn3yp_1W4n#<ty zd-m?-mzbDxi3i?2os4N8d@PwptMZ_AjsM%{+`rT|*1bt=L0MDNI#z#4Hafy9EtEU| zHDsmsWPZL0oNYp;f0CeB$EGoVP200jTr(u>MqL;8{1mUEY+pKtHi&|v+*`}MH{g4!10$m98je<3nr?_K}^`biaR^b!fms{|Ki&?q9l zHOE+`;k?9)dYtLnCgCy@FAKCozgGB>OB~mh3jOIWNB_ArC!BK{(^NahV*Sr-WvZS9&n6#Fx+)IY8FS@`N9?ywnZ~Hg zoktx;+$wJSmR*#wmj8PVzx;Sf?9lDsOMVW`cZLJ?zaA<7`fTTLz9T5aCB$H^9!tLy zRsulwcje_eEA!-93qv@g*Y^1V)%?W{pj_blrr6KDS$DtJi3vvS>yDxeQAWW@Mk<>3 zFG0$Zb1pwOEIR@|ye?t)w7Ge*y)n<$YTixjAFluX%csKKTMwm(ej+`%h*Y;<$;NA_ z6sEDbE?ck@YK(54?0Jz=0p~~K=!HTQoy;+Vz|TihH;a9_op6Y0ozp0n2Zly8pI0X) zTTRJtyjYuFx_l9=b7FE;kH^bCU-Qa=?EX6;2C;v&0)wNK{zr+bnb(m3U@PN3Kh+*E ztNZ%no1?_gItLX9dwxeMG$M;MBBMRdpW6m(*(sHruslbj5g)a7s=CI)Z}%ucKkiF=iX4-vFW4)p&Aw;|Ig6!{ma%7a8l2WooQC**{UI#8}~z1 z9qXcnfDuN@AON7iGu=V$ zC!iGx0%uTvV8!r55o#l1D9k{rYv5p@Fp5JY53-f}MSR`z2)+ z3me~jd~<96z9|?;Lc}3<=Wi9s7@221T2MQbc{I;xTbB8&#=h?Czjvb8HN`&`F37Ty zlz@eB{c#sBO z<@Z(jjjz4#{A!#u-!(UiOB$(y6$bm-E9s2zSdZkv*s#}hfree_otNQ!csX!KHqG&x zUA81$-r*|ml--r#jhc8T1Sk46hi!A~*2B>BpohR7W)ynlof)~6pc|w zHJ#$G^r_lacsO=@T;jt{(FZV7Q}XF>>Q5X-pD`dzE(&OPh*}5qmq&q3C)?HB>2YZW z07_v+*ST@JMxu&+7E6);bAK&=_iBjP2vc4xx+!mp1xMUW{|7{WC+CcUg2vZ~#s`R#VehUV1)&b#{Inxux2P)@lD*LVv?DkhZznU_A@KB4vIx z0z0gR$UFnW_XK$WijIf2i?5NjJ%M=V3S1oMRmeuHT?8n3BG53hpr=GnaYz07|L|wj zw~N@>NwZ`9eZOh+4GqGqK7RZd)Hz@?^N>xt4)FU4ny7LYt1obCOr7srmyed=JRt{$ zZ!AhefH+@Os@6|@OyU)EB6E`c`|UKHMr{QnBJ)Ssm%kem#l(Kp-y?Ni&ceZsZ5t1! z=I)5;sQTNVr>!c<%l`%tmIRf$1qa*Ws&%>C(YuZ!-~iNk)gkN-_DF>m^s>d3g9n>$ zPw_+YEt+F~r5^-nYB+ywCMVvsTQM^{S8F2FOeLvs5EZRn*aNcbKjbSPOZJBor~vmj zHPr(3&u2~1$TRQ2HUUCQ5G}u}(d4LMpTRGErdEEjGIi@2yT22rJX~{q#Yr8h%+EN) zPPW5w1uhu{L9FmJ3WC0WJh&NRM+|=i#)ckh{#E-Iow%68qAJe&uK@j6w(TRV-{qV( z>IAB(v(rf#1MW#qH|TdLSpcYei=%{rwm-(BUaWijRf1l)!?jI~QQO7EdrGsCfZV9z zz5~{HM@jo?^{%2zfj0?Gp+I*k9cHn6cB)&X_RcrHx8!#Vj6i< ztu539$Y&UaaQ`rD% z`7Yf*$2MBT(LCvV7>u;-g7-pi&M<&wTgpM+kQ&VxNr!OkBhB6~`O= zfEEL}gv3-kx`^2{h@oNgt_Vaee|2rQZ$H#xqyekqpIC0|06AJ`cE)TC#yRvb=OQW~w(k;R^B`}{xGImher8Yjl^=6T;|J!{?fz3wHcrKRk`*?2j(haoLiJRK@$ zEmJ{dlumWtxCNB^(IDhL3Z z!WF4yuG!~u_v{$&?WvsV!MfFy7Qb{-N`#c^3WHi+6ptc15mwp@9Q7P;9sRR26$Zn> zXrG01j3+&(8Zi;io^_a8B855w&YwRY+rg zK7Cd)^2*VZulnGe(lODQB^=W3{AX=@HJUChXarA3y6N_DSSk}p_bJGF;WR8z8SQXJ zpcLtnoR!95U5uo>yb%`_=E9K-s(%5TFP=sLC;Hsqp8)j&&;YbP`bAzfbUF|zH4D1{ zyoH@d2CVSRac8CI(TQfankE(m_A=LwgM8Gd+(a$Mz@+oN#}TWdwcuU6zJ!&jB%dEX zFv<>W58cQGFYyDc4xaiv*nf`B{w6LiE|=^UMb7XD9vN*T1TUa&tB0SL9T_0{Q=2i4hOf2k zj#`hX#^7X4?qu0GhwRH_1)^TP3ZdI3VU+!^L_Z1^D%u)g9vVh#^mMGoT0W^o3KhK6 z)-|h16=1Z!+=be(eGA?>i+kDXn=g|jaurv-qwuxZ!kJ%S2;*# zI3mJrHOspSin>7oAsl-8oc8)`Qk|C46%XqBg`T;m(sEeuvrMq`gFK0Q-g5Oq78VvB z1~+bGy!Y(Ondoqy=rAI7DmduoyLN~p*$dj)MiJ+~>Li*+hb*JtSrj4Z{=oya1YiAU zqe|5}Cznc**|R5S9*~IbxoN=}c6dKaHKG0ci-BM2inFvGwX8hzf<@EIjTDp?TAv;w zVHm86a-Xr{D`+4Z?i;C_SM;uWL0Pgi8F*))|0$$Zh=#TFmS>3y?HJ8Pd*@|y=L#yT z5$iRc;|-c~3^Mm!4h{k0*@{>5}eUi=Mc=W)`dP?+MPs`m4NEzTji3xcl`f!!DWsg8(JmTzdoD^t?TUO6L6H zHo-<{cjajfy*|hi3S00e^f16~XORN}2}C*QaM(&e=Wz4&v91D(-(woTf1mDAfP09u zv=7t+TF1M*-cBx_OG-gZ?`j&VmppZ<4W*4G7Z?hbDtfJbx*qjZ;AA_d`P&&Cz#f(g zsTH`ZDg!H_+>H!YPM}e$nT}3vW?694#X2lbL#5z z(~^F6X?X3tP23rS|4qI+~TOH%JyK z^Yk5JZm2bRbdPh3P1dj}X`JIfFfXvVI1(lySy%*u#)<-GI#(yqkwb#L5wQGMcbY=!fX+ zCXne3E+%gWFW-v;2FY1ajo{EAUV-VQTI|EF79tIdpMiuc?RjkughzT4lTcc zti?QSu}qv!w(rPko)!qv>Kv7|VSUNVWq|>R77Fj%AkNeG;dvRdRNa3T>-pW3gyGc| zrUCzG8ot2row+o|Huq9wMB7DYO(Yfa-LplkI0ZZ|CMh-DjDB-oENMN>dq%1vU^zTp z99LRk_fF5w)NW_#mF$n}?q&}){bEvIAI(^|v&i~0D*yAYWmsI#c(ag}@6BCLVZv%K zQR<=!s~5Krx;UR*(z_8t_>axtwm7AU3F3I- zomWPwPn3EzE2UsQ(*-a4HD*#6CRD{c8nR)5hY!q%TXKjQ>L7c0I%@^;k8yuur~qeJ zVN)90NGFke8N@v+g;upe*&%5f**d|XwRrPnU_A>?$R;)m^%lwDiuA~NAzhPI*6-b` z04kcb2J+6@qwf5^L#_lTw98njQ*$Dko2n86ue9Tt!C9pwfnS#UdR>lR(1_0Z_3pEO zrJN69%Ruq+_Ozf0s5CI=Qq4m>eOzx-PYcG6b{00H*+%zxw>lw4m@3CFO8wq4hI12r z5h!Tfthzl_xAVHsRNd`=Mf*PzxQHM*Nh|o3Q1B$8%65&RI35L}Bam7+87$3f0=m^% zV40*`R3ik_9<0m8abXyKerlN$1A|j4vI!)v-Qc(@rzshjM02Jr->NR8N2V_Uat} zs@2fk?aIgdB=Pxr%VJJNO)97xsh6T-smJezR^+`J8@v#cPm8|#OjppPH!kGsqrd{K zk1mzlHymM|zx%-vR%IWWRLIF@E8Y2`2`U&~>w)3nTbkUV<4M!mAjZO+Jw_LoG^i08 zc;|=(CD0Z@V>-oay1LZG*0%7|=g;M#>mg|bm_K=RPdG&|Tiw?MCSO{+^{FvG(yN^y zT@@{NHBD0jPn}(Pi|b8oBmD$8YIJQeSqVo(t0* zmuLBu5@*63n=JAu2#Mc(nLYlRxs_T7vd!gykx&lF!2ill)Q_K{+4a?|WwA?f zNZ}dNtXezc`$eyKp9gOEre~W$Idz?EBlWeTt9S+rQ5oHJ_%o#-l=%>J*#XH-bf@Jq63O0ZL zDj*r^NB6*tOm^Y3r_Xofg__UK?b@t{OTV5*N}PT>9flFP3!B}+;Z1PYF#m7wk#k#u zqRyOv4ZJ=1pu6l|EnL5cg>eERC8_$r;hq;Uanf=M6Zp7{xDbzi|M^vghyUjn*@yRM zwlO|4jXgah|G~BH9}3b$Uk_DnJSxB%{vnXUFEpyDYEDh@BVkxvT$`K6+vtiHW|Gs)o8seb=Yyiq%lZ((%t zTbz9O=lx}eulmEcDY6()@7%V+Nmpj>AGwpHE`|_7NE|g?yGz$9Rm-Z3?f9NAc|>3# zZ42Jrsc+F?TI=Y|i~kBXZjqoWI_Wd|y(3w54rHzO4OO$FR#^1Rvd&KHMZwdcs z_Qcg${J_|2eQ{j9!_cqJQs+qh({ap`9;X{_NokWrV_+(|zBnSh#z{G;iJh1Q8tVnk z`!%H7d5D;h_K|oy87z5GI2iJ7!GotkZCHk;TX=N8Gq-u~*|x$;g1xB*LNA12ZWHrx zYBw~;MQ z_vGT-Zp%;ktZQ#R8Lpo6N>%~gO#a--gmGsnp$k?Q<_G+>&8)2UMO#L+u5s9CismByWw;J)X~^e(zcyrk zQ?+zIyMV9RTt28dRa1TtsguaZGf3PTaIm@Wrg_w;A%{78yqS8_T>rJMYAEyDD$SOu z-SCqz=YA(Zim3G~cSm!t?=)_)Ds+1~|FEFY4_P0Cp5L|3q6AaTcDofc<0IJoTb11k zH&01YV~u2gO{^fli9Hb&X&}vH4LrO-ZXru|f&hYx;3uQw6zc4VC7jow`njIV#p{@G&yFdtq-|)bX#(?RnN-^mFwR&Uv6?Q zSSS<7pCByU&9bSD=*%K#Jl)M2kG0qX%OQpPR)`OM+#1VUhu?Un9(Db(N#M*C{nfSY zmvnT-BdJ~sR@7`gXM+Saf>4gzwSf>7h)qifj?qMJ$M&=Wu0-;WUz)xark@`n-#|A3 zSHI*&uD@KmnI}kEM#*)Uv>y|6=Ru+l-hA}5Yv4=7l$4*J*5I&;!zzNyXA!u% zeE0??M2DMM>N@Q6r_&;04P(Ew*$gvs=`4V$U-*;y%kv zpUr%bcgvrem_qD-x?5H|_sfm;$eTm|u8+V;<6h=BZt0l6#mE>KJ0!i@cj7=S9fV~% z%-Z!ES25)Tf2CMQL;1eEW8JWYo`c%utXFjS}Uq9am;%X3A4%60zZ~ zbq$P_ z1YTBFo)Hrhzsqq{rc5>Yk@|z;;#6z8fG{|5{CKakd~U}CtMcp3{5E%;n-%dcv%~q- z6R&<^K+iRMR|82m5Rbt?KHA|M4re;L;pcSil$fNb=rn2T!)q5fuRpRWyKjdVnSuFY=z+w_3b0a}?NfbWyzVuqC3;C_FOzMer5tdO$zuP80lAAc6?eMXTG zX%+xpi*KpWwXfRoEWxJivZYJjq9D4d@hB8_n^^U2^?*>~h_6-(ORS3o0@nLN)(_kV zp)D_wV?q2Eb$olC%fTI|zV}+R>cCT;lvnFOLPkmiZh=jPFc^W1s{i^`3gW&!;kjr1 zo&kKK&Lv&)XVKcju3gt>%aLLeOQ}_xSS^gU&(6&N!wt{e-As$hjC$EcpXI}vS&U9w zQ2^M+V7?DQW7%+Brth1fiOg*W@V-P+Ir8Lg0l&8_ zYiwZwMVByA&rlSJ{J==Nb+7hXR+Y~f3e~UN%@BJU@dRd@CUB8V7K7TvOnGy}^1kE@o43-^I@IDDW!!hz&&*L zOM>7xSzb3Zb3#Gn!V%?_essB1FnfO1DkjZ-u(a-SsX+7d!=>o4X&XWp5F+C*b#-X( zsZ0xK#kLRj!n4Gcy!yX8_mxeQuF|$&uCu}YskNSFzsUswjAm=VSZWcZfK+FO%77t^ zZrzjLCd119al^zUFLvJ*9)Sp>n|U@S%W4}$9D|oH0`q;*0%wKgJ)$h7pzs-G2etA1 zf{i>Ktkk(3Wm2#m%C$5#4gC88!ta1^UiHGcbN7J$qf=_A67oMpTIoHvUkNKJCi+EMtA8=Fef_}U!#^Qbxp8lz60Hg@8m#M5k)PAt7 zK>#7*D%$*kjou?+TFi!a)QQBv;bc-!94^yd$bZ7e=`X`njiHiF<0j7#4d*N#uJjxj zcDO2X%;oOmefIdjgD2nh@K)bE72&7)-w!6w?p>g|2@jYnDq5)zU80861B6JoUt1Pt z^)WRvgW1}a&|z1MBox=xi6`i5?d09nwbbdIs4j5axPlojYkhEnrMf`|@P%!h@7O zf@|*vJTrl`Y?qc_*O9NCf4U&NR6{yhBntC7T{+ckR=I>wT(|p_Pa)Zjobg%LtIAqM zv*w(AC=DeSA^3Ru?MaUi*2pHoxna`*lN&dT(;Sq&LiE;`hnS1%l;MVno>dytcYN9y zCcN9m>&am=YK=E-+VnIcqHWSjAvPhvi+FCz(95g4_|>a}@pYdR3T>;tk3`mO@diu1 z_|2`+iwuhJ`dI18kTBBZ^5t(~QD|{>3CrcJTRjs{XB4e?t6K*%%Le_#w)U8{-xB>T z!kB;3!SYRLJ^zrL;_F)_?<6*ysukkgCGVd3SdW>18Nu+fa%y%S2((pXj(mHJ3=yQfkA zd;6v)>G_znHQV*I&!OJSn2sZW=z_P4N15?XF!! zPWI~Rs?H>uN~N;A2G(cNt3%f6qe2SSIelIuOy&Ud2lso3i|@&~7!H8QVmG2E0hA|< zr9qLR;Oh~3Lt8d)R`hUN&$wXugW*S*{v|CJwU}c~Xgj-rRo7sbreSMqZ(Ky~su1#` z;sUj)xHK)c4~H|riPIPFNFQ4!`w+?b9af2Fj(N2DJpT>9Tg_r3v-3JO`hMX~73zV0 z|Cr$G*Sf{{R1$qlK8oVU*EVEnmFTvEyQj~Nwb-|Yt_>4!m0Rl$vnKuUAuE&XCnQw7 zuGaJO$B&1v&#d1n?e4X;GJ@GsE2OklN_r%R>?5C-Hy)iNO&i!Q9HKE;&Vq|J@b-SP z#?jS2{F1Z6)^_i@ci(7PUJER{OMRG%{+7*8A-2OWMm5p>&2URjgr1 z{j6TJ!L}5y>9!dud%Kd?W*tsuNfCpRI)+Kj>50mj#n<%JkR#YRUafcgLpF$SdVgtX zn{V#T@niIeB$0DM*K$K&bZZ1LM8Vr(zQcK4S2$$8;l*fo1v5QlHH}0QUjOMeLZ{Pn zXw{(Qda$-^ADRaSoaY1we*DNIvf1lK#MQ4P-+IB{J&RkaM09IAOSRcw!=mQ6cf>hO z@X!KhW#x$1M0R|rwv5lv!Y}w@OEadJ*?i~u)<@``=|O8T(fMFOl$RH!*l3X!m1IxJ z){7U6N($OGa0!z|et+1UTIauO!-nJ0*$k~z_Zj_j2~VHN)3*<-OGF4JO)e}qH8#%U z))z!ZMn~|DwNYCA+3oG8In&$M3&Ccz;=3e5Imv(<6vje1zh2;P+7tW?Lf2A=okR|O zoowgd;%)k{h5M-?^Fz+yv8a_dM98=B>I4jPniFcGum7An>f!cod~AKL_XW|6#}{rA z`KD_ZG{t9AkfC}L6Mrgt(4encLQ+P?OiNrC=3`=Wa&!U$wyL35o`c^w=KBrB)~^RQ z#A_a|`=ds0)g1qPXGdu9b_$M6NyFFz5pW^3w4}s1$v)6D6ywvBj+a~p_EreR2G!n^!`h`%I=JucKn{2|KY>g_%Rm`4^uzG zVtB}k?9!%Qer?xqXDKUL&G*j9yUd}y+~5c(pRfHYpsht78A74x^Al5~`8HkkxOisj zRA)=je48Por`+Zl(=Llo`js1uj=I0+fU>eO?b}O-mFY|)-{J97-R|$H?=8F>N)+i7 zVt{dy3NfP`Wr;$J-ptN5deP?MJF)buNesU7F6eM!SKrBYe~tahh~(_txcrwN<>g&t zyYO$iJ_x-o;)bu&J39|R5IvzLWeq3F?A)ShU|>>Wq73*@m?9eZ`0S)CJU!RmK1yZ( z3C$>AnJ?)$c@6Rr$rC3phpdITdAOPfUm&=l=;mg=h=Ap-QgXF2_~2oA11+r-l$zm_ zCo?;D@y4YQi9td7rw{Fp7DeIB4GumN5D*~4hKi1sRs%n#E5)%UbXieo5rbzc_O!B< zw|B|T-Me29(}>noYY6^k#>V--e*KZ6>3_}=hZJPo@lL|WEz&jp2eQ*N*+W|OHFGU& z%mOjw)Oo&*JLkinfF*+Il5WrZ^fnI5J{U(jwa#X1(?$k3Hy%iBP)YHGN!(^Iv$>I6 zL`K2KnCJj2pqu{3#>Kk}5?}cL=`+%=-5^S$+)Pp+n>C zrvewI1oZMw&(BruA4c8C+$J$tSzvpNY2##!*_>D^!W_vCb!|7iTgtGW3vDA?L$eWM+8z#v|jy z_0~-SVY-)(XdBYIicKzFe0`;<*sZB+(Hx0X?Qy*EVs$l02~874n4nPDckk}#{@?z% zYw(2a5=&Z@)pz^Z!3x^yO1)$f^3}I6G_08(vb0XG5MU8Rg!jDfmL?7VeJBCmT zU4vOd0^zoq65flLk7yCXLPe93^UAy=NaftTP_P5nT;exIfRi$CABFyslr)a=v>srd zfzA6yFCINTxPN4n@A+9rkf9I5lWY{Fc;MZQFE2+Md-i^*!8@T0b?TGl39I!say*PF*57m; z48toP)l$d1_Zx{+SXY-rJ5?S5Uh zD9ZNAyGTLx>>{fFb3cUVjS4Dy>07*Y*|ZQM8stPjKiBJ26=dic7(nGPCyFR9M^_l| zsHv%$fP33ZQFBYm_@}RL+GNL-KlSY0O;^HP3#`<;d-wHFX;=O5zZ{mssJ}tdwk0ms z))UuCmqK7q6+jgRjp|$E#VVL~gqKJp_PthyBkyDqSQ(%FiygQ+-T`C}u1QbHQ%Mz#kP_@0jaUErxH~i65nKsfJ2aso@pt}m5YlpP5--e>{fXXM$_-> ztk}-o)0W;1d?}iN=TH{%9Y0DCZti6+l-#t5-YPCTPDxa1ihX?ux3aR0K@GiJRkfAV zrF2Zzywu6S6t3|6-}gIjmE;u^SX*@mG&V?H4GhFLjV+nqw_`tkd|UQ)XnbA75zy+N zEiWs}gS2q_ULoUvOWBit>@eD)N!4xJwz+(Lr;ji(@F>!vE?y$ylstNFWoPGHj;_`4 zGqJpatF|4xx|Q`nL~fTRPZ*%>NKM0{3NrF$0?gRLrikS-*q$rsTbQfYF%zfdx%V-z;Sh%vFcv zWCyMiI5+2ng3moD9_WgNNBM=RV^zz`34y`2OLq{&O<*dN=cUD|92yhaFI21@cqe9U zEM6YEZ>mWSx=)EjVvfFHyr_~&g_MLu=-GOSzL$AW31zKvnj`BW+#KLPlKH$_VZNc5 zROU4>7TmJ`=+Ufjm16UiFJ(fc?5yX;b(4p|`=Pr3?pz9T(XwuGV&|@1fK_#+_&7<6 zIX)MaJbnDUk>>K_kQppLi1(Fza`L!_t$konCK9t8B~|U)J&}Cu<2lT&4h(2n z{Q@WZ(WFF~#FNL&!M)&}b&=Z22trB8Y9*+d{Yu8{d{8n}ZGMdYF_V<`L5lS8 zjqT z^x2AOA4{D~W4h3_?D_fBkK*|v?(+;U%@M_Eiutu`d9^e1LjqJV;=K(qsjDJwxzzLL z*sCrJP^64JoW(32?mkqAYo`Rt66q=bq+dqKFpP2MQ){rk+J_aCpf z7VzmoC7*?iT4c?%sicLxvAN9bVcmgp%Q#-|ietI87y>G?l{qiXZU;@G6yV=5#+w`@$^%Iu z!QSI}jZNxmZ|mzW(y1hk{OReFmani$9}9!epDz>&^t}(jT2D_w&8m;uo%u=#0F6!+ z^q&!^K!V@ydQ~UK5kxd>RS{8Y63Fw#zSJY4|lZ zH`kFFrI9CTx`KRW6O5}4B&bylB@Pv{1d4qo4Zu4^$d*_)W9{V?VAR~K8Jc!`5zV(D z&U~;N=i$AZ@c8kxXCE)N0JU;dJ8;1l@sF-f!TApPR>=u*wJUl|o2&$dTm)?grU+SX z2CzfmQR2^!Y&Z)hEA6jcEBve*nO{(#lX2r%qMEAR;lm1|=m!UdL+6ii=DY11oGKPy z(=h=L0eEhAO<4($U|)V3nO-pX50eij`UIerV;7Jg(N!4x%h zs|u~IK9-Wg9ym0JD=YK1t{n7SUPk-=Zig8-?l?BA`pF~OH#^HLjlq6Htr@D%J#(H6 zw_2@JFP;Iet|6p~_c`D|M}Ce$U23@)W4KG(@yTk5fSXYJA0(qI`Blx$KT5hu#|8ReBvAlP+x2XJfU%Gg_C z66$-q2mA+`t06&{F!J)G?_aU4`laP4mpWifsV1**u%R@shU&xLd*p2rC56)>?YF>f zXVFG0%N~lVkF@vc>1>FkI(z<>Yo}Dn zs;2Cps=0O-;Z<91heoSVnn)x|Nb)Tpj-D2XzznX9tSrkTZ>v#)6P_L5=a2)38v<+yJw zUoNk3B<@dDPS9_BtyxOdh<~a#-_WAjx5nphH2^c!g-HHu>Oo%uPF7&0V1jnk-RCeT4HROAs zw`Y3Se7t^DN2i|j#i?|HtXqj>rmFj?Cz26GZ!i-}AT73e{hiFS1zV zFGtw5eQsokhzxMyRO_OXo4q6lrN`#Yr!?xNEgQ|u$@^)-X=)amHW7#mW%+_1-IJPg zl98JWT6X*4{I(TSi!2ChbzPp|#p&ki+VP$toL5lYGouE^N6;K4usc#d!gq>Xe9kg| zh_eO4KeC8cPEOwVQZO-YE?q4C;sVpTRjFqB+&a!cv@hhvao_G)+*&P~0U zif+%AgG%vpc$BK3B{Wkv79;Uimht62%JHI%Zl4bmwvZyx04}QMWMw@#N7g!EWmUBj zCA@A+TANhm3=V|m82K}WX9O@EXaa^Bd2gOQl9rvgpOF>cq`+1a}UAhuJgBILSIjCGGi2yNz;n4<07B< z1bW)+R(qG9_4P~Gle-HTXgB+*MfZS{)aV?f>HU!lD!vI zo%+T{-A^YdI=-8m>bQ}S5&0M6>$6L4Ztv!;94znmVYS)JF_xG1(vy%Srnb)m2V$dJ zc|qmU)1lhZLHCOnD>H|JjN3+#iv@k2mKF=mixjl*s@e|V;TDL?`;?)r81dq{m}SJy;@O2vYDan|q^ z9LYS^f3L#UrR~0W;e!W%ReF?Lo6p1AqFhkSQZo!3kg%kjXAop!RoXylstCO=R!D^Gd*D3q`AO0Hgm9@3!VdRXY>i zUj*+*a1A z(7eaOtFBS5Q~HTbfdojGa>2DY!eEM{$2+A}UpQ$ii`1}>Rt)aBC8+IPUH>}Y)N5NN zsdtAy{^p55vX#HRGXRT%70oRRZt<)F+7Lzrp-a>#hJs2I#vM1y5!S8;(2#;~kk2aB zN6?i_0zXmeb)&T2ozv11B(8-nwM_{LzQvPAGARE&4FXp737`jYckaxUb5iyOGS2sy z+n?*J%h8F8ja8enk?F(d`SAvqzb|j-(!Ew#HR80d$?Dqol-2LGkyO8KCC4k6v6MZJ zlGgvclX)R`SmXH*bg>_{ynYVsyX`Ekx>@JQLDEUDQ%A=iV1C|w=|-n*)6k`~l(qQo zk`mahnYzd#Ec=wbU(~d$kgK$dJhCU{+y%r=-rw7R#U?~XL=n61^^vQ$;TLo|nw82H zN|_$YV|_Q&tJp_zu}E|1;egy%QOhHFmBS(T_G*r+deCm^G6?7ybe&+JdGz1+?;j8l zIBq|XO4NDy;K3QyUpd8e@<30;`}g@Zy#W+E|G7?rnk`+a^Pfz)vZbxQKI?Et?wKj0 zD{1MKKMo;GK6cNk1$|kzH@aOsT;` zpaEyPyl@NxEWkUT#Hio_8`$bP`2&1{zkbqlTWRE1SvlRw0=yz4kiTze$IJc#&u{|G z@N)@rYH(~Uw_Z3fuYRbxxu~a_#WBv?=&$i1n2Tm{3xwy_C`Q*$Y~HdZHZ@fo@lH`i zS|#^-tn{p{tZWg?>O^}K?i$kYX7r&WySu9zZMhGnN6)b5rty5!c1J}4F@$r=aJ4P< zDg`z4Xghdz$<%ZFu;#CFQh%aK(*l0r2lrM`Z{Vt9@Dx<#GCPScZC~z+55F#cgWQNG z>>PBFe!$w{k;P-F>nVB;9?H~{mOktRC7;BgAT_NkEy*p$Hi7IDmcvzh1hsOE{HDCA zL8+D8Rg%uOG`F$>!eNKO+S(9(FGoYKjwk|Zs>Wut=4Ewg5mxqiC$b7N5OV($U(R=wCdpZw6WRV)N#Kb*s0+u)*XI8Q3h(J1M4|J|G1K$ibZGf?k#Qu z8t!jv>mH2iMuV#lrH6I#JUppQZurAf^vTMeBbjZ!b7!#VA8ygr6vr3m7z~CS$LCs*LWo4XzUi_y1G5XkGwz>oTEe~UEC z70B`V-q;c&y1caH(m4DiS~N6NJ3A*bPSmdQYL>VjY0r1U;YQ3qj}HOO-UDQMo9j|H zpB<%%k#uv11{&ADhZ{#-O1lAuomi@XdKe@~@4$Lg!&cf-hQYq8g0y7V^~-ZYJNu8e zdU$M%tJUdS6vL1}ajJW=6PkDK*<*6yLS{#a@yq>&QAhRw)D}to`7^^asR+-g?)^e%tn~ELG)` zCtJPobW-RU-(cczzpdas`ZJ1yQ8w5iKML!%_{o#$q_J?wPv7~(Ro$%>b)|Lsrf9@v zBVxoG>u*)h1f@3l@S+Jf9Wuqxzp2Y3YdXh%6x(m}cKRz&2`aXPMY$KEDQYR%BDzME zCR6EmX-I-#7fO`mO-@~XYcNzGRdtLIQk zq%C`MK|tfy*Atg`T8vOXH^QevY6GCJZr;^!?DyR}TC)&;9{A&r!WS>DD=D#r8)2*Y z_b$vG!K9x&apEf0QkJZe8gl-Kibnd6AFq#*R1X}stNtaV9pZLC`X#u;lS4A+K(ldb zc4mUZ)`VK(Xoa*C_EC8;O%BZ}H&UlVp?p>EG(VWhy(g7iuw&C`*7c0x;VdLR4N4WU zsH#k6)uV}{dV08mH}6*JyrFuYX&atbM3~^o1U`ULi&eSO@269;PuXN`Rch^?&vGBl5 zRk02Gd=^t8!>lYHLJm_4h_0DW{xng7=oUOiN~0-VRPNOm&vcOg;jRo1 zrIHld7m`-fb8D&y!IUeMLgc7#Z?9^pqMVpjG@JoNzkWsk#9`Aji@~@QSoA61WI(c% zH7a$aUpc#doz>3`ERRv|sx2-?!c>a9eFp{JeE5VUuz3~j!X zJ5(nwAvsis0+|k9GT~(FyX{|07io;QNRi-3_M)&!L)>YEiN0?v7@AJ{CPU>_72qs} z4tjaL*==m>b=BQ_L%Yo5<>K;6+LrB8#tVvzTf75dNY3emCDO)Od7j$|6?eAqAV6f2 zpSs#3j)+c6OA`vbb-xQ|m)%$(@bJ-t`5l8oXYO9+<>hRD)oW!5Jk;8n7csZDcQd(* z+%Pa;KV%Xe_xcd?Q0PRvC-%y{O>0k|Mn*?X zI9WD;5TLO^cD$mlX0;UV14y~8qRU)XnznQ)Es`lL4WIx^>22;o5a5yfI&js)EJGu*su*~tnl+>Jj zQwh~N=E{8O%X}0m#2~rVm{J$sC3Wf)()ryi6xiuNghwE*bpE|{?!aNdbsL*y!3Fvf zZmtT&*c%dZ@raPE2p|ORCBJTFqJNpaf|)6R-M>*ite_aYa=@5ek+lPfoo!EkE>vQMM3B*Sisxy;iJ0j>U@F8ZV zrqr?Islog}mftnpOAXzmzTzTkKV@<4*Wu;Y`qpK)Q{VaA0b|Opxubn|?fNR#cX1P7 zK%1JAPnjS4_)#rrzS6A#1-K62JvnZ5&y(&g+u;M#uvjR(VGxicP)n!4@sv(nDVF1p z|K`LLQeOYn8qWnr+LJd#+YfUfoN3KX?%f=*Hm)K|4OshQ<2oo{cJzFmiGo zD-QMyunW{CjSU|hA2RYo0fzvP1Slz5d#g&5BW0^hawy1wcDgMP0xP%1gk#CMgWse3$v#y2gmthjw# z4X{2Ln%YqU3Sw?5j=ghx24PCTmZ$E~rHOp<+No}jwlo`R`+vz-KtdLiX<+7S4~Lk}i@ zwO7OO!x1`~Wj{A*F0L{$zIlFQ=<=K#3qXAgxNHHK#hRGbODTa(?pMGOID+cvnO|b@ zq_LDmttpi;Y+%gKv(mo{&jCn>A!>ml@EcuYsCSVgdI0|a&SPMyS3+5o)~whXsaq)|)p?jo)q4e|OE-{5z2^j;wZPxDrx7DJ;3G zNk$MDnumacf*fcwK=2D^>Z_|Uf~uJF+xHwbu?fUKa~9*~3Qrr|WNQ~aBMgKxKYR!~ z_b}tgk)$#oinDdes?%jQpgb)p#503ckEBG;T;}e07m3SW{WmV-ZAhf=LE~Rk1sGaL&qXDM&TXeZ1%PUJ;aXh6V?lUX0m6 zUIM3{+LSiCbj-}$ZfJG}j#_Zqx_bNSi$V*m6t)0jnR=e}>USq-sXuYz%w^E^DJhkM zbG=Zf?bN}80tK&soFI2~VQXq`nz>yo8}}@-D%X$p9dC`tc-2c}n&^fCV*cs#dA>sj zvYeh>Do3LsxjMplHw+~alnD=i+n!Gdz8;9K?CN(>gurbGk&D9%1b87BGxx>g>|;vXs>6cJJo=;)27iEU(JDzmSYPdY26@)hA$7 zwR*0a=By0;EEa2RMT_9tsO8iYiAre{2!f)WfbXuj{1q!tQzow|KZI7EIs$$|}~SBv40>G^$p2+%B>V2zY2i3~Oy~AALzVYSfEuFRDUnj?I4e z@rBKuU2YD@WAD|CXnv(2MBS6e!6GYG%k_Lk{pM8;pD=%>q-i^zlfSZM``#$ufZN9H zf={5MBu*5<&D`S`bjIK1!iTz0^Kbua{uuYm9`b|`@cc0p*+qoVOT?j_J6q_}FNZ{~ z?%(!#atw^L_g)4fgD>iiQU!lk$n|Tuw$D8q)fU$)4};K&k!kl^n9TqP#O6U#KE(5f*f=9P`kheJkqDjU*Qd;*JKal8gz{)-;$W0kyP&`59@>w!g<$)-=Li#E z{Y58Mm!G-0xE$@&m^vLB%ru+|+8gkTeF_*Kg&ZSeU*9bFVQ|e^px=NWmY&yyy@Ub724BpR-E^1#W!&@w*)^ZBb~Yf4Mz{%UInrD4BL)s-_=5=kx8bQPn|A zBg|e!gScEP+H=f?ZP$oZN`M{*F$FAORQ}IA7SAy(5Q8rr)GO8V8F+aJ zhtp=gc8I(;JvT@8C(iMH{aSZM2!%q$d~KHjj14BP9p1f-ThYwSl&rA4kcWL$?EI}D zj~DqZG2Y)o!m#@NUIiyoYc(1>JeH*}>T>eX?v=WZDAa01WNmb3YinL&j~YO=QW_fi zFoL6z+t4<#JZ5hk<2Zaiep=>!-{RD8%i@b%f^cwfFpJ@w+WGpFKU{*Bm!(6@rA>Cl z4`Cm-zs|bIS@^2`{IZIRKT<-2h4H%m$Cq02?y)t2MkY_t9^S~@f0j2kfVp{@&2E6f ze@jmkwybP^NF=5~rx8~i_v)5h=c^*-UWe@e^~m$^Z2KD^{$m!!H8`84D#-D<31oGS z1y_u%XJBmIa|R*UH?6E$gXHo!c6&1VX+1qu)~XlM_{#sq-g`$ioqc_ysACxcv7yq( z0W2VbQ~?1SARxU-7irQ#dI>l-RGJ16=^(vEYN!E4L3$@ZfPj<$0YVQDLf)Nu-rqCx zy!ZZd*Sdecv(~I}oG~OP=X=iHpSD}hZGQ^%%&E95B$U+6Jjelvb^D1R7Z;cA5FSJ> ze6iJu8N3w9=`?MyTqV?UO>qOPc zpiy?^b1)xRM^=6?XYK4!o01`G@pB)NxhWH zx6`5g^mB|D%2p4Ykip z$clw2V7W~^xRkTzThE=mDPIPrJBTl`%t_VWpM!{tKA^MgB5duv!X%ULHdNNVT9tj8 z6?DvNwUk(!W&ZaA#KFO@BI6~1o5ovIi?W4R5t3wCxwz(*dj9!VnfB2Im|L;!@I<$| z*M0f&ppJsa!o9>5xrifIo)M}s0aaZT-$Bs1X*AyF+TH#68cWOpF6^p#jrg#QZ-#V9 zmaL0wFyR0|qKlimU|QLzHk2f~(srsrz(f~LT|Imc>GC2M{tW$zG&VM4WB>x(ekLpj znLb?QU%TAH12VTM-!WS>I0N%@a~;=X@BvDYp~_a0MT|;J&dsp#th%WHI*m4|D!!$y zhlX|Mc@#>Ff>+Ms^H;ErJ4E zoj4GXDp+LGNdW|@y7Ll%7|m8l2)({LJ#~i3#B8ZX!*_LK>|upp^_OhlyE2ZE=@VlX zGXQ3@e4emxswQK*+m%iOEAD{TFP;uXnXV;4+e3d;QM%dud{LH^HQN@KG$Ib#OZ z8ea3gJtguIs)JpaYJe05+zaEua> zI?4R8rfxM*W38+QvNB1gZuIFl_yu5X{1vSHyW3httxjfL$>TvIW_c`?kc$ z+?b+|<3c$0>ZR+bQx`ke1Mc5^_>}jdXJxuxryRMsm@;=k`Xn-nq~+%Jj&JV1flFkU zQP#|iEmy?~KNB=Z%~3td`AT+@^6O8RlBAT8rR0)Mvtx6(kV-WdL!Z&ui3-GeYZUKh zRfCPle=jr4tg_IO_ME+kc5^UnZx`B`jT()hc!=5_bNtU@KCakKR!aH4BhkAqx2h`U zJt)8;i4os_Q_-=p(NA7u4~O(R@mpe<>IQnAA3!^54YnIIT{;kHV>4aFsr2SeMBl=Y znD~Xu+UL)IU6r7;xDbzgH?1=-1dtH~IzE@{Cn72VbcIpF$eqB%#AjRDnoq4nQ9{D2rBwl5IVQM3XyhGBKcUY>!srgLii~^Vy@} z#XJwaGQWH3c6RDBh7Fe?WauMS+!InnVp98(EjixFTWSa&R5J9DbisZ8f(B=kdf^k- zVyBHUL4hN`9qGW9? zl=6DnZOGVI-`F^pN{NJ91EW6^2JTuRDPjG+hGCh1fs22)fd8jnLln=~|ji(QkPmW!~N~a+gYAPQy)!Vh47n%Jx%Ftet!gyyM5Us|^Rf{py+Tb)`O0Gag zx+jZ+va;giCyjTxNh=Q8(f96S)wkB!)XuK=vxTA+Cwv1|OiEN4_ z%x|rXDKsoB*mnpYTfhZpHwkmxhet19$sEytmRY80ZI=iILHwcgP^ zz15xlUqtehel*#`B1T6W&Su2myMBZ&<<+tKu`oAMAHVo=)uXs-cx_GpNy42w4g7Hb zUKSVsZdE-|uEEsk?|M3_(s5(>FS6Z~$IsVZk3?o-P8^z)w*DR`c8*vpzD($HGiMCP zgdDo_sXupn%}(H|qyYvqMzi39RT2;q?EP3!t!F{n3$FeL)H`r2i=XURtgkGfl6V%>v+ggjTn}z!|k74W4n%zUF zhq{UVYS@FZM?d!K zFX2ytP#kq*`^%aP_(fO~gms*q6JdVc$y2}J5D*mUvhV>8tFf_F25|v}wGuWL*Hlq? zgG|@WS}XOz-$FHBQDJ6|VPZDfp-|>llu3rB36uWNc!Ff_u71D7(V_>O(Mr#soj!h? zz5@zC*+;%r81!*LoOw%mhC-i;{TGF#)0n=3O!vW4?&8Wx^BxL|&d$aq4li%tz9JJJ zgKB9JfY0RMWSo2D?Vb4|Ws*-&&Sxl*FM4R^0sRPHBYTXz z!L@7J5S=m*rFkPeD|jOe=a8o~c1H;xsWfMd=yzJ8a<8qpzj!A1LnvfY#&X@?HN6mz zb*Nh%VLWye#vGR3zI~@}q%Z#$Bfo>w0+rEFNj@aZ_+nU`g%XXUewOe~+c5%NV@8r? zT9+12z~1I1rG7mQJt8W2!yP{1I*~fr6xrX#HncO1qCNas*Zj|3^aaJ92-Alc!`!Jy zIZ8idZ?n;*F^N8bDm=lvu~>z_#`@#>1rsdIp$$Zj?P(om;qp3U&=jW9ZRRsp2t_}L zDQ3q424)p#M0<4q9Ed*_N1%Thz>$K|YijmONJ_RiNeYVoX&QQ_L*I7hw2BUtrJ`b4 z70eB6L)hjKv0M}>$oX4-YD-JpFt)XS5qfLLnKzpnZs#U%!<3%9Ke{=6AB~jk;o3= z2fuAmh~+q!p8VJ$Fv}TxQ3_-b^9w1#%@4mkC`j#qq}Xt_sQmhR2B4dC&Z>Wi8HKbT^SU>M@RH>v?l` z|7mr!gOWyO$xh|ekxFjhl1Md__w4>#KJo86&!-Lio;HKkqH@%mV4t+lwff^HjBkXz zdP~3P9~3{vfPBhxd({SfNe@7LXI>`-xR&%#FAyZ@93QWAL56_XYuyq9aBK{wk)>%#@5Rh|#j-AuM*I?Q_!(DLX6fsX24pmVSwQ`q!oW89m8O& z;RIP9a};K$9JAWTi{wlVeG}DKt>+7{km|kHK6|x(ts0AXd@YKRKf`}JdILybZCD0A z<(qR3N}K#D%8^N3-DwxAY9;+Wb&?NsSy~RbV2QIuQU0lp8;g&cIum0I^d7_DE$b!${ABUY%|zDVph@2#KLBydhHIQ7_tBP?aZ zi~cJaShL#%=pR5^PyP@Wmk-bPkH*GJ0s`NxevUXvIxZ#(CN>3;o-^kb6c7h4T$I(_ zXDD#g1&^t=AP|H~nx~lN(V#7@tjpI0-7H$NT{3>4ZsV_ClaHd2NIs|)CL1&6w%+KP zb6E}dG1*w(*MdBca;H)IEor8{sjxS33D>lxMl0?eKE~a>gC6twy`s{p?3Kf4o~svQ zA~-AuK6|0^4Rv}hLETEPD&M?G)6?@z?X-%Q5W`j00)-d32x%$edo6K&4rt=b7;~TB z`RL}6N`ghV4{e46!QFI|?HUOp*@{at)sS&KPLCAtqjIdPYVrw?x-FxB-?KPq?l_Ny~{(+JnDX6jK0O)=huxO_4yD?Hc{`u1|kp= zL8ftyk!n#cbw-McXG@)%xVdilez#J{$+_H-atkbb-b*9Ia zpHYK1W5v!fVhlnm-QDla!#94upOo=lNdMN54}A!njbqOf5$;0bc*5>dOLpj5T&qUUKXzf|BsQ9&w&za z?593Nqh${G*p@_oYbb!qf=|V$H%kESth(SCzoErVZVD7-o`VfB$KjnsaLMHWeSDAp znnaJ~E7vdmWDBhE%+YT0e;^B%w5foMlMZ{8YupV!(b4?^*DbQUFE=CDTj~3bm-m}Z znas@moS2`3%ciOG8k`iWs!x8q6cu%{u$5uG+v=hxQAo{3GG4-_g^zwqfk~mdltb)- zqMGZ2@m*PAVSGmWJ@bn@#c;|xNlK$jGBx$xR}`H`F>2G(tVclxx3yv=;nQ({8<2P72z>cfV%UYrL~6`c zY0T+ao^jW~Pyc#?B_)|7ZayjhPt&{L&zxP2wn>BrD#(|6!@xCwz^>GNSi+u_a!$UdQg8 z;ylw2Qz1*egpXH}gjImfoP$|+Yq_Ma){CMsdATDY<+2N7C7P948fr*(j@rE9$Djxi z^?LQ`TZ0)kEU7JCwwKL(K?T%c^b#4hlaH#KI+-0W5|qF6z@s{U;CMK9Vm`D`^Cg|= z%|T5w=USYwpy#ZACji)f66J-(C1TSA!q!FXU(c2OnXC^V^4`RoECbTkn8#46jUCJO z!+RAD^sUz6rY zUA|~nC3YG&Fpzo2<=ck_IGoZ^#9Hf-8mV}hIzF$N`$9w3VQJc>fH0~L0>09F9O|6eZH`>Ny;pP|ov@Y4B+H4WT)Lknf&vVx5Zlm-X zqHY|Pi{z9`t@MDf1RZ45Iysmu#CrNPOiR(TVPzE%EYPyIH;TMP{5souX`fjKZlI^f zYAZdbQjFou_=LY#ztftj<(INnY47G$rX0GUcrg*(3NklmXXoKM+81wc?_5Ta2c-bQN~3mAE8z5c&CHL+_^Ja zlNJ>jY2-6nrXz3o>>>20mJjqYz;1dYW^BKqq)BK?>*(Qixy#or(!sM6&d!C`*vk4PC~4e zerJ8<#>F9I&b}CvZz`Oee{%vm_LIH^jG@JlbPj7OUopg_^Qu5z3pbA0P^q1wDd|Kf z0590x|G=l7bVD0*<2;ujSWrMPuPa+Sf>=hYx9pcPeKKcPfjYIc?o%r@Kg=WlWW>lm z2)8Ncg2Kg$5j^MnjKGb5)q;nIzFkjU3@H30_m1$cn|@C@e%bH|GHaE{5+))HpHn&6 zRiAtPYYQ;!d;IX61Ed)Z)p*Fn`)IhTfO?@nEa5(!SJO-aS&>|6Y|_1DKFI_(MF=I} z_J6qev~hLiVzRxbD%iY#@vH3Hx5us!l;0T1Lv-o|uV#rK5N-&xMsi2>1;p0}R=th0 zn1F$wPLlKvD%fvApU|0i_6#ds6IMmT$VR_Aj^YnJJG(l4!FjCM<6a{>*#4ThtD&ET zneNbZ)K1v&4+BsD`Z{`Szz$+Jf37zny0T zY#}GGL0@JN(Ae16YFtZ4P5gxC7haxV;j3O^YwNg_zzM)~i?wJ@L4o@A_RHaFQi=hp ziq7tq%tbNBGqD9*FncCe4Bs3GNGqfcI9J!mj8#6nP5*Lo2rb{0ZiIw+%eAAvntHry z@K@kuL;UNTxCy~h^+&XfjWfw(w7c|LG$z@%%c$99Ab)}TK$p^E2ZsUIAaXwB9Haub zC@9%j7WWxcm>0q{u`gEUhRXtyp*;JVLOy6!VD6O%nl!NJtS@*7=cK3S;jT&CBE z)<2&;hLw{=`;eqX_^d>J4YLf?xVn@#+m`rY2U@)w-gK`8k&~#ywRR8+6?pnlQ8v_1 z5Y7N`f&qyZ@8~(^AOU5TwY6ZjPMmf4gIMdih)u|BP3(*UohsTn%>NkkN1|mA^Guu= z=K$q>TYNxvos7IkDF4r{pnpy}CL#Lb>bPO<(e2I)M$*Xi?=O6z$vyK~6hEP^ z8g}~j`)De>AA@ha0O##MEKfXIVocwkpFZ6K!dZ}CTD!7)TJF_gByx8NG8r+rS+5 z&0QOf_|MQM14(ojT2)%WKjMJ;pCCRzcL;=F){?ROD$FtLUUNu7Zq-3TXu>{lRpgJ4 zKeW52h+J?^yu}`M-(ll;#1d+_N(O|?bKjdgusyv^v#Ey%WbV8&GSaO*D#S5l80=MR zZd+2>RBl_MNubo^-;_T2-Rt+nWZ{}FE8Mp=#g4^7%Z;zLlChq&TAxdDh%mmK{Cv~S z-8n@w%*a$f#m?GJ25k|%W|kk zx>xSqX^}4S?faX_)3(gfd{AeX#)w$QxNj^UWKq^72|uWg|MK}Fbg2of6c%ooTnN(? zKGx-BDkmi&^T$lVUmcW9m(I&fcY#JW7%9Jxv~t004;*$mP{yV7P3BtHM$mOPGTXTw z&MGD1T?<0Y`>+HH?4$NMtUtd054jK}fi%KU1XnSXH4|C(pxvVQk`lwmGL?TB5cuforD9;ViuY&RV6YWWh zM{EIUW)8*FXdB|U#g_K%?H&ESC_D8lffL^9CZ}QbkqX6asj*QgS zjUz=5{GDViZ&!W15+|%8<2(BrQW@~h!DeOznMbuJuWx>LSTwbppFc##{EU;=@6Ci- z;jQoo2pRcqg?sm|!?DZD%=6+Uk?bbw*RNkYoK1KHx;heKCr8FWi)lzID)ZdLZ&WNb z^{-WgTTj)Hsu~X+JbcX830o2$BkUD)`5_!1GWcmVfu*n)UXN0UweO*kN_2cvFq8@8 zgIxKdf1PBdJs`fb#s#Mki55ztPlC)sC9)Lg_u&R7y90oC2ekD9Z)wt2$t~~>;dYRT z2_Ea(``7USIbQW5TISkwOgA-w7~=^nz17~nB-o@LkDM4OEGUR-AE-407lH>4s+XL! zsAvnV*>V)TLH(OgllkV_h61@1l5oznPuDJYB(J%+#>nq&iIw?gUm7j@l~g?+ptNa| zzFPd&C0H{NnaU+gt|;Y5M5cfF((zQ?$Zt9&qLmBqAv)*=#`k##VyO+T&?VU?7`Sm6 zbZ?K6BcNg2xCpkWUn2lVtPOVD%QFD-ARn0L+<6D;L@Yty0*@9kZ901x?geNMJhzJp zDdg;}3U10Un)VJP#Z&$6w7qJ)9<*HK16)tR5Caq{bR~ik5{f|>3^%C=Dlvc&t~8%; zsxkKSBOtEvTYWBf8k$?FfTG3CCC1E4fvw>4=QRvRjQoHV!*D0o(5hF5U-~u`S|_

%%_(p1m%GX&`RZ(u;pq^gQ+yAXv_{RHwcRTYi( zTut6{(xg1c+fzj?vwybxujh$hf!(+Zu_GyB zf!$vp6+LdB$BDy(uvS(}Iaaj}djRrwI=reZhC@!(Aru&sO8 zpYqm^&qHC`TVo(C<_TI722L9XUmcs$s!4*`7i|5bDtoF1m(JTaMYlN0ka z<*vY;T#rZw!oIOlNhPA*1VD8c98@(9&7b``>O3?P*_Mj}Jl0k-;yg0geF5eNx&lV( z5d)U19Ni=Z9pjq3p`VsH5X`IY;ZdS7BO`wbwkEwZEgzlzz-MDoc?hca7CEnh40S)h z43Ild{QB_WZ_2{i!m26LVW7>4uo9X4(+k&e>2P&^G<{bCu!{2*G`QknkZ>b5z*t>f zy>=7PIV>SsZ4@PXA>43h5S&+#j<&is8k{=iP-15%izaWTvOhL6s}*B7Ys=h^Z*%!P% z*R8eQqc>RQ9y`@z7VWdZ9X*_q`|ZE?34>wUegNY+0pvYMV3d&gal%(cL!%E{xA_TV zcHkw+t53NJI7cfsYwcopUT$s~qS`UzQS%8szvaKmkm)b9-FKGofp>r_%Y{1}J^*`E z+-HEh?I_Ql-3AYu=ZDuu)@OX@JpnQ9JLki6Hq|hIC|tr7qFwH8kZE5MX(c7ui?w;Y?b`$T}kMXV_ARp)e8J}?{Bx25b z$zH;=iVu8eeINLy+deHNJO{pM!SH1R<_P=<2Y6FYZ{7TbO6ukc!gDbj#MADP9$|Sswmj4C}<8Sbc_6}h$a!6L|rArz6 zDMovfhEPUgB@oA%f#C! z7(i(f9j$+q>1ZEt&VcAJ*;UgGA1kZV<`l-?iyrqx3!fCnIYD6SS>bhCBv(WdK^laQ)n`>-mS8&M(nADTNg%&)2-aI5SK?{KZ zG3Ng0gbyh|VhW0WbLZiN+o#}9r^CFY^J%1}uj0X9od{wRZ9_M>ZcI@*IU0210p4Kw})NUa5BvTgWI#QwUUG3;)mzI3zd8G!E zbr4(gPdaVY(SN84*#B9N$JbFWl}7LU^DO)qqTe+(R!G=eeanrQlIOq3f8BzGnOXaB zw&P2DON#?_yD~=Z&`JbXXFpvnH{I&;4rUL6uErL2QMt3W`x|rL6^Y16n4pN=m~P_n zeDxGq{G*H|RnEfF&vc+)I@gB@^Iq^Rl;3iD4lGBX(IR+7@2R}Gb=8@L^ZMsV1-6kd z(X!#;F@Uqoa9V_@9XePf1-9DR>`NfgH3KFaD$(fa)Z34~Zr`rm8OFWhL|-&Kd+R=j znDOeLpG=L7XHLGB^<91qCla>ra?}*)6*yw9_JpgQg+c1A)}qoVV$tM9Fo1ct@9)~% z+iPj=SST9>lGe(T|>~;KML1Gjp=bKwLlCug3hW@W**7S;|+Ukxe?bLzt0M`iUtZ`GzFo7YJYq zSL2z?{~!%nZZXpIf47@;eb$is0$&Z%Pg%XF`c}lv>N8>$&;p^|g|Jk~F)5 z>BaEpqLt#HHJk!iRXL}ptOYK+e^84D=`+Z{?#4@!wA9o(qYda^>+bGtAtOVmVx^V zz7%a$RUNeI2-0eAaj?=5hb9?-@yfEP*1$Y)&P&KtM0t(SX#j!%WCB^y$|?;~ShB*C zox71g7+*d4bijLj{Dj%1%l*Y1XYF^lJRy}SxX(~u-x0St3f&}XIAEI-wnQGJAdk8= z><{N5R7IRl5Qb@(vR=Esg%gB{H={0Ck6@#$Sg{gupG~Himf7}pNjl+}>@t#{JI{`y zj6%nK|H51*=v_?WL4D%|KeCqg^$_a0vzVQ=G@861H@k7J${&Vy0DTY$Z`tJ5t;EzT zKpjJU3e(l)d$}A9*7Haczd%}2^}x8$M(Qf0-poPHn7ko`?3eY168bi5rx($X`)PqJ zwO{jw{4#Q7r0|Z6Wof58J{Ov3ugP%6MAn&VfyxKD?eRYA#+QfSCViNHN~sMtiFDl| ztM+S;YDo{LJRVnEn+JBqW;z<0XT6!y%f&bx$nO_60uCHFATHN?;x}QzJkI>{|Ac)m z*Pmu(m0i_ws`vVX@B8Ovx4dW1pY=OA8km_q&!rrnAqoHu2`U-j2&~aKeIxzF6~aQm zsZ(5+)m|$CLI*jghPoalxdM4|25A@r1REG|HBU@T6u-{v ze|qAdPH6eezQ=!`>uzSM=UB7y=X8OSp44Hhxk#7D2nzZb^#Ak3x;cYm>YrQS?_Y^L z`V<25KL3xu#sQ!CKYoiriRu6T-v9q=fKR0dyuRDN|Jf`5BMc&s*^?#d~@%cFc72m`8@ZDf4{<0A{XHJpMZYt1wac3 zt}_n8S1~E8t1a>yr$7}#w^+%@+&&Lk`@&LiRv9#DCcx;V%I*elOUYPqArI9^X9(o+ zoN24P#nXDvt)S2laIK{}k_KRyDoAA1Nj=CR{Ka_oaGx;77qU+iaQ0S8r~c=|Eiy${ zUTQu1b!jx9lMDO}vzt4zB*cOjgA5#Y=-|0CHtNX&B|{6vw3%r&P)S;#YWE@>92}V0 z(6X2yOqJbYE;nw9aQ5U=%r9dz6ok>*ZB^cR{P^e7o!<^M3>I05%R#yYb{L2A0%Dr3 z{{zewjHGOzyUk=%Q_l!lc&9Z{vosRB!9jtVcwHF2wRN4|TqD`cH-D|e}>(z5wB zj1SDt&%X_`N_jq5{d*bfuhT0G+poLQRogQhQ^d($lKlStyX$2rd1$geapS_()%wwO z+cwl4bg+)1qJCXu$oni{^FogC=+2ZjBf*#b2t;M)erfM3%+zS_lEUnzH&MO3nEi>1 zX;kouT(_i#mDAy6hD|wxsS4&8IfVcuMNRrm*disuF zUAz#%{;d1o%gJyjl&(jk$BBT{@Vjp-jYCswS&$}2O@Fcf0={v z&7N;FHN5m@XGL3t8A1gd*Jc)>WOqo3`R+)qQV^#g}nEeN;4I^RlJe1JQklO1b zE-9ohJrs*fM^IQIvl%?$_-;>1oOBO1GJ0va?=g=BkDUU%fDA%LQ}f2&Lp1S>Ewesk zgM#F0E=^M&1}EbD7sOyIRAUQu{QZsp7T2@0@?-&AtZ!hR8e<-G=dAGNu5>0)$&{tq zJ3x21kS|{3zXXCa$olVTMb%u6FUY`pG%gzkriwl0ImNYjjiJ_I+Igo%HxfcmyKF2J zZKC%k6@oJnlw%bO0ppnJ(>y#VSPlD0yF_v`es@C(EK8rCheUWX-t0`4QnsMXpW0bi z6B^y$)mxG|kHhcg^38pJV@e(8!Q^$PMOAYV#{!o6Odwk)nf429o`j%nko`Tq@U@N` zKO4wZ1vS*ZsK|NEPE1M+wsvWS3iuIiwSgo^X&D~2+{k6!*@_AuUY@%k+-d83ZdQUB zQdL-Ja{(RiJVIa{y6LVBkT0%d@!M5=m&twh=zWiagM$&YfZ((rp)=KTjl3*)6s-6> zje-p}L{RJZ+W%d0iRiW60lSWlib~Gj`aLnY*5E1wnNhMEY`D^=6=N1skAc_rb(AxdH%>^mKIQon-BMd<%?f zAJr3RiciSH{r-EEc*n*tClp@H(Kf<7aE}aDiuqfdMx@~?JwKKsZp^<_d$=53uoei4 zBP7;*0IeX1yXJetlZjI*%4O5D+4rbZO*F8Mi9D2CQ zIb@7BZsk0ne`eFHI5ngw+xa1qORG7@r8I z)3(Q0^mTiUx_y^MVs}>R_DU=Xa-YH~GK-43jWG*KDmp=Xeh4yYPa9^sbWq1kS>RcW zkCEN^*)CT=+HEHTvQHeC5!95+3|PY40hq?o)ISdNUqr0*ztcgO_@TnUzIn+=pu13d z-r_zsV|}cH$zTPO7JOCyUz%RxnfhqX=oiVsE3Dv>x`zZG7_rFTO{vkBunC4A;+deO z1)>D6=0qIbcn$t_;lZBa@u-6-b zv9I2h(__SpmBJgOQ5Oth&0|0`u{W)Z_c^3HO0V3ht&tdEV9dZnObia%i+8*}j4`Qj zOWLJ!C2w>l?>VC%=JIjLc;4--8{c|%cleWDMx;8i9OiW1R?&HM+m6BC^KIJv+CYIU zpUUOtHysESl?ZRH=(N%8AH8+(kw1dqD6n=Xj`tq-nKPMCmyk5RL>}^&M%^B;NY5_W5+_m7+yS#K1Q>RU7-1d$hzKF zbY-QYas+qGRJI%Lgz7?z1G7F|>keM_MyrQK#l^-44R615d+bs^_p|$y3mNK084pSP zwvTnXBZCa6WexIpvwe2%CxJ0gjyo^ySt+3L#qpimNU`geRN5sw5kRnYEAzPa!QXly z{BH8b#)orA&EbPqb?uJq@B}aW82dN;8RHFF*Yx16Jv=x^@*t_5wi%V|Fxfc4rK7I; zy!)aXSIkI_x{pAB2DICu-~n>OC1Pd1HOM8e&&aXn&Y=7L@S^5Ecg7H21!pg#?p zG^cWxa)tlL`iGr{4v~0M`N3c>9;QIIZb5*tzmtj4kkKu%5p@RPOCnvBX8QWQRu;93 zbz^<~{j$%!m&C=zucHs_nh6S-G{`74#lxQ9C2 zuNB-}xhQt8f9J5jTuuLQkCDyT!b}j;GE0SMgiKj^Rh_DQ+1WY@zsdJ5S z%aRb+0~w+3f}lL61esF}oA4MYa^QRef>hgwoWx)ltIA`eQ1Q(?nt-yV6`aC|0hvaqk9NLDXgmU2W|+$ zyzamGeFY$W&CAWrP#Yx!JPxeRd_F_shuQ3f_~hWR`coo<3@GK0l$1Y4i)Osr<7L&- zlzv?P@S$ujxmR zXf3rPg#I;vDFm1>hncW^7iSpg;I+G@;*7^AmvIZ^o3orerIb_FU)WPDil!)REuR8N zXI?gV!z;Gxg`RAA@a7tR{anye0quSEcwlO#IvNv%l(oH*#z*eJadRpqcsH`#1$XUx+CQEj28LN4`iK5JWUC93$yDw` z;|5T5L1Jumxm+o(ETC?c@onkDy(=)LN8|h%H8izb`vhG!W!u@(U1c~x{NAj4fSr8r z!^e-g3cI9UWO@j9km`@Ksw3qAlth-P$;mvZ$8uayv;{)lfGjCUWPIdx^%Ahy1Ixd%5aorY6UN#j!?tkw-e8U=>x1(x5RwvOGwwDq%a zaT$Q8+1pXf=EuK*PvPpEAYe9KPlslRoi-+9 z7W*iRVDb1)E1n>;CE6L2(AK?(w;kX`5)cxqUX4rJ?`diI9-itCpj!EUS6OjtcXz^M zpMMIr0W>k23v9@AG~UmGL!Me=G5Ofl5i*;1Hbo-1H<>RgwG}}=1HoHI=Q9j+B=lVH z@>ET3@?LqAbJhGxKm7`Y3wFLk)?gkO2_0C^I0e^2P`SD!Q~ss2U^wRhr?T+5>d_^7 z#4t#{svs%RxvSB-v@q2s*5AcZ)JyWQqla8mt9nc#jy^MCla+O7Eofamz=S{;Y;78J z1s7^=ZlSw@f5ht}*vUw_6v&u7xJ7ZCFKr&iAjp4kQ9~zU738zT>{hK^OB{&l+Y+)*_f3?tr(XIR<9R|yBH1@# zd_0A_$vp?0MS#Iw67T!GULlG1%m0`JGeN{ZDKq2xesI=u@u?rNAH$p0uTX zksAeQJQj=Dc>%I7h*~)`!t<|lre14Sn(Oz6-*Zr|ZZ66%LN*2^|NQJxSDl;Fpx=#CRGbcOBX3LygbZoCLnzH;9>J9 zD+8E{SJEjdON$79Q+1(;RyJ}}uO>e<`psKuu#N6MnH`FluQ zkOAsRjY~<{EX|H2e_%?0h~9^<9#SWPAM5z*b&c8AfBaGFqj&t)Z+h#j#}&WUy|g`W z`pTu0r?2(q^yg|Xq#i`RGnbSb;-dsT_+&jKjvRHmV2X`NSZ#~NON+&dQQ8nmYmM=| z4h6Jv>O6xd_NrNLo?T=OZSU8KB!zSOa&o@Zjo#Ed)nVLgmf@k=^SymJy+}SPg={w0 zl)tp{U{$Qb6>r`em?XSdjeSD&C5-;yik z&S1XMs$g)KX&WXe{_pST>7$3gJ$}#>#-UIoGMc80UY{M4u#(T0`_rK}Q@x~{Ao*Z^ za^S`G+RWKZ1?MfV;nEzVFK(7C>=LTUm|7hJgO3Lp&s-^6e7cW%v^VOb=r3CxuDB8J zcp)U;;RUqM-jhYV*mRnfC9>2RccF3KaiQ{X&!=g~$$9TI@^V2(NkC}9uJb9~K8NcB)wU90Ydm7J?M%FDoEXMWH7R&+`pSa`YirA+K1#OVzW+#G$qTq~K{E+S zR>C9)At=@~>SlARaNWm?Nr6>gFkVCh>AgYTlD(Y+F(wq+9$C*eS@L3gyi{w9=8X&V zPasE@{{)WaOf^Es_71iVN&mGM@nc@VCj;Yn$%5j=auXr6djIF#8`L%riI4q10R12X8#i=`+8p!)eHzarhMf`Jc0aNd3Hc{%Y(v5mE=lXK)bjYlI2;rWlq>1VgNi>ILh7PBg-)bU`+e~|2+wz2~&CgfQdn-M%oG?R5ey~j(%N&PG z?T_7fNGf&3URCjWNR2vk)%4Q=mO)t|lhU7`6+>661!;Pda&_rK<%1O>a`ruaGQ%7L zFBpS&pJ!Z6hJg`RD;IqHyxwU0#=yY8Xdc=W;pM|QZDsDbw1^v}2Gu@zj#&KO5=(2~ zo_1Q3byxkqxzRkkrAoqH98vyUVkg3-fv=gu6ZoaLx^#GNHHKKSC~1;u)pTqqxDHWF z3!?NncnnP&PDRMlbU&d&Cs)7~d|rW4$JkTh?ss*!*^f=u+q5x)D|w*6B*PGubXm~x zLtYRqE5UnSr8OTh(`Hopt0iwO(KT31Jzlb1N>hP9D|pf|R?N;NPRjM;_L`u_qJ`^E z6$`l_)`+%THj$LzApeQ&R_7A8iEj%H=)m`HV65W=HbSlHMW7eERSX?TU@e)FH{4Jqf2 z+-xvTQI1@a-XLBq{4$zkF(?rbWQk>OU8*Ff-q;yD^>Q0dm#HLpY;PUb3mFG>O}uNu zPIMTzm?SG}Z*PrX$@)@1uZ>M8jiRvKw-$TdelCp{n+1@NboD|F0z@3E(J;INjSrG4 zv28ExUrO{*p828qn-$dTMNI6a<}IzQC8Omgnaboszqx6w&NYNmoRs6ouSAxg@6JDG zTqFgLzqc9CzJ~DM^Bz@~-kiPDn{QO|Av)To{^{P!4aPH9+X`qkg3k{N2aEUal;<{q zUZ-QoJ;d3eOi;*(^NZ_fmHsj5Ew!@Y`w8Owk@7orGQ-b@xdUh6r$3;|a0hn9^wb7m zYgXsuW3O4+#z{EFnafq%&2~8-oghe!olAOi>Y|XB3FJUM@*eUFfx>fU3dJ37myEb( zCC?c{KNw0&wPE*0t4p#WgXSD-hwKDsYI!QI>*5 zf~hzKQP;6z$GvfC_y1z=y`!2=zxC0eqmBg_3!)$}N>M>TK|s0=gG@j`q}PZ@Z_=eC zVPud|nucD3BE8qpgD4<`-Xa}=5D1+R0wLx1=KDQ!&bjx$d)GSauDjOxgQZx=r@Zfe z_kQ-XpXc)37p+>8MS&b`C$cY|$kK|j^5Iy%X_&X<3R9Q8ekH)DcHE%|NC8WM0B-SP z!G!vrgUYj#1-}Yd->V=rC7WJZ8t@tcwvJzg;w+-65`|Z zALZ!@7(RJ|TIx-+r|+Mv-`$xl8EGk&B+aWP8pX;pagoTX5x-S#!SXhxQtXf1ko_Ou zulOx~F!yXm)QssT-@eIeS&y0WC1XwNB>y?;YK)-~Tp2}t4L|P}uZ~uAY9Gr=T8iUX zj)mLCTA#KZCj^4Sr~bz#=J}7d@S!7T>2xwWC-DDn*Dyb6EHs8H+WJ^e!LD6oyuvSd>W?4(^?ODLyX zybkS%MZd!#o7dhyqE0);o|$e@dda}7diL>qo`JbE>dFhE!nh|C2m}JI#C|0I&H=dw zS$1~L`bl75V2t|4RxuvC>5Yjq!b*EIuWzUvu0=|?HRj%R9&&4sCfzTyQT1L&muWv? z@Y#?%u?|Cv4fT+Zb`g7uD@C1oK14kzYC&(`;$bYaT(8zT35ehiRoF#TyNt^Wi|5)8 zeY#mtNLxb)SWGVr78pkOu1?6YH^%<L+4WyY>?qtXuXVkRI?1g1>rX`<{Ye zlNCJm!c_ht%9ne{9JyPZz0t_}(R4WTZLMj+e`OZysSVt@iiKnytTKaW^(s`l4yTbV z#R;SZ+n(bdlO>faxoDldnDyaL(6|Hy9rV23S1gD<$5;n^JI;SbJ5Kb~K;%Wq?nzH< zQl}tN9B5F?XuErk6^{pQedyeo)4=_#Q}*j4n^~~1OG<1I1PaKV?gX#37>>b^_oD~9 z;z;ie(LuBWm4gvk^ST$~e|`fGjD8@kz|Pb9Pw0evAr18VLq1Z#R)Q zzBSAwe7x;4l6;0={#w`xZtWzg44)0f*@7hEWi@(9jSM~Ahy~+Q@CfO!KgfQFGyzFt z@;$E|mk1ZY>)zQ=+C^4^#|fwHNdzoA%ZG~Xb*+I?857o2)iiasD31}*iM6%P~d?fe~93D&$A z6cNylmI>;axV&-80jRfu{j@$*XdijYto(Ld-lN`X=c0=144;HkNK%bw7IZoZX}A^Ws1z5cSSqm@RILD@yA zVlJFsX65|ZJEq3qMo$g*(>5T85bdH)6CW{4kxJn@%_S z=^mWR&MS-QJ|L5SeD^B4VsQecy7yfarnddvz7(zCi9j$Wi^iV*lQ~0Ng*co-6Uqq} z({dgA1)+suHI&QMujT9~s!!S4-#7E~;>@=qW-ZanlsgIM@l z>c~^~7xO2za^Q#la)}ARXQEOw^{%hD<0%>p?qfz<{v-BdZ>ldTq6#l;3- zExF#ZtadV3>POGs=NXL-Bu^>+1B>M=g9o$!$E1A8aD_%Ap*C=*z%d-nk*@%<%AQxp zPV+aM`R5X9QXzL+wK_uX$yYhzuY9N!HwLD{He!#G<4tYg!gd`|?pgn()D0ZIR|Y=# z7;EryK+t~Ry}K@-gOGZGnT5&JKcZ+)MHA$G|G5zMr%32P{yFNwcQ*js^dFxMez`vg z{0RQ*+qbX(I}PRW-T%%53;lTYzx&-@;QSA6;J^NNmJ^@@{nyXY{J%J&|3CW3ZoZ;! z-*f+P0UE$>!1K`4Nczto{`dmeJ^%R~t&IctM$w-|N6Z4MT}H1Rt})u%=m7hfYm{g`<-0L;vvIwF ztX)H4@XQSlcAk2SHK`TKa_rw2w_V)Sf(DfdDX@>l_2*?!ufdhwwtv}AtaR(2{_ z?Z3wup)X-f2_0!3=3f#xz(`Q-xVX9!D;*SbE#+=)f1Nxc0`Xg3^F${!?@b6pcd*wp zd(=JXjz>*@D&JcjZZvK28(WbW9iy6tCS=F*Jnc0bM}$&nRM^U96G)>ynvZ8?2!}qy zgZz(x$VJ@qpHjLUI^&MF_Wu6igcww+c}?NS0NP;Et;#q?mG(`fi!9yrPMt+)I9~em zQ_9KvM_1#8(y7gSrfyOe)i;Lo^z|oe-Q$b`dO{8@XCHydDo_b_ycR#71c6rhhV~z?5BUtZ!J;^4sEAF}i$ST{j4V;~AY4t>#ZxkffImMmWpN0|BK61l_X2wC9&09b5ayut! zhfo#*Pg&Kckxq_#G`gcigda9p2ZSSD`TSHabxF?UQ6FuO1zl-{-H$XXk zHhha5Vb@7$B1`?DsTmHrdtcy?No_H_baUlHgivoYRc)Y^hvP!*(8I{117J1v12psb zQIn&O>AMw2MzH#m+_&Mho~At-zm7F^-7~}BXZ2a1{-X!m`Un$DyVPgJBWAonz`n`^ z$aiL{^5yeEp8Bz0a}mbZBTO z{g6>b&XHcv5IMkOJ#hWVKyy&TlZwam3voWbL!r>x32(AnWmg9qTTH#{i~?Erqtv^i z(#>>jfC7vMP4hmT9xSz`d@pZ0SlY_*05(f1u>H1;ao@P;p^t%)Q3485)S1Kk51T0dg@q<9{hu@r_r~E= z>tT%NCV;`kftqq}xvlWdz?WG2zH~*2=8RDTQ{o$WFoK7-_6NOy<{{vm@F+z_(zBW3 zFUZ3@t4OJ0HDsVI@6br5S6LPn_&uO;Q znhinKr^wioo=k?+eHA*huI9qsI%I6p*DspHR8pAn)zL^gQm#ju$kAzmkFA6spc(T+ zj{xY+A#vn}nobO)8IYlm7ytBhIxChZi#cMfe&5_Y+iEMDT-iws&_U5bTaKuJZ7%F1 zStT|oY5A^>$*Hr(^=0<6Z_f@zUE2015g1OAcRTh_3z08@8WEI z#}M+!)D|_(yFJ0waquH;Rr>>D96Wt(w%rEWPmyDj>$NU;XU!RoxQ!$4WtwM2mMmF{z8{Ekw|jq5PGDg zD+U0xi$rb@^L?lV1-Jdq;>{E}R}MXjU)5Q`p0^VxR_)JpdV1%CZ7?v&k6sGZ#C$&O`U^jsL# zYQ2ijhIQq;TmJ+?i(o`KWiy?@Xxbl=2&aS{=la&q+`PUrUKQLBu$tPuHzmw!pwtEx z@ZYfE_qca2LcsrpEwdwynULl|bL@!WM@v~N5xxm?_1i`WCW|O8&K>pc%m|S78(AEc zbD9gi;yb{E79I*pu|_?;C)We=Dl28!y@HdSLUGdQmu{%V0S&Hi>K;wb1;Q$2rP6B^ zmNd$0+9%qz1Xt#M5;w;xN%d$r7^9ODEoHstKkbr6 zg|DM`32ph45>?W$?{?HFKdNf|PNA!eZQED&4tLD-vhbg_&lBn5rP6+5zM!CG{nzT0 zW;;kNQkbcoO}=bJ&)V@0#r>?KD3P)znuSNh3arLOKpinJLp}P`T?E2eTm``QLQt_0zHmdKz&jsx&6f%F3sW-*9W_NWAfRi~%cX(AvA@E& zRZ&9etOYs$&)tRT!a>vx<7C9{V5jR}6R783Ej@|pKK=c#`8|?F*a#A-*6C?RdH)RT z)+6>YTE931<5}vAx||j%U9TdbRE^tcTt|r<&^H>#bRa=-x(eM9LdQ%t8&=g1h?0?E zbs%HuHO6Aa(%fsk{L{ zQ_M|%cGtt+!>-FN2=s~a&sOT9(mY$_(mWfS{b%7x0M5Mza#h2PhQk@9sYn+;ugv^y zW*wDj71A{y8S|Qx|9urU*rW7tQqH^#$x4_=Vs~VC>HDlxWV_B7T%h{Cmj|sAHk_?R zWnUAySD_c=Ylg&0vuQj5Fiv{FsJMBuw5OHL&-Y_Ndx<*Dnv-!oe&Zrw^C-Elmehi7 z4ZjFn)T~@ zq%A8Tvmy(&&F;#uGPLs0qwzU_!R20sQk;$Z&XS3y7ju>UHmXPDzr4mK*nXKUOuP1{ z-;-IjNR3LWdfhIBC=f7}5hE~h9olSt$(!7ZSz1c-wDw3-n7xwOD?bg~<$q)ckt||2 zrcVvWs#qL)KzMj~0ypY;D;HuklIo)o@L=9tG-7Eyr#5g8{ORFFqX?7+I_V?wY^W5| zN9*uj&5$p-hHaeZ*0DAP8;i-fh~H~H%8j?cf~KAdZJAYR!(u`co2A3uRrVj3*n>fp z!0q-V_DdjHDEsFpxsQy3_$LJ>@W#wR+f@mm;@Zy6&f?;^j-=|hzAHoZA8YEJeEbCV&fQyqu4wZ?|!Lx&+46?O|>BrNK3$j%?g` z{V4e{3sVM5@cCL))|?13H&9SLPbTW>$~Cg*E!G zpBg^jAfi07i)F)$Fo-FRVo%+qJ-3K3*0-R zg<#1xV=W<|X37y7?lR21kc-3NR_*!yC=HNe9J}}$3l`Q|TFuLRAFVKRBUW_}+mlf= z{1YVfwIAK9^Fpq4(=w5LaDrsIJjkNvM*l!>gTwG=fuX$MqMi)Qufh*a7AXO_gg3mM z4xif!gUWvVh!Dbk0bGV23j0OQeU9*%KiMe;1wp#Grxr4j8_RR!xI15xBQ;aC>cQdO z%21QJ68yqXX~1^01JsKOfM)c?uH)0QHwL2kc)3CDEh=&<-P(WrPMUiIluynj9&~(V z0%j;^C1j?)1$g$$gF;I*evh6zT0-wd#+~y%fG^TnDCkDW1nnlPu-YQ)88NKI8oW&z zp7!U2|0Lp=Da3-$n!b{;V*% zN29+qr7=AI)_Y+fb;@dqAgqAt%oydpzumBm+Lf6S%kJ*6kTb4bt(3}`&bKkcsLtJ( zNe`QCpE1MaMybB@5@cpRZF&BHw1=BbDlBFOyBT9DYMZ1WuCg3XS+7wds%aSQBr!?X z5QEawV6|OV_jGvJpJ{LccGk2nQo0$%3k$Mu=j0eWZfv}QucUQPQ62PjN&7yRLSLKO z*f1RT>7Y~4Q`N8VLJm^ov6&G>%yV?hSn73&(W~n38)dannAK|m6RPHQ`L@WvAE7E1 zF2<6ELH-W%*ZpGW(Iz{k+V!}d9kH6R-b~B)+i|l{!mAqf$HlQcW|r=Nq0l$!LUxDiin6DIt`;Z$4Pz~JpW#=# zrX}EI#y-q!13^W@-fK3^K}(?wY&^dw%~Yy>*VNBiru<6q`W9&0j4KX1>UMM5VJ*#C z5VIe~J>tH^iRd@cW20WO-n2DX*Q#7iFKUgJ6lisI4*UT zsisDVMHS|q;%x1fxCw1R4ml|P2>pz6S18w?bi4D(E;G=tH(V!?FdRiX`j{6y z=8W@uhq}l2#{KKpFG^Y@pU7+xX9aDw85v7jcvwO%y5c{1Ndel@6{V-m35ZAR3UG5w zE?^U*?{P`(8d05WmD`Vo#Tt(p%`zaej#7f=4d6-u)tm;-KklWLLqSlZxXZj`{0S7b z^nn>ItaipFa(8)973#fjdxB7dhXJStRLi4nj#_Dn8F3=LX|*L4P0n$N;VUM+$|Gi` zHe=NtcIrBa-E1Gw`&ZsS4gx*cSkH=JoQFY%LAPftG7Q&T5PkTwp9B$t_{gSY$~#m0p@ zheGS=*0m%9Ra4^xS5N-T>o!-Cery^)?-vLnj0nH}I3~RamzdK;?{{$Q{2URuYOgT zto8X9Gf@X*pTl5bAL=(Mc_u35vT_kE*ok^katLFLTbnHN+*`#fNOl)(+IWx9!YTua zdW!RzhvXk?A_xwDRP~`uq+I zkwL9q&t_h+MZ)ukFIpH>JTF@aIofZzBJe;f@5NmMfq%1SONq z)7GHN@@~JE$cN!!*xRZI`d>=H9-+owPGy{@e-NQ$2LgNHtY0=ni5#`4=^unEXm^T9 zdi%~s?=0<}Jiss8a+*uET9h%4Gm`iadZ@%{<(%P;Fqx%oTI2Lu_e+rWJTa-I_bpx3 zkk8)^yV^Q2zt6x+|Alt!Nkw}ycd-Yvk)!Ir3N|>h(tUQ8!fSc0wyipJ3t6|j%pDQg zTVV?crq&r?%&y8LO|2OKOJI86WK)gB- z+{(#`u3Q(3D3RlEVlOneF#mOLvQ6+Ovetw7?J7s2sWcXa1(|JNq)Y54mWih3Okw^v zMt0|7u)a!y-I!3atit%cj5LM*Iq)C*5r>Nd!5jTs&J?PEm){`==&ScUIAI*|3-2wn zck0ZQG96BJ$|jIcRJCn;tDcqyY7t>B^?M*41tn}tCUVwatI%YtaW3_q)-9{L+Zg6y z%x~o-fFUOF@~3hodA@QjJyrUI(xeU9p`^;Dti*@*9#6n4TmiRTfIbbTdz0fikDDTA z01?E~CU~7<;m)Q^f=XEdB|@DFM)t9OgyW`W=zLcf6Zkc;I>dHio7jUFYcgN@ODt+E zHvDs??tJu|ZD~%L$kj);Gx!X1N7Nj3(Rb|L)Hd%-ta^^P43~MLEpUm|i+9#XtF$tr z=B4msFoorVvpSR8b=fn*E9#~o9Rc_T?C)9IxRj`r=mn>^myOm@`1M+V?PwoF5>m(;G z~~ zMt&@lk+c=Z)!hRQwdl-V3j)i%Tm!XOayiat@jT}FUR%4sEpEM+()j2QQPy|nyJCDH zyaoF?CeV8*IVz)ZFUsmvdF^Gy0&?h+?&{suk<#*F=SdUi;W9@Q@Yi6>u0Pl8qmNNd zzArUknRmjDbDA9;g%mfbOWo@?hQ*oM(ZiY_5eFS|X zn%Zoc>^bqtKz?mjN}3y;9v(^Bd$u|>gmP=9T!f+kys|Xuh&Hr?#{wnb{+Zy5OEAR2 zs1h6zs!tjzh1t@u(l%(**#^%Pcg*yD`I!xjh7xAA5rj@fn%SQ<#zco+@8NSlh=*#I z65@mqcdK|u1lQ|et0v~;K&#rt%0U2b=`ZTGLPO?&36dgt*bc&<^D9^qAL_PoE-qy$tZa3Y@%B-^QkC~7xX)U`|q`6;k5ZkE=!C@$`>xx{M)Pv z*Ba@>=?}Tkf4BfPLx3nGi$vJ-DM;x9zDpGVNeKX@x7x6C=&_=Q90kESQPR?3sV9|J z&EIKO?GWy~N$?=%B*7%Dmc8dVVBOIQ)2N#|e|=OKf4SgRMM(@l;7!`O%o?-ww2$m9 zk_OdSzOXk7)Moo3GY)AbmD?RPn-3d50UqOr7mv8r*XC5nZtH*kjyNDkR+9ThvF^+@ zViM>lj4{u^tUUYG7U zHQUb&+Cl+yymZ_zDFY=EArSNeKt7uQ4_*yn#K)mT^3eX;S0Rj=IBYtIw>_Y+UMg+e z=qvx>34hys%naD(+(&&1%x*VD<4yM!L6fANB%xf8TYK-d>o4s^%u1_ld}kbAT05_xJ1k}&co_Ef$_tum3d}s%x5d~GhxO=x zaAE_^u376A2I!QDA~JwW0ekWC$~5SHRd4d1&R%H-x2GbSe`2I6SNCG2m8y74HEKD9YWVMRZfQ zOfl%Rvm;xC;A_}}xiACVq!r-3p8io8yWTV$BXQ6|dY7gRuGanK67{evQSxVzGLja*`R^0w-H(aZm#!iD3m<-SQ)s0}lHLU}! zO-0xQH8TADpAzD^#F^kExWTh0s)gWDs-&u7w+=+Q%A~RPJh}KY6|0D2P64wF?xj8D z4Q&+Q#;Ne|$PVS|7J>9%DG5fa0xZf=fQ@Jmp zp0Z+wa)2J_75Xq{nepH(bp$-8WG3T~(0X)Fw@XW4GL}Vs6UC?6Febmj?1!^~<~Wg~ z$9JVe)bv4<nMrjMtd%8%q~SG<(+#=W~q*0d`6;cS2d z9f;Ene9+jNb}aktTkXeZ4+URteI^JF8yf=%MwFTZ+PC%FLdae?RXrJSd-F`o1jk+b zRfoXsh;#vus!UTG^I+-3vQ<1`)D+eaD$HSkWACm^D`TeU)F?hHrow+jiqu9X3VMV%3vL#F83s zK_Gms0EJ+8u1k9SFs8GlhU&YD)Wu#lPdEHDFJTwWRNz)o%-u@PrV6|hO0Rqw|B<74 z@_GM!FoXAWsn&D>CQU~_s!m0cRiLBCn74UNG)(2DuoVTk;y{Jua4Yt z1eGv?*_O&5GCy2LoH-%${M&IwIylfpCf%CtqX+M+@9j-)F11A>Ok5AHOPZDc%I@Kk zMUMdEg?@`Z;lfd{5n_}#`T+m=9cbmrNAo8Ux4v+7{lL9!QJA@sX~}|jo?1PkmLVxW zCEd9OhXS&=wV6DU`Bwd?$ym?0;ZQxCRqZq;g=Kx-@5z)^#ZNQVRahPCetTCyUDyu@ z|2KG9J9*Ay#pa+_fnJDFEi`F7AQOpnA;zbUd<@R}pse&8Kp*2lR}geXif_*|P%~HL zi*Ezr@03~kUUz!H2H?uF;mOj59lYFxK6sU^y6wae07O|bCDPO&ZFref}mfV8M0Gw#sB+fTygd@T2%UB*lF)= zX#~4ZZv8h=fqR|df*FrKiV|CR=$ z_uIhe_B81wH^~I#ckKHWV)?q;?k;&uH?Z*uxe36M&X!wBTGIjZ^adsNk%{+arLC%8 z6yP{WaiBw}7f9c}?oEFK!#Ts$Z%%qN0@GgJST|MIv0|02xi9P)23bE+_3`3!zd9Q| zu}IFeQS?k%cev#hS<#V`gL6BvWj^u=o_0oQ|1>)V>$a4Fk2Hy`TjY_lSFTVLvLDkEPlKed+>c>{FB|`8MjIKJde8~NQ94> zSWk+f<0R0Wv7)6l?G6Hci|8)w{iQFM#>5^n(*}ha6K&-$6aos7Y zYIzS@+C8aq%1HV`jf8P0DO32O6rHs;9q`f!pTXLW{bX=CI;iZ|`@WAbD^I7CJLI2E zJ?o4KmQQRfw1sZLaH>KyNFdHK3f4m+ivInModjAEnJOlSVW=REN9 zr1N&e z{j_7Tp9@E|2wV9$V9F0I7`Ts+w5h%i#8DD`1OxwLg}IJ zCVq2hNHh`%VY>4yxS3-RVL%URcOY!`@slU72ro@dlYnbn?sas~L!bq&zS=xI#I>3k zEHt731X)h=_F;-i(omqpG_m!$GiYg?XarM#@qu<83x5Knx@i;En435910LjjSy>69 zZ=;b0JoKb({#hw}4*x}(An>Y(&u?wv?MQ`2w25k?zGd@~I@MONlMYow|;e{^wOO^AzA-sL_G@!xtI z6}6&tXb*X<@%!)9A_dje-+G1LBNSG(tuLd+p{YBchra}s3FB3+K_|?7t!5QpEUi0S zISVwm|JuKIEaIo4hQf!(!owfMLoVI%5Q6{7bR|834!tcYTYagK)4SSnLX*^-gV9DI zQd=}x-;~!X?;L6U<01-`k3%2635n)##J>F5qBnYCw>8!nO=MmXfILL?aF1!DW)4KO zW|7bG2>Y_J(xal}bE42mD$i<*ko#Z_4tU?OD(>6Z*Z>=rAtaM2!Jto0bJJ&7D!^L8 zm{n4qA$nKDEc;=s39yJFt*$0y#C2LhAW5mRgyYjlG8y%W$P7g92@1R5cPU0j=aEZ_NErFl2r{+zWq!3H9u~(t4efyT0YQ`N)(x6`qshM-?ROHC0+^!ZAm^ZwQjWqVx%LLc^ zdTQn}>mhqiRYvAFL<`f!)zes4H~Q^c?Y@~`LZ`3&hN|8fVwpHSE*8tZ6)z&tJ@qMX zwoU_eH+Y8WDw_AeGzKxm^C@hs22}U!1Gxj^CN53!@ z^$>tD=Pq32E9~eHvnMVVcC|}ai(D6FtA+qsX@E=lm%Ve7Nr{&Uia&BR8W=g_->kXS zV7L*Y)@)1k^G|p#IT<-aQ+7_D>Ys56^%4>LDgk7DI323Iy>~{Yrrb=fOS$ z&a(zD{&-hd)s2ifn-*k4f3%+(fb^Mja&mU7WD+)PA)&gRB=KA7v&nZ2 zuxhnmmi!dND5XnYb+4uMAOm(ZMp1?h9ZWRl-tT?+G`f~S+)STQfV?bJJ1vg8@KiNp zzle9wwd3m_ny-&J4X(XpEZ~z|mOkCk*(L`d$vN^KI6RASr~)Qm%qenLfU9;wVuwx` zWbW3#7DAd~r#7makH6-sf_|UgjoIG5L1|ve$8}h2sgP`k@Tn|7Cond54t@~*;fACn zTitAWJ!9MRcOYch-JMK5ZdFiNdSAMUoF%d{F66%%KQb}qq^RyUwR4pk_<&@ecw~93 zcBXA%mg$}&tRYaHn=4k-_9)<+^o_@eKNmD-z?)jpikX;h0V*Y0F;{%L?>%gzLLBMJ zgA)^^cx>AHR%@j6v11J0sDd(u)w!#yO$9|oIuYD?jV@-8U*8;I>*8X*n_3qi9c@p& zELf&&=#c+0$DnNS%41>yS#4(s-_bLI@9-`2sBEl!9O0s!uJ)N5k|`eY?5#*aMU!@# z;C@AFchMD6{{`M0iASKNRJbR)xyY(r zOFzD^{D`mjRP~0uF0r%Q4078j?#mZBF_GUNQTRqW1fh!s36=}41XzIDMrFTAm z?k*?`V8S7-8kZPhlIm#pBvj@RQdaAht-yH*r1ky{_Qb%8Fe&J)>Q=>=WWDagu(B5A z!1pbRwwd01g@gq{&SLB1%bYDJ5g*H<7UT=r_I}8K)>GfVL<;7&ydomeqFq)`J`5f0 zyi1yoym0CK$_pgqMQI>XCsWW|S@*+dMCzS8+J~`w8&2Mp-;3pD56qB=kb+$Ozm2Rl zikJ-E$r^VUHSv>zP3p6H`Lg6^=ODz|`L_#+=rZe_

  • 7lim`}Q>SC%x-R8*ryh~R z$lp*3iNE$+T*G9%V}olQ9%=82@(^AME>Ay2ai z#4pBS!3Xt4{VJcII-VIi+|9W3obR~=Wb2WQ&5ZWjjBDA*KhSo4>D=qW;<=)er0(dfF)ES1*jpm#vdfF%Eb$1p2O+ku-hNJLO1YfPCP5LRev` z>*X1aqpd%0VW`;bx2aaWDIsAW+YA?7w>Aw=3|XI?9x%&By*&djUys#R-+sfdbs8dU z`mq25X{cz1e_IzyQPW-8^M_kJe%Z0gci1$C$1L+f`ZV#9jF?hOnst%gw!!)IG30N) zp0LAC*km|DaeZ&h;y?JDZl5XqSt+k&W{14C?9_}P5GHJ_ zKBCuWic|>%`N{pjU`qsmNM*sDL7qO({`VT4J9|wmb~wJ#<6YU!tr7WJ>mAP8nN z`_mWVf02Z0TLgX3Yoc~#^&f^OkNCN0wh)kX>6#S&1m=!%Vc$1_7VcvZ;gNNG4rx0g zI1-Jd%yTeu^tXLx6F#`ETGmLE7zMUdQWNnmR8*XVoLPG{ zf}?Dun)bbnF8d7I9z5^IsN576cVqib>+(i(L8h{c3%YiEqVC+Ur*JyD$C;StSgO(g zn9Y#WAkWW>KqKcDc0Y6CU3`GoP`!PcxMgvG=$kuylIltFa>~v?TUDkROTj3`*F%{I zG2E%%cAgZyRcyYBA^0p&s6O26NGXo4A?P_w|1&g}T*U$OUN%WI|dGoao&;3Dqz4(84ySwEwPX$StVs*zs|-Zq^IPC4YWXv7=O@5V%&{ zHH|kNaq{fYiQy7}sC1KxI?Q?0S^a40@+hUVq>pNSS?TU~{Uqr}8+d&K^Dy?~fLCCi zzdUgULJTdsNVzDRWW1E$&{8)m$ANgD9g`*o_OWDgg@q@QW;iMr-{74LnR{X%-F$2S zMBo#~z>U{3@@-8`zL`&UHpNp`YQQ*K3yPA`(Ho;W^bb+7Q_g;j>TiF~1k1}Q*)BqE zdxH%uUba8y+?jsR;q3`a2~ zedG6tqzc2!l!PxTzdG_Wx zI;b6;*yX%_NujL<{zNzoFU5!vxvA!&0>WvZvvWc-IZNmKx!oUK^s6*^2owB zwzjmfsFen9Bb0om%J^o3=*jIwGLgV$+mcqsXT zWHfYe)@OLPO4u^O))#WSl7OTo?5H!eQNF9ZW!k9J?qZ=wW6vpS--;CnS)Z3v`V9SDXKw1OuueaTR`54rSb%dMNn?m9)kF&Ndi?l9LlkH0f8%#OgEQ9(D{y7K^K zG&)tRYHz>x^~7A$*!QW%xvy2~SEg|OOG~L$Fd$@e`^>+i?|b7e!&~kBJdzF!RRvvh zigBTbeURjOpIG0uQ&rHf6Es^-7Ldb?tT3cA2fF4pxT3)PPOJQ>ERH`=N{tOx z-sqO$hZ)|zsvJ*6{*XDodLoWj2Ni=iby<%=7F1-r53H--#0?hJN9@wtEbmI)`;Fzf za}I89kUzgf`@tszd0AY=BX78W1~T^Af*wK(gtU&s>ljeD&xi_eVcd7pjEJobw0`bX zDG>pp*-X`nR&`JakbVVPh}HY+0izS+EAcHyLqc{Z+fR!^t*xzTh87Kmv}2*~*XHX3 zK;l|d9HSXdS;{?(bss>_HfizdH5+`+U6$Q5HiUh&j$jphF-M!lXh!6jAu^MP1ngY- zv?;Okn~Ju(U~P_pw>T#=gDQQ4etOW{lrsnqBc;YOQNmqM| z=4j;twa7{Veq&%%rzK}U{|)ent_aMiFX(2!$oG;ZmD*+Yg7}G(3km1=KF&&;7e`>+ z{IqA0EiVhsg}oAiB9#+A^h{Xf?Sk{w++xh3H`TY~2tDjpC#jcUg1_=@|NaRXwelj? z#Oo)TuPH4f`0qw}QB&@FoEP;RZ(kajnc0YM2?=Q$9sH4=gn~fIdBILTe!*>P7^!ca zd(7YsSfGl{tS@r*dYk7V4-4x;XWo3;o!F7J3Uq)^?yaAP^eIUCaqe)ReDa6($6?wT zA@zwN1xPx#SS;sotYE6$0Ry10qcFU^1%fs4_9U(BL{L9#{(up*l5vK;8~a3^&C}u& zYr2-|9g$|@P_U4F3GX1S!gOk%b*g|s&YK@U8btky#Z%LEvojh+q|2%lYIBNo)Gcr8 z<8yuck6tM#9E6@ia?Q34E2Smy*A>rm{YX{5JiCV@@PV@()yOECwv> zB<|$|YEUG4v*B~Y{sDM+HA>tdb6zs3IzoTMMsgPXuIA2=drNe`sNRBHI!l_IsCHNo zgICWhwI4rD`s@t$Jl{vpkOs}AtEJB_udNmllca}5y8vyevg2V1!zqq0qnb}_?Mu_q zIl*>9inf59{+l~-yTe)m1w~bc9ihJ%k6z5Luc4YJCM98>34;nwTcT=obhMG=8H#*& z?YDEIBUabLKAWEX08MQ9fdFdN7I54tFsE#Ts&=KSw2S03W-L?+uI%nNB<%i|a4P$hX1z*DGPzxi%$i!xFlP-$m z#r5O$l>^p|bvY+Bj=U%lE-S@X!gS+Bj$QwH~M)TF_C0PEzyQet&K3TkruiaMHnGJLPU@GZ5Er3ea}x*r<=7$M}Lvu$h?Qo)${0sw(d%U2 z9%aZzmDgchknewlVBGdo!^Ked-%h-=-Q1NI@?eFXfA}5U^?V4WIc--fm&eD?pKUIi z3q|edEv+tBEPRoPX(_FvKMM>F1j5qRg`VZefCr99Q?8RI$AIo{32^61Fg0?Trp3np z_^KWYT%8#6rpS?;q+5gGK8`D;%J^l#LjIw75fZ9n0O8C+oAOS5x{cj1)>OaIJwoMj zdqOmysG9_89e3SHiw|I|6-OAm9B{07@j3kMn>Qv1+xNZD1bblO_59RGMb9m8Sk+l= zC!MG!1Y-wB8a*EWp^c)yIwC+S%PD=}S0Ap;nhZ)KRDylsJwQzs(>A&B;E&oaz=8Sh##;t)*0X}>0&}j7NHt<>(-+bNzY@~!_ z(gY;-DyZaoa{sYQ1uZK3uT8AC?E-@jo!WPV(kZm5wxi-jN_TGAr6}5^nv_^vWev}F z_FNvWcI!H#%4(Z6vR+RQ42_gZ&skjj-Pp*)0HO-n(aXXQl3c9oJ;G)hNZvWG!<~&@ zyt8TCk#U!n8jP8Lwk{=M`XJ0))1j(KL^S!4_9+=>k4I67B41q*R>e7G3vzN6&0cY* zdiOTR6fUWH^(6v&wytWo5p=uAO4c$9v^N@oVwx zQD-13reIAO(Fxyf@d-wajBL0I7Zti*I4hX@^{=|K=DUNyNLyXyptvIjYJhk!SdQI& zCUcpvxhX9qgtl#t%~r8bZ%t_;Lt_M+;i*x~que6tF2bI3D((Gsd2TE8FXK^Duh$TN8WVMGGbr zTjHnkd2Q$s8dUabS_P}AOKqBx{#%%x?|9P*!umSA!&TNBvs$*wf}3@m1v}UhUmXAW z=|BrSq8hly$;r#zl#+j#WbI!8{Gk2t0YYL3FD71I1;Ezkt}XqZfMTkxqMH4V2-|WJ7yL57K!B@{UfKlW1Ay~^ znq#!h!6g3|=5PC^jeFwqV%orXU_@7CiaeeH1&Um+HvGQTNqt|K-Qe5%& z?m>u0xDX!R-rd(V8Xy#G)YbJy8+tis zYAWuFJQw6%+{Bo?YKvjMAMAcBn#Iy&(p@TL_XxyI$$W?kS)2)Ix;6>mN$Sm?O1`g@Gd#%ahs zEI2mqqpCmb*I6hjS38Utxl5=QnEG%RXjpFuChC0c==g@mXLWaL%gIq@O(fLROhD5| zT%zc)D|!?X)d6AK2>};G0)gig*QqC@{fbjtRAAOuQ>nL@pPiizL*5xQ*dbF5jqPjr z(o*A-FO}`rnE?p==H~MUSe`;X;=)3SK{g7kcQIJl1As!?+w*X#mj85{t|Thp^W<}7 zeRi)iob$TYJaB(5o}YgPMEZ+BRL!{e-nAy&9JbeC2YHbP2E1@{3)#hSiru{u??j@7 z&>u@+YxnGwgY8xyPaZpd{KmD9tS<|Ddt3?e>2UtZ4^zND)#_pP;7((bDV1~!TqFY+ zS(g~Xo7de%78dLhl7Eo~CY7Ujg&WGmPd!K7?A`SBHj2HD2=_Lkm&EhRZ4_W?54Q)! zcoNbm{jpUEiGj>D`Exux1$Ez@4rm93z=;TKWYU__Vfx?A>sD743xd-qz!`Z@IY0(` zpC=&!EVn05yxT%q7te^6Hk|emxyM^mqacA@ET7yE;^qB6?7eqXlj-(09Cef#6*)7C zNFNnNK}329MP(Et0xAMhqtd$&dT8n>DgqLkbd}zf-qAsN?>zx2p@m2%K$3Sqo|!r4 z{MP!`x7N4T_rC8RCu_N8gqS?fbMJeXYw!EIPGXu`7MQjxQW79w#FLSpg zo_rVY9QQH-t{AeO*WiX4L-z`u6+zFxp}f!qt@G>ZyoqCDzk)cn5UerCMTNgP25k^g zpg&}lWfKdNJK#1I_eC{U^mIyWJ;Lnj+YHq*-6G;*A%UUN+^biNrUqy2zgJtSk zMF|;f)t`2oTUh8RpI7Qxoz;TR%YFX*0%$ngZ7bgz?49*yJDnr{OD<2G+v{f$!RRg9 zL$X+fwISYUxmjzwfh~i~34Cx?|8{5fB7-)wtwUxM(tM#ik)){F*BlOcf;OmYHHMyU z8Xh)?i;IhvcOeLzm5~9uGGq6;^>wdQ8LmBH^7gNQwK^js6WXeYZEYG>|8QYfX*|x0 zTuej5L)ew`pFV%~Eq*BJZ)K4Z4 zX5@6j)N67mZLbbJyK?O`P$ti<$&fm~EKR^Tl5^$?i0gKNLXr(`w{_Fx3;jy#r>(^b z!FFLiA2-G>^pU9ZbbOS_g`>5_ak_Va0`+ziqnl0p=9@!dgvkiF(=G2a_?WB-8zg+hL=Yx%-fGmgC>?o$mFPZVh-C)=m} zRxl76AwuRkrl`#q@6?&cWR&f$%wrhTpf7MkS|tyDI5W5GMtZtQSd_SE#<@?HO{2U;`x3}1v?yjke-C- z&D#$iR}BzUk`59G5QLDbO($3X zT~5wlFGo6jg>+3rAZ@tjhG?>H;_i2>FX6fl}>u1i61U)z8Ihz*CJZrG16HwCxn<<>W8<#g$kVfU*d|Y0{zGjd#_IuQt?Rb?@GN1J7YFT<`iihRB=V zys-?_5JwaADc7 z!uB1Q2ngmRr^fpbt=SW8a;t7MM~Zg3^c%hdK`gymas05ye$|PL%nPc!4<=C1u2zj* zdqJbD8`0cpO|b18uc?1VucjSHQSZw2p@&&p;}1-p*Q>hQZ|GeLTy>f7NHgl{h!1S> z6TO5hD~OGFTzTrl^tS+d^2)Dk^)X8QMs8UVpT|saUO{boB|=mIrQI`PJ*BttUo%cF zH4!~nR9BYXDaGu$;KUht6$}P!%ZP4dMZ6{Bc@vdE?nRjGliesmm?7>K4qRVlviJj{t zV6khPrY!D4eJUMWwX23fqQaHTNyFq;qZ5e|WwHkI2OF^ta65FPUF9l8K0 zPOI~XVurSABk4nwLEOc)HP-!5(cW?D^4lm9 z>EQux1EV;VvMnDZ;;o48sOMHMxmMd* zyHSX#K|zOTTU$cY1tVqkH*roVa?p#zw@+b!4CP9-h@p(nPC95xupcqKy=*n?J3QQ` zCCjC2^i=Mbdz(FQk>L=NLA&dL$$OXWA%oS>`s&p3YS+vu{Px3vr73%4Vhah%;9){O zBk}}&j=e;8DQoAAkxxZ-<M4Yo?8Z@~h~eRsd^Rz_A9L0~VJox?{h$2l)-s|d(a#0WsYj-f+t zY7#yLf|SAC10NH4xu78-h@a5DcI~-9dfxl4Xfp@@`AmcZ<9 zu0d0G`(7+2Xk85^eD3hgKaDrZ%*0*OB_WCpoSb^}qOS0ypdgKAA;BsiAL(H>DM6Np zi$DJKQ@B8Qxb`Z#zq1n*0r8ncl`I z-k;BNjIa5nQ`;STlk;!$Gu6Vir3gI^+&H+}vGfOTu5)|bGc(h!eqYV^pW8sQaSv~6 zeH04}YA{k?y0Y=gpTJzd@w~e;krVPpo(tGQnP+cym+MO8q&>7dG%Gau8imsAW2OtC z?p>1~vh#f>=On2D+&}qr;%-F)bb97CQv)x&{B1lD)JSw7jPUtHa6F(t_DjC}BS=5f#jBP07 z$;@@Q>EZG8ePUu5h%7?j&XiuS)6pfrUTn}idGZfLrtpfY>e{e<9{Ezk#b?o>dt}{? z9ULvS!244u*~3F+!3ptr2#%q$zx-*gkOR-exyb-N#}+Pcp9M@98H4#LHX|iRM(KqH zYe2}3Uh~;AWyVWM{hA$kYI5O)uA;C4+nuDY!NF4o$Gu9X&&9^M{ldR%^@BxD&*9~Z zswieNG>V!waRXk0w?qzF5Cbw zpndb^WB~&buC>(!EP@X9wnTLw;ruzrapJb5z2VEFA{EyH1JykzvfaYt0bSns0fm?4 z%Khc*1c>yQbZf0aYSo{-u5+ySh)voe(VxBlC|-=)~$yV=1=|og`}l* z8Zk?d^Q63gf1<#(^*Vh{Ml(Gphd8RT_hCFvz*v#CioIh@^xYOi$#NtGahuR+FPlY+ zJX$w<#D+F+F(>|M5jkZDXqy4BBNSM5!AhMV4LV<3N!8?IR^3SXw96B^>{C{b;c~xx zT61#JoO`$>*7`RnyKUL) zN?)cAzVO4z`9eXMELztJ zm1X!875}lcM)A>2GV#lPIFejerUQ7Eb@#N$7M(x-7EE{S-n~0)$b$V6>fvviUJBmj zHGEG3gP#QI+NBa7@}J?`^QrXS&gMr#5}l=Mb&f>J+ahs0R0(6CjFg@4%Uhp*oYQvP zf5ivVqHq9PLEx*97Me}xYrB4udY{EFB+QOu)c+asY$h3GfYpx-Y(CLc)I+>ynM$nJ2P`0H^Ol5&9`0y?QKaRxke`i5F`|~FKH4wzdqF_7w~Qp z@PbeIH>&Tw^z!n8fz0>I%R-sB952_YEQJnsQfORA@E%#;L2ebUxpG~_px}9C#W|3i zBobj+VnbAhR(R2(qF6%MKlssUJ3UcylV5&HA2pHHyD+l^G#x(z*c`iYh+jzvqyJ&- z@+rX$pTVDg{IR2nz5f~kCO}Q4V@AcR_?ktH->dRh=@w^K>Qz6b5O z3En~)i%6L-mFaLNa-^7703~|@W@X36@53D7P5VVmOuuPdLx=$6x;8E&#DTdvo-WRf z%&Y~U8MOD%q=b?rWl&oDxEOJcbT%Eoy$qkj(_Bq73hv7U7CKwbbVHt)gkl5 znN&VpJQv8>&7E6OSy}j?wpaYvK~Ngkxpe8{RP4zm>+PUi&;vi2D0VSe$tjm_dtC*= zKxbJ<%m_$T_SusEUUZ2~8qIn=YgXT5!iA7zRKC8B-&GUdo)a6+QuY;}C$a2^B8%cp zu1zo5V!EKAXrLpzXjQyiXOrbBJlo7Em!wGg*xV$1=kA>o06~cui8HhM(Zc?(f`Z`e z;>0No%b_Gz>nv#fH-TOcj4UDeH;0j|7VO8hR@KGs#!#BIwS8yWqyPXAh=eO(P|h0{ z)?hx}axUyewCr(h(7R&Zw8Eqh4GpuKil04y9!Xe*gwFSVG>KZ|6Y|4-SybGX@+I<| zI$bv(6YQgM<;p2*d;43LFUQ_Ux#;5RI!9;aU$e1rs2D1u3nN|}s%E6*nu8RqW%??l z1XeBS?*m(7lyy~};IY?sq44U*NJc$|H*Y5O=?n`u12EqWRwuPs0PvhiW+laEd2>S~ z6N4c{x@|mZ2CR&}W1(D7ww{|?^)69Wg0~EdMOP1HLf=UQTlE{rjpPk^ZbXrdi?Z$! zTu!KZ2(CzZ-pPaG1;)JcVL441)_Ym@S~Z2efE-f?@W2x%{@7O$d2-)(Gh3%%t)|mQ zR!DsGXON~NyYtPPGkbsdA+0h|#euP+m|8eLpD!}({CRJv6r=(Sa=r2g+t0%j>B{sg zfLHG+;EM&^$!aln7Vzq9Rlcn<2nc5Owzq9>6o7UYBnZjHq|tlq<+1XN5Ah&6F@(Ud`)1OkT8neHnDQ72TPmNCg3iQqq?n)G7vd z8joiFaS;$*Ev*2?PD?F42Iy0kM{VlwUzT**1fWV6#g$;yf<0qSSwY= zC=;HVIt68x+M|bkFL=B^EQR{?mqTpGU;gy&i!h$d8~c|xQX=IUgyy}sv-OL7KRm1w=TavKPHUQxJ8SvF` z_C((Oj1kkFk^p;zcl9o6>;{6qouPcV{P+nwH#bgb@%QHTc3q_Eq^uQ-2#2qmIaDCe znz6@f|BpZ3gw62FFJHsL-n@lRWoM6^8wE~~Nc6^PdBXRa#Ew=Qz|O&)2&e$iaSl&Q zsjI8&7dgKJByFf_Lb&<&B1jF1u94WH8!)90=tl$$X!QQz)BZ7@$*T}%uP4&PDQ5%v zvU77>NBH=J5VmI^r;XW6@VR&z!FeGh)hkbk#gWtyrm@HgnR7# z4x7bPJZh3kwP4q+9xBvzaS3W=cWHOwi2#Y;2k#Tv`S?;mycJ}R)6>#^bEsO2@9GXl z+<~D>9hv(&dit8*xR$PNE~vMkJ#perNJz;31FT511$rOhk-C=jXThtq2nu~@5M4hM z8fiDo2p1R6lf?HqBT9Le2XEqV`AR!8T6TkFiC8q)JPsGIPSNb{6Rqk2*vPtuCB_-* zP{8RC7zfUWtD&n#vY#nsW(=V%#^)+RH-CJkpy>GN-h~EOz^bY$aOAfyT?$d6FC3^W zUQ~$@_AlES443%J`WIU3<9-g__3GfIOQ%H)-9x}Fg0~#DHJ0$%xf{qSp)a^{z~$UT z$y7loKf}Rv0QG2ne>!BW~u5Pho_^I^7nKZo1 zMoP+6ncyE}eLb#Rt^43FJ9hf#pF0UjU0o;Or-}e@zE3=9c40FuBjc+1O2duYkj5VQ zg-3w@NigIz+D1lpYo`@avN5^2xscOO+FVFWDYEa_%eh`B49&df-!1@Te8)&D5$2Fo zm3O0JCH|RZ8F?uC`YIy_KcEzWi5zRIcW?K68iPTLvjUR8RqcB+H#g71tFmB6wuXil zRQhFQTbHJm9IZOnu6>A( zFo1rL}$T+&R}F>n;>(AOJAazh1E%u&Qfa!nu09h;Fl$RVnP2 zQAweH{{DLr@Yh>qMGK0G&MMO1l>7gAGb|qE_`iPp_kSc7tbY~p{99qe<1xr-y*F=LCz|A!sA|Dc1MM*5G}nAqK2Tn`h0Ou~)r-!?Ato==G% zcK_`t{`yD3m;U8E{`<4Po&XB<-{&h^cUcGKm! z3R12vb_Cy6N>(7z{&<)$Y|-|oUh@8D|K{;gZ|@-}{NJztAAjk$@3y}F&^ znL}|a+9xro*gPwL%7ic zW2DVG-=6|zbI*wL235K^>I+c$n)-U6y}9GvlTZQZ!994=3M3BU8gcT|VWGk8A*wv_ zp#LDKl`3_^@o$YT)R8=ZeHMGwd4J$(IA1=w^NJHUV~^cP#q%|o1S=b*R%R7Ql4HF+ z>jKg5iStGFqtV;Uooki-(vv(0<1N|2CwUsm;Of)4)k+TK6K+XY0`h<}qTFfn=frtR zZca{He!F^$ksGPH-Szrn$2%jX9U1;>a5s8owjK-^UXNO8%DSW7&j^>dyvO8GaQ}3O ztY741X8dN;DQ+EDDJCb~coI(5@x{YY-w~&Iqo~cVNzZnU{r#+OK3qVtwziy#tU&wo z$~%?Q5$5jgtnS-)!W6dd()YHoim9rUeO(R^f3W$Qvd+QIJ~fkA2UI@NrR!w|T#E7V zxV^@_Ba;pKms_IOtbI28V}Kzn!n6Jm&wz4jtkc&+v8hDJF*ZV-q3+ z&MOE;8hRPS(~tL9IV#ye{`-pjE+?V5q6c2-;qrWHw)E{leIZxdZ>LAAJtdDQLr{^|9_dVw{p`zJ$rww`wG82R?gDh3f?#1=>FJ)^lJA z7S7bkZsX4>H!H7g$K1YCK!GFBXi>1LT(_1c?M$~4BCO4g4f`p%@!w6H80GCv!M@9x zib1@8-^%{D-IkUQM5|=a!k=^<117`Un*BI)mdKLtGT#^1Cgr($T&n&6$N9EgUz|6b z0S^@b3iYN~ppqdgZ@&PU5pk%ODh&1O&!L%nrjd+p{cz`&R1KFmc4B8Fc`Nj zwm0H33wqX*tVWKW;LM-ynau@G%P$;)W`oP&dSH40lyFJK8&@7eH1YDyn)@mgtmZW8 zIoR#CrMCOg?$jJ$<24&vSS2et(&+LdAh|z zM0LlE<6n8O?9IVlT+t!#E40tWQu>CgvJ=zn9aLI6Uq_rX(!nXXF5I{3E@PaS@lFPP z1`MIlL3lSQL}tD?Gebzr@J?9B$!F8Ng1nJ#vCg*p*tk9z*yI{lp2u`md6$ot9s27Y zP|=6c7x!!zMq=X$HEi7B0<#0}yp1>~_=+svG?9uWwMt?2`O|jghyzYV2XA z^NbZO3mSbDIK=%Rx+Q_7D?pO=3K&UxvW)Z16TWB_#Gs8iIP8S|_+%EJ!#ve_U!Qwk zK>G2z${TO7I__`U@s{DS;X!R`Egl)P6#M&DF5+$@max#`A~O=LYaFGWnVpm*u!m)} zSyF* zBN{_flMxl?&9O_IVz$=~$(jDv)Ko+lLPJ2{L{E2AfVgU6?M)-?IDxZ5=ZBJ)fWW31 z0!Pts^@FL5)w#iwz=KAdHM}xdWRB>#XT!naKVPl4mfHML3s`uVZ!n=L(iKY{i8+;y-O~aXAe-eft;%R>62QPe@Z~MGE6={>3DZWbdD4G<6za%{y&V@X z8I(DD04H{5^a{}brnqywe6i7!p$(3~sLa{^HBG<37TBAC{Y|4DCfjMEVp&$N3;*>{ z{}C?jg1T5Um^YPuVkj?fi%B-p(b2(GxY(E_wshVELGjnH=N!=KIc=euiU!mYaHhRH z92^{}R|1X(7O5?BJG7_~Ru$6lL&Y{R;3r*W87ll<4)U%uuge+kSo2oTLLDq%v)#!! zpzqHJ&vYwW?>PRXsP=*zG5+^|*=Xxy-nZ{K_43W_S{;M=6z6zxH-qh^>N9owM=>SG zr!CI_O=Xh@IrbwZ0ge;%jh6SKb8I{!qbJm3WMuT0aD;QfVGOdQ8X$=oq-;N;Q7>vx zrZ8IW)*59I8x7h?eFc%>5z{+v2~ohNlcirMzrt~DWL#miEZkz?14MAqY6UZ@5*(!b z^L6CH-j0q}plve}J#jKv_8bU5W%Me;o(BeCw^ka=FA~q-dQwYDWNWWnLGO@r!J(X+ z7a|Vbr@cSWp052{tU#Q+OUrC(mc_;y>f_v++s=M+>RGx=n0#0M7XeS zvK#fuarhzC}6mXZ}xWuF+S&oYB1%Qq$`m*E- zT`T_1*ZlUVTWPVNBx5ZSV%{9qJO*Mk1}Px9!OtA%jar$r(aZNVDsvImBvm=EpBL3D zJD1z?W)v=-oXB70lxo75jbTQ##uWF&>gwoBy%3CVEN_!Nh3U%0CU}jnB(%)uwzdwr z%!A+BKFBp7nw&iB6AtV|Qz5De1Ig~|b%JL!n)@hvR6|aKL1f$N{(j{w^`2|Hp+y3Ryz6B%>q2iV2Puaef32AKBmH7 zi-5X$?Z`zHi9GYxp%LfiIFI%z4iH3^tgWq;bN}{Mexffsvw=USGMQ-P&;20i*=ob4 zpcB}Q+|D;31w+KWAvv3W1rELmGNQD7{=@_za0lIr z)01m5)4iPK=B;HT4=3!bI^Ul&YA#PINs`Uf&21biWM6Bk0N;`A^!29+$hJ&C;(k z7b2$%LSS$|Qr_G$_HAmSnr$xqqlEBG9ho^E_^}81tl+}!76(t$ox|x_S=(}gno;p) z&5@~~|71m~@_za9rS(>kjt;O>%9YMeqs3i@UXAKIIDlU_`x+2>TmLoz=(>~1mlSUU zi`!W(?BKW*x zbgMdKN6rf!$5nZyz)^`1w02!(V?ta`3;uMEzg?5YE`mp~f!uC94&u9Qu1{{S-eusO^Fu z9J=fJpM@yz+xyBN+?O8z`<1NH@p9jH~*h0Y5%f6)c?jZ`oG`#{~f=$ibTtQ zzAko`FYlqY4qC*8ta0~|1)-Z8kGFrI<&Pu+YeL}oj7NJeeR`Ck7HN-xv%egYjsj1C z3Ncaq;YFrA1;QYbvOwt|C1KuX{kXxEBv#K8w~M}g3Um5~`}y3eb^Y#r@L4P+6G6O@ zKJ=|VE2KuB>&=w#{rvt0kURGFwoZHmivL6j=Z4Q1!i&S$CSeJ4X~*ACs17eyo(sQD z)bW-!Phj!%Kn41$M0sV~B9Y9-DK6X3VDw9o12{w*J;>Oy;rZRDPmkWJ1{-4t?7|V! zjsg$I^7csH{6KW6vy$70m2i_aCsypGHER0Cv+#Mj^Is{nXj58w73(oQiy=UVyIKhL zo!@EK^+9SAXKF%OjhI@bNV8NmUB$kNR^)v|djg3?yW? z(sm!u$a)fRu%QS02t*K~=;M$XNH|)uq#HW{7mr;x}O1qUg)I)vZu1oWi_(nStxL#4^?578fxQgg%XN zS}e7gin2&(uG~=U2)hqu@7|e@^$9IhA2<4;`mDR95BRTy?(A?>x3&8&&3hN|A$NEd zxx){pjd>M}Ep92Yghg9hVz6R}jW073WDZ3^^2qlA-N4E?VZ?{tj-qJERB}Aw4N1A2s^H_FV$$+Rj=B|d%9vvzfO7UF6aY%=w5@I!Ko+21h+Zv zpKMUPTwpsWZct>~m}e%u{8dfV;*Z}gwBT&W$|t7y^zVm*S{>{8LN$2N=Cw`Qp!fHa z$JfSGS07l$51Kn)b^CjcTny%sQveVM0p%)9n#-LLRE4l>&sMUE|GHkOlNqvd{(L@? z+)(r+udPd4AX1c^kd=4(^G6ri4VjOF=n}PxMciaM>yEDO;uZ_+&UJ1)E&&TU%L>VR z(RCY(rxaFPSW2d>^E-Zdvr6sHo!;qCQSIZ|0P0*0BX^%To1^Cp-=rq^nEN|IRQZ$x znSrE@gQ&oO@vTcx6DTo022awd<#44gz9K$bTKvuGGNfqcPHsXHAYDW`&vk_}%=J;< zwr}@Uin*I#apo^?VCnTOaWcOCV!FDzWCPNsXPd&xsy-#s$n!qbt5IH4RHkF(0Rd>y z)#Zb3p)J&=h_R)motc_4%5z^L=$pI@2w0KfFB9jl*xIu;&7H8lz-&fL6Fs6>cHAif zlCvpgpG}9tmxl?H-qcD367#dKd2__pfnu-or;{!PoOzh_9HMI3u#*;`zsj`AT!!BJ5n=Sam%lc?aZhuBkCuv z$5vhl@?f=g3~zm`(jk2WKPT$FB!R8~gCCG7bDoVoD$-=#)}Nc3CV_masnYT|Ti+x% z$V#;}hn}{WY6`~?)+xhN2BaPTSXxf^yy13AWvd=YOe|AL%zpVRKZ6;AL;K(+f?Ol0`TW;7X>{YJw*kFaS$&Y+ahqtqv))vEjHFd`#?IB-Rx}FojJ@ z+fw!t)>M*um%{8)6K&y8uBNc@RPQ9Wo9_UFtjDmL%H%nVSVr4H9@$?px1kgS?-vaa ziJe}#tKB|Ut~a^9YXRU5CDymdVDSrRR>y>D5DoC6^H*tS+V!OU?9t91xp@0vG0_vB zJsMm8O70v7*7*`)atafQ>sDv#lA~(_A{V~{St9yxK0IDS17y>kw6jm6a~^yrnT z=16iv6^-%=o>C=#<2qPET@Y`aGGmFz+XgW}@)ZDFNgybPDXQEx{q{*2IZL9wVJfmx z8|B@cUEE)~2pc0qjzOX3WNPok3doSW4C;dT;)=B=>=Ctr} z!a9grwyjB<%Jnh8DdE>+y{2*4#<*hAo4#xV>T>vx`!)ZX1!#v5Zg~uFd2N7g;7ccIjpK>Ek2WQ;(J}U*~+4;WKWg4n= zzh@9MEsx5(EylL36|~37p^J&GsybPB^{6h$#Ug>0w@uM*NSIS%>zhFOhKpm2j6=HI zVqc{}CGLi=nR!aJIL?2*V)nB?Uru)sSoe+C&D)?VSZEhl%EU)7HtXkC0iKj^ zSKPR)p|G^cn}a;@3cyOjBJ7#i!6G<-NOawxfvxb&d=Ic7-uxz6t*BBu8%kvF?e)eF z?^J-1du(TWgDK@U;^CHU6z+y7e-OA|hMAk4Mx%-?!>xgH75l)LYCs;|c-X zv@NqWM8dqMy7c8X(B+{HIe)i&5MgZFRJEZ#ykI@~+>1(L$iWtErcsq-kggD~8{=bJ zp;5VOYdC+&iF#hQ8=h+mr8RL!=Oi46j}7sofskF~TqDFfcP+`yCfT8dAdp#rN|FcT z>5I~p9K5*4a#&oOZrt3cK0la}VY{-j*vfRfCV|J}%RsbD{?4jJbxUFYXvygOOQKGI zS3j=|)j96H=gY{@%62`X@j0NXfg7oG1&8RU2#aY%{4_-wA;Tg)e4EK6W$lvOqm`n8 zgcht68F4#tVl~eWHLmp=hYFvsmZWd3t0UTepmR#k7iRmBxvi?)+Rt1vh>~~l%wC7J zSfvaLa>V)?`_P*cCRY-+%PScGw128{sgLt*K{lw@Hkt$i&^x+P6vT~GqR-1DJ!{f~ zM+mRCav%3de$WNMr1;N!wk^iq?wai@R0AolSQ+A*g!+Wo*X+VN6tBlu*L%9#A>ayu zm*7-zzX#aAb!GGxH)+#a-qkiipmaNUK3^RW z(Lia$NHf~Ya5N8~Y9^Yy$t?nnR#b)Wgo6v87_knv zqxw-X4y7UqTa%>In4GOo+eHSbN{^L@VvUzC^T5fE>Sw-lipXdxc71LSS6OMIUvzGm z;sI9v4@x_&Zt-CP5Nv*Wl;cS1-Y=;J@(3-dO0@B4|0nG1+~Rgy!k~;&pYORAEr5Yw zqNUTf>T4oq-P2HD`H`{<3maA)5Y^QgV&$|u&cECsV_!}W%EDGFVUX?5<2&+kEy$qERIk;Mq*UpKj{_s=%Q7;+jezQaXZ=B* z4&UO5J&^Kh{Yb0J8RDm|WM*bq2L%M|*dVPJuE>_d10ttR)zW5Eb8diaI03lR?|A`8 zNovjY@iqcQy0{=-v;YPY+TS3f90+@t;*g+38zT9>n+AUQ_>A6a*U_@*i6<;<*FhI@ zuH_anS&isT97G$ENxt85J?38cFZUc~n<50XLu2mSxQaD2kBlqTmiE9COS!omSZE{c z)IYk>7mM}o#n1C@jI4d!5+mpUxE~ej5%a|zF9GPp|Q*yp^@{ibue-s(@ zUw^6Am9N{**N~81n~~1f`}BYo!>`BnKmm5F*mlr zhIHPf{#B!@l9vkv)Ud8D3bV{>VM!1Xl!W4F#VtVtHkY_ZQ-STs2eE8};ZNIgC0*`5u*pyHP)#i40sYYwub`dqs`t*)Pf~0w zqNSS^u%$$+B;`PT%%D}uBF;l!!<+m9QM`j0MrG61Va8VV@jwW3ly&VpcP0&(JEJXc z>4FT z(`zSDiVi{=?T|;sH5cWp!7#&-GP)dT2D0NghS=9 z^6Ehhv}Cc|Y)U(p1Ut6sAkO0KP4}*(bQm5ecKx;F+hZJV4qeQ?2kCrjFq&&>z3)RC zF-i{SnnR-$dbHz}-}v);6c8Oj%Cz2#GEZu$@Hk*yh!k)kOTkl#{M*j3F)C{cAr!7W z74;Tj(i0PqnUb|2KoDGVDi`a#zJ?|=fS915sV;f~!2?wMc?>OD za4=*MuInoq{L1$nR4wVf#gL3CY<8S1U7iazhZ1_;C=tp#0on#Xerx{46c zt8o9UadnX&nN^5&+-iiK;OMi-uZ$tU{(o-hGR2M|K#jl_#b|Ol(3@YZ@W3m!6Hck+o4$zQh7#*= zs}$)P0U!<7RyM)84$ouF=KHcCtJyMcg-HH1qjZ{kaRvTQ_8;sr(CQq&`_ajakc2N@ zKElr4Tfm%?={Nr~i-gp!VtRZGdN7S^GCNpAzy+BM&O)CBTxt@&?BYa-11!wN6`)HV zFq1HE(I6n;$rBdVG%K)iJD>Fxn>?qfjOe0B&>mJc#N(jImhc*QEzk%?xNcPe|91Q} z@)3bozNZ9rV9f#-#^R$MLjRxS>6SNa*$ZC zVe|n$<H`Rh}HcU#_zG;|)9lUp1}1X)*2_w&%rOm2GvElJZqeka?s)xT1bgyc?# zU{@{-56Xh{0}M|cIYNUq?zoHG8stl7{r0)eb*T~K91%Dz>RZc7=~Q32_0}OyDTRQ@ zbIo1J&DKMRCVaPs;@Oj`2h|DZ^b2IGtJmwixncP6=0CnaB9lKKQ8GNDcg>>ZzpC=e zoo+0O`$&i6*YoKS3}fC@bp#yf8>882=H1CqTfD6Miilg^D%NSqL(Y681XDzb{swbc z*O)%uU6raEckbMQ5tu7_C6>Stnh`DFgF@Y7o+7~oaRne(hdRV_Wol|PGe-VL*_$r2 zT_PAjd>)?;niPTDwsb3F!*ufbI}8ET36IN+I;pBN|BUHI4p0Gn3Tlu+V51Yr=dCC6 zDHc_#o7cWfR_FPNrv!#tdd@&uy@*M*3v0Mc)9m%7}+M#&d z;n7Gbxx3KWSONlGE)zG534p2*k`{$0;McBK|L1!#1jmV>hHv-R6nM^^Z(||)>K#%v z{p}Yv7I^5Rg8zPVH=ryMXndFWOb}z<>Qgv9eh(C#K$Ga@D;_b)7yeI>sQB+z$Z{GW zPeKxmDPag#4nXi|x~{hJj;%@*#Ko?E9sdC1=Jc@X$BIVx%t4XRp{#OGXBJv<_LT)xbBZX=Ux zFp$&twx9PovP}|$xnjP)e=pF8^w(hr^q^h8z1_v>^zAKgTd^(erNYKe04tYCKMeRe zEsV>hS!{%bhnuZsNJWXec&6O&PQKK?3pK#8UyS5lK3eKKLTZ$8%}QUsvB5;4@ETPg znXerd-XI3>_C9PGkk=e#-h0N%8w;~@rZ5}RQ$THJV^KZGPcR8qwDvavhpzK*ZwJ(l zjaI3ZiNvbX6%N~tOQd*Xx3`>9PIu3jul@lZwk(ALjH5$7yH>|`4bw}jtm;|@S@lug zJB|}hL=Wn97;wp6PYmXZ(StJC!Pm_jQs16?^Se-zMVDJ-hy~>v!P{tBU69n3_*n)& z>K?U0W}SaIkT(=@Hj;7+@wQ+w$$sOg2;p_@QkU zSeZuH@F=c`-GE}2ZobFi7Q|Gi!a4JuS{)oL3;Mr6m{bHg*lY7T*~qvo2=uM8kQm=d4+BGQhPBgj3D?8RNPhk!?|E zHy_4BiaN|CXv1Jmv06Vnhr{r$vb>B8v6{@zYHWf)<6%vWv~C%|L4k^D5 z->8%>0Guwlz_~M^w5WI zN-(Z0v-tAuKGFt*=^9CZxE!+Q5swq>yP#qb``KF!i#m$SCE*~@^J;t#%bI8%v=7?69 znGGr%>K=Bud%Iq-cVs2-a?gNA%C;}cZ%?Gx9wcmhB5FZ%Xf-+l_}21ib1MtYWhWmyUp#v37P zF|b&V?O$HO?wki&IVo>sknMccOa*ubF^{p&ORQVg`J+^8jRl4?bwnU$0}u%-yeO*T^5Kgw^UDq@AIqsl{COa;wx%B9@%bp%WM(hqAu{z z1rDV1f;`IS5lchn?qIkv=ThG>X|$a)>d@U)D&-2aNr@;ioJL}HVfo9vY}D3(NpRCpp$(2iXEUpL#yUG( z>`)gZCkgOH!&8pfxOs=o>V4R90I+Gx989c!p>@OjsN^j)360%}{(R#O9W*0yeKO-&gv=5!V=0HYUhl=T;Kynddu2f?)A z^=D)l8hYQ}3EAF@au$d)ur*+gU840>@B9iKBcjDT^;##^)ej~U)qOS-l%edX#d^kr z5N2h3fT+Bv$yNSn7;uweXP@VGroeE2l6GbUusScFNAnDafgeKQb4$tGtoN~G*6kDR zUT7o+L+Vr4)+j^#o;vM+3JW|KRekT4eHkstYZ>O6Rj#tauxvJr&DK@M7blq-9ACgv zoxY{G%nQxgX%d#t$AL)c_-tRcjefzI%-;UIDaO*hhs%>zQOdse0ZWh4^jc6!PrI7{ znwzGbUr%|^*JQ+9=3d#36r|tkTZBRIR!3x@F~?|_o`Tx?>|aL)M7?o~m7A!uQ_-rW zsT`T#F0TmQ@OlEVa=Gc19&6b;R4lk*itqJ}MWdAtm0%dL*7ETMPg*%0?;!bmN((oa z@c-lO>3P!EN(c4zqPhr#Ft^#Bq+DZeiRlhQpV_@IcPfPW@aclE3lXWo;?O|uTxxFaJ#2|#>WWaA=fT0b* z2X3!G#X<-`Sui7CP8Zh6xlZk4Nmh^P3(R92*0g6zIXm=!UmEpPfXbeA^nACvQD9p6 zpd8|`fJRbwztL2%xCr2x%(t3I36oyR>>q+HpeE|$c3n*kNA+f#9#`el1Qw)hG15+| zXev~g>PPO|LAL`s-b%0Wi{8HiW_be!LA#1I+j+!Xoy%M)gbhQV`&AoHn1lB(eT#Qr zq;W`q(=M>Pl#eLsK3DV?u*C0R| zDE{vL%X3T8_dxv!$aovcdn*x{1gstvbEtZ02dm4PHyx2}5Go*2yfS7r=I6sHW$g|{ z_?>QP=b5(!(2<3Ts1?v zIKY-h95-s69gxxxhaOENonOd>oPTfn1Nt+7?9ny;Y#1oa=Tx6lQCkV&uSt%2t8DVoq_kyIfb@ATuJHAP`fu@+}P+vF0uk-Q*A zC)==aR1VfSG(c2AMi=XH<=G+5f>B4L-8|Q8LlAhfZ#}mgY3KX+(x{}oYEODRtkByJ zQX5{0|B$@%J;U>(2YMyW{xeNN{PwR<3!lJ|L8bsPb*t{J>}q3b$2c?JjdaO1b|Wf{ z)W~_P<=#HIUw*y#%Z*=af88H;EhOx>mqqWbtm1U$1q>o8jqwzNjlM>|xP(8mcDcM7IcXN}r zA5ti>BUK&d->se?CL&>R18r9UnCHlfF!g`2_ufHGrtjY%Qve`dnW=eE3kb9Ey7(%Q@mTT3NvVO>qlXIWFmGnaIWX81aQZ zPFXfFNn<{k1l&`yq1_;p(|};_4%$N>pl=IYcX!uY4kIrWpwVZnVK;42fello@kvSf zB}+XMTS{qF^*4{UEGYN0#7;^MrFa;(!SWcsO1OG>AhyJeC+c0? z+@Q!sysVEG@>SRCOOg>gAGvbM%4BS8TFcyrQSpfh4upz=g2G~b9UW)p>>hn*qVpiB z@J+p1#14C7da`VOZet^)E?~tVG&D4?0+aLJ`z~PRg6$a!*f7E9CFY!FW4dy=b8ddV z7{I%9KX?dteT&BBCP}qF*bjHTm-ad@uO)18MI#xgSgyJi;5l0|T#7C{+FGr|!+|ZAU_d zIM@rlh=>RcMl!w3X~|b;nf|bmvf|q-4m&_;WG-YYd)?VN%~x1dxc&`7O9$=JIpD!S z&d_KWP^-xMYNw>|_K%Ne;KH^Wifw&qOj|N*E2iKTD}p*?)_z=Qnf!AEuRK_3bZ04Tje26d6)s_GZixjhWNJU_$Y74VQ)H&;+_DK0hj7AVjn=oAvD z)*GUB@%@@K)|;Y_i?Q}L`I5b*RIQ6GAZkmjBoC385marDYG%s7fC<1+XLM&BTY!76 ze4%lqvQ`TaC;`wj@R6+9n-V=DFd%(XY;@b z%15K6aizD@rN`(qd@IWW<-RJYMC`XG`mcMu2x}nTseSN(xjXkUlD%qcb8{NcT_kIF zRWnZ?Zs6OWBd2Jj3$$Rs;z&AuWzTI$y~?}V1_h!^o?Ot0LRP%jE)=V#akp-)p+RkH z^ZCH=pkw>MkiILtZ9u%FprD|8n-n-bMsG+V4SDkRtyD6Be|2$8dwuF@=igWWI@7zt zmW(^oe76`Siv=OYL;`D1z6I}R%iIA>*TT@+T9qT#hju4r_RF1kc{;Pz(7?c7)F9K&N#kieTfWJBW->>}NdtuK>`|Z0d_yf&mey2kD6(*FJX zgV&vn01!3&S1?c?yzecZmdU(V|aD1U~Fn*(q8DG^X0`I3kpdQj+kPg7H5<~TK` z&ZnzSLWeIK_&ti!b(bnD<*DuFjp}#nU|5n(jlr&P9e7^atOK7XB zy|rA#vgu^6a?pMwg;P_o4-q^;qYPAXu5ys(WW{bpXCq70KOt zw?Tp$^RuxsVeAz&PSgo+pyk?tBJldiU&K2V zTHmS2SiaMj?jZWc5Oi4pGZAhBt3LfoM?lZnd{|-|1hRi!-Hm-$dDxCt2Cg)OkyW{l zHZ*{HU#AVjD1{7LfpF$fpuo8le1^0f2nGbKQA$~71$QUdeYOUZ@@s1g@096a<*I&RVeNBFn`8We+0gh=neSf3IA|`uji8_geoem)Ktk8AG_KH>nj$X+) zRdiuvw!6E&rrejlROLWj7h7kr3+RJ^zQG0mboMKyv&mUc*h>B=xx2dnP;UTaQfvpt zAMo?OhXn>2X)y5gGKV&inl+j`>re5CiJTtN(h?nA9Ul7qIW1v!kAD7_uwj18%=#(n z#P8wMvegahdvInzN2`Lo{2)M{giBdeT+~an21W7XRXsopw)GWB1Mmj=f>-X}Psp}P z5M$%enoPLb^K5lPVt#3S|SGi3JF=w z`Z(O5Z=~m1VD%S~6u6l*Pq1gdZLB9}!t}n8P^htwGtZxY^EBe-=6=|L#Ujv3+K!IR zFPEeRrv|pg4APX&CfQ(iKEHVI;KJT~TvJTBJ!$#l3;?OJ`q;m|w{OB6`2L73z^=?H7)pF{79ZuNJ)kA*)>fs;^)J5Q5PNgbfE0$Ao&7Hc_ zlJnsis)1&uwF~P>1kW)QhbRV}0_>Lk>Q>!Yu=xU+Z0O$qRq0aI_;yQ@+jtq`tNnNW z1-2G5sIM>ia@GsB=T-oc&TQ3Tt{2mGhhuk5YwJP{trS6g#N0Frhky_RiUd~X-dLHu z4rf(9VD2b)!k!2ptMiP|peYLCK75$3dPk$x?zTdhjDew{G?^9*pPOo0OUpBm9W~U| zGumC&adviQQgk4w7~@afV%Yg;#Tm)K5t$tt$b4Yn2n`(~vA?J1C3V9IVEDtl1ot-E zZ~&7*ii`p^)4#5$l67$Az4~4W>1^amN64$>kJjnC&to~?_)`YbkQ4PS} zjV>}ncQE&}_bGW_1*FTM^H!$1qzWw!8eqDUm@{WUW|e{GQ~7%_*t|G(0V!RiVb)=< zeSp7D*Vov-(7mv~cP#lKJF%~y7;0t@g|;b~e?MlcFXm^IrKpnC?|uTYO!SF>@Ah!w(q;Wb=Y_hUtT@^ z_X=lLr*}R$bhT+m0=OuvFSK{Q|7D_5Q}k#{#_3EuKI|@}c6N5?2h;Cg^r1${>(0u= z`iBqqlfes;_kq5izprz=Kw)~X>%x5BIpey$i@!(b_j^-h9BJ>!T|aP4wI1!c@hkeo z(|_Hd=>CI`h5n!ST65pd!gs;BKPh(P;A0SbPko30O*nsDXhhlFiyHrp&;J)BKB8F) z^Tb$+>fS1M6^u3-8*ThgB#q9?0aAOw>EQpF`Iwuwb3d?ko9UUtPYx(L-qkV{H8aUC ztSIgg@fP4;S?TzOvR5APRZAssIgPJ%UrqXcC@$|iba}s4c5#ldxx8Ul?s3aox90Ai zQZ&tVbS){=b2qg)Ii%>g^$HQ32M?sl-kAHx1w6;L&n5Gme&O5hGgh+>SG7!$({}>R zIsFB3y<=1J{BzP?{%J-tBDt>T(7N`HrYYy$%N(xiIhiOJzybznvhin4lQwgX6)jdVbr`cd0~((gLr{@Nd}PO(j!sVb=} z|B&F7r4+P#8|eVn`+>ywhq+S^mXxRd8TVk@Whf!Ei`jW7UE21@@s0!kWbMLxf#u+- zf8N)Zcda7!v^j2 zS!-1Dak2c`;@_Y?#b_~i&Xo?Hs?ttOx8{M;pZ|QrIWv%ocEXpPk;$!;2hO7N2pS~S zL${Z^j07xEz`;Yc9^S_`@d=o>i&1+QBV5N+5Z_*I4LWjL5aiVLQu9c#K#wnCiV)NL z3x9s%{kubbGRlI?o|U}3ybu!xkF@P;vTdM&2CZuE<$iX~7;k^jYV+;*{ewZt#Jox=c31|feQ=$?Ggp`r(S;bkVG-653>@XV}kLW&^`^FV!gELjLuykSk zn77g%seCWdo`HQx%uuQp8Yk9h>}|9YWy?LThP#jT6}u2q8Kdc>l04OX$<9a53)Sj9 zLj(NyPnR*$Wu>5|78Xo6Wp=|7bW*aeiwa<1AW|4KG~R2VmaL@v?}6HK{WlK9oZQ^t zVm(zM$+%QskS&N4rgI5&;=;G}uPi9Rc+ZHgyG8?6TIuMM zXp7LIu^(K|W*0Q{^js+lR5|jv-nM%vc?<^<=Vgm|WSx78j=Jup4L+dTlDN%N7I4&E zV2R4N63#Ydz{Wdq!qX)=uCVNQRZD19iIU`Q*e&x2&xUtP_}%8*j$6jIuZF8SyA=b| z8Wn>B^{rl>t~_2=gN1)(+Z>gZHq_MeN4XLu|F)-EnPr7w4gr2pzL1G4wN3{WDW|3u z1AB|1({;232_GUHoIt;TfQGyudXj8vl^WN_0`p2soxDahV|O>X_vd*cI#M@gtvn6e zUOX6x1EmzgADITmE@FOJqwf9!^&&yVGlaoWLtn1SsV`ov*c!H}k^{xrHfN1c&)V82 z-FG!kl5%Tm#DOC{TJZK94iY{o>m2fSYI4>Qjk5t%OJzyvNnrY^0?P*_HF!m+sOnHW z!-W4>PmmB=i5L%+^_{KQUDe@=4&JGGIo#orHqR)|YLKcL-Ok(G-pZ}YSH!86FI4yj zHf;?%;LmvId%|D>p^I&oj$Dr24vPtd+`kHZGNtC}Lc+%xgC)2SM%AA<-JWuc_?Pym zw{9y^4Aug*ua5DE-Hg_p#NONQ?$`p=YGsuoYS11vZ_Enlejk)Q2_~h2yqRYkEBr;O zF*@6=5rZ>m0+`N-Zi9P<+tcXujk$GJ3sWs0WGwxKR7w)jBNPBcZC?lO_SsPTk{ZJs zxW3g7)CBc@s-H?4PXrkawe20;`sd?DxkW`18?#+)>0n(B@N{*Aj&H<-ZTnK9+nBb? zqbv2Q0Pq*QFwbq#*l_W|-Zz$T@Ai#)8vOYI8~LcNwW(HF=k4MVZ@Q#KwO4pJV|+9& zcCSCb)gdAqs}bo-OdWp>3Z0HP6qc3Ei-979ZtL*ZY&71FN5!LL3lgdr~?T&;o&tN<9Db*;Y0 zDl7Y~+!?3qJp8`+_InkmPp{U-5$nMkddYZLgokZ_JqX}u$}<`v#{FvG?oNQWF5AqHNUvnuGr}B6d^b@1NX#A zg5;xS$HojgOqR9j-RX53{jI3Z*1fg<`6Y74r^g%RF1^2B!jHS4aY?C{5N;mWt=UZq zh139U2!TM@-x`O4W36p3M=1wb1uE6VWe+%#7eWq%_si!pWsp_fVr-z&Qk75s4O@q(bbEZ!uJNtsmAgn1{-|c z&dAmU+y&5yvc@`DU~vorBlh0G$sH0hxwRUtw2<;_vJ0<7p)d8bc!9VyZp?nk0ukgH za`0daZS~I$nUK%-p8R~lDnJv|w85mF$tXG~Pdb2XBDGDk!*}{i(Vm)@9B3Q%&VblB`KCX?p3tPB>kpTJ7%ISq3mq z381_VI8mo_BH6(=?rJ>1Yo_wH7+=ohybXvc(Dy5J&W>{w&r@o>aY+)g= zyCwv>8~B|^DEj0J16wkd&Y=)J z)RqnM`Zq1boU-D3Ui1cgek$3&qdbf8Ngej50Nikql#JW>#>Mb(b9=&!TG0pX#utaN z%PC=-U}h^jXA}cBds0g2lIp@x2@x<8JWIYo>Pk-qhIf-o_H3Uglcw$phqvWx_#`Ob@JV8q62^`cxk{1>VO-#&C zzR}t81QmC;LSx(T-z??&D*YzzSb>R-sZX9H{ZM1?ZMz&sTmt!yFW|P?uC! z=s4auVUUKH`MNgMgu%b}H7c!{{VXjTH4TnE+-r6UgVCfG?a@7Tj&8)x(!y$po^>Mq z`37Aexen5!;eIWQ?uzcKw$j=D7)rS?3*hlQp=bkBHI@j3+>t>usM+jmcJ*92)qi8v z1xE^^>^~>W@`;L7%cODq_zbP%qNLJ!9+DUCeOPF#uQK(gn~BH~dHb7V8umAl4qaJT zyw~Pskb0?pqtkQCG!9((be2nhshpUz^e_;+Jna>&+&*tXVO6|2eDv}`TTWDAEX^RJ_*nU zFfgkPaOZX23-vxY_`ZigK*B=zLX82k9w7W2o7)HHmzTSnd0_FBdh+rmA`TiW6|q=M z=YxCVj0EZUr$l%6B9KWu+xG0;y{srqH_|wqQ{&<%MF3W*Zm6i(Orr$$E#k|4fRWDw zIV%{my-J%e|AwK!M%%kNuk#Wx#O$`RuM>xq5kPD_JHFsMF_C3QjG3jW4HjGU`vO2z zA2_GD}9Zey3$as89TwC!q!@h`Adag z2Z`nUQ`grOaSXW#>?XY6yaW!7t2Ou#jSK`|w^VV-D8**{^~&Py3?t9r5UJe zpiq?M4$`a8DzQ3$6~~&3a&twX>ws@~Hg%jfW)C0R$0Eaqs0+t5I0_(tb$L#l|&vN=|+K zqN(<|deB~q0Em$?=aq)V(+@2QaY^L`<-igg7nO5pjjquIos5;qOVB0AIX z<)C>T1xR$GPlaokgPId{5Eg6>bnix7q(Im9k{B~RF+Toix7;`$?{B-HHa^@@b83lE zoR=p`-xmA=%#aU0_eD8^9s`=&LG4SvK;02T+pw!3`PS3-Me%?WoP-aomyKBKg$@9vj*cccB1>1(siEvn83Bg*AWQ_MeOJ{ z@~$DsX`-s)(Rs=QN|GAri|zzE{lZ(%ZeJZ>fLcgZ8leR8HV_Uk+-$w`wxh*Bk6Iz< z&R|{lVLt*Q6)3X1w3l(AEIV5WdW$-#WIz9gy20vXgcEIn!hDu|Su7YaF5OutB=hqifwl4wbYID;sKP&4jsEu@h znXRZ$P+CGEFkW#UOCvR^>|6r6#>Vzz)R-!gaR%)w&4`Mpu^g}F7bk_Gn0t7QNhO?G z-FcQ_1>Y6Jr>t_f&UXt_)u3#gSK&0yWKf4A7~dvn09mxb6T$o@+?K&00cI!vnARe0cx#iF^>MzvtJWtMbi8>I_o4{7$ndL77s#n z(i>XdCf891Jsm4E!LihpMQh07Vk-}lKV1hYY_6Y%)Dy$7G_kXsmA~=Kv+?UAqw<{( zK}kA5&>aUh?PGv+RT0@Gg#Af4fBSOjwlEZ-HJB{$S0dDFAFG&!bO56nACeV3RNkuv zb}zE9j#X8G`rD@TyI)p6JlL;^;aBpy2z*6xiMor<4@yf)y4jfwx-noX0Af5dpPzx) zGCbgg@gEu-%&Vn+#)YP)iec{(!C*TM#nG3-^C|&qEPuGnXOM85?(WtZI>v~(V66mM zK~SU}H}XnAAv%0WJ}SX8ft3t2?QV8iaW*v-i*ap! zK3FrgZ6#w*8`#|?8RY|XB&a&sTTT^dhibIe34RCyxU5rGRxU?0VC>;#**ytS-N)Gj z9rQx2W(SShb)*{QbwXntLrJgH#=Vtr{G%Bef+NYL4OyN;9ePHOav)y4eSBo{=`w`b z(&yt#y1J30o<@5;KyB{U!$GMincXMsHR7*T=V^E>w*O%&hO<;Pq(YqzW-I|bWM{gn z1fVB9okhWm2YdVEgZW8zRlwbwzS;>f2W{sx zqtgQW34}C3#@x3{3a?w)!Ybb$a3w5WU>O>4YaPOOn*N538Vy|;829G_pF~SPm0Y4^}qTRR3Hn` z$pu;Ozc;F~es6%(L~XF`9t4X#Gb84qR!3Rv65FG0h)YUzfIbkz0h@5)dwV;o{(WND{#$JCdtRM?e*X^7k4J95lQitma# zWkhvamNZ1ZVS`=ZP21Da^lg{)KQApE;QE)($5~k{~0-_&7b^N5(%?PQQI5sgO=q14{HPUOkG^w9p*Y&V6<{H-FN#MH+O=| zd=;cxz9IA1xXH=8B5m+P#r$|%Lbf_1xe<0B5wek3!s*!1(9MpNeJ;`3&rjZW+4(Fe zXM`7JT@gQfOokn^>xS$jxH+09LL(!B7oQcDoW#~SKO7q^zv7+uv;UDx5|S5m?WaP; zFDdE!hGZ1h6hVx=;ihr^d(}({M{|B3%>mXD{Bd0#9#T4g4-9xp6>`N!ZOYL%KOb^d zvJsUPGI#;mBfMN16b#=eZaeWcr*nLKT;p(2{wf#|8P$zb>09Jgp6-j2t&$p9oETcj$Ko=N1_T92|!?w*qgMDj$$fIRCa_U}4f) zvJ+qE%x+*-A%0CPWolFN<16#g0n3uZ>v8%GBftH<7;N`rZrr&2c6roT-s^IN#f2@< zH&o)Yu?}|rg^Vox&_nkjj~;NI_}jspFu%6V7x5`67vjWwP!p2S#1LYtqrrpjy@O|d zzC*8bKwz8g!Yy2r!I(GJ#OLT#j8c4Vo^Yu3n{S)0JPYx0^n&{OtAG|D5eP?(s&GEx z7gFz``j2Nn7QTIZuT0)2oOJ}=x+TKZGwAwj**GwbZq;0OomD&CaGHDUS2;yrAp)% z>N}onWnV80c~MrxDNZ=?X6|w;EJ5(?dGl-cp`)5z1dE#^!l-YT3J$NX^*`wgOBe_q zBd2^{RW_rld-%f~0&NZ4<9C0g!}pBh(=XK zU$we&iuISEy&EZ^q#8N!Xo$0-qshESlcQO z@xxq4hPU}uO|9JTy@8vWRtkhpQk^>N-(>rZR+?87mJWT!+&ahL&g?`gEuHEXVA;Y)_&F~#h zw8hA!$FA8ow&vDW*WqRLWzGUEF#H(wPn#eR z*npKWvS0F&Y5;4$lTW*H9(o9jrCH-~n#<>>S(a7d+1pFyBBcip9GHJ|r2CaH$N=hq zng!>UB;ETlAWz8w6BxUa5$Wq2TyHU@hR>cIB2-_}J9NmKzoH5*>;368ue#f#^F@X7 zE0t9?ylr)!<72!kpSP>V(T&vZxt~R91V|~BXZ&biZ>p?}w1KI2nV&vAL<&P7Fq@!{ z_~%h_Mc{Z6F>p`HHkcn;VW*jR@Tii7alUotFlo<*olDFT$i_mwubBARv$pc`@<3JC zlyCX9tf^VPY|GieWn2JlOU;VsB_-Vl!3=`*ymGPVB;gssb3OgI`WzZ9aeQ~=BnJn_ z`HR0ktCsW*a0>M{4W-En0g4}s?d{~`S9y`l-D~C9`gI{{s(|jri|3cjeSE`{23gy} zlUVLtT2n3B1YXH!Um_GP)js5`Y_;XGcSx5bx@Qo3&=s+^k9&xhQk$O39^e{ww>IRl z)_gB>sqQdjVlWX~JBTU}Ra;U=S=k-jTOU$oj zLPxLuHbb7uR_7M6G>#{~8=D+~^PVQ!%zx_*aq4k^UuYZ7stEmZy^_f0Wg1|JD)T*Ary6B8pPnoVB{V{GDl7dBlR)w#_DP1ywRy1f8xa|HEt{erCTf1AO+Y*6; zwwkTAFuv!YNm=VAdYW0MZR9NM(VNt_=LrN$#{9=d0wklSs!CY=%Za!6D(mrk3;;A4 z+#B2b>T02#0++Ka8Z68(+wGhEYSlET)JStHpr!X%fpM_4k@VC1Uj%XF`&q zm9sM+F53X5W;^@zr={_XBFBz!x2+I_jO=3np*3$HY2=BId7iF`)}9&KE_!L{qqf$) zC}!tRy*F=eyT-NUynENfIsD~}aJ$owT%d|83@Hnx&!JrKnU08gQ z;fqrl9rld+ov~j|rnF9@d+hG*s7kXUfstyH^1g#E4*FcVkvhbVZIk%9rF8eo_nVjF zwgSu(=N@qA`J1Ad3&)QYHOvaZo-8bsSlU*YdxGKAA@XwgvHfQfU9@q=ataOkPnz=t z3d-wm3odr_tv{r55F!64d6cA(kPQ-UGBCS1;wqtMsCxr=73U@3o+*``;95b4Mco^ zE{v*<+F}A>n3GbwS5*TP#-#|g*!K4G>Xc6sDxvE?a`1UQ*9--@p~jPlZO0c&Hx@>w z@K~$o+D8T5^g5Yd4*&=$2HR%`N^Ztz+tC>^xhTCZpy=&e>TM!XgP0A@Y(S{D7VFFw z?>a^^f;>hs*?0$^wyh~Vp1kjV-OlIFpR1VNM?Q!&B0j)(`Oc*N3T?;X<~Lu*B>HbJ zp6sAYhP;pa&3n0BApbhjjGH82--f9>i+Cz3j2HOou|u`z9tLASc)3p%2RC&m2_q7* zrk;zI_P3*Sj%=py5{P`P1TaQMXrjK8*N;;k^WZ=6yb=qvnv@9ujq*SN=kA-r47NqU>$AEBNJ+(4Jf zN3!ur5tofLX2u$~m|$paBXC$*b4lO4`NhcK zZh9{+1^2JT=YWy8;M@a`z0shpKfNnKNwp3e6!*t(ADvSKWY6SkmQKWWJ1%nB5ze1I zYk@Sm?V=j1(MYxH_Jr)eC?Irszf$RA8vECL8-wt~oXHd+Sf|hr220NiL@GJ92*&tr z*juw9D-#RTq3tD$lJ)>Q7N6E%S7|}Pdk)<<+Z$UHI zAQ@Of*9kV_nSdjQ>mlBf7cQ8PGF02_P~u;5qN3!WeYIPuU=;0COwZB&YUz&fU3}Lo zKMp=4Hntc+;eR)bumwGJ>@afy>@J5nIEGy9wX3ahln%2~rxHScOxEX>d)wGw23lCj z3<_j<`e_u;N#4Sd5j94cTS=Yf{4YuXK%Wk^Yx@4Cvx_U@Zt>djaf zT|F}Rq#iKmM@(+@$up(syDv6I_^&2?TKkc0V&@9W?IG-K!~1aTbvK`req7eG;vku_jOHPbp(P6n6QB#wF7tcVL!E26);*QE`+=gZbR>{ zy-<`hnjlhGW=XIyPZ)dyelEj6F7Q3v`-bWH%Hk6H*VW1eb}Oj^YiyVrdo`0D;)OGAX->8z>SO;}c`QBd46W%EjAA6KksyNyoH`8zf4|ttS-6G)Y3eAX+{fZFu%a-|MA6aiofmSy+17ewm|q0;2ize zb$==z4Ic1+T=$2->;LEPhJXFdzY1$WSU6Lt7I+?mbm zIj|?TS_H4BKd&F0a7|aC5^mRytMpc9S}u`80%wuWxLWM{piJ z>Q#U$&6Ke5EnV(d3+VcYt?iQ^Hrph(2B=lVVkVUFh@~myJP&zDSyBAr#dlDA8eVas zh6L*aanzUj+ACM`PSWpZFAzGW4xsd?>e zZC^tsSp`KDXVBjBUa0;l&g!tLo+O{tLW3ql_S=xek?=EgF5p|Fi z4)cL2enUTUoL2kNb#tFfe0q9OLBA9z6jIAXIJ3iQ7p9C(vi@P}1lTTi@bl`BHWaUqD`IwK)0o6hgF|k@@8P=;-Sv;`D zE-r6{=K){6&voGU0xkk_ZX@S0Lo-WG0fTDJt1zHI*Z?4fig3ok z!GVD<4l=G^CO*=X^8K^fV+7Q1H5u{efm923QoIYkOmJ&zo!jl5r4@hVG5?CzoO03g z!os&z0~UezN%rr1#*ho&RRTngg>T+m1(>92HXK?RHP=>XCDwKDC{6 z;`tj3@Dn!=4`93LvaH<<1z)dZBPJ`WDVE_kdDH*e*L?O(Nb@{s3SH>|86S?(B#+Ts z@_tVxZ`v;rwg@%u=H>L@yN$;O@{K&9edaLfo1pEiyo2}S__$G=Sb8oP6CxqgR}A`T zBrVoU6dd6mgS1P%y}XJZ#UYe#5Hh?5OJ516b0QdBzi^*8p-^XQiMHjv_#uGvq?Bbb zEkv(w(ksvkLRy3Z*){AiE42nYNvW!C=xw#1ergOT77Lz7J->D3N>;c7E?bLu&2QhnftYn@wySZWbp|4Tn$dfm z|Go4rI$GJBb~RfLp1+lHk*UwYI{-U-_G~*2l9@Xmjt2Zg6@?n+G%g`pKge(zoQlX3 zCcbT$>ONI;Oz4R?r1A1z7zL28zO@EY-}m|1)L>6f=7A#`>a$eut@;WIVdZH6p~@%^ zEMwuLM?W55KTyzp_okg)7VDV+8%0$RUz7vbqvfdg?O(2&n>gg0KQB??PYggXwtfL= zx4@lkA`vZEB2mdDw7NL)0>{ zpNw%LCu_h9J?gfc4jeqF(%E>8MJq25wsTVw#`KNb0*sTzHC0^!mZTi$aXHt%N1_{h zTPu4i28%AZ20*n(M%8&HWn^R|hSRa2Rf?-?nam$JQfO?b0R{YlmaA*|T~f}EUcWwt z-R&y}sptHrT?urx^!~z4WX!*IdE)GT zf1FK~9#X-hv#*JZk)h`uq-zEG*D$|zIpsT2_U3V|`th<0K|ivy&r8RuIT#hX%1y27 zZ8EqzHrvErY}V12k;3q?vgXwvU4+oe(AuDd$F=_5U57b2J#LwSx<#Y_^97iTfp5r6 zwcBjCzSKBty>ukn$4}L!G9+3cY+`R<<&g$wM`xOMfdsAMucBV zIYt^99nX#`YxgUT7cSPP%ufNAn=RF9Wj6v<@VU`Lwe3es2l>J*07-Iy0|zr#iy4_SO)}>mX+EkUuf;8WfU_RMmMis ztw#mZyloVw-0}}`CFwWb4jc_m6&4p$G=hZfJzVEHaZW@;>v&77v7cWBDr6E}OHCwJ zdSIuXdM6~9s(4M@XOyGS8F%gNh5yI^*w_p@ghp~!rVY%rH=H$9hghxVA(Hz(<-iTSHWPMQ^wXNSk zF0udK?|7E@lS{Fd$fes-d#;ZGc`~=4Ao`|IWCu{?-HWLvpu$0c-1h2JLdL~yZVl^! z(s~h~bxBuniU(z^nA4H@6i7yN#Cd%Fi8)ZJOp-WP}WF&c9TGL?!0H6)1l3oXgb76O*`=*BnYPN9=)9+90k2p1&VVHK&HgI%f zFa8xOFV&;o5KykZu??mwGpMgxP0XEp`2#y|=HQEyWu*rh5h2xWG;I`*c`1;?dEkx^ z;xf1dF7!Q#PBpQ#Ja2p_lcnP=Ev3^{?mZOmU{aa72Zlx?f3TCRc)55siGPA^_TOY`xJ=7l4%z}nIs4|x3s)9$Ds(7p;i6< zvNC;uLCGq7w6LaXeCgsR{9Lcl)@ED;+?%dFU{`TdNN4UL*U7o}Vkk3B&4&o4e0k}B zfue2x*RQ7xG8DAXSY}>R$gyL`!09(Y-b1H0(#OV&3Ky=+wCpO5_0^l(n&|45fuSCp zHX8PcO*Fyq&G#C+bM|x+*mlXCo^742q&)>A!LYABKGCn}>%G4l$vahmPUX1nl+1`d zQiQ>RhWq5JSKGh}j?8`~xGjF|P4QXF1oO)GeNIN^maJ6cwUq{nCus6jEq1=PXXg;1vQkNH@UAo$%(2XM?G{ zj>w*FAG~S|NUBu-({D`!!17{t3A<0TOP@b|+9Rnrehx~g^UPSTp3;bXmB?Brpl9D& zRh<|GUE-G2tFj4+{4#2s1q?=rxtpN&1peczW0ZSFVTMoEaQ4eI0Y?*l9p9~g$(>mP zDtyZC56QGguY{BkUxke?B{%!{$mSVi$^6Gc zj+ZWW05q=YjT^s0RVZm~hU9dmHGOCwa3m|?!~h}9fQ4uIDF5r9wPNgUk;Sm*qNS7 ze{81~H>ys#eS0p%`0 zoS=^cUUQy|tSn!ST(%O#Doqf^T&yp0n9vQkwzrp2D+DGa)&QN}_4`S4^j%w=s-+QhbA{TwpFE`XXT z&`B4dDJnBRj3S1YRJI1Po-a&at7x%zBM(a&CISBI9tPL01qHI|4C?B@V^g~V6i)g~ zP)NYVNz~Fa%E426YO~!dH#hCSFWpiTOoJ^b0YECCWzk7qYED#<=$a(QcgmdR2;j1^gsZpo5<@<9J9aVt!15FqqlxV`!e6uE&1 zc9U26?xO}elbwUFXPl|K{rJVnI^|FHfO&dQLK=bwajl3l1QOm5e)zDDj^E|sL2lYr zAv%}Qyiwh!;8!^Y7ODQ!PItc*17@zpzk4<`Wp}_uFSe% zb| z0Ag_`h{%reKRMgLSY`*Yxy1SNx+IjU#W-T`VRwp>Ez-m?HOaXzA5JOn=W!sGes&Oi|7?(VLGOtOh_zuX~_3m0 z>g`>Cs12of0D9NZI7cw}4*%>}P3fA-Pe+g53rN1)|LW!z+09*5O*C-*bHl}7KT!Px zH^bbqQ|jGw%%)LBff$NMXl8m^TujUevb&l+7DDH@32BoX4Q?#~$q0a=a;4!S#udX- zbz)pM7`4I&*xB`v3kzzh8i>X% zzg7i_t!d&)#ogw1!|3;wYBRx|Pq%v${v|4^JFIeRHVzO6afK2kq#;fMw(KkTj*7RP(rgaf`Eh)=^(uY=_M2u0qLDU zXaZ70=rtj5cl=Iq-uwRX`{TFPUF+UAYh`AwNXqA&^EqelXFvOSE*4fZ?DZ=As)Z=N z_R$twW&n2tI{7->1Q)V(Uj+mf#RouGcnN1@W%banzVkQxA(W(OBOXsVc?Hb2%;Kx= z*QKa6wpbKkBa!BoFTX8|>)o-!t=H7l9RKaNQI|-E4CPk)D<7|5io19n<`fPu&>wk& zp4}!$qkL{Z%2U)=Ugesf(|`iCi=jMUcGD+AtxkPkOBAk;AkHMNc~erBu2F#6-n z>;sXbMX%CigIPP|W%IdY&dJHCjn@V0+`sx1r}+2Ew2Ig;=lV&~;RvhPjqedH$goKo zgL|V|rkt$iOvk~lz~YnQSNo(aH5@eOfpU%`b2Hj{<7x1g<_7RjIw(go0x8*^=g_xoXJvUC;HxlNG-x?5a zd6aoe3+3n)G#R^$?&CbEqFlK4>_7l_KU$!oB{jRHk+7Y`4UKQFP}_ORE`+xo(s?5X zrFScwS2s#bx_Wceluymwz?wLNT~LXJshD?WmwBqOU9_p8VH%vyMz7zm4faDS$ILcb zZfhwO>pNVgjIAXNa3157dAIw}(s%aNRBex|v_E$XPx<$nOzRm+XNO;U9)E{rbK+RZYKz_dWXz*b?_7JlPAI zC^v$>^6si-!zHnWpjeskxQNt%G?Qcqd1rA8zV9*0*RD;!Bw}@|eY~Y*BZ91mO}=D( zFnW$t=e%@nW7yfx2h5Oq^f^pxsvb}nCMKo^6f7+)&dLuzy4d}EQ<4HM39JkarZn6- zX9%}ZXzpe>k?41_M=)15_eOz!S}yq>k1pHp}}PapUz`>~t>hsx`YyVQ7K z$%XgZ^)g)$>FN*VL*#P?cvIG8vb%0m{cvP04OtOvaTjb5>0nt18#g{FST-Ro-opgi zW5I_q{OPl2;gKO&0QQPSs=-N$;gxT#e>kDiGZ?i#m#D92dJ48a@6C>S8k+ic+RE5{ z)FTR6BqW8T1aTU8b_erBjbnW(1&e~$XJ^|Our>>FI6*nS-Ew4fuVY_24gG_lA-so> z0Ct+Uz@+7->T7&hfN*!MmPjFKqb;itR zqXX2yz)?!amt%4DudTS^`hpRi<{0(R;NW0xey9{^rN!iehm&h#@!#77;VJC^O9alM z{~zypZbFNg*PcS|9Ft0Q+M-18#ib`C{I0_#P9P*IPM{#MS@jzkN&EWs?vV2aXE_Bs z0MZ31uvB|`duNL9BkgmB4tB*pXJ!|V1LKfPXlphqtgUj~0kG79QOdI6$T|H?+bGOT0x~knQc`7N(JiwbNzvO>>KYhB5?f^) zNbQo+?{eCrB}|i)kLVd0#R$itLWJW8F2_Gr%ddyYZ$5lNU4e1JmrbeK6t_Yu-P1f$ zv4)z0D+zc zsotP#jgxo18Mkdef(H{@y|0^$5~hs0O0KPXAZr|Dh z;5r7D$Gw{0j=>+QUOmr3_p}{K7Rbj$?8)8Ko|_8V8f+y0>p3icWRu0FIQyI zR;E}oBCpK1N=kw{R!TBAAmjFgY|QR@=1mnwV;a^?xdSm+{BKl08H$#Mj0l=Y zS_K>-kxD5*uIXF|kBsa{mQ~n&yt6X0vDW6cYwz-f!F8UH(qv(jGcY-r3{6dEB_;cY z8A0?MMkQiqi+MM7aBxt!XR7tj%>|~K)G_Sos)tfK^B|>g=(xAYZM3-(*K=aoOlAYTwB25_V^i;tBM-+~s8_>QptC3*mg#xKpqE#irR7qS%wy$nwaY#MCUeO*q-k^9+0}KN$y)ajBExll zkSD8$FoSeQ!|s7pQ45D8ZrT43u&vXyLb(DjqE>InQIqYt%PG67?_qvZqR%*~0SjLf z9jarQcUDBi^u>!84zH53iq?X}?rlrTEl(ZbM(7!Nls%W`+NJF) zzsJ7rr0AC41O#Q0sgw;*QqjU|ng8s<-F3ia3`fUg!^+9b&K?vJ{WB=|khfKeBpy6i z{L$(V{R&V2&LyY95ug2cWfxfA9j80o3xzdc{+CuVXn?(ZO3pyVnR_dDTIDSBxig^Z zsZvcHZs{nU|L|l_FiT}QZCE<9^fV}|AK%?~d(mM56$U9O5A^aNr@!*z^Q!i#MGO+! z4zja_gnRC3(=IoIFNPWJh_LkJ&(x%BT|GUw$g1yW(^!8Z616sK9q-Tr=TlK0)`cX~ zMf+?QOS$h+XGS0tF*6fB$SD9x@6U&o5bObT#zAnpPTxMmgD~|i=(bg!dS-d^rmA-5 zs9vbVp~VfYh2hBZ?fe;ZGN>2e>_4l0xUrf>SHZe5hYfHB&%-|--#@7pioR#hpTCSQ zd^mbvLiriiV)@&v@|ikflu&L)Mh6Xqd-dB-oPImUA4`nK4Iv*n{C%ojh5}$hlN7*( z=;XOM4p10vZRz;xe%r(Z8(y8=OI#W01aia>C&S%e$Xay9zy15`A8s!KP~`Z~?bpkC z`1ba%tIl?IPr3OO6wfg?=kbmmD)o0hrD=`*dwtMJo>2Nx@BjP1eiaA&=D+;i z|GY-GxjQ%H<{sTMw7V26Zr`*#92D!$)B3yTo>OM%_Y&s8`N~gc=hvUIUSvR9m5Z%4gdLz?fpMWg#PC>pJzY% zH~iMK9mKYV}f@yNg2Bfh`(!SP=zp5I^lD)BGf(eJM)qafqiZ%G>zg zT;NrH7=`Vp;)B@dGvnrY1KfzUiEt-v?S|W0X_KkETzUsqtLZ`!Ck)@5yXU&^rJsP9 zpic2~s|jBm;>d(fl`!LN$Wnf_(cJE!hh0s*Io)Rqh)BfU@%Mnw5)Set02ZQ(@9tjJu*3N&`pSvV3csX!x z=depgUa<+<>|oK>4?2XjH%!ZXpH9B}=cBpBRfI9QDnIQnv`EuJn-v;(8U3xV`2Dk2 zIsd&1spn+Rsa~*D*E6!qZnkhPp@?6)l$r4UmZ`8|A~@;EXVaLTi+!0!G!9@mb3nqT~6;F9@rc{U&q=$G`t@>P^@;%N zYbBm|(d*#{7_ufGCgwK%Gw1th1SIJ9K(ZHYRx)}HjsSuY06G8){E(@QZQQEtX{*j{Vcb%gwn(cj>8(=k5~DzMDI%v>-_vh)Ht2$? zM2)HFEy@A2Ll%g&dp#$VFMxh?c4?5;)G*vfeZxz&ZP zdB{Hdm;l(Ys^UxT>4G+lhurqu_aO}{-$hw4=2;I{T56Go_uxFU>~p zQnF6_F>2LT8TDXvS7*FVc)nNF4cPO#YABI%?O(%~Gwk3L7FuBH>q%ZP$*4WX%9?>r z?#vgvu5|4zzoAJud3?@1jA&aLt)md!Dz5C+xv3V2^DOgyJTQ+=|Bvt`4Cll7Lr z4xt?B$HvBSQ$wSt!lMA>!ZQHcRnFa=imj@m_tv!R{X-R+bWiX5`DZIo%OS51AVHo1 z#OrRwdarw#C#TKktB7}+M~P09@OFvs#0^n=B=gf(7xqvEm`6bFjA27dGC{&FrHqj&bh*Gl_2*~&QT5F?$}3%ZMsue2OJ z@h(j2!f`YUpVq?!iZg7dTxGF7J4QThar5gHsRWb`mvH-&w(5|icLZV)+c}FDDZE** z({9;b5?sr^a9mu>`||z-o(ycdcG>lmD&c>2q{qzq=PUp|$tuR0fPH9BWP!RK3KcbS zAi=S8MdM$f;-KkWB`>TsEna3z^M;3OSeiOmtxz^nM0wk64W$PHBQK1Ta-0}Cl6 z=cj3asey`VwdL<9{(x+j+cWj&Iu@iIX$Hlg@2KrIaEftmp)%&*^VPZbpOm;hoP2KuMK3LsyNBQqL^U3Ud^tW`Z zj+#k`4ZVLQq0Nk5^P#Gl-#E68S7+)(%;L_PSyjT@8<7KEDQz8&s`<6`E|Mix*_uKA z0wvCb&~bAOL|DgtUB&A~jcyB}Nttz|vf}3~tT`uoPGB-J%5~SnuJwnGGyR?+JN3qf zfq7j&_V2xj#pqMbr;E;$F159FjdqvZaHbC)Bpo_Lzc!K2l$kvjy%DFW<=1F-(WdPQ z2WenrxyszaA~V9f=o4w{(W5-EXf*+TejQ_DZ@=~D&)r!A_cNV1(KPzd#fq`eAUq^Q z04I0N+?$F!WHLh56mudSPQ@GrLCh_kZ2~@mU#a%QP1xJ&e}hyg9c1D#%~k z?Ar6F=;)%cXh)es$H77!OpnCb^X-|RpSzP9T4#{4anowCidtwMM3Y|jY`X{)H7M(x zw+sm9kFZB{vi-4Ihdg-bP`Afa91p@^d9*$g>$)qC6-nmi<3D}cNEp*ZVrbmj_kvJ* zFBJUV613`G&x*Z|4i`6C{}pVKM^d7qS{IV|V4Wx>UC}dol&7kc6ydW}#Bj2iS(yi0WH`^WkH2G@QaX|fK$>BlgttypUo-HIGtw|!-AiW* z676@}8!N1?1b=E9GG{oXJy~mMU1MspK3f`c2I{*hp6eNQ8$Jk4aIQvo{4QneK>7T- zsdJhr5+UB*o0wzOBy8_9uVCY{;`fe^dcW>4zRys2;h~THqp~XUQjPI&NnESM*m41r z#62@V=lWqUeZL2|;jL>@3%(umZ-qp4(+1sNebV$Rl$p12n*A^(Shy0nd$Xm5L{Q(F zPdUVX>6)R5yH-fjg)m8^5pKOAoV{qGR${_Ndi`MkaPQx-)Z>o;`WcMiD7yRhmF?~y zIM&eE*xk(^x(dbgjO5Rhk-n(Nj^(NeV914sI$uVIN!z_W{L@cPoEs;jxs82EOaQ2M z(2G>Q1K7vSs+!u%ZdS05*ibDGZr}`sw(N4KWZ~-7tNjE!X<$-fr%c`T-9|hD;Go*s z{1VzIGF_^FTSLe+&p=SmThNkkp|ZTI{cC%)qfkUr9)dwRX}}p4788qAc>fr6BNf-w zCH?H_9&@pxYXQ=RDm*FXH&xhqanw7%YxYH5%$TS-9IIdW(1fVo4P)+&i zvB+-_=~}&DPzmeKUzu+l*-<-U9$v{N0!L!k?uM)W+^IvHQC|a(4!ml-iQx_>HHRc+ zD)nq#KlCd2cHjF`nP24FN4FEwe+k5m-(*$eg$-T1p_N05IBU*uP<@U7@fF?~6Dbdc=qj>_M2c)!lcFQFinnf8gO zB0!*Wu&}0DGY+3F5V}+Jf=6KRy;@+=5%@vLeM|am#dlMU)G@BrTX_Sh@w6bV>Gq#V zyDl%-GHs8Ch-!;Icqw>~EGe$7V>@Nb<^DtDb6EYqMzd~%;$;^A-;a{Du7JAQMaTJb zQ0W*j$rX{S6hr zN4Vm!DkAc2wksS(UbT%UH!?uwL_!enLO996k-iAIhtR0H=~7|q<0R8Zf*Ji2^6o@w zr@_1DS5Z|-;CUBrkcD~wI+T%=Q(5o&GC4OM81awD#~RJs#Us86Bz3(-P1RI05GUJhoJ)AaD8 z50mA-omFKlX!!8`fPCEIDr((Lfo_op@{;{IAI|vR3b$NXKyK=zXQ4B_+%|rUFYxx= z=WYr}tNeWpaWTKA)g<~?3RRn7NX@rzc)dQYoLY9|a=kiB|51Tf!ES$3*$fMOUhJ!l z3Q9)m&h&|W*Izb8uw+Hmyr`Vl%eN1{S$ywYBd{YqF!)oxyVCP`l_!u68HIY%`TiJ)gX6mDA}Dz%=&sEp>ef z(a#_+D>F9T?pO}LCxsRhew($rGI`dcSDWNRd{D*~jWXu@`;6J2_FobRtBbRRT4@5p zm+A&m_$4K8!GY`~$szsl{e*YZTqWf=J9|>|>x;Sj6D;E{8Cw(}uGiM0=GrteM!)P= zPUb3=`@r3J0?FfwYHnU&rgdj7=k2)h76K+uiL zDk2#oS6i!_`LN6&8|kp#EGv1flvL%sRV&7FnC#T=#3|XnEPLdaU#y0#tBQ-U8nbh) zsHNVuIerb(HmDY8Ya6}W-rf^{l> z9`cO%#)!dK?e(z5 zaYJNKNYA8#PI_lxy8G>RiL(cxO^YgLI^1|fpg&)w$KyKbQyV&rvc?uzF>AysWYIsC zq=S4vF0w>X(C=XLkr$X6&kgq(j=4l#t~9vt&ulSfea}vavTypr!S)t`p7Z!}!EosEu>tEE=7$~#B2gDS zQ@Lll+9I|}qgvOJ)uUTd4{(+0J$a&AxiWu%jPhu6-oSXl9`t}4ruLBLx)O{^gJ$D8YLrp=KB7E3?zOu$!0l#8*w{Hj7d~hCk5x!9ebxZjVjwSvVtsHaVqFa zNi9d|U$ql+U1q#m^rDtCNQ%hTeMd_Ww%h#y8@s3eK6B|VD2HH@)S)w;!k zmJSjbY`Yk-p|F{Y#Z4ZfMfa2ioFa(Y>v(%Vd&!{ITY6@6`{m31pt@1n^oWDOkAp!O zGqtJ&4FRohJu)HOGV!Exila8SSCd=gvf+SigUxTP*~Y#Mw6pDdo^BO{hq2@q+2Wff zoS@aS-FA($?XR2T8o-l54-OVt6LxWm=)zey-_WG|RArL)DI2pUk>M1XjpCEe+S(b? z)pt%UbsTDA$?PQclgP)h16cpP9*0yA?cH2M=ai9wnNIXN-lexD`YAevkNjegM>r>! z3**zsp7Mx8uY+!ltuE#BeE#BYba!hys5|x%=hnBWr6`KkhU4FN#odH2vTQ8P!+2D7 zW+ttd*8kP3-uL$!2nBp;p~G^{tKJf6T{Fc)6E?k6(~SnRu)T z!30F`Ay1q*F+*~bzh!3^Ccg``JPs5zKMhPp``DZeYyMvxQ^u%cM>Vt&!_Xvfevu za3W1>(DE9Adl`yhsMd3lPoH)l)w7S_FQxJ=9!(9>dZ^lzhQH7Ex))lCNv^p56Ui3h zDu>#?9dOXqGLFq3G0e=Q`pIGYkDknYv&n+cTuncG{q+rddt3Qs>zXgwg$zGUTVS-Z zxKK*gQ*jxa4G)+&jwPo5M%&sl6ARKZ@>10eO}tcI5KBB#X!BF`+Xxh}x`V&pd` z=^kyp5$w?UUw6a*k$hUrYrloN|G;p#q`6sD3q^k4TbzVzv`9|#0L&rywbC&ucO|;)? zDT5=$bxk^F?)S8i- zZdW;UM{D5TYb@c}gLlUhA~a4ba~>3s2-CBr?6}Ecr33_X)?CRR&VzBs`X^(GV~#M8 zd{!RNp8Ej6<>uE-$=eO#`P`f3@%J)+jO7@ry(lU9r9RxnREeCo!Oh$pPg;+Gf&_5_s!qKl;8+wh#-&o zAH1)*(o%WTda*+Z72A7EF+C~G%@H5If~k5DlDO__Z&9ERPOl*^4P~wJCzjH^$P|d#Hz>72% z&>GTnzhkmxpwOG5Zi?wr{}eJ*L_tgRsu+5E$jN!j5jEmW+IHp-&{e@!`yoN|Ju2Xk zF6b!5t_?bJBaWny6pRhL&+qn;6t|>J^hifim^wok5AAGB#=ECXehCrUJLyxY%QLX< zVx=MH8DaOR!=EKCb4;z?XQm{L`-@dxI&bxzz#efDS(3=69jnJ}>zolzT$qgEO2}N1sytkaEViKpP&!(g>bggsZ41zHLkBOc#0l7Yq z_|wA4=jDm!-!X7?7nYm%9*D)QpA|=@t1H--7dwtXkj98W&#nJz{BY+q6a*UWT)8sm z9hptu-_YXKxMgpj1Kl;(nNBcmr29g!IN4^~|DqWBWfnxXN<|vy9>uDusezUzaNE*5 zHT^(Oa{bC}2u;JjaU-1A?dE5^%KTQoxkxi{aDaf4HmN_f3p)fcVMmhE4KkX&MR z%H!+<*<}pQ6ey)vRb5#)x05hHg)T}n#yne%brUI$;Uff4NG2xwaV-?e!J3iq-5@vw z_QXni;!Ft3t0-4YTCU~r-GY6#qfg87{a#wt0=QY^nLDuZ7dqJR6{?DN6EYe^dh-wL zrkgYF+!o*B+@+Jbjg#0{f1UrVn2~UrmbvFTPA<*PvWd$(M1O`9lqM{z*yZDuRX6NZ zcDSB8$|ZdcF=`F83c}O-xvMK-5|dd%0wLVB>KsDICFBKVuH1Hjocwr;Z7;l4V{@*l zwQ+E8#rFu78~&3Os5=~Bn4ImX-^rxOL)8ZjH4aBRkEzQaXJX0#?JcQTb)da`AW3=V z8_p|QC>h|!4TGLfTQt9Y_wCH66yng~!`;`JW6mHN8~SsjqSm8%02sKLVr~w|Y4?!Z zP&U_~Z`ynrLo9mE&ILfpm7(S)7d8`-sPg|zt)8PvTgB+0b8F{o zRxMgntFiA;c)o#9Y#i|~NOBGs4G!JmF4TaXSA@z5tMsz_rdT8UDu?8p&R8inr@G_U z&5L$xWu}tM{W;@PraM@rS3)}f=$2m*Lulf65nxF3Rvgb4^R=QX7@~nLaD_il`i{iK->Vx`O1^Hb6 zxxG2~Os|D2FVc8y3C~Idd`TyIn8F*5vc2njq@O7yaN8L1;%QQi43tLxwJ3bxCJFc4 zJY$OK5UROgI2<7BGX0(-w!$&8(ybbb3fZgUd%nx!cJm**XxDdZ$QdsL&CdxC$A+P0 zuU#^H&&I(}B2z)TO&m2U6I6ty^=kYq&qA~$j8TEwMDM_YSHh!l^`%+w?HzBov%I`@ z_5&Ke%a@7vel>~lyUQCn3c<~6!iYnK1~icm7wH|4T3(HpV8+=u-LUNs?1h>E0Qqv* zN*URll`f6y_Vkb%CP-On3(m=jAst&~(x8Yw&dLfr9Zz_IMSgzATBM2kK4ZCv@IyrN%hXHkQh3(N+|Dm=Ux35jM_sX}X@-kuj z-*M(viEwTNWTa6KTNJ#u*w{6c+Hrq8+Q)ZQE;ctuu}N3(vS#Dp%%Pw_2Z@o6-;233 z%JhjRcm49Q(t1mwb!V;Xenz01X0&&rga})+N7X0|v9YaPhC|q6e;=kSm&L>kKsvr$ zFjK}ecNWgfxHKF z?RmI0uHDffW;`GQ(Fz*ISj7zAT+5qIXbaW&&ZEsfl%bO()hY+75)>W3+&ypQ*eV|$ z;c8;*h_`fv^k*2088&L0pKT#VN`Sdq)f#G^5Ibc3)@XVAKWI?+9Kk!JY$^b0m+sQ!j4!T1xdR)@U9CVeO^w+~3LvbdT3hZhjnscK@FG_;a);>CrAN zJN~%c-zajnn~TdMpHW~nrol`TYT>H9&j6Xz(b6JzNdENGPpD}(1ruY}94r6^u0yhmI6ASrmW_NrJ|ws^Kw2EVa^P8)21hT>SO!#00YoF8Djfx+XH> zaFLlAa~2^W?BKx>JhB-SxcIoL9vCzGbD}Cv>x=m{#vM_JM9t39vJFQ_+2dEByqhWgb%2rU?=?$TZ);`MU&;rOa?%cLI@}E%=NB*0K~k=1 z5=R+|^Ss~J9xTsV*)gs}}KR;%F z@Vl}5DanUBo-d7ma}v)2b9VM$pflL!(`KH~`0*{@heF(+Q&-xI`{v(@w`YErZ>&=@ z)B&J7ptE9f+FQSB%#cgZ$>}R<58Zip8Uw?M$&rvz6uhfAGu4b15>=l}_+S!7X({uLshEwmsp?#&+hO4L+8WCW^_`x$9aXwjIOj827hT5i)nS4W5Fd;(RX z)x2t(&d*D<4!eH4dDoG;IwdGJS(D0-G|I({`01<%H)aY{E_Mo%f&enkOh~vfF~KCC z*@!#oyUc2ed8j%1*Twzt(JYstz$mz}s9YsnGsO|4WIt3sgg1vjxJxHSV^>N3X89~> z;^mVzJ38Zo=Zj^P4dVYn%4nbS@F*=M`CxlCH;g;-S&lsRXO_ModF3E^%OAc4w3N(f z0rq1(Q(YcJzS2ud>CwA)>HUMyE^|`(ejC)3VABw%J&nhTfq=w+cSJ&N%}>nB`}hvEC+cu~e!V7Nl7$B+Iv(28i6dR5V^@Ga1&+81Jil%xc* zCepKYzf%IjDCPeM$C-WC2?fqZ>po78x z@89!(F~#uz<7od6LG%A_{^)nQEUdf3b_K0(GgjIRll}b#%48TFm_scF(h4*S#F>b;pP5m1*$eKAGfA%qSxn#vK;ktNvngtT9ubEBYhFEqSFPZUEW(cwtD=20=bl09E}9)hz7ISO}@yk8QFVu zQT2c85P?_mTc~D#m55Qt08hf+?VPBniBi&S=!XD)h^<`Q$N|%zM!LKE?)e~~lWP_) zT5&I-`^m8agT3O&Sr7=HQmJNHMba6wHWLd)Oz};PjcWrVa@q94l6v6)DvrHML&>Fz z98O-#vl`W>b$@U}Gqf7ih0$FzHui31F?<}){+pizowdjYbG>4j^_{WZvzIREX6fY; zS8~1ET3U$btGT@kysUAYEELpXN0zfl(7{g1HN`Vmms|@NG3+EhYQI1FX$zeU-A~4D zR3Qj*7p*u>{&1jtxNE+M1Osc8mX;O=M>xksDH)M7Aj_ce!?}L`ZQfoWW#vGP zaLrXsEr(H1O6vLZ=T^O}X|{du6QRllU2I*TJ_DSWa2hwJM6_HC)qNw7Mbrpgh-xw5 zlyy(XKXBz05Yu#?Z0=kfjXae1A{`{9a2sYspyL7T)Mp;$OPF6 z7;>AYUJ`jMAh#Qol5#|4!y}?Hm&Og}DQHG{81n9I4`*6X2OPJ)ZuZH;hbcmjpSyh0_)93_34iTJO|A}_Jf6p2TH-66&}vC?ntPE z+7DOg#Cws;puJE-GJ>xS^JKg9hl6ta00cnK48XJx!bo7q>1b|Gs6o*6RyoK=%+xxb&E5onoM9fVSKrEofAY3$VAhVKumn^ zTAzzs7obkHl&tyragp*4EWVU2xyBYPvrV$IaX~IH;JMibbdRC`g&jk zf$qf`=9=z|rGfawHJmfHj4WfYJ$#h9?#9?OF)Q0QJwr~<&BgCksL+6$n>On{t6w#w zW5;(W3mtu0bx;%0UJ&*H=AM~Zvd{KcMs5UDd57E$eeO(z(7-Culv=1vtcfOegMJP8^O*wMHFaRgp=?VjhwK{Y zvs|qJN~%Qz&J`0fdPb?Z`HCL<_O)Tp`EJwQb&%3)wDmTqsoWS&S$u!K#2#$icabu5 z-uc2+Hvux3+1GamJRl6&#BH9>S1-i5l9){%#yM0@@KP6_?54%y1{D@OSG=TH zxqi6UFYlg#5W77qJ^PlbOCdOcZMHG^lnOqnKPIh*7 zr-YzhMs~JpUS6Iwse2~hAm@8?wU$+KyIq^C1M zJ#S&!{Rq|!t!WYP=o=m$9<-+MiZW-jA_H1cccwkDCibQD05-VBV~b4A%GJ4bCmH0~ zTH+;?LqbAO-$q9^Y`j-LgH~8N*m8MSt*UU!d)+yB@L-e&IRoSeXqpV*>z6yVyMrF6 zs?_I#8mPfi%gq*9>cv>sSr=%_ETb+e-M?Q^7bgZ1#~r>3;axK>hD}emR#u-K*ohbG z>~f*lOV~C~cc5l-eS3GQlcxY>pMkZb>wf1gIMd3I``dhn600S+FwiV}8!&tFT!Ba! zCO!=$L}Nu)E=IOYs^k46@Bji0^|DQ#{Rc_nus`YN&=#gKI%;h}&t#!M+CRX5!fJ)h z`eGCS(EUlG>ZxlT>{@B6ozE0(sx3gX55z4}RlB1dD>@T{nKG@ocjD0x(fFnWE!nGl z`gBkK(`V3BI!V;r*1VH$yfNsFqlr8zl6M$=vw?F_Sga8vc_mnOTc+RcBWm#@1=-Uh zvEIPK=d@*xneNs~Q$a&*qDdZ@(ks<$t7-~x(BT7S}4)ya%(jzX3TSFOrO zgY!8tkJ*%rF>&AFfARb|s6OQO!ewD6CnsI0(8FzruLl{-wHhvJ@a$~RzHf26lmOIZ z1|QH8rkBXm8T(eIW*f5{Z@I1y_dWoHJDPN3BJ6u0>BmIg5X{ZMVyld+hp~(Z)YQ<1 z_{wU;*Qrkhlf)wYZdYPe!3mCmD@9qct??e~9yyy9;ZzQV-cBKi}UQ2bITivm}0TRda72}s^D8vEeE|sx(phM!6 zya{b1gW8PCK+Z@Hnuja%U4A1Gd81PYU+7x_#E1^|+{!`H4jeJlISC1Ofh<;LPtU-h z`|#mIzwuaatHhFE6~nYxX+_7%HS9^$rUnT+K+{|V3ugU}j@!^48?ZL#EK*3s4@@;( z_AvV4sM%?u6>&b`9mw;X($1|{LD^2PO1M^}B~DHS`hC%>ehTsg;nH&m5KY0;i|i<( zLEZ>E8X4&=rlhCoccg2`t?2n7Jk~fDv%6*!Oy0O=tTM+>y?OI%Agf%aqJm>~@I+HJ z99$vXGlR}2%5oe(J}pBh8{^ikKsGOSIC%T%r%NT!8*A!IZOKC|8w6dy?mBp{cObZl zi)+gn_b`ha3TZ>-O^@J<*dYJ_(gcL#)H&x8T#W*?c?8+z=7?7tsWkT3dcjGkyf+_n z!94FR55jk|CZ6yjJ|$vpDNUIR!@UpOPG-CLy*2$LVpNGqCHWf`M2%qij`?aJt0Ow& z=7Q{fQL6{#$bF{dlw#%GZ8JZ{*ow{0CTaKcdYURX$nJY5w*Kjvp<@}OCClxC{QUfA zZm0twiI+;{EL#Z3oDx4^&0vQ}Zt81wEU}`Is^lIK-^>pmF2e>e!(2^P^LvkxupcN^ z+@*Axx_Yz@$9U(IfE+wIJK~8&S6z!ku;V~g!J4btwbJ}xTP~EDHwX#aj#N<{>if#= z4T+?%uy67Mqm*Xs44dd_t;k4L($!Y@?@bMr zP;Q~EuQkF`Hcdk-W78pzA3tWWANDc2~trw?I?I|uv%IgPaWBm;R)qO@K#D3y*jOi zV!HM7^^{+pzOx-LMj9K}7G=w0k$T2R5)x^Tvt|A{LsK#2G|#WU@IU8HZcfI`kUV~4 zxnKsqqN==nM)>F$%1+5hS(~>7Z+zRvc3bN-37?X-q3md7Wz{?$tPrBeASKCcyh-40 z$d*}XtngR?DQvR5et}GzaTkboCqu?HAP<7=Tc7z9aP>B`8QwEHeR?@}^b3ZHc~l}S zTB3@;C4Ml~-qEA_&tYC1CTBpUuERENjfBk^H(l<87tTA1tCrd$TM2bmm3?gkyjLVI>m`fGb z-;%mJ*f;8zCj^}rycxee9}jMk%5~STT554jD?+L~B+p^u!@0|*+BPgm1SD>Y+ZB=t z3~93yQyt;xGcK5snb`c9*6@aJ>Ck%R81>IP@k2lqq|G9Sbi#E!&0=pSDNW)=c2L0jf6HxcYul9C5{+Bs z=NCqJY@Ot{jPD#Onp4+$&~-9B)e^O`Q#zZO<@{->-lFgQJ*k<`$DHf4Wt_W1l+EV) zL=k8O6Ho74{79_#C=!8|f|-#p+9|HsQo~HV#>7Q{mKE5FY{g#&vD1#Q%IBQr7e5A@Ep-Kiay$C0u&)sCF~Hf7;jygV z&K`8{YVj^#SC(IWq8fEkw53I8Yi_FQjlz(Z3RunKR;9)kYgQl>EzltLG0Uwq=NoO4 zn<%)~L!HqZ^(42!NZ1X`X&16k;Y_=b75@U$2yT$w*0N|R$7#gXtdQt)o18(;J4D25 zNpR~Eb2S=f*n&`K-S0~-jCuI!krX-o49Fivwt10{l}(RHBp18t_+kuyT~GPfrD%w3 za}w#^?-ddfLZI{W^GWOc?2?PQ)aYo6Sx`<-Z+2s+WrX?6g{I}6TyH@^y}I_V%_K$7#sVJlAGqjk(Y&DTHtuSM#0qGF=O=Ttd#M0nX7J>oy$Awfk+#V+uDM zox43Hf#%+OfT3AMLm@|_cbh0q+W9@xviL79zz#|sT7DqL2eO2x%R*?uIqtYPkP81EzNwMH>D3pwFXFfIy!kPp{_1-4*HczT*Ny? zHceDfi{|N8;=l;69a6zzVVi&?mKUzoQXfs|;JN#cR`?7I45Og}&x62vUt$@;tk>aF zL>9V6w%0de70%SZW*y?Qm333g$~uq+rd|W`$M}NUNF~E>1{^C-flk*)`qqBaYEu)t zjRVG{PD%>1DF$rMCz>y4wDpNTK-vG*v$(Df>l0@Un+Kx!xdYibXt+I8`lN@M>(n^j z)aiDKx3JGp#0o4*54+ZJZ?XEILGQN8vh{MIC+&JM)chFKYyhfl{%$s^vU>L~p8&9oq^j z!o5@CmE$Bw()xB*n@q{d($js$iml#{e9!6UUu2YDeTe?{iQXiX+k-q)8@aN1Oh!&w zgA|ynqOs7Wr6DJ!Kpu5lPU7<6gH&{;xD7P0WfVUH2bEBHwPd`be$qw0>f0DSj^sQq zl{@Vv>o~m498*y3Q$42wqqDY7NhvYG(6{=k?QBf##D?yu4GsTTP0d$ym=FFH>R3-5#l( z(g(|lzCOMnpZ%n;uoyX0@bKfO?Cgx0E-93v#%$^ClLIT=S^;AA1RoEg;c(bbd-s0} z#+$@LbZ_=)bE#}QFt-i-)6v{zOR3QXuoUcjNr6%(Uwn6_B%HP%)ti6*V|43_pZpuY zlPK=!l0(VsFMG1|EJw98%*ln;9z)lezkNKw)yi2gKWE1!W_Dv|p3|}6xiA~C&>^d+#09WcIy_ zqkfdJGBXxLDb9dmLsWW4Q4tYPs)&FEh;*eS1QLRxFerkCCLO8Ll@>@KQ5kv(y@!ZE z0)!+$fRF&mJ((Gqna}tBdDptXwSG5i9F@d3=Y7vU`#k&E`#hUpo8+~yT};Wv+z!^c zRWLnn-BgzSidP_eHJ|JA$HG%rS6`5q0(ZSiDuz7t^&MSMiKEJ)ygAC_2TtF4E03CV zB^3=H_O5;n`sz)-wC7&K&oNWm-rvkvz_alhsjC?I<%dNQoMtP0N-=WD@WiRZujSVp z2#$j#lh$~|WpKDv!Ld0rIyRQ#YR`vi&+$;e8VIX3Qk3pgu3w0y6}46hdp~iRUBeHq zqNc3+vz%uwd^KSx1QSQT>bvxMZJ4SZL!rbL*wv^5(Jru!1O4aTUg};xcd$%beq>OT zN96~`qd1K^)CCp01mK&@Ei8O103SfvX}{)$8RvfERZjK|djU@wH#76u+0&Eg+fS|> zCVE0sKr;<}xCZ z73FYM@TAGA*;DL>(R3BNxX}(#X56TsEtInU7kv7J&!{Bq6gut$bobj5c@zl_iCJ{b z$jE(Y2&(ma?;l%3-+WK{Wq9pja)K+*4+)f9ab-YU@0bo}nm^(0iLwGe0dp(rMJt3& zP;5L5(bdaJ)XB;R_Fdyu6;z(EWUEd!bsc`65oLhhzPFw_@W39uWzT@l&I6h;Y&`RY z=E~PvoFWzMmTyIPEP})GtUg!@9W*7;ixMVsx1&|?(}e>6&^ zt}b+d*LIr+j?ZW$7#76&42;KT?@j*(39fZyum!l$2W7envJppFSdNfcMgj zTp4=up8bv6w-Y)u^quC?K3ih~k)Z8HO$+Y)a=my!F!-?}aG}o5?rqI6O4c?0zNok| zllSl6H)e+IHM7-HO{C3~Jna0YsI0uq!HU`=?(sM;P}k5&tmb|JUOgcv*S1M`Hg#d) zMNdybYl-72s6T!bt7EJ-nTeP*w1Ix&|4s-z)=z$97D*YjU0=d30&%=hC!o_PcFQuh zuP?U1784k`ttAx{L!^Z5m35fv_$9a4voZa$NzS`>XN#xtKB$h~(S<+*G{CM-V61SL zPP4(v$KHgVKdtXmifK=Aex#U!wn2%%uMs-){d-p}qRuvdduJ5LzT&gW%as5Cx?WOZ zPaQbi(^oWu`EdYW0NRYQC@@0pLW=3)EGiTDZPv=TcYvb4CrOH0GczJ%U~6j&plY87 zvEMA9meys{2lj#`T0a{m=IrLy`gH3~fGzDl3Z|cP}H#ne3-UJqHh!HJ^ck#f*(tcdyv8^T(T#F3Rii(Ph-byxWBn@(u0N&H68`2_o zi6A>AG_bv}(B0Xcr16wU9JF~l@Yh&{KFIE8!~Q|b327Mqvqa$dLBUYIzjQzUOAKd6 z2naAXUN_GEGyfa(6qIFbyeG6zAltI>O1t|n0iCTtRAu8mq5pq=<#9T5vyk0Yi{7ui zGaJ+4XK72y27mW^Q@Z6}PZ|`v?hEH~+tzJ2XqW1T7M6Qvm_G}l|5d4Q{K)^C!2kPZ zdJQ;)AY{|mu975GFVl$kl2J_-dfh#vX!zlNO4zG>Z(_nxXZq9>^XHAZPfG)MX%X#R z&qyVTL$91PY{4e!z(!(GRL|Q{FBxl&L%;s#?**o1F+$!dv=5suBc(kzf#AHia@!j? zIu!m{+G}y{aX``8jrBDkJh1-bcambzj5~+alo1QKlF0+LHnT}m4$Rb zzS?kHb5HkZKXf`~G6W5-Dl9CpjR}uawy8ga&7aFWMI z&vHK8dHmJjg#3xNFGj8`EG{sf5`iC#gm+SWok=c?&PfZw!#+0|yZ~;iHM;p7r@1)`&^qy^RVUJ!0?+%7AQtJRKdgQr_<=>To}5bYYw{U_v3OndPv$0ORGK7YB;ow{y+ z=um>cyH*e~UPw__Bw6+TeRSr#@}vAxKPWm=wFx{EB@mxDyLu8=rLfv6{by#xy$2`s zSReI_X__h8etve^Q2yHEx@zs*c{#RSRa-=3s8wVWK`PKPd(WX*jERnVsil1Tx@#P} ziTmBE0x||8!O1AsKIkyCE35&sGWQPb-$|CJW_cTj zWzTBfC8e#nQxwP&@gCV07sBerasX~P4kLSmICN4enQB#N7v@c?q{pjN zv(OYk>`zTK%1Yvnrm6w6Xo`kx=?>_6+UCky2^kVU=XPgLp=3=hUQalhn)I<-;lW&^ zd0dR6k(r)TadBI!%K_*%f_-CSNpm$~woF{r&wQK6aYI{ILyG(D zD9)uH1?IoyvW-^JX^D<3dW*?uvUEs9)7+C7wZ{{gYm3izdi~z<5zP6uzZOE3^W+x< zw?o0q+&nrxeeq??t3>Zh=rPCr!*o_iQ|M%{a7y%(qr*Jxt!^JLp5tOyUf*4wTB7K?THM^l zGgT{V;sLEU1(5@1niv=Z*fP6BTza}@R+f#|Ue~??dId3|+*QGSD0q%{OpgZy&CoC+L$d~-^=^2%B}Y6Z zQWD{UcCcQrAGy8i1H2T(Yw~sV{n_yFgSX;6YbGqGeQ=cQBFOYSf?+jn zr2gv|yzSYR?w$&7+>P(GIX+Zh(VAlo;1DsT&HgPQBt!vjM2%g*r9I3i&8rXLQgQ2p ztU6+#G{-w+cYb@~8G;Nbp2*G3ZTpgwVr*t+23SI`#?Qi(Pw~PNa-MiaGPga_zuYOP zCRy;fY%%u(!9Maa_srq9wV@GhwKOYeg`E4qTa{7PFoN~giqbK2Zg5~E`jjO87g(`% zf((3VB|aG$*HcU$FPGBtxwA*|{7Hj_uUPlmmfox_tS9lZ#|(JY-1xQe;Qnkpz1W_% zK;J4dk<(!R*1$q*C+2tO1GRhm753^b*YLira4psUH z(|ph)e&dC3Hf}Ul5CJVt-OMet?S(9Do^==aldnrrd)hQoNrS2ew}&vK=YAI6$TKyo zD)DN5mk#uMUez0pExr?+EPPmkJlQm!ReI%gt7I28?e7ay>t z=e%_C>}JkB?pFBV&{F&xQ7ceCZq6Isque{0$YQpMMj+sO4xT>`cFK5JE;0ht$t^4< z^`sD<%`L)|Je@m<8cZw1lpvfHsmdg6DbIV!U;*@3L9*V8b2$Pny*5twa%RME4$= zO(oo&I(`p$r?N+$8-Rh<|4EzMCf`H3?a4R&7PP>9wr%BsAAuasnKYF=%$ z`U#DeX~M>altF#mx>IDzjcWJ?hJWo6U+bI)0k5>=S=5XpFD;AvH$FKbFK_bT0n@|7 z$_97R%FfsQN~C1Gjc0OsEk06f`BJQ$%^SeUOxmLO6<~XSOg0bo^z>1{E;11&OUe3} zn`=eea{NCa1kso-X$qx`hps8DtxZ_;lNuI zaz%v~GLylO(9`n%gYEsfgLv z-RhBqx!;p*zYyb$)n)k1jyR}BMPZGW;f#s{FS{J>qi4i}%+Gq%ehIi})J?fVxPQS; z>z zh!&!8^#^^{_Z?~RcfdV)jcqNy21M0(G<_Xrg^BfGtS8sQ*?Ew@l#s|**T3v|84!k; zX|tOj9d}k`XG?gRV?c#(feq#t&>5yB8OBqSTKgJcs^xGMoR~}ql??R3xNbeZPcTPw z0BS(D>%cH9R^J#^zOCjIzh-Jo%lg|H&U||4!pwFBU_sHa>pJREsKr>1)$%gW?I+_7 z#4J8c28ct=y(e0xNj_FH!PT;r)~QLCu;EGEs9slx_I$^hbBLwgyW>?KUqokO{jVy! z4Gd% zYjOwn+71C}lk@PdSbxjnY3%6A`~y=<<+Gp#;pj0{(m+r2P(p961~ z+CVN-*1!%m&&fqV*^@G2=N%pa-oMuXElX`P(bF4;QsKR)i0vS5Nz@ZjJC0&e3>6;T zWei8+B zNbM!?GmwR75DQx5_szDN2@gO3(UEo-cL;Q;TpgWlFwsfct8aA8`SIh?3UViBb{qz> z2cLaUp=Gk_>eOH(;U2I@frD??)r&s|@Vsm#Zxhl>-OXIRspHeZA)c66!($R9wYO78 z%CM|?S6T7sRPFVvam(Lim~R6Y_fWqBX+J!f#}I+DHOR5Rl~bjqcsT{PZbp4Wr-vU_ zQQp$hY-(kdda4);4fWn7rCh+8|MX-FnDUtHY>LO~D1t>-TS?n;-hc5u;%n+?OirpD znj2CNvZcS&hKd*g#}mjJ&hP5$8+9Bh#I!XF5a}j`WnzQQ^5>p{0s})=*KHOSj-ba6 zX!P2?uwN-&0|Ozq@4oRKFhX;hao5!|4OcjSY|1GrUC?!U==<{V2NJ2iH_30-ptpSSllI-e--KKDoOfK z(y}s?jcr#tlZd$0F0NH>7~nho39*=F<~`NXsZp>Ln{YYTb#Or{nBA_b9cQGi7<1G{ zyG<1s8XhsB=KRooFec|aE`KeEG7gj1E>MW}Z=Y;w0lB^C>&fTUKyD&lE5I1fEoTCH zc}XK_{0TubdU;vy$t0X5Dqx(4>BGS6Jp)$>USy<5c1?yf@M^@LLZ#54;J`_cQ(u=_ zlkt{Bc_lt>iR+)hk@=4g$lXqQvG8QjlW;=qZvA4ol#Vi;PKU3&(PstoxduZT>QLrW zp?z=Df0Dkk+9ZrD7>A?oYQi+VCfC^o+DxV)SUN`;tDFYNB;+LoRzYxX21&=yIZ#P{!(P}Vu*(suFa1R!tXO-U z!f&6UaNW6gROde-C?87EGCKYqzF70!#|^!0Z18G)JmTx8y-_l8r1iws(Ho^K z(S22)!AY_sLI#&En14St;@6q~{x}irBhH>CeL%jDWSfLjaA^E$Zei(A*F2OgsgPEt zSQ{c@I2hDbK>lzpQKg^WQ@OCPuuDwy_|dGR&d$zlSkN$j@Eo=nkd)UzI?R12gXWR{ z-rh<|!3lwOSelVji%Cx}TGHenoI+-;+X{UeooGGR4M-Kf8aw687_}u;6s&Vq?CU!< zZg!&@L2Gu1kf$Lk>;{VS{lNBAnzilQf+g4PR(XwS_1CEmQ;aV>-9y3h+}J0QKsclL3I zAt_h&TwRM)=Y9__JSa$Qg6Gz6S?x2Pq+)p!xXdMjXhV`MFRN=Gmv2` zKl=zQRToz)`ois3uUvoc)k7M(wUT#*r%0u;upr*-JqYD#nD9{pJFi-H*T86k z!*FuyPPMg>y)j3(oUih_8WI}=F*Ec&;9=JfcZb%$eG+^=*Xz;kA&rvs$7U{xP)5{C zn^V`$KbV`_C7<6oLFD4C@9-f81gjcS1W5g3!EWb0 z7_G-@Uyr0~^6&4|ku^Zt7*afLcaP6286}!^mj!HEj*wv~7m^CqzF$q~I09^gZ)&qGa$?tBx<*xP@Ax0c}m+Zno{SfcW-#n+?A_$qV z=K}PM0O>OgPX2s<1yN53Ho(7M(AW)*5B%`|B`5yBkb;R`2C1eFGrjzL@8%N?klDAL zC*&kPbdQ4^TuDi`GOQIVdsxL|rgq17oFO#j4s32;rl5LFia+LQcJtEQ@Gp^&AGVH?ty{I?guR`E#9q$5!W3Z zFSo0+%GPH3t(Na)ZCK3Wu}u$ywC;l7OzkdzT4UCX-0@b!iBuDM@+2ZdKk_U0ajbkj))3a?Q8>5+e0B$_s(;CMFq%q4`D~^!l z6yQ-uVC3Lj+wzkJT>}F>_LEB>#C)+!T=i^S-BBdgxYHxTzU-j^es!JCh`GIStpRT^ zt`y(ToZ{xG&J2IW<@tuxR!~AY=&_2mE1{a39)B_#zmxHAIid-mR)`d1QBz*U~M zTevRpe?66gO!K#Aj|B;1LGc^7e~*S&VhZYjOR~m6jSd~EAieP)sXPY|O%pg9BuLZC zh=Jo5fL&5w-3P>265*{=Z-&Fkm5(0)4xzfG|DCL$3?tes0u*XxnB7*to6}HUQQ_cd z<9(^AW>!{l;9S6@fRdZa*h2)LsNyEja1yYA zcfydFo~Ei*#%hT(ivoDv6T(tHx_fliUiYXdPGAT)e~fE>p>53Nn3&`SaNOhCOLfoObo`9zfvc;PJnRtWA!A(g4Qd05D2%7FycO+WJjtyQUn0yA*@G zbz;MKaMuLG2GtT?{UkP}$IN&nmxroeu2)o2Vn4QM0;MaHrTpbdx1q8s*2ti|j8jUNptTlabizpvp4-b;IcAyB&|Qv;_U-4NSX$bRvls>i zG9!~KrK!9)!Ezw*+AKigoa50(GTT_Y^pILuW?7k{Y_gwQeYj-rKUVICM5xD0vmnto zhdru~0{BvMmr!RIO92qHTKMEO55hPDKoFU#@d#X$%$9At8sFcP#!9Hr+5@o9QllLD zBw#roKb{akSP=V>_l*#no14)xzDrA5hX`<1tgEf^hk|~W4Ufd?V|BNg~TR=@#%;sQp2YRmSKv=5*g z4L=KEJMC?VBvVE{J#iE}hq#4y7!NVB9ZDG<^amw-!{I~L>==ycW+Z^Ezz8i4ZM<;e06cWiG3Rua+i5? zNC2Mb;6Rmk?JJDK;KF{bl<};b)Pz!dURV40rY#j~D@azNyrF)8KH32kByzsR;Xm4x zYm-x%>xR5#x@Y^hqeKT*c3b?}#|SEeh{AG~+erzjtWWlm?H?P&<8#e&CbXD=`mv|2 z6Y8_GK@55h#Hr3M9_N6Q!> z^&WWCEJd_pSt;&Z3(twEj5>F1a}ZYrO&suF_C`!-LEU7t2}h(aggqGyAmNj zI0$zuZp z%VN7I;8f@l&K@4^%;M@}!2Xe78-D|iasUnqoE})xuw~bhif`iH3K@=iq7{V)EDN;z$HF@e4Nt&j&)EQrS}Y<2S=cu*KZ(+sQ|EP&3ZGiw9)E`DQU3q z-*g*meQaR0f}hwG2-@BJ@d_FR_WD5t%Z*gYOi%w6w1AjmnsV#DTUUhk49H+BoXpDF z0`+%ubv1Nxv7G!ID%n&rd202;Mum`&KTu-6E42tpZC7#60>S)V+vK$I704TpK{?gm zuMedFO>F%2|3sY+Kec8xA}SrvSLLO+!!Z4uayNHf!2T9=Vf?4hv()5CSkDUb>01#dk0HyeTn>g4#03&&-U%QW_sg^ z?hVt%34fta;$Y-y5|;TE2VVBh{v9QOPX=jSI>dQiD)&TJ_w(IPv*|)d#=)I`lH8?N zGB}$){4a%{65GnA)XPtTOnynYuthcKRx7AFD+#-I>+;Iq{hz=ktG=8Kw~ZGSQhfxT zYtV5$%Tp@B|9nJ^a$q3awD>~ex`MAdb6_Xz$-mw=3b`QuysNTL*n29%^oIptOWnIDKW2DH+-D``>=& z&+kP3@tp~W64O!gpu_vee{1Ooc}amuyjohHt+lzM&Uo*se~;n_8&o$2Del}Byx>=j z&b+VwU#s}FfcbxY;YkpA_P?&%=={g`Zri1L$r$oRQ?fWOACq9a=lih#{9Uch3A#z> zwdWbatm%GLadhT4^nD{u#8ZTJDucI<1bP+ctJMGfPrMEb#`|$E;hdE*A{Cs# z;9Gz$RE~S||7pL#f^`H7X5T8u2-EEH@hWJ*VWYub-It_thH( zIpZUBqd;%(ZG$I%DZ1T8Qwc3S*yi)6CHwzshgXU>1zLKQJ*BP($R7zg zTi(7cz8(P1>xcaf3;+0<*QP-b_*{K$e+Xgmo0jUZkn?FK|@-!95(Xvi|^^Azb?g@4ce`{*Y65` zXm#M&kAHJvf4sfWv}u3PvBmH$ohc1k_x@31D|q)cZ9!eg=&RUomtp_&oijp4Q({xq zjfm~fFFt$s?^UI1+?Xh&_!)9U?Ty-{9}oRCkiUL&C{5ak(QoZ`?H3rO6=>PnE` z#44hZJ^v0l3)}eOub-m!N~9@BR4eXkI=jo@oo(c+DenU>J3r%sR+JQb4A4pKPuKCv z2X|dVUwk(}2)+2r*5LT<$MyR@Qj>%tG`&i)CDktHTOtdr?ACJzCxs*qDb3svNX%%b zng~|W;H1MV1o;d{Tl6C9n@suVqU@ z>q>CX$T@dkZVD1+2tB+}VA%%g?_DH(KA=Vb0(tP^!D_gLq? z?O)WYjHI-x;Yn-)c2UW~!hOoz{Wss{CfS*^;5}W?v`q!@DwrHTn($ z%xAccj1>KCmT)kdrvmq_{1bLWw+{G*t+Gy)~B1c z2LG~k%ckcSe|diK!q(u=whcKl*86vb?EB$aTu!Nm;)SW%b8b^Vt`j+smPx2S{|V~L zy>TCg8pNfIYCp@CS}C;5_ptGcI%cAs-sUYhw{cd8&f-ii`CHt+o!4iXswrub*uTi= zHspKr2I+WITLN*rE^{nChc5uk>vPSnQp72@cgxLINqHR%LU%-w&=jtCw>$+#ZE%Tcl|0bT%(*B-|V z$0%%?@+%UNXkv3-Gsjz;a#gkW#>{+wijd&zOv3d-c`L@>h4-&33wG+}Xsn4(FNkkH z56l-lxj{GRARYW?jWG~1K-Yl>TiV$7xonApba8jKIP&A`d;f93*zDh<7pie_b^UYM zws+evB%Z>0vIiRIap+ihWOT}8(3QJi#~^iqH6P@c`c9X&gbK9O7C&An-6 z0oq4mvB}(i!lslv{U7!NxApXJ`|XSs#}3_zKn0)mNEc- z2kBsWoI}kjGkDbC^IhG-LTwJB5M6d^Ldw)N-~Ng2S)~UA6VF@4^_eyoaTQ6HtPS3=t!Y+MA*CAZd+C_H@griXiv22?v- zJ4sus-|j7AP<+NVPxr{{^c~*zc+WVnDt?Saq>2D{Q6}HJs@2ObT*aAO;Z8KK1BCq7 z>*u)ro2Gn1UhLK@xxHH~0l>N>O( zH58CidnoZsi>A>kzlvVc)ioDzNCWgaq`C$yF?yXqqB5_TU`LMk+NJZ#f7$PQ%`}ohQo8p=)u8KJouS53ieClf z5*d$F{Q6pN;`pP7Hgsj=!t@NZ7U&%BpXtoN0lE4!&o8Rq@4Dnpx}7hmC`K!S{XQlx zj;6!s6wg3sXCEP(+C%*+41qV@qv4hWdN5ennq`P+uiOYkrJ1oGB^Q>$m?^;ZcDE;M zSlHkSNkkw-c1ZP6kj(T;dHe{KJ5FLICu0H*KWhBtkfLAEHN$J>Rv100LhdCG46cn< z;z(%7FCBL^`IU9NbN;9qb@auH9lgc&E1K2!YPZ2_+=IQ6lbLUZ#J2(^iE8UtuZ-MN zq%{4~3M_3}v8Hh3>u-r7%85o9S0Zk(8KlWasz3$Xy!>8D<8H5fw(#r@!?k+L{)ra$_Qk-7Yz>~?{xrLHlw*3xOxCksdDac8GOP?Q?ay~A zw$d<{ec5|5r#rW#>pYmI+_JLkg@uI+{xv1HGM{e7ZGMu5&?Y?n+`&i{GQRmJ-lHSF zzh~gM!Gd1``&M6>tF;SZ;2eD6C+vJqKgQY)%oQ^!(C%9Q{!Wai)cQn2zw^X9>HD!D z(Kyn0+I#JvAtadF>-|OpI*se}}TK1o_X1;fb$r^?rP+FQ%0!5xMjO=M)mVBSS6taL5C-;!85SakY^x)Q+m|K)zpeC_vqQgBu1+7r%DdV^? zY`&6H>-)EpU~WLO5^CAh6aD*)bES?1m}T|&u-Qe{@z21RA^IMb3>S%jYuEewL6GuC zEx_z9BzMrIEfM87%l($bL<>aP^6UwC?3lhz_hC;wiNfLISWt`TIM>m&Q{^+`CNfzR z?uVTA_E3(s!C>^4ZY3*XlX5nqirQUCUbWD*Fvx|(uQ1{shPcs<5vY%7%pz<<|P zGVT&ujT+Vnt@0f!1`2)mk`pjZlBr?h5c)LeP!Ykpq>rvLX^&YKl?SGOoyU~^?y7~} z)|LtRfMNs7{GaG29S$DYzh7KfLopw-;*<@Q_HyG1Q{xLK$5d{uxH> z-7ec2q0J~cbz_diS2H1V+MUl8lS-;?Tit=XFU}js83LI4GxCn z06D$e3fU&z1=bBN4aRYT@YsEHFLS;Gb5j~7?CrleRpdd!K5U6iKvqkiYw8m_2U(s+ zgK>KtzCL18J*FT;K zSGnNTmI`~rW2K5L)}AWvRih^5hJ{LHD;Voo5O6enO_-K$ZiU;STk(N7W<0 zN!ge9RzMgdtY&vKV6{Q1F= z;Dey`Cx{!2IwzX+ACAfC^k=b91B}(3J9qll`dUqsP&GM`I=o9+S&j)BkXZla*>^xP z>oOI^UOmTBk6D#ZVIhtcF#%Y~=CIC~f+ifsNYu4MVYUGfXOQrxqli-Q1^YOEfD{|Ur**;0K%SSs`C+-ZQ!p_qC40@4& zi%*xto!B(B_&gP08@ZBOy8wO%+I+bmKIn$rYoxMri#$RP!F)G0A0F%rQ5DHl-QuQR z+>l`;8TrES+LsFt^`D4CLp3EM5qL|`@O+&Hflbe)-Z^PAR^)*^G4jK+G=&G$_GJI@ zSC|LbQ>cPDd zy{QgsNT5Cm3+)v+#ZyJqR{~M0;md1eyPi;~qOM4f%G+^w5bx~ZzRrtP=5IBpe@K2z z;lFpH^}W3h?&^R<-2I++I4Nmzs*|%vBdTtYZnfyPl>G{KP=81(@^8&i3x- zqDjSO#=H=Q{o;*8M(ez!e8B4J&X-DF6YoGHNPln)z6(^rUrt5?CB1_Zh)13vw>?wt zcxri->xZHKd27Di*+w=#P1UFFtN&={-^8CuDkXMg#UV_Jý$60jPXz#-Qu_Mgz zX?^Fe%rNY@3e1H?aVf9hD$dWU4dC*0|f9i*}x7tQTE&jz;$PgVWSO7ADigAfPapKC)@fd*A z<8$(D`ip>Y&PyPjxA4~cb8CFV6jgTb(UfZhhZt=97RQQ==SGjRHDV?d30at-P=@bM)B3f&=z+}@2 zeneJ$B|bn?QVw`g@=CpU@>v)(QE2hQJ-yyyPkYXKpxipOL|G}}amw3A1FFf(^wI|x zXUMoYln-#;?h2k!YT8j+?Z2ucB@E}`_fbFB2zHdX9|T}(~`*affhvAR4S%(AH6tsB1r)IR^dHyvXEAzxM2ZQ1Ery`mw?~S!5 z7pGDM3Q_3AuX-O_OtdF`Y}NVA;jc2ZK)2^;8LjnJZ}-hoFXO7yB{ZO56`2A#T$5@4 z)IbX7?IZKdZIKP9K!Zw9PWihgyE(sjI?vUkLk=?{xi>=Krvt463HD%vp6O?Q-Pg~# zqv&vOJ}!rM(qL?9$rXq9i;SyTXO&NK>s1I@X2u$3{iH_^(9%=4N_?+%ji`+=UV{#P zv4;*VqojUhE^B`KZEl|Pdlbr<#i9vD4}ww@OEFSoWb>t;ke+l^z~9AR+ZHXN-x zdX8z(iVpXFyf%n2kcu?U0R^{*f$lRG z8D7L1H}n}bPVuRfnVy&B{k%6jDig)?uBvd$jRhSdK?FQh(A~N|YTXnrZwk@;1I*h{ zQ@~ai$Q`GFj#W$C!=BT1aj|7^qw_4%_2gs`aCy@eO&mM${qRRKNHWVdnOj)O7Q0r% z#tYw<1Q2zmF7ALK`by*>G1i^DW&1TMr}Ylmen5!f~Iut_-fdt%x|hu>IAYGJmqQ*aKG`ft@G#?t5-MhiUTGb8f0ws zL)TfEmvtbBe)-bS0W;h)GEyLCled4@w(b(U?l&il@^2hRVRf0gqjbv$dci=Usqo8} zi`H@nax-rf$@kIPPKmlS%UR`B!r0NpamF2kb+o1WwTUt!HEti{)-B9~9BapNTABw* z#voDI(}T@vv&rktGIU(@%`JJ{U9a4-`AJY-zN_NH4xyOXdT8^gm9PeZTwI@h48TCp z9r|pRZw1B{_<4{gHwSJTlm$f>$$z1_d+hzKP5JWW=jz<4X>{gMdDxk$pP9?y;xJmi zav408%WtW9-@1uIiE}GhObu%HRonQ!)lPf|$%bTm0i%ad-CU-%I>pn~ol2 z#;t!fHA)S12u($fa>$X>9&$%@@Rz7JtH%u|16BFerHW3i$KA$z}UrshsP!WF5e!;ko|F z04k+bc|)e(2qYH)A21$~_%WZ7m7G_lfUePYmUH=J4^Li_60M?Cg;C(#F7J;_097-ms}~vm~a}0Nb_b z%gNbss-W?We*T5&luE7$7~zj@-GH4KkJ0ns55*mSy6 z9n;r0(@pGq*wgrvy33Gx&D=K~nddl`ak9EKx(rcATP;u>T~BGOeidGvF`iWuQPgzR ziC|?Joo_`D5;6u4NQ}1m*b2~GxoZPp=dCVQ=bMRT~pR-j9 z@NJen{#O1wJz*LH7bqP!TnKU?t|aV9SuHc-j`u)feAma&Lykge$o+CD=$L2~&jMv@PGPP%c#&}wxjcYg7?npmy z{T5c9H-Y%4`OMD7>Ek!XetJ+gNzPy7Nr*UenSn9^kNr0%`UGG3b(@G}rmn_4!&I_7 zxEo9R{OuNWWpBzkZs$^eJ6Q3n5)`eiI-CC^0nJwDs%&{y5>of#sJ^^y6$Jjn)r4d=D zt3riey?&s2TVg7p`7ivXGMiuQu&yqg@t?Zei>$EZ+g4tU^1xmUIjR*b`=3{SAlTkN z^x++y*ar!2Zr)*wqHk6chic7zmG=MElrzb0fPM>@u7gvE~~3}wO_>%dqQQwsUzTCpo|gt-Djy! z_IX#?dR383ubwQ?Fdxasx$s$GmO9*|!K_U#?(T170Ip$BV{$Fb2YB3HTiCny8NWSi zC*TtEhSC~%D-MZ|qN8y-S}FMykdUZ7y9g>zUB>I%oNY`D4J!kRx(5Rr5ILn+(LEzc z31@Hu(hg&I_TXz7wPEYjHm9hzJ|1iuom&cUlQzev+)X(#TWbFvZG9e2;2{v6n&XB} zqgU!h2MGa(HJqX&0QCWOxhmoz^;6*N&mA!khA9K-R?w&tv_-H@SVz zIc2NdOH>?DRJ&q_MrF~y1N)veM14#1FQI-;D70Bx*5G$ZM`{=C#KXcW`g@zEeO%Tm zKc2{yA6$={S~-stxP&@%CWsSVx_%1H2jcG_`OpK&@Att{+RxGgPgu`OLgpFl>#4+_ zWAaz!yStG8FklY@8zJ>%+wRi`^ekOhq!I-C_s4?^ewumt!*42lzkNEegU;BNTnxoR zn}=s(Hec|a?l!CV_G=;1KTNrB4K~ylFGAY?=51{391$kE@4zg(yth{go%v*oaM|iw zr@mCyI7A|+$o8a}Raz$Sdl+j=bMs8Q8h-EcxUfrst(x1}RgitR zZ63QD(~Djwo2dKz{>Ug8Ww4(lU**7%3Y8JBb>xDxK{?VXlcu129;^^|gV3}Ok{xiu% z_w3OmR9FgC0g2?#Wkij5cM8l2Ab5te@$8KyENI% z8#V(7rKoCV2S_T5Citz~bGEk6CVoFFD93%qP5rP8J!-qY26>VpZ0A(mxAb0F(3U*tk*wnKOZ79 zz2;4{1gg{zQWBMYUwGiBuLyc*?>JH3&?O-MR_4dkBi)trK^X$*6%18UR0M=0F@xwn zP-%cx1V?d{Hyr}`C|WTt&#Zw0l8S;McEHvk0+-Mu*`#Zz8@@!e|0(|w|QWYYJ_sML<lWOpkUbi)v-X@RQUB&VQ)u7ZV@XU!q1M>;amr`7)3az z3yIXd$xK0y2AS#TD3frwJs>lR*2WM9U4Nw?Ld)v{kXIe>sow2kSaynr@3={^XLq4L z=qaMU*zt%C+O*H~Q0XS;SLNeb)nff@QK;_|Nx&yP{hVfJ%OzI2Fm{zP;xbnvhz=2@ zmzD!gw(p%E!5lkjwM}}^CfNaOkm*zLV-Z*=(CswZa+3_d)e%KbAmrKgi)(U7EnzA6A2z4IUV781sdS6v_ z)Sb5b`Ni|$i_Z(*+?KCdDM$+VkNB^vMF85|UM8Y`uU?i_;hB08);$u5ehcy@uJz&0 z0BaWT=D(E!xBCjt>w;TN0oL~#I%S0CeAM>nrSJq)EWXp2sP!C2Uov&MNp&#K;lbGK<_5b~^gpyGbg%He?r}Xhh`QzpF zdOgqUJkL4La~_X#Ktxt{xlR{ce$;&--M*#h_5>>Rel9H(0^f+8fgFYfa*FzP|iBo_ke z>(I1-wM0Nu&+F4V$bmfG4LaSu8Y7VC%Xk(0F+*`9f)JEj(qNLC!jB+j>eVK@<`D;( zZy>Pf1}ZlGsgT073&*VkGX#8p{-xQ@0nkU)6I*|j z+a+U~1xh58EVnc+&&+_b$KDupdkIYz}OmJE8_`5EeyI4Z#^ycRZ?+JL@HzuhwEgB{Lt* zk5WUJuLX7_-ceV+=4riAEr8lr3*PU0Wzz5n`?>Wl7zBE+61!S{oOQEmiAjKf9+(TU z!F$M5zrE_WgW(F-q5fmqn~l7)^WUv(?K_u#b|J}5pW-uZ(Dfb*Oao?ZJ7{qCOO!gq zc%#TArLci)J8~$Mj@@5UK>)g!)Esk_FRCdsR%Y)(f4^x5KhHPd#Q~8Yl=Gu3R0(@* zEpts+ln&CaOCJ>!$PtFpSp)507g7n#m?Y6O{G8qo3yZq->pjb*hfq>)F+&l$H5gIRX)2+Vq^#1%AuM5w7z_(B0qwc zK}9YVooVKj{kkeJlJ1VRpPP!JyV9w@j_QSu?1A$7zE>3{EOH4bkIiT4Z=!e~CFAQQ=ae zPH4a{1;pthp&$jHx3(r^U^EDZmdo&Ya^4tPjDR<|6MlxhKE>QPsKO%XYuI^eFr1 z!9!o)M35j196plG`0cV-5xW8N9ox54z`T$gPUMlSlRL4^9lwnP2n@^Y%uwk)8+k8I zZts0Iu?OduuJ{#D4l^Jjp#rhZ^7l`|%Znsb`uvrq@0&PupDBl1#y4e_d50$8&O`?~ z!e9ml{ow}ld1tDa=1LtlpHNlLL~W1J9R*i=6T^Yx&?XF)^1^f7$#Q`Gp^Jz_bqOR8BBRh2-?-mjUyqRGI;n9xT2(z6RB}r0#bFI-o@1iR27oeAO>8{ewPX1M0~?1JL1=n>p|;Gf z#BNR=dA zxQ|2(q!cs3PK3jsY|nw}r$;)8?oG&Je5)%rEgZdU7@D2Qmdn7fU{h{l^XUvNg%DYM z+Di3Y&~Z>5btZV0nTW`m1zC(w3#_;&zK^rZFDmN5`TbrbE0$yIl;jvV`hZ%IEfQ>j z{PJ=GN=izAS)NXb+2a!z5&QO&5!6BeAf23)xVGW-1r^6T9--mJxX!aga)-AT_Zk3f zNpIjk{B*E{-A*`;QMf-;f+nPFq|q2wSu4Z7Nb>RB$Z|rNKeNfrb7_)46moy#34g8n zV`Y2xl~)h1=n7tW-FTl>)xpV0q?&Ew-#!cmq6(f*PWlX3j!cOYn0PSmPf_6%uH7@3 z$~&ZDXbDum%FD{QFPzO@T33!`=0IWT*K+~2QJGnSW2H-S|K~ZkT4;}~?BQyxt|8aM zn6Y)~3qK10`UiMmu)o4ByGj0nP`+hbIb+4y_?%9?s;O|y83CI6`s(iQ!XjMCO*Y$w zyFLIek!xTANPvRXp=aPD%ZBdm?)l%&lqzJOFY9-Bg*n_;rC0cirohVPey;h!g2xz={_#Z65 zFX6T7xpG_)9t!Ihq_}!7o48C96bC?8aOVbaKv!>+@0_RJ&ZS~Gr-l)B3TX5V=F5tb{_)~dg!8e&M=>slfil}@?hcdSrz{GSBXvI@6 z9Yz0F$AA6u5G6U*&|kPx5Ko2lk9ecEJ9H5sgW?kSp$y2S*x&$-1H$$Q-fs8cupYpS zi%mu2c);3vSE?_~_r^Q`kho(I5+JE%@$#qTT3{P=3=8^~N>4tc^?VAIoSpgYC)duI z*k<}WaMShzN^lXN>*7iQQs~an{yGR1zR&p+hsufUgE26eUckZGweuz#PM$y-TYP;! zxazLeTR6luKZmL^04UW678!F8MBe%vVZ5e5N0#2mMrq6b8I1|5=$ZWl04I65sW;NP z=%F&U?N`PEx1ognLOE8}%35=NFkQ^M9@JX&%mp4+v`W8hzs4Zv-M+ZFkXFJ6CREq6 zt1^f=J#YKniSyUG;#edvhlph@I%GE%0s_kGR3Z;$!=k+h2&k8Fd+B+BI-R*>cdrQi zD&YWVdwz*GdfDq@$UXBH?TQ7W(gN()hJ|^IYUGrC?wG62I`N*A)!O|5t~EA!oCaw+ z$2G7C=OLh%4klb|0W%OJwh(1b&sn1bGom53kd5{{1);6 z$fCyvD9qqg>``8aBQbCkD}L2x&HyWLCf!x8HG`@2vD%A9i?{X;nSb{1J-%IDT=^-S z%15DN)G|nI@p$Tk=ntsLwLaIaua~qj<%8yz)jH(! zx1!?ylgqY729l;DuYaV^`|MDjWZ2mIUcLOuWXYq1v4HiCBq1BO4Oi9+I)U}3&Xww$ zo6e%*mDV*Htjk_qjX&!|)|uid=W;uD*3jkf?09zNw|{d0c0VUqd~-cZN(^7h@eGOC z+3gB0&eMy+QcHrzm^`3s2=wK^K=t#d?5EROr76y^L8ChgImM;d za8nFYUN0Ngm-R&LCx~F|9`Emm5;DQltRgNN?lk=P94~3 z46s)j9q+`I5u3Ok#p7oJ!R-g3{ws6`XJ@fbCaibZPT|JJ5UO%`ZP>EdSP;Rb{Ba+h zCP+?l1+Je-30R9fKHm2ObQuzTlFYz5=keW8!rIMo@ZoWBP3MVGuKv4@QJ_Ykng4uQ zpH|;Hmj)i-g;jw_UjPmx9n^o>^Ox&TV6A7fxOT8sO+rTdEQusd@1E|q^#&rw2V)08w{ef{F>hrA z--C~GC8)!2bH90xOT5Ir0==gsg#UT=2+ZWI(aQ^DSmvGMxdv*VW(r=2uLU`S{`2w$b}b$VX4 zW7FS4Ha?DA1J-|SH$aF&@$*A&-pvM@vhB&&W<5cXxfT5}`>jgn%a&k|1 zH@E5yo{rS@-(dA$lBm8}K-?YQTr87j0oT{nb$TQUD}oZ<@K;SJfUo-)fKEUHwixjF z7hBnL--27a#S?k3PGC2&>4`JbFPovKa6*6c|FZw-SUSDMK}*gJ5Icr2pBH*THJw89FLpMCID9iA_0Oes(s% zr%G|8jbkv`Z+MM34jShDlEED9c`>X%qW?2UknF!HRgo6al^rjd{@!y5kdbQOYTY>@SKGe~Hizj{@ zdi))|>p{>B_;D${<~u`z6rlQzUL4|gK^g(OD3d0;DA{E9ZZLt*h#LaVHn`0}?7&|B z4W#->-@@uLEk|TRenvrv0&uWEgm`@R0x?kQj9Vhqgb^V14As@6z=j)j7MJglraec_ za@N11uuLD|h`}c$CwT_q)inaKE#l3lbM1i^@^`BGmsD9@jNAFskxxHA87%|hCtTmy z_*kWU>Y1jSrc1X1y!EbK7LMi+@M$$F~T{bK@rB12O|K7l?Vk zIg|qo$(N@a8=xPxELG;5v^9C7|;4 zf3NbNe^tJH0xA)v0A32-^eBFC@eROFfa85v>Ya9e%2wYyHQtCuX9W53aH&H%g(|y% z4D>k3(EFdi|Ns9(W*!omy!a>mnGgkNv3nmdd4W}17%q<*dVgK}>b)IpnS#O(`u8p4 zDc%`Ki5Z}=)BPc@V)TwA`a1sTkM!$itT(ubyLGG1dt~#%^z>dSWW<3Y&pCPGb2#0h znWK@~zKPo_g!U=z?RFyJE2G=0G-u>>vwb47S=BQk%y1>+7(Vo@nv7T{bL6Y&sWFq0 zJd7cHtPZ{6R{)2$ywRtzvw=A&ITqOr!TBUIHB_^?7udMtXnANRG%tLWXA*u};BP@e zbb6ZT45@q9wqDo`EsGisZeeD*T52!ay)eV?J0<0Sda{9`$qm&~>97UaAT_1v z^5K1D6dyZ68yrOc{IPytsC`bEEx~ETkee3s^XK!MXhU$C2eB}~-uFV8<31(xBa+^k z>g(%68YklfZWl+nup|_eJup4m*`8jj>VvZVc{?Jae>F>Gi1@COhZW{&bvHH+(o#hn zGd2*keZ>%&3eVS0PldZ$^x)-y#caNVM9o?pZLFG?m65*PTkLSQ38Yx^e-vaC|^QN(fD$0kEbu^SFEb-N9A8=!hw0ta!Kwh9bA z%Ioc=+1#eK_wW!IYvOFSHhhJJo_poEYvkhkDgQ(QB-nkt2(*elM-Xy9B;4bQv%zf*}^JXTWJP1N*$E(~8x8$bzL`xoKV?5-S({?G) zyc-Nr{o`k-r4&a*$r?QEM6@SqI{A^W?==&ZUc8FPSy&PkK;O2`d$cX`!3RwtdG~^j zsz6Z87T=1AwK0BimyJBarH7ARg{@K0T9OH5HiR_wVj_d3n5cY5nSrwA!PpJ&X6QV~ zc=&LEnsFrIbm+vFr(M371Bq{j5Sr_M%$WdAppd4cqUXXzxx?0HL>t!PI2Wuqg0}~P)r@s2pko8L}O{8 zeWe}G@VProRJ^n`4HQUZ;d70?X_G&91S{|N3cisXf^f(_9B^dkZ=w@mBc(SL!>=di z0Oh5nh~d4$ly+8CS!yXMhbN-%x?*z~C?D2npYpx&2#UYwY}%vMQ~~3m`D8tkpy56{ zN)g5>^;UUm;Edm#025W4$Tg3rfAUo2?jZ~9@0}%D(;PpO4wze9!z;sH8%6Z0FYW^7 z@AW>+U2l6mI3M~&Ybg+_;yf?ITe8MDipJ!#%F4kv(KOI%`j(`#$;uXff|olG?wWtD zB~Gj2rurpKIrgOG;16UWW(e_9xa#i}MJf=6@B~ic_s#nDl6$^xS5K*AE0_6Ki>|h) zdlYLO`U`ajUt1TswG=5n&Gtj-uh?FQ_!RC+j`#Bh%Z^8}b*%@v*COe?bvZ94ZZo1v z+I9h?l3jISu{L=v31AZ?B7OtPercA!lF^{Gkpo0K$9zho@Z2A22YzKW za1$X)X`^{(&Oz~xX($kitnoQ{MTmq%A5`wf)o|;P`C_)!oCr_%)qPzKMuje(qFapo z|LIix|Np2}%J*_eCsTO#G~~@WMF9;5p@ud>!=Iu87p`eik);dY@krX=bEep)d9J0b zH}#eBHfnf^B{rz*iZ!-ov;MT677-=sxSZaR>{#lPM=j9;zo;~QnO(Ty zuUC$<7Czk#GLW6`Ax;A-!IciBS&^@Iq7s)@y4ir zB}Tp3=c|Na8!r|%P)B=Ma|sfVV|v8U=d zikZJ$qQd9g*$?Qu&|DU5yIl@_jy3({**u5Z+tE>LDdFMPHFXX9X6F2czRJK}Qdd%j zj`N|C2rbiKd3TjoG*!L;PTgDe#2dxYuiYfNQ#(o}FQ9*T#vG{)Z+JIKe|3GnyMMpASzhs^OrRi!mzURq?SztrEX7Bh!hMIPb0~DT zdgM_z(==-~utliXE|?gsxT(YUh=V9eh#8{K&{hDek8`9pgC?%oHQs=`qZ>=mw{x z-^pADjX*4MHw}2fiVaPLFI{;(vqf>&^8_XaA3$EOxP1gKmtIU7C2p&v^kJyEirD9HtDEeFqXoJ6w{!h#K;~zYf0B{HMG1 z&dRf=2w6S77^GEny6^f`IEU2Odsg``r4Uu-2;q*C;f~x&k}y%A-W8#_jalX&B%^B85)5(gc7-z>G|-+O}ZtX$W^K&XyTR;zJ4?BpNE zWh7S)|0_r_RFU$+M5x=E++%A!jrnQk2>PGVN25Ep)_c1>3S*Wt_QVY6d~9zo-e_){1U;^{+zi~A&LD^F++S!@YzEPkN z+rkZ4wbe7g95m3G=Dcz2-SjJ8 zPK`0HEGPg}XUa--cxUpRT&tE#;$%PSk96x<{t(wLF5#5D%KAA4^})vOGa0wuy-Z`D zet{g`8a6C{qu*K6usD4ApyU!+Kg&k$$NZ)OZ~8@dKF3GJQldgt=|mge;<9X&jd`+! zCuyBKr2-ckT&E5geImc*B!l7guoiuNk2_yc_-JBxQgN2UX|3h&4a+$Q zwv~nN02Uw#(&Rz+K2Ll(o!@rBC1rbbEJK;)AUm0=<>id`xNn4vOn*^s&`-tPgV2AX z#i0fU?^D08OH`X_9dux2M6r6Bk%Wq|B4sZ;d@6j=&8USgMcUUy`)EqGZBBj3oj-Ng zm%UG9bNW)6k-Pgd^D%;-?w771k7Gd@>Mr$nrfl;vMRGHdFFvD|Np@Y9T&YtWSvmOU zOZ;;x<^V2B&_V!@(#QX?`{f#vjEE#rwQ|~aIK%7qJm(dPRjP*==>NDA4o{m z2fbJ3Qr_cG4Y$nytd~h}OmF}j|FAEKu&JCE2^i=#D-*r zgr(rC0e591M{Z)ux_#4&8Gs}FeBA&*t@A#a)AO5axf?`HJ$dVAy{%Z|GQtH>Wnq%Z zjAk4db3?_%bPQKd$rvZRKOh?iL@ig6)AD>4#%@zGY)^(F9*y5FDEG|s!jHed3@h5(pGxbeP7UI`J~f70ZDD?cp1}}U zJQk+g^vjR)qf5Hrk*)$B{(QU z3(RXaT$A!ms~SSI1|-^1Lt=4vsci3}YNfXRNM%e#H-R z-)qIjj`qC8aU*8?Qj(j_CBrXxbAmYMys~mVn%BSu230~*I`vLNx)Rr@Qlz}i?K)Et zkfH~loCcN`7aTxRjypD2aCR>|ToT}f^V8E=XZ^e_Gn_4IO}sswdKL_p-6XTH%6fXw zz#%+WM!?WUDXcRMm5$m$ zZI(j<#PA!Sa8df$N;+!SJ1lj0Dm;J(A?7_Q`7?QF@`nbMEaT8y&MwyaDRy}F+7)>$ zsrRR#V4H|x^?ci#ER-ah9$l98e($AbZ&m7xEqD)Cd|hhZKb+tQ-2N(08SzXw;i$#H zyWrCY{i9{vJ4q2A_X|I@ol)8x{FR5<>ju|YswMI3Q8T<~F?&aYbxky?p>!(8Ms`6W zbB>?M6ywu|X~;Ct6~PUT1scLJHBN~djd7Q!5Ur^FHD=)Ya4e~N88tOA*s??K!)2qH zO}hD7EJ12>BRS=h^YU>2@cM37{)blKa;Jw6Rf3PGkCcyAypN*I_;VZ9Wg806^+t4o z<^*b{UQ;q^aAusX*9Il0*ff9nj3+9fa4{=pWhGNR7Up*OE7fqWLY=3d^h#ccZ}5>^ z`@tR=X<5Lo?qzxl>SoR#HPWR7c~EBjkURGhSaWalxx^cn-yB=kQ2A)G)3++Cee#Un zaez#5hhOp-HNH1yc?6<*>}pHE)b1NhR4t=oz<*?sdBcpX6jO@L=KIhkODWqyZN>lj zs`^Wt`RghOFWEYE7_3OrckT5+F|??s=hHu4N-^7q>YG8oKP%C6?bkAwum8+Xwuf;f zS;mOE)iDSZB%~k+*KuxWKT|?*l1( zWS3E)FXo`9^sK>igIw3VLk-`|2Y>jA8*NrK2?d!qX}#@vdEdlr6c=iRE48LnK>or~uOndl<3p_~>L ze-TSSo$6oRFuZq9d2`^iw=ERET4i8veuIZ$R14v=n0#0o0c)ZvGr5G$^{$65c^a2y zD_!$;zJHshCpL{RRM?`5Xmd|3AXK3}7iD3Ho7IQ9cfkhyrWVyNiUi=#8eEga-5{ z<)Udsu!Pl^u-(|6z+y!~O?QEfO&R)=MZF3<^_mL zSA8(v1%>|>U2Zbiv!7nTNX9mDs7lV&di4bNGaC`AbYqYOm-(p+$Se=KTV}y9<~#aMlPUyt6<+0vD-|M3z09 z;q3(O@wr5QHj#1P=?~0)Pij}{w=aRT{@%`1-vaJ?ayLV6)c<^hcO+y0ER=G#?cCii zN9(mh9(V)(wxy7LY4?GzWyN5*mZyK}%Criq!QBa^P$6pNYa?IgK0CA8^N!l~%EC1k z1LIJChgwzo{YH4Ur;R7Thi7FC7F%QL+BkR(ZvyR>9J^EQw1!0|@W=Pf>)&S5YJH$| z>3ZYgVd%R7ke24AiEN#0ZRhW1jx|MX_H!L4_rZy2{$N|G52~wizNinRTSuVW$Nh{n zzWIRT!*$iQSh9_MZ3|31b*w5p{|0(x9Q@%2 zTE$&GLpaZ**ds|utD6%Alp9m8(mvwYuX_?fCb0NafI+JPnXlUK41zJQMSm8S7uEHB zFU_a0CJm~gtsNQ2+00U$SGa`LZbs<&g{#*vr9a3I<~&D4X&45Lu^&4L#q(K{LG@BT z_YbeMG=3?vk0Kq`PFqPd5fc*1Z)hk0O>G8BZTP@vUoqY?f7|BIPZRmo%*D7u=++Sc zL2dM7_cI;x3p~N5I>05)^#U=kVti-R(t_6ZqX3X_V7$JBE1(XP4Matiq&}m5A`v-X zy5JaD!R+Is9QJT_&_GwWQ1jiuW9%XsoxUQQrpsec3;L*$$6;EYipE7any~rV*it v;D;taE?1eID{u#~a!>9ibxXv3~ literal 0 HcmV?d00001 diff --git a/docs/introduction.mdx b/docs/introduction.mdx index d260daf5..2fe3a682 100644 --- a/docs/introduction.mdx +++ b/docs/introduction.mdx @@ -1,57 +1,162 @@ --- -title: "Failproof AI" -description: "FailproofAI gives AI agents 39 built-in failure policies that catch loops, secret leaks, destructive tool calls, and more in a single install." +title: "FailproofAI" +description: "Guardrails that stop AI agents from doing damage, on every machine — and one cloud that governs the whole fleet and shows you exactly what happened." --- [![npm weekly downloads](https://img.shields.io/npm/dw/failproofai?style=flat-square&color=2ea44f)](https://www.npmjs.com/package/failproofai) -Hooks and policies for **AI failure handling**, **error recovery**, and **LLM reliability**. Keep your AI agents reliable and running autonomously across **Claude Code**, **OpenAI Codex**, **GitHub Copilot**, **Cursor Agent**, **OpenCode**, **Pi**, **Hermes**, **OpenClaw**, **Factory Droid**, **Devin CLI**, **Antigravity CLI**, and the **Agents SDK**. +AI coding agents fail in predictable ways. They run a destructive command, paste a +credential into a model prompt, force-push over someone's work, wander off task, loop, +or quietly stop three steps short of done. One engineer watching one terminal catches +most of it. A team running agents on twenty machines, overnight, catches none of it. -AI agents fail in predictable ways. They run destructive commands, leak secrets, drift off-task, get stuck in loops, or push directly to main. Left unattended, small failures cascade into outages, leaked credentials, and lost work. +**FailproofAI is the guardrail layer for that problem.** It hooks into the agent CLIs +your team already uses, decides in real time whether each tool call is allowed, and +records what happened. It is one product with two halves that work as one system: -FailproofAI solves this with **policies**. These rules hook into every agent tool call to **detect failures**, **mitigate them** (block, instruct, sanitize), and **alert you** when something needs attention. A local dashboard lets you review every tool call, agent failure, and recovery action afterward. + + + + A policy engine that runs *inside* the agent loop across **12 agent CLIs**. It blocks + what should never happen, redacts what should never be read, and refuses to let an + agent call itself done before the work actually is. Open source, local, fast enough to + sit on every tool call. + + + + One place to see every agent your organization runs, deploy policy to every machine + from a dashboard, replay any run step by step, score quality automatically, and get + paged when something breaks. Your fleet, governed centrally. + + + + +--- + +## What it actually does + +Three things, in the order you will meet them. + +### 1. It stops the failure before it lands + +Every tool call your agent makes passes through FailproofAI first. A policy returns one +of three decisions: + +| Decision | Effect | +|---|---| +| **allow** | The agent proceeds. Optionally with a note back to the model ("CI is green"). | +| **deny** | The call never runs. The agent is told why, in words it can act on. | +| **instruct** | The call runs, and the agent gets extra context to keep it on track. | + +**[39 built-in policies](/built-in-policies)** ship in the box — secret redaction, +destructive-command blocking, branch protection, infrastructure guards, workflow gates. +You enable the ones you want in a single command, tune them without writing code, and +[write your own](/custom-policies) in JavaScript when your failure mode is specific to +your codebase. + +### 2. It shows you what your agents did while you were away + +Every decision is recorded. The [local dashboard](/dashboard) replays any session as a +readable timeline: every tool call, its input and output, and which policies fired on it. +The [audit](/audit) reads back through your existing transcripts and tells you which +failure modes your agents already have — with a score, a ranked fix list, and the exact +command to close each gap. + +### 3. It scales from your laptop to your fleet -Transcripts and policy evaluation stay on your machine. Data is sent only when you explicitly use an online feature, such as authenticated audit reminders or invitations. +Connect a machine to [FailproofAI Cloud](/cloud/overview) and two things start flowing: +policy comes *down* from the dashboard, and activity goes *up* to it. You stop asking +"did everyone install the new rule?" — you deploy it once and watch the fleet pick it up. -## Get started +--- + +## Why teams pick it - - Block destructive commands, prevent secret leakage, keep agents inside project boundaries, and more. All out of the box. + + On a configured machine, a guardrail that cannot answer **denies**. No silent + degradation into an unguarded state, no "the service was down so everything was + allowed." [How that guarantee works →](/daemon) - - Write your own rules in JavaScript with a simple allow / deny / instruct API. + + Claude Code, Codex, Copilot, Cursor, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, + Devin, Antigravity, and Goose. One install, one policy set, every CLI. + [Full support matrix →](/agent-support) - - See what your agents did while you were away. Browse sessions, inspect tool calls, review where policies fired. + + Drop a file into `.failproofai/policies/`, commit it, and every teammate has the rule + on their next pull. No per-developer setup, no central approval queue. - - Tune any policy without code. Set allowlists, protected branches, or thresholds per-project or globally. + + Policy evaluation and transcripts never leave the machine unless you connect to the + cloud — and when you do, the CLI tells you exactly what starts leaving before it does. + [What gets sent →](/cloud/connect#what-leaves-this-machine) -## Quick start +--- + +## Start here + + + + ```bash + npm install -g failproofai + failproofai config + ``` + + Two questions: **Recommended or Customize**, and **connect to the cloud or stay + local**. Everything else is inferred from what is already on the machine. + + [Full walkthrough →](/quickstart) + + + Nothing about how you work changes. When the agent tries something a policy catches, + the call is blocked and the agent is told why — and it recovers, because the denial + reads like instructions rather than an error. + + + ```bash + failproofai # the local dashboard, on http://localhost:8020 + ``` + + Or connect to the cloud and see every machine in one place. + + + +--- + +## Where to go next + + - + + Installed and guarded, end to end. + -```bash npm -npm install -g failproofai -``` + + The whole path, from a tool call to a decision to the dashboard. + -```bash bun -bun add -g failproofai -``` + + Every term used in these docs, defined once. + - + + All 39, with their parameters. + -```bash -failproofai policies --install # enable policies (or skip — `failproofai` will offer to set them up on first run) -failproofai # launch the dashboard -``` + + Your own rules in JavaScript. + + + + Fleet-wide policy, observability, and evaluation. + -See the [Getting started](/getting-started) guide for the full walkthrough. + diff --git a/docs/it/agent-support.mdx b/docs/it/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/it/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/it/agenteye/cli-and-agents.mdx b/docs/it/agenteye/cli-and-agents.mdx deleted file mode 100644 index 61107bc5..00000000 --- a/docs/it/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "L'intera implementazione di Failproof AI Observability, a un solo comando di distanza." ---- - - -L'intera implementazione di Failproof AI Observability, a un solo comando di distanza. Controlla la produzione, genera una chiave API o riconosci un incidente senza lasciare il terminale, quindi inserisci tutto in uno script per CI, o lascia che un agente di codifica lo faccia per te in inglese semplice. - -```bash -pipx install agenteye -agenteye login --email you@example.com # a 6-digit code lands in your inbox -agenteye --json sessions --since 24h # every agent run from the last day, newest first -``` - -*La CLI `agenteye` comunica con il tuo dashboard. È uno strumento diverso dal collector, che invia eventi al server.* - -## L'intera implementazione, a un solo comando di distanza - -Smetti di saltare tra le schede per rispondere a una domanda veloce. La CLI `agenteye` legge i tuoi dati e amministra la tua organizzazione da un singolo binario, quindi un controllo che prima significava cliccare nel dashboard diventa una singola riga che puoi rieseguire, creare un alias, o incollare in un runbook. Hai a disposizione quattro superfici: - -- **Leggi i tuoi dati:** `sessions`, `events`, `evals` e `errors`, filtrati per tempo, agente e ambiente. -- **Gestisci la tua organizzazione:** `keys`, `users`, `settings`, `alerts` e `incidents`. -- **Esegui analitiche:** SQL salvato più un runner ad hoc `query` sui tuoi dati di eventi. -- **Chiedi all'assistente:** `agent ask` raggiunge lo stesso analista di sola lettura con cui chatti nel dashboard. - -Installalo una volta con `pipx`, accedi con un codice a 6 cifre inviato via email, e sei pronto. La sessione dura circa un giorno; riesegui `agenteye login` quando scade. Usalo per controllare la produzione, provisioning di una chiave, o triage di un incidente attivo, il tutto senza aprire un browser: - -```bash -agenteye errors --since 24h --aggregate # what is breaking, grouped by error type -agenteye incidents list --state firing # what is on fire right now -agenteye keys create ci --add events:add # a key that can only push events, secret shown once -``` - -Un'abitudine da conoscere: le opzioni globali come `--json` vanno prima del comando. `agenteye --json sessions` è corretto; `agenteye sessions --json` non lo è. - -## Inseriscilo in uno script, integralo in CI - -Ogni comando accetta `--json`, e questo cambia tutto. JSON pulito va a stdout mentre lo stato umano e gli avvisi vanno a stderr, quindi un capture `--json` si collega direttamente a `jq` senza alcuna riga estranea da togliere. È questo che rende la CLI altrettanto valida per te al prompt e per un agente di codifica che analizza l'output: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -È costruito per essere eseguito senza supervisione. I prompt di conferma vengono automaticamente saltati quando nessun terminale è collegato, quindi nulla si blocca in una pipeline, e ogni comando restituisce un codice di uscita significativo: `0` successo, `4` non connesso, `5` autorizzazione mancante (il messaggio la nomina, per esempio `alerts:write`), `3` dashboard non raggiungibile. Uno script può diramarsi su un `4` per reauthenticarsi o su un `5` per dirti esattamente cosa chiedere a un amministratore, invece di fallire senza sapere il motivo. - -## Lascia che un agente di codifica lo guidi in inglese semplice - -Ancora meglio, non dovresti nemmeno dover ricordare nessuno di questi flag. La **CLI skill** è una piccola cartella di Agent Skill chiamata `agenteye-cli` che insegna a un agente di codifica come Claude Code o Codex a guidare la CLI da richieste in inglese semplice. Chiedi "c'è qualcosa di rotto oggi?" e l'agente sceglie il comando, lo esegue come te, e risponde in prosa. - -Per Claude Code, rilascia la cartella `agenteye-cli` in `~/.claude/skills/` e viene scoperta automaticamente. Failproof AI Observability fornisce la cartella; non c'è nulla di extra da installare, perché guida solo la CLI che hai già installato. Accedi tu stesso prima: lo skill non può completare per te l'accesso tramite codice inviato via email. - -Poiché l'agente esegue la CLI come te, può fare tutto ciò che la tua login consente, sia letture che scritture: creare chiavi, modificare le impostazioni, risolvere incidenti. Il prompt di conferma "sei sicuro?" della CLI non si attiva per un agente, quindi lo skill è scritto per indicare il comando esatto e attendere il tuo OK prima di qualsiasi modifica. Tu sei il passaggio di conferma. - -```text -you Why did session run-001 fail? - -agent Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -Le letture rimangono istantanee, e ogni scrittura ti attende: - -```text -you Give CI a key that can only push events. - -agent This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you yes - -agent Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## Correlati - -- [Riferimento CLI](/it/agenteye/cli): ogni comando, flag e forma JSON. -- [Ricette CLI per agenti](/it/agenteye/cli-recipes): pattern `jq` copia-incolla e gestione dei codici di uscita. -- [Skill agente CLI](/it/agenteye/cli-skill): installa ed esegui lo skill `agenteye-cli`. -- [Assistente AI](/it/agenteye/assistant): l'analista nel dashboard con cui comunica `agent ask`. \ No newline at end of file diff --git a/docs/it/agenteye/cli-skill.mdx b/docs/it/agenteye/cli-skill.mdx deleted file mode 100644 index e5de4d94..00000000 --- a/docs/it/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Competenza CLI dell'Agente di Osservabilità Failproof AI" -description: "Chiedi al tuo agente di codifica \"c'è qualcosa di rotto oggi?\" e lascia che risponda dai tuoi dati di Osservabilità Failproof AI in tempo reale, senza comandi da memorizzare." ---- - - -Chiedi al tuo agente di codifica *"c'è qualcosa di rotto oggi?"* e lascia che risponda dai tuoi dati di Osservabilità Failproof AI in tempo reale, senza comandi da memorizzare. La **competenza CLI di Osservabilità Failproof AI** (`agenteye-cli`) è un'*Agent Skill*: una piccola cartella di istruzioni che un agente di codifica come Claude Code o Codex carica su richiesta. Insegna all'agente a operare il tuo deployment di Osservabilità tramite la [`agenteye` CLI](/it/agenteye/cli) da richieste in linguaggio naturale come *"dai a CI una chiave che può solo inviare eventi"* o *"conferma l'incident in corso e assegnalo a me."* - -**Non** è un servizio o un binario separato; non c'è nulla da distribuire. Funziona sulla CLI che hai già installato: l'agente esegue `agenteye --json …`, analizza il JSON pulito, e ti risponde in prosa. Tutto ciò che può fare, potresti farlo tu digitando gli stessi comandi. - ---- - -## Come si relaziona con le altre interfacce di Osservabilità Failproof AI - -Osservabilità Failproof AI ti offre quattro modi per raggiungere gli stessi dati e controlli. Si completano a vicenda: - -| Interfaccia | Che cos'è | Dove viene eseguita | Usala quando | -|---|---|---|---| -| **[CLI](/it/agenteye/cli)** | Il riferimento comando/flag per `agenteye` | Il tuo terminale | Vuoi eseguire o scrivere uno script per un comando specifico | -| **[Ricette CLI](/it/agenteye/cli-recipes)** | Pattern `jq`/pipeline da copiare e incollare | Il tuo terminale / script | Stai integrando la CLI nell'automazione | -| **Competenza CLI** (questo documento) | Una porta in linguaggio naturale sulla CLI | Il tuo agente di codifica, sulla tua workstation | Vuoi *semplicemente chiedere* e lasciare che l'agente scelga il comando | -| **[Competenza Evaluator](/it/agenteye/evaluator-skill)** | Una competenza gemella che progetta e costruisce il tuo servizio di scoring | Il tuo agente di codifica, sulla tua workstation | Vuoi *produrre* punteggi di valutazione piuttosto che leggerli | -| **[Competenza Python SDK](/it/agenteye/python-sdk-skill)** | Una competenza gemella che strumenta il tuo agente in modo che emetta telemetria | Il tuo agente di codifica, sulla tua workstation | Vuoi che il tuo agente *produca* gli eventi che questa competenza legge | -| **[Assistente AI nella dashboard](/it/agenteye/assistant)** | Una chat incorporata nella dashboard | Lato server (nella dashboard) | Vuoi domande e risposte nella dashboard sui tuoi dati | - -La competenza stessa non ha privilegi propri; converte semplicemente le tue parole in chiamate CLI che vengono eseguite come te: - -```mermaid -flowchart TD - YOU["tu: 'conferma l'incident in corso'"] --> AGENT["agente di codifica (Claude Code / Codex)
    carica la competenza agenteye-cli"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|la tua sessione CLI autenticata| API["API dashboard Osservabilità"] -``` - -### vs. assistente AI nella dashboard: una distinzione importante - -Questi sono due strumenti diversi con raggi di esplosione molto diversi: - -- L'**assistente AI nella dashboard** ([Assistente AI](/it/agenteye/assistant)) è una chat incorporata nella dashboard, supportata dal servizio agente. È **di sola lettura più authoring controllato dall'approvazione**: può bozze salvate query e dashboard, ma ogni scrittura si ferma per la tua approvazione esplicita cliccabile, e non cancella mai. È controllato dall'autorizzazione `agent:use` e vede solo i dati dell'organizzazione che stai visualizzando. -- La **competenza CLI** viene eseguita sulla *tua* workstation dentro *il tuo* agente di codifica e guida la CLI `agenteye` come **tu**. Può eseguire la **superficie completa, incluse le mutazioni** (create/ruota/disabilita chiavi API, cambia impostazioni org, risolvi incident, elimina query salvate), limitato solo dalle autorizzazioni del tuo accesso CLI. Trattala esattamente come faresti con l'esecuzione manuale di quei comandi. - ---- - -## Prerequisiti - -1. La **CLI `agenteye` installata** e su `PATH` (vedi il riferimento [CLI](/it/agenteye/cli): `pipx install agenteye`). -2. Il **tuo URL della dashboard impostato** (`AGENTEYE_DASHBOARD_URL`, o l'agente passa `--base-url`). -3. Una **sessione autenticata**: esegui prima `agenteye login` tu stesso. La competenza **non può** completare l'accesso con codice monouso inviato per email; ti dirà di eseguire `agenteye login` se la sessione manca o è scaduta (codice di uscita CLI `4`). - ---- - -## Dove trovarla - -La competenza è pubblicata nella raccolta di competenze pubbliche di Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -Niente è controllato — il repository è pubblico e la competenza non ha bisogno di credenziali proprie, perché guida solo la CLI `agenteye` **pubblica** contro *la tua* dashboard, usando la sessione in cui *tu* hai effettuato l'accesso. Non devi chiedere a nessuno. - -Nota che viene spedita come sua propria cartella e **non** si trova all'interno del pacchetto `pipx install agenteye`, quindi non cercarla lì. - -## Installazione della competenza - -Il percorso più veloce è la CLI [`skills`](https://skills.sh), che recupera la cartella e la mette dove il tuo agente guarda: - -```bash -# Claude Code, questo progetto solo -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# ogni progetto (installa in ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex invece -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -Quindi gestiscila come qualsiasi altra competenza: - -```bash -npx skills list -a claude-code # cosa è installato -npx skills update agenteye-cli # tira l'ultima versione -npx skills remove agenteye-cli # rimuovila -``` - -Preferisci installare a mano? Un'Agent Skill è solo una cartella contenente un `SKILL.md` (più riferimenti opzionali), quindi copiarla funziona: - -- **Claude Code**: metti la cartella `agenteye-cli/` in `~/.claude/skills/` (ogni progetto) o `/.claude/skills/` (solo quel repository). Claude Code la scopre automaticamente — verifica con la lista `/skills`, o semplicemente fai una domanda che corrisponde alla sua descrizione. -- **Codex (OpenAI)**: Codex legge lo stesso `SKILL.md`. Il `agents/openai.yaml` in bundle imposta `allow_implicit_invocation: true`, quindi Codex seleziona automaticamente la competenza quando un'attività corrisponde; altrimenti invocala esplicitamente come `$agenteye-cli`. - ---- - -## Sicurezza: le mutazioni NON chiedono conferma quando un agente esegue la CLI - -> **Avvertenza:** Leggi questo prima di lasciare che un agente faccia cambiamenti. - -La CLI `agenteye` normalmente chiede *"sei sicuro?"* prima di un'azione distruttiva. **Salta automaticamente quella conferma ogni volta che non è collegata a un terminale (che è esattamente come un agente di codifica la esegue), e `--json` la salta anche.** Quindi il prompt di sicurezza **non** attiverà per l'agente. - -La competenza è scritta per compensare: le è stato insegnato di dichiarare il comando esatto che eseguirà e ottenere il tuo **OK esplicito prima di qualsiasi cambio di stato**. Mantieni quella disciplina. Quando guidi Osservabilità Failproof AI attraverso un agente, *tu* sei il passo di conferma. I comandi che cambiano lo stato da osservare: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- i sottocomandi di scrittura `incidents`: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -Tutto sotto **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) è di sola lettura e non cambia nulla. - -Poiché l'agente agisce come **tu**, può fare solo ciò che il tuo accesso è autorizzato a fare; le autorizzazioni vengono risolte **per organizzazione** (vedi [Chiavi API](/it/agenteye/api-keys)). Un comando per il quale non hai autorizzazione restituisce il codice di uscita `5` con l'autorizzazione esatta denominata, quindi l'agente può dirti precisamente cosa chiedere a un amministratore invece di fallire oscuramente. - ---- - -## Cosa puoi chiederle - -Ecco come appare un vero scambio, una lettura, quindi un cambio che attende il tuo OK: - -```text -tu ▸ C'è qualcosa di rotto nell'ultimo giorno? - -agente ▸ Esecuzione: agenteye --json errors --since 24h --aggregate - 3 tipi di errore nelle ultime 24h. Il più grande è TimeoutError - (18 hit, tutti su checkout-agent), poi RateLimited (5). - Vuoi le sessioni fallite per TimeoutError? - -tu ▸ Dai a CI una chiave che può solo inviare eventi. - -agente ▸ Questo crea una chiave API. Eseguirò: - agenteye keys create ci --add events:add - OK per procedere? - -tu ▸ sì - -agente ▸ Fatto. Chiave "ci" creata con solo events:add. - Il segreto viene mostrato solo una volta, quindi salvalo ora. Non posso ristamparlo. -``` - -La competenza mappa ogni intento in linguaggio naturale al giusto comando `agenteye`, scoprendo prima i valori validi (`list `, `whoami`) quindi non indovina, e dichiara il comando esatto prima di qualsiasi cambio. Altri esempi: - -- *"C'è qualcosa di rotto / fallito nelle ultime 24 ore?"* → `errors --since 24h --aggregate`, poi un breakdown. -- *"Perché la sessione `run-001` ha fallito?"* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *"Come sta andando la qualità questa settimana?"* → `evals --aggregate --since 7d`, poi approfondisci nei run con punteggio basso. -- *"Dai a CI una chiave che può solo inviare eventi."* → `keys create ci --add events:add` (dichiara il comando, lo crea e cattura il segreto monouso). -- *"Chi ha accesso? Rendi Dana di sola lettura."* → `users list` → `users update dana@… --permission-set read-only` (dopo confirmare con te). -- *"Conferma l'incident in corso e assegnalo a me."* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`. - -Per i comandi esatti, flag e forme JSON dietro questi, vedi il riferimento [CLI](/it/agenteye/cli) e [Ricette CLI per agenti](/it/agenteye/cli-recipes). - ---- - -## Prossimi passi - -- **[CLI](/it/agenteye/cli)**: riferimento completo di comando e flag per `agenteye`. -- **[Ricette CLI per agenti](/it/agenteye/cli-recipes)**: pattern `jq` da copiare e incollare e gestione del codice di uscita. -- **[Competenza agente Evaluator](/it/agenteye/evaluator-skill)**: la competenza gemella, per costruire l'evaluator i cui punteggi `agenteye evals` legge. -- **[Competenza agente Python SDK](/it/agenteye/python-sdk-skill)**: la competenza gemella, per strumentare un agente in modo che emetta la telemetria che `agenteye` legge. -- **[Assistente AI](/it/agenteye/assistant)**: l'assistente nella dashboard (da non confondere con questa competenza di terminale). -- **[Chiavi API](/it/agenteye/api-keys)**: il modello di autorizzazione per organizzazione che limita quello che la competenza può fare. \ No newline at end of file diff --git a/docs/it/agenteye/codex-capture.mdx b/docs/it/agenteye/codex-capture.mdx deleted file mode 100644 index cddad4b1..00000000 --- a/docs/it/agenteye/codex-capture.mdx +++ /dev/null @@ -1,56 +0,0 @@ ---- ---- -title: "Acquisizione di sessioni Codex" -description: "Integra le sessioni locali di OpenAI Codex del tuo team in AgentEye come sessioni ed eventi ordinari — senza cambiare il modo in cui eseguono Codex." ---- - -I tuoi ingegneri usano già OpenAI Codex ogni giorno. L'acquisizione di sessioni Codex porta quelle sessioni di programmazione in AgentEye come sessioni ed eventi ordinari, così puoi cercarle, riprodurle e valutarle insieme a tutto il resto che osservi. Complementa l'[SDK Python](/it/agenteye/python-sdk): l'SDK strumenta gli agenti che scrivi, mentre questo cattura il lavoro Codex che il tuo team fa già — senza cambiare il modo in cui lo eseguono. - -Un piccolo collettore in background legge i trascritti delle sessioni locali di Codex man mano che vengono scritti e li invia ad AgentEye. Un collettore per macchina cattura ogni superficie Codex locale contemporaneamente — non è necessaria una configurazione per ogni superficie. - -Lo stesso collettore cattura anche altri agenti — vedi [OpenClaw](/it/agenteye/openclaw-capture) e [Hermes](/it/agenteye/hermes-capture). Abilita ciascuno che esegui; un singolo collettore può catturarne diversi contemporaneamente. - ---- - -## Cosa cattura - -Ogni superficie Codex che viene eseguita **localmente** produce gli stessi trascritti di sessione su disco, e il collettore li cattura tutti: - -- il **CLI** di Codex e `codex exec` -- l'**estensione VS Code / IDE** -- l'**app desktop**, quando esegue una sessione localmente - -Ogni sessione Codex diventa una [sessione](/it/agenteye/sessions) di AgentEye; i suoi messaggi utente e assistente, il ragionamento, le chiamate agli strumenti, i risultati degli strumenti e l'utilizzo dei token diventano gli [eventi](/it/agenteye/event-stream) corrispondenti. La superficie da cui proviene ogni sessione (CLI, IDE o desktop) viene registrata, così puoi distinguerle. - -> **Le sessioni cloud non vengono catturate.** L'app desktop esegue sempre più spesso le sessioni nel cloud Codex e mantiene solo i loro metadati sulla macchina — non c'è un trascritto locale da leggere. Solo le sessioni eseguite localmente vengono catturate. - ---- - -## Attivalo - -L'acquisizione è disabilitata finché non la abiliti. Installa il collettore con una chiave API che dispone del permesso `events:add` (vedi [Chiavi API](/it/agenteye/api-keys)), e attiva l'acquisizione di Codex: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -Questo installa il collettore, lo registra come servizio in background e avvia l'acquisizione. Verifica che sia in esecuzione: - -```bash -agenteye-collector health -``` - -Al primo avvio, le tue sessioni Codex esistenti vengono riprese una volta e la nuova attività quindi viene trasmessa in streaming in pochi secondi. I file di Codex vengono letti solo — mai modificati, spostati o eliminati — e ogni sessione viene inviata esattamente una volta, anche tra i riavvii. - ---- - -## Dove viene visualizzato - -Le sessioni catturate appaiono in **Sessions** e i loro eventi nel flusso **Events**, come per qualsiasi altro agente che osservi — quindi la [riproduzione delle sessioni](/it/agenteye/sessions), la [ricerca](/it/agenteye/queries), le [valutazioni](/it/agenteye/evaluations) e gli [avvisi](/it/agenteye/alerts) funzionano tutti su di esse. Filtra per l'agente Codex per vederle da sole. - ---- - -## Privacy - -I trascritti di Codex contengono la sessione completa — incluso l'output dei comandi, i contenuti dei file e tutto ciò che Codex ha letto o scritto — e possono contenere segreti. Le sessioni catturate vengono inviate così come sono, quindi abilita l'acquisizione solo su macchine e per team dove centralizzare quel contenuto in AgentEye è appropriato, e dai al collettore una chiave ristretta a `events:add` solo. Vedi [Security](/it/agenteye/security) per vedere come i tuoi dati vengono mantenuti isolati. \ No newline at end of file diff --git a/docs/it/agenteye/concepts.mdx b/docs/it/agenteye/concepts.mdx deleted file mode 100644 index 3517b456..00000000 --- a/docs/it/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "Concetti" -description: "Il vocabolario dietro Failproof AI Observability — eventi, sessioni, valutazioni, audit, risultati e incidenti — definiti in un unico posto." ---- - - -Questa pagina definisce il vocabolario usato da Failproof AI Observability. Se un termine in un'altra guida ti risulta sconosciuto, la sua definizione si trova qui. Non è necessario leggerla da cima a fondo: puoi scorrerla o tornare quando incontri una parola di cui vuoi precisare il significato. - ---- - -## Il modello dei dati - -**Event** -L'unità più piccola di dati. Un event registra un singolo step che il tuo agente ha eseguito: un `tool_use`, una `model_request`, un `hook_completed`, un `error`, e così via. Il tuo agente emette event attraverso [Python SDK](/it/agenteye/python-sdk); vengono visualizzati in tempo reale nella pagina **Events**. - -**Session** -Un'esecuzione dell'agente, identificata da un `session_id`. Una session è l'insieme di tutti gli event che condividono quell'id, riepilogati in una singola riga nella pagina **Sessions** e disegnati come grafo di esecuzione nella sua pagina di dettaglio. Una session solitamente inizia con `agent_start` e termina con `agent_end`. - -**Agent** -Un attore nominato all'interno di un'esecuzione, identificato da un `agent_id`. Un'esecuzione può coinvolgere diversi agenti: ad esempio un planner che genera un sub-agente summarizer. I sub-agenti contengono un `parent_id`, che consente a Failproof AI Observability di disegnarli su corsie separate nel grafo di esecuzione. - -**Environment** -Un'etichetta per il luogo dove l'esecuzione è avvenuta: `production`, `staging`, `dev`. La configuri una sola volta quando imposti l'SDK. Quasi tutte le pagine del dashboard possono filtrare per environment. - -**Context-window fill** -La percentuale della finestra di contesto di un modello che una risposta ha consumato. Failproof AI Observability la registra negli event `model_response` per i modelli che riconosce, in modo che la crescita del prompt e la imminente compattazione siano visibili direttamente nel flusso degli event. - ---- - -## Qualità - -**Evaluation** -Un punteggio di qualità per una session completata, prodotto da un servizio di scoring che gestisci. Le evaluation sono opzionali: finché non colleghi un evaluator, le session vengono registrate ma non valutate. Ogni evaluation può contenere diversi punteggi denominati (ad esempio `helpfulness`, `factuality`, `tool_efficiency`), ognuno con una breve nota di motivazione. Vedi [Evaluation suite](/it/agenteye/evaluation-suite). - -**Score key** -Il nome di una dimensione che un evaluator riporta, come `helpfulness`. Gli alert e gli audit possono monitorare uno score key specifico nel tempo. - -**Evaluator** -Il tuo servizio di scoring. Failproof AI Observability effettua un POST della trascrizione di un'esecuzione completata e memorizza i punteggi che restituisce. Non fornisce un evaluator predefinito; la logica di scoring è tua. - ---- - -## Trovare e risolvere i guasti - -**Hook** -Un guardrail o effetto collaterale che il tuo framework di agenti esegue attorno a uno step: un controllo di sicurezza dei contenuti, redazione della PII, un budget guard. Gli hook emettono event `hook_triggered` / `hook_completed` con un `outcome` (allow, deny, modify), e hanno una propria pagina di observe. - -**Alert rule** -Una regola che si attiva quando una metrica supera una soglia che hai impostato: error rate, p95 latency, costo in token, o un punteggio di un evaluator. Quando una regola si attiva, apre un incident e notifica i tuoi canali scelti (email, Slack, webhook, in-dashboard). Vedi [Alerts](/it/agenteye/alerts). - -**Incident** -Una questione aperta creata quando un'alert rule si attiva. Gli incident hanno un ciclo di vita (acknowledge, assign, resolve) e una timeline di attività che registra ogni azione. Puoi anche aprirne uno manualmente. - -**Audit** -Un'indagine ricorrente (oraria fino settimanale) che estrae dai tuoi log *tra* le session i pattern di guasto che non hai scritto una regola per: cluster di errori, punteggi bassi, outlier di latenza, loop di chiamate tool, ed esecuzioni che non sono mai terminate. Dove un alert monitora una metrica che già conosci, un audit ti dice cosa guardare dopo. Vedi [Audits](/it/agenteye/audits). - -**Finding** -Un risultato classificato e supportato da prove ottenuto da un'esecuzione di audit. Un finding nomina un pattern, si collega alle exact session dietro di esso, e ha un ciclo di vita di triage (acknowledge, resolve, mute, dismiss). Failproof AI Observability deduplica i finding di esecuzione in esecuzione così un pattern noto si aggiorna invece di accumularsi. - -**The AI assistant** -La chat in-dashboard che risponde a domande sui tuoi agenti in linguaggio naturale, sui tuoi dati. È read-only per impostazione predefinita; qualsiasi cosa creer (una query salvata, un dashboard) è soggetta a approvazione, e non potrà mai eliminare. Vedi [AI assistant](/it/agenteye/assistant). - ---- - -## Eseguirlo - -**Organization (tenant)** -Uno spazio di lavoro isolato. Un'istanza di Failproof AI Observability può ospitare molte organizzazioni, ognuna con i propri utenti, chiavi e dati. Ogni URL del dashboard è scoped sotto il tuo slugname dell'org (`//…`). - -**Collector** -`agenteye-collector`, il daemon leggero che gira su ogni macchina con agenti, raggruppa gli event che l'SDK scrive su disco, e li spedisce al server. - -**API key** -Un token scoped che autentica un client rispetto al server. Le chiavi portano permessi granulari (ad esempio `events:add` per il collector, scope read-only per una dashboard key). Vedi [API keys](/it/agenteye/api-keys). - -**Server** -Il servizio di ingest e API. Ingerisce gli event, memorizza lo stato operativo nei tuoi database, e serve il dashboard e la CLI. - -**Dashboard** -L'interfaccia web. Ogni pagina è scoped a un'organizzazione e legge attraverso l'API del server. - ---- - -## Prossimi step - -- [Overview](/it/agenteye/overview): come questi pezzi si incastrano insieme. -- [Observability](/it/agenteye/observability): le superfici di observe (Events, Sessions, Models, Tools, Hooks, Errors). \ No newline at end of file diff --git a/docs/it/agenteye/evaluator-skill.mdx b/docs/it/agenteye/evaluator-skill.mdx deleted file mode 100644 index 243583e2..00000000 --- a/docs/it/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,170 +0,0 @@ ---- -title: "Failproof AI Observability Evaluator Agent Skill" -description: "Da «penso che il nostro agente a volte funzioni male» a un servizio di scoring distribuito, con il tuo agente che decide e costruisce tutto." ---- - - -Da *«penso che il nostro agente a volte funzioni male»* a un servizio di scoring distribuito, con il tuo agente che decide e costruisce tutto. La **skill di valutazione Failproof AI Observability** (`agenteye-evaluator`) è un *Agent Skill*: una piccola cartella di istruzioni che un agente di codifica come Claude Code o Codex carica su richiesta. Insegna all'agente a capire quali dimensioni di qualità vale la pena tracciare per *il tuo* agente, quindi scrivere, testare e distribuire il [servizio di valutazione](/it/agenteye/evaluation-suite) che le punteggia. - -**Non** è uno scorer ospitato, un registro su cui caricare dati, o un sistema di plugin. Il tuo valutor rimane un tuo servizio HTTP sulla tua infrastruttura, esattamente come descritto nella guida [Evaluation suite](/it/agenteye/evaluation-suite). La skill insegna semplicemente al tuo agente a costruirlo bene, così tutto ciò che fa, potresti farlo tu scrivendo lo stesso codice. - ---- - -## La parte difficile è decidere cosa punteggiare - -La superficie dell'SDK è piccola — un decoratore e due modelli — e un agente può scriverla dal [contratto](/it/agenteye/evaluation-suite#http-contract) da solo. Non è lì che i valutor falliscono. Falliscono perché punteggiamo la cosa sbagliata, e un valutor che punteggia la cosa sbagliata è peggio di niente: produce una dashboard che tutti imparano a ignorare. - -Quindi gran parte della skill è la parte prima che esista del codice. Fa sì che l'agente ti intervisti (*«descrivi un'esecuzione andata bene; ora una andata male»*), poi tiri le tue vere sessioni attraverso la [`agenteye` CLI](/it/agenteye/cli) e le legga da cima a fondo. Queste due parti di solito non concordano, e il divario è il punto: quello che intendi misurare rispetto a quello che i tuoi transcript possono effettivamente supportare. Una dimensione sopravvive solo se è **calcolabile** dagli eventi e **discriminante** — se punteggia 0.9 sia sulla tua buona esecuzione che su quella cattiva, non insegna nulla e viene tagliata. - -Quello che torna è una proposta di 2-4 dimensioni con il ragionamento allegato, per te da approvare prima che venga scritta una riga. - -```mermaid -flowchart TD - YOU["tu: 'voglio valutazioni per il mio bot di supporto'"] --> AGENT["agente di codifica (Claude Code / Codex)
    carica la skill agenteye-evaluator"] - AGENT -->|"intervista: come appare il bene vs il male?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["le tue vere sessioni
    quello che succede davvero"] - DATA --> DIMS["2-4 dimensioni, tu approvi"] - DIMS --> SVC["il tuo servizio valutor
    agenteye-evaluator SDK"] - SVC --> SCORES["i punteggi arrivano nel dashboard
    e nelle valutazioni agenteye"] -``` - ---- - -## Come si relaziona agli altri pezzi di valutazione - -Quattro documenti riguardano il scoring e si passano il testimone in ordine: - -| Pagina | Cos'è | Usalo quando | -|---|---|---| -| **[Evaluations](/it/agenteye/evaluations)** | La funzione: punteggi nella griglia delle sessioni, dashboard, rivalutazione | Vuoi sapere cosa ottiene il scoring automatico | -| **[Evaluation suite](/it/agenteye/evaluation-suite)** | Il contratto HTTP, l'SDK, le variabili d'ambiente del server | Stai implementando o debuggando il valutor tu stesso | -| **Evaluator skill** (questo doc) | Una porta d'ingresso in linguaggio naturale per progettare *e* costruire il valutor | Vuoi passare da «voglio valutazioni» a un servizio in esecuzione | -| **[CLI skill](/it/agenteye/cli-skill)** | Una porta d'ingresso in linguaggio naturale sulla `agenteye` CLI | Vuoi *leggere* i punteggi che hai già | -| **[Python SDK skill](/it/agenteye/python-sdk-skill)** | Una porta d'ingresso in linguaggio naturale sull'instrumentazione del tuo agente | Il tuo agente non sta ancora emettendo sessioni — non c'è nulla da punteggiare | - -### vs. la CLI skill: costruire rispetto a leggere - -Le due skill sono deliberatamente non sovrapposte, e installare entrambe è la configurazione normale — l'agente sceglie tra loro in base a quello che chiedi: - -- **`agenteye-evaluator`** (questo doc) costruisce la cosa che *produce* punteggi. Il suo lavoro finisce quando i punteggi arrivano per la prima volta. -- **[`agenteye-cli`](/it/agenteye/cli-skill)** legge punteggi che già esistono (`agenteye evals`). «La qualità è diminuita questa settimana?» è sua domanda, non di questa skill. - ---- - -## Prerequisiti - -1. La **`agenteye` CLI installata e connessa** (`pipx install agenteye`, poi `agenteye login`). La skill vi fa affidamento due volte: per tirare le vere sessioni su cui progetta, e per confermare che i tuoi punteggi sono arrivati alla fine. Il tuo login ha bisogno di `events:read`, più `evaluations:read` per quel controllo finale. Come con la CLI skill, **non può** completare per te il login con codice monouso inviato per email. -2. **Un posto dove il valutor vive.** Viene costruito in un'immagine ed eseguito come servizio a lungo termine, quindi ha bisogno di un vero repo, non di un file temporaneo. I valutor spesso vivono nel loro repo, separato dall'agente che viene punteggiato — la skill cerca uno esistente e chiede prima di scaffoldare uno nuovo. -3. **La wheel dell'SDK `agenteye-evaluator`** — leggi la prossima sezione prima che il tuo agente inizi a digitare comandi `pip`. - ---- - -## Dove ottenerlo - -La skill è pubblicata nella collezione pubblica di skill di Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -Il repository è pubblico e la skill non ha bisogno di credenziali proprie — guida solo la `agenteye` CLI con la sessione in cui *tu* ti sei connesso, e scrive codice nel *tuo* repo. Nota che viene spedita come propria cartella e **non** è dentro il pacchetto `pipx install agenteye`, quindi non cercarla lì. - -## Installazione della skill - -Il percorso più veloce è la CLI [`skills`](https://skills.sh), che scarica la cartella e la mette dove il tuo agente guarda: - -```bash -# Claude Code, questo progetto solo -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# ogni progetto (installa a ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codex invece -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -Poi gestiscila come qualsiasi altra skill: - -```bash -npx skills list -a claude-code # cosa è installato -npx skills update agenteye-evaluator # tira l'ultima versione -npx skills remove agenteye-evaluator # rimuovila -``` - -Preferisci installare a mano? Un Agent Skill è solo una cartella contenente un `SKILL.md` (più riferimenti opzionali), quindi copiarlo funziona anche: - -- **Claude Code**: metti la cartella `agenteye-evaluator/` in `~/.claude/skills/` (ogni progetto) o `/.claude/skills/` (solo quel repo). Claude Code la scopre automaticamente — verifica con la lista `/skills`, o semplicemente chiedi valutazioni. -- **Codex (OpenAI)**: Codex legge lo stesso `SKILL.md`. Il `agents/openai.yaml` incluso imposta `allow_implicit_invocation: true`, quindi Codex auto-seleziona la skill quando un compito corrisponde; altrimenti invocare esplicitamente come `$agenteye-evaluator`. - ---- - -## L'SDK non è su PyPI pubblico - -> **Avviso:** Leggi questo prima di lasciare che un agente installi l'SDK. - -La skill è pubblica; l'SDK che guida non lo è. `agenteye-evaluator` viene spedito solo come artefatto di rilascio privato, e a differenza di `agenteye`, il nome è **non rivendicato su PyPI pubblico** — quindi un semplice `pip install agenteye-evaluator` potrebbe tirare il pacchetto di uno straniero nel servizio che legge i tuoi transcript di produzione. Questo è un problema di supply chain, non un errore di battitura. - -La skill lo sa e funziona secondo una scala di installazione, fermandosi al primo gradino che si applica: il codice del monorepo se sei dentro il repo AgentEye, altrimenti la wheel di rilascio privata da GitHub Releases (serve accesso), e se nessuno dei due è raggiungibile **si ferma e ti dice di chiedere al tuo contatto Failproof AI la wheel** piuttosto che improvvisare. - -Quindi se il tuo agente propone un semplice `pip install agenteye-evaluator` da PyPI pubblico, quello è il segnale che la skill non è mai stata caricata. Fermati lì e controlla che sia installata. - ---- - -## Cosa puoi chiedergli - -Un vero percorso completo inizia con una richiesta vaga e finisce con un design approvato, non con il codice: - -```text -tu ▸ Voglio valutazioni per il nostro bot di supporto. Penso - sia a volte cattivo. - -agente ▸ Prima che scriva qualsiasi cosa: descrivi un'esecuzione - andata bene, e una andata male. - -tu ▸ Bene: risponde dai documenti. Male: inventa cose, o rinuncia - e dice «contatta il supporto». - -agente ▸ Esecuzione: agenteye --json sessions --status error --since 7d - Leggo 5 sessioni, incluse due che chiameresti cattive. - - Le tue parole dicono fattualità e defezione. I transcript - concordano sulla defezione (2 di 5 terminano con una - consegna al supporto, ed è evidente in model_response.content), - ma la fattualità ha bisogno di una ground truth che i tuoi - eventi non portano. - - Quindi punteggierei tre cose: - - deflection regole, economico, separa le tue due esecuzioni - - tool_efficiency regole, cattura il loop di riprova in run-004 - - groundedness giudice LLM: ogni affermazione è nei documenti che ha recuperato? - - «factuality» avrebbe bisogno di ground truth che non abbiamo; - groundedness è la cosa più vicina che i tuoi transcript possono - effettivamente supportare. OK con questi tre nomi? Rinominare - dopo spacca la cronologia. - -tu ▸ sì - -agente ▸ Nessun valutor in questo repo. Dovrei scaffoldarne uno qui, - o ne hai uno altrove? -``` - -Da lì scrive le dimensioni basate su regole per prime (gratis, istantanee, deterministiche), le testa rispetto a una sessione catturata reale incluse quelle vuote e mai finite che fanno crashare i valutor ingenui, e raggiunge solo un giudice LLM sulla dimensione soggettiva. Conosce i [limiti del dispatcher](/it/agenteye/evaluation-suite#configuring-the-server) — un timeout di richiesta di 30 secondi e 8 chiamate concorrenti deployment-wide — quindi se il giudice non si adatterà in modo affidabile, va asincrono con `JobPending` piuttosto che lasciare che il tuo giudice sia cancellato e riprovato cinque volte cinque volte il costo. - -Poi distribuisce, imposta le due variabili d'ambiente del server, e conferma con `agenteye --json evals --session-id ` che i punteggi sono effettivamente arrivati. I punteggi che arrivano sono l'unica prova. - ---- - -## Cosa stare attenti - -- **I nomi delle dimensioni sono quasi permanenti.** Le chiavi di score sono stringhe arbitrarie e la piattaforma tende quello che invii, il che significa che nulla downstream corregge una scelta sbagliata. Rinominare dopo e la cronologia si spacca: le vecchie sessioni mantengono la vecchia chiave e il trend si interrompe. Per questo la skill ottiene l'approvazione esplicita prima di scrivere il codice — prendi quel prompt seriamente. -- **Le fixture sono veri transcript di produzione.** Progettare rispetto a sessioni reali significa tirarle su disco, e possono contenere dati dei clienti. La skill chiede prima di commetterli a git; se hai dubbi, mantieni `fixtures/` fuori dal repo e fai in modo che ogni sviluppatore tiri i propri. -- **L'agente scrive e distribuisce un servizio che legge ogni transcript.** Agisce come te, limitato dalle autorizzazioni del login della tua CLI, ma rivedi il valutor come qualsiasi altro codice che tocca dati di produzione. - ---- - -## Prossimi passi - -- **[Evaluation suite](/it/agenteye/evaluation-suite)**: il contratto HTTP, l'SDK, e le variabili d'ambiente del server che la skill configura. -- **[Evaluations](/it/agenteye/evaluations)**: dove i punteggi compaiono una volta che arrivano. -- **[CLI skill](/it/agenteye/cli-skill)**: la skill gemella, per leggere i risultati piuttosto che costruire il valutor. -- **[CLI](/it/agenteye/cli)**: il riferimento dei comandi dietro i dati di sessione su cui la skill progetta. \ No newline at end of file diff --git a/docs/it/agenteye/hermes-capture.mdx b/docs/it/agenteye/hermes-capture.mdx deleted file mode 100644 index 23c48a6f..00000000 --- a/docs/it/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Acquisizione della sessione Hermes" -description: "Porta le sessioni del gateway Hermes del tuo team — Slack, Telegram, CLI ed esecuzioni pianificate — in AgentEye come sessioni ed eventi ordinari." ---- - -[Hermes](https://hermes-agent.nousresearch.com) risponde al tuo team da qualsiasi luogo in cui già lavora — Slack, Telegram, CLI, esecuzioni pianificate. L'acquisizione della sessione Hermes porta tutto questo in AgentEye come sessioni ed eventi ordinari, in modo che l'assistente con cui il tuo team parla ogni giorno sia osservabile quanto gli agenti che scrivi tu stesso. - -Un piccolo collector in background legge l'archivio di sessioni locale di Hermes mentre viene scritto e invia le sessioni ad AgentEye. Funziona allo stesso modo del capture di [Codex](/it/agenteye/codex-capture) e [OpenClaw](/it/agenteye/openclaw-capture), e un collector può acquisire più agenti contemporaneamente. - ---- - -## Cosa acquisisce - -Ogni sessione Hermes sulla macchina viene acquisita, da qualsiasi canale provenga. Ognuna diventa una [sessione](/it/agenteye/sessions) di AgentEye; i suoi messaggi di utente e assistente, le chiamate ai tool e i risultati dei tool diventano gli [eventi](/it/agenteye/event-stream) corrispondenti. - -Il canale da cui è iniziata una sessione — Slack, Telegram, CLI o un'esecuzione pianificata — viene registrato sulla sessione, così puoi distinguerle e filtrare una alla volta. Insieme vengono il modello su cui è stata eseguita la sessione, la chat e la persona da cui è stata avviata, e, quando una sessione ha generato un'altra, il collegamento al suo genitore. - -Le sessioni appaiono non appena Hermes le avvia, indipendentemente dal fatto che sia stato detto qualcosa, e la risposta di un turno e le sue chiamate ai tool mantengono l'ordine in cui effettivamente si sono verificate. Quando una sessione termina, ottieni anche il motivo della terminazione, il costo e quanti token ha utilizzato. - ---- - -## Attivalo - -L'acquisizione è disattivata finché non la abiliti. Installa il collector con una chiave API che dispone dell'autorizzazione `events:add` (vedi [Chiavi API](/it/agenteye/api-keys)) e attiva l'acquisizione di Hermes: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -Questo installa il collector, lo registra come servizio in background e avvia l'acquisizione. Conferma che è in esecuzione: - -```bash -agenteye-collector health -``` - -Acquisendo più di un agente sulla stessa macchina? Aggiungi il flag di ognuno allo stesso comando — ad esempio `--hermes-enabled --codex-enabled`. - -Al primo avvio, le tue sessioni Hermes esistenti vengono riportate una sola volta e la nuova attività viene trasmessa in streaming entro pochi secondi. I dati di Hermes stesso vengono solo letti — mai modificati o eliminati — e ogni messaggio viene inviato una sola volta, anche tra i riavvii. - -`health` ti dice anche se tutto ciò che il collector ha acquisito è effettivamente arrivato ad AgentEye. Se un batch non può essere consegnato, viene mantenuto e ritentato piuttosto che scartato, e il controllo segnala uno stato non integro mentre c'è ancora qualcosa in sospeso — quindi "integro" significa che i tuoi dati sono arrivati, non semplicemente che il processo è attivo. - ---- - -## Dove compare - -Le sessioni acquisite appaiono in **Sessions**, e i loro eventi nel flusso **Events**, esattamente come qualsiasi altro agente che osservi — quindi [session replay](/it/agenteye/sessions), [ricerca](/it/agenteye/queries), [valutazioni](/it/agenteye/evaluations) e [avvisi](/it/agenteye/alerts) funzionano tutti su di esse. Filtra per l'agente Hermes per vederle da sole. - ---- - -## Privacy - -Le sessioni di Hermes contengono la trascrizione completa — incluso l'output dei comandi, i contenuti dei file e tutto ciò che l'agente ha letto o scritto — e possono contenere segreti. Le sessioni acquisite vengono inviate così come sono, quindi abilita l'acquisizione solo dove centralizzare quel contenuto in AgentEye è appropriato, e fornisci al collector una chiave limitata a `events:add` solamente. Vedi [Security](/it/agenteye/security) per scoprire come i tuoi dati vengono mantenuti isolati. \ No newline at end of file diff --git a/docs/it/agenteye/observability.mdx b/docs/it/agenteye/observability.mdx deleted file mode 100644 index 79a41ead..00000000 --- a/docs/it/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Observe" -description: "Le superfici di observe sono il luogo dove osservare quello che i tuoi agent stanno facendo in tempo reale e analizzare nel dettaglio ogni singola esecuzione." ---- - - -Le superfici di observe sono il luogo dove osservare quello che i tuoi agent stanno facendo in tempo reale e analizzare nel dettaglio ogni singola esecuzione. Tutto qui è live, limitato alla tua organizzazione, e filtrabile per intervallo di date, environment, agent e session, così passi da "qualcosa non sembra giusto" all'esecuzione esatta in pochi secondi. - -![Lo stream di eventi live, con codifica a colori per tipo e filtrabile per environment, agent e session](/agenteye/images/events-stream.png) - -Quattro superfici, ognuna con la sua pagina: - -- **[Event stream](/it/agenteye/event-stream)**: la traccia live, passo dopo passo, di ogni esecuzione su ogni agent, più recenti per primi. La home della tua organizzazione e prima tappa per il triage. -- **[Sessions e execution graph](/it/agenteye/sessions)**: quegli eventi riepilogati in una riga per esecuzione, più una rappresentazione in stile git di come si è sviluppata ogni esecuzione. -- **[Performance metrics](/it/agenteye/telemetry)**: heat-map di latenza e vitals p50/p95/p99 per i tuoi modelli, tool e hook, così uno spike anomalo emerge dalla mediana. -- **[Error tracking](/it/agenteye/error-tracking)**: una superficie di triage unica per tutto quello che è andato storto, un click da un alert attivo all'esecuzione che ha causato il problema. - -## Correlati - -- [Evaluations](/it/agenteye/evaluations): valuta ogni esecuzione per qualità. -- [Alerts](/it/agenteye/alerts): trasforma qualsiasi soglia in una regola di paging. -- [Audits](/it/agenteye/audits): lascia che Failproof AI Observability trovi pattern di errori tra le session per te. -- [CLI e agents](/it/agenteye/cli-and-agents): la stessa osservabilità dal tuo terminale. \ No newline at end of file diff --git a/docs/it/agenteye/openclaw-capture.mdx b/docs/it/agenteye/openclaw-capture.mdx deleted file mode 100644 index 531e7456..00000000 --- a/docs/it/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "Acquisizione di sessioni OpenClaw" -description: "Invia le sessioni locali di OpenClaw del tuo team in AgentEye come sessioni ed eventi ordinari — senza modificare il modo in cui OpenClaw funziona." ---- - -Se il tuo team utilizza [OpenClaw](https://docs.openclaw.ai), l'acquisizione di sessioni OpenClaw porta quelle sessioni in AgentEye come sessioni ed eventi ordinari, così puoi cercarle, riprodurle e valutarle insieme a tutto il resto che osservi. Completa l'[SDK Python](/it/agenteye/python-sdk): l'SDK strumenta gli agenti che scrivi, mentre questo cattura il lavoro OpenClaw che il tuo team già svolge — senza alcuna modifica al modo in cui lo eseguono. - -Un piccolo collector in background legge i transcript locali delle sessioni di OpenClaw man mano che vengono scritti e li invia ad AgentEye. Funziona nello stesso modo della [acquisizione Codex](/it/agenteye/codex-capture), e un collector può acquisire entrambi contemporaneamente. - ---- - -## Cosa cattura - -Ogni agente configurato nella configurazione OpenClaw di una macchina viene catturato dal collector di quella macchina — non c'è alcuna configurazione per singolo agente. - -Ogni sessione OpenClaw diventa una [sessione](/it/agenteye/sessions) di AgentEye; i suoi messaggi utente e assistente, le chiamate di strumenti e i risultati degli strumenti diventano i corrispondenti [eventi](/it/agenteye/event-stream). - ---- - -## Attivalo - -L'acquisizione è disattivata finché non la abiliti. Installa il collector con una chiave API che dispone dell'autorizzazione `events:add` (vedi [Chiavi API](/it/agenteye/api-keys)) e attiva l'acquisizione OpenClaw: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -Questo installa il collector, lo registra come servizio in background e inizia l'acquisizione. Conferma che è in esecuzione: - -```bash -agenteye-collector health -``` - -Stai acquisendo più di un agente sulla stessa macchina? Aggiungi il flag di ciascuno allo stesso comando — ad esempio `--openclaw-enabled --codex-enabled`. - -Al primo avvio, le tue sessioni OpenClaw esistenti vengono riempite una volta e la nuova attività viene trasmessa entro pochi secondi. I file di OpenClaw vengono solo letti — mai modificati, spostati o eliminati — e ogni sessione viene inviata esattamente una volta, anche attraverso i riavvii. - ---- - -## Dove appare - -Le sessioni acquisite appaiono in **Sessions**, e i loro eventi nel flusso **Events**, come qualsiasi altro agente che osservi — quindi la [riproduzione della sessione](/it/agenteye/sessions), la [ricerca](/it/agenteye/queries), le [valutazioni](/it/agenteye/evaluations) e gli [avvisi](/it/agenteye/alerts) funzionano tutti su di essi. Filtra per l'agente OpenClaw per vederli da soli. - ---- - -## Privacy - -I transcript di OpenClaw contengono la sessione completa — incluso l'output dei comandi, i contenuti dei file e qualsiasi cosa l'agente abbia letto o scritto — e possono contenere segreti. Le sessioni acquisite vengono inviate così come sono, quindi abilita l'acquisizione solo su macchine e per team dove centralizzare quel contenuto in AgentEye è appropriato, e fornisci al collector una chiave limitata a `events:add` solamente. Vedi [Sicurezza](/it/agenteye/security) per come i tuoi dati rimangono isolati. \ No newline at end of file diff --git a/docs/it/agenteye/python-sdk-skill.mdx b/docs/it/agenteye/python-sdk-skill.mdx deleted file mode 100644 index d46dc12e..00000000 --- a/docs/it/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,132 +0,0 @@ ---- ---- -title: "Failproof AI Observability Python SDK Agent Skill" -description: "Da un agente senza strumentazione a eventi che puoi visualizzare, con il tuo agente di codifica che trova i punti di strumentazione, li scrive e verifica che siano stati implementati." ---- - -Dì al tuo agente di codifica *"aggiungi Failproof AI Observability a questo agente"* e lascia che legga il tuo loop, determini dove deve andare la strumentazione, la scriva e verifichi gli eventi prima di dichiarare il lavoro completato. - -La **skill Python SDK** (`agenteye-python-sdk`) è una *Agent Skill*: una cartella di istruzioni che un agente di codifica come Claude Code o Codex carica on demand quando un'attività corrisponde. Insegna all'agente come usare [Python SDK](/it/agenteye/python-sdk) — non è una libreria e non cambia nulla nel funzionamento dell'SDK. - -## La strumentazione è facile da scrivere e facile da sbagliare silenziosamente - -L'SDK è piccolo: tredici metodi di evento, tutti solo keyword. Un agente di codifica può leggere il riferimento [Python SDK](/it/agenteye/python-sdk) e produrre una strumentazione plausibile in un minuto. - -Il problema è che questo SDK non solleva eccezioni quando sbagli, e la strumentazione sbagliata assomiglia esattamente a quella giusta finché qualcuno non apre un dashboard e lo trova vuoto. Gli errori che consumano tempo sono tutti silenzi: - -| L'errore | Quello che vedi | -|---|---| -| Nessun `agent_start` | Ogni evento arriva. Zero sessioni. | -| Ambiente mai impostato | Tutto funziona, archiviato sotto `dev`. | -| `outcome="failure"` | L'esecuzione appare verde — solo `failed`, `error`, `timeout`, `rejected` contano. | -| Un nome di campo con typo | Accettato e archiviato come nuovo campo. | -| Eventi emessi da un thread pool | Silenziosamente scartati. | - -Nessuno di questi solleva eccezioni. Nessuno appare nei test. Ognuno è nella skill, enunciato come contratto con il controllo che lo cattura. - -## Quello che fa, in ordine - -La skill esegue gli stessi tre passaggi che farebbe un ingegnere attento: - -1. **Pianificazione.** Legge il tuo loop di agente e pone le due domande a cui solo tu puoi rispondere: cosa conta come un'esecuzione (il tuo `session_id`) e chi sono gli attori distinguibili (il tuo `agent_id`). Raggiunge un accordo su queste questioni prima di scrivere codice, perché cambiarle in seguito dividerà la tua cronologia e romperà i trend. -2. **Scrittura.** Associa l'identità una volta per esecuzione piuttosto che trascinandola attraverso ogni sito di chiamata, e sceglie una forma thread-safe — un dettaglio importante, perché il collegamento ovvio silenziosamente mescola due esecuzioni sovrapposte in una sessione. -3. **Verifica.** Esegue il tuo agente e legge i file di evento risultanti, verificando che `agent_start` sia presente, l'ambiente sia corretto e che un'esecuzione abbia prodotto una sessione. - -Questo terzo passaggio è quello che la gente salta. L'SDK scrive eventi in file locali, quindi un'integrazione completa può essere provata su un laptop senza server, senza chiave API e senza rete — ed è esattamente per questo che la skill insiste nel farlo. - -## Come si relaziona con le altre skill - -Tre skill, una separazione netta: - -| Skill | Usala quando | Cosa modifica | -|---|---|---| -| **Python SDK skill** (questa pagina) | Vuoi che il tuo agente *emetta* telemetria — "aggiungi observability", "perché il mio agente non appare?" | Scrive codice nel repo del tuo agente. Non legge nulla. | -| **[Evaluator skill](/it/agenteye/evaluator-skill)** | Vuoi *valutare* le esecuzioni — "cosa dovremmo misurare?" | Scrive codice nel tuo repo; legge telemetria | -| **[CLI skill](/it/agenteye/cli-skill)** | Vuoi *leggere* cosa è successo, o gestire il tuo deployment | Guida la CLI come te, incluse le modifiche | - -Si passano il testimone in quest'ordine: questa skill fa fluire gli eventi, l'evaluator li valuta, la CLI li legge indietro. Non c'è nulla da valutare e nulla da leggere finché il tuo agente non emette sessioni, quindi se inizi da zero, inizia da qui. - -## Prerequisiti - -1. **Python 3.10+** e la codebase dell'agente che vuoi strumentare. -2. **L'SDK.** È distribuito ai clienti come wheel privato piuttosto che da un indice pubblico — l'onboarding spiega come ottenerlo e installarlo. La skill conosce il percorso di installazione e ti chiederà piuttosto che indovinare se non lo trova. -3. **Nient'altro.** Nessun login al dashboard, nessuna chiave API, nessuna rete. La skill verifica contro i file di evento che l'SDK scrive, quindi può terminare e provare il suo lavoro offline. - -## Dove ottenerla - -La skill si trova nella collezione pubblica [`FailproofAI/skills`](https://github.com/FailproofAI/skills): - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -Aggiungi `-g` per installarla per ogni progetto invece che solo per quello corrente, e `--copy` se il tuo ambiente non segue i symlink. Per Codex, passa `-a codex`. - -## Installarla manualmente - -Agent Skills sono cartelle contenenti un `SKILL.md` più riferimenti. Se preferisci non usare l'installer: - -- **Claude Code**: copia la cartella `agenteye-python-sdk/` in `~/.claude/skills/` (ogni progetto) o `/.claude/skills/` (solo quel repo). Claude Code la scopre automaticamente — controlla la lista `/skills`, o chiedile semplicemente qualcosa che corrisponda. -- **Codex**: Codex legge lo stesso `SKILL.md`. Il `agents/openai.yaml` incluso imposta `allow_implicit_invocation: true`, quindi è auto-selezionato quando un'attività corrisponde; altrimenti invocalo come `$agenteye-python-sdk`. - -Esegui il tuo agente **nel repository contenente il codice che vuoi strumentare** — la skill legge il tuo loop di agente prima di proporre qualcosa. - -## Che aspetto ha una sessione - -```text -you ▸ Add Failproof AI Observability to this agent. - -agent ▸ Read your loop — one tool dispatcher in tools.py, one Anthropic - wrapper in llm.py. Two questions before I write anything. - - What counts as one run? You handle requests concurrently, so I'd - use the request id you already have in ctx.request_id — then a - session here and a request in your logs are the same string. - - And who are the actors? I see a planner path and a researcher - path. Label them separately, or is that one agent to you? - -you ▸ Request id is right. They're one agent, call it "assistant". - -agent ▸ Done. Wrapped the dispatcher and the LLM client; agent_start and - agent_end bracket handle_request, including the error path. Used - a context-local for identity — your dispatcher runs tools in a - thread pool, and the simpler approach would have mixed concurrent - runs into one session without complaining. - - Verified against a real run: 11 events across 4 files — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, one session - per request, durations paired. - - One thing I did NOT do: your worker pool in batch.py submits - straight to the executor, so events from there would be dropped. - Want me to fix those too? -``` - -Il pattern da notare: ha letto il codice prima di proporre, ha posto solo le domande a cui puoi rispondere, ha riutilizzato un id che avevi già, ha scelto la forma thread-safe *perché* ha visto un thread pool, e **ha verificato leggendo gli eventi effettivi** piuttosto che dichiarare il successo — quindi ha segnalato l'unico posto dove sapeva che fallirebbe silenziosamente. - -## Quello che puoi chiederle - -- *"Perché il mio agente non appare nel dashboard?"* → percorre la scala: gli eventi vengono scritti, c'è `agent_start`, l'ambiente è giusto, il collector legge lo stesso posto. -- *"Tutto sta atterrando sotto dev."* → l'ambiente non è mai stato impostato, oppure è stato resettato da una chiamata successiva. -- *"Aggiungi token tracking."* → trova il tuo wrapper LLM e registra il modello, la ragione di stop e l'utilizzo. -- *"Strumenta anche i sub-agenti."* → una sessione, etichette di agente distinte, nidificate sotto il loro genitore. -- *"Scrivi test per la strumentazione."* → punta l'SDK a una directory temporanea e asserisce sugli eventi che ha scritto. - -## Cosa guardare - -**Lascia che verifichi.** Il passaggio che rende questa skill utile è l'ultimo — eseguire il tuo agente e leggere gli eventi indietro. Un agente che scrive strumentazione e si ferma ha fatto la metà facile, e la metà che fallisce silenziosamente è l'altra. - -**Accordati sui nomi prima del codice.** `session_id` e `agent_id` sono gli assi in base ai quali ogni superficie raggruppa. Rinominarli dopo divide la cronologia: le vecchie esecuzioni conservano le vecchie etichette e i tuoi trend si rompono. La skill chiederà; la risposta vale un minuto di riflessione. - -**Se il tuo agente propone di installare l'SDK da un indice pubblico, la skill non è stata caricata.** L'SDK è distribuito privatamente. Quella proposta è un indicatore affidabile che il tuo agente di codifica sta indovinando piuttosto che seguire la skill — fermalo lì e controlla che la skill sia installata. - -Oltre a questo, il suo raggio di esplosione è piccolo: scrive codice nella tua directory di lavoro e file di evento dove lo indichi. Non legge nulla dal tuo deployment e non cambia nulla in esso. - -## Prossimi passi - -- **[Python SDK](/it/agenteye/python-sdk)**: il riferimento completo degli eventi — ogni tipo di evento e campo — dietro ciò che questa skill automatizza. -- **[Sessions](/it/agenteye/sessions)**: quello che la tua strumentazione produce una volta che gli eventi arrivano. -- **[Evaluator Agent Skill](/it/agenteye/evaluator-skill)**: il passo successivo una volta che le esecuzioni arrivano — valutarle. -- **[CLI Agent Skill](/it/agenteye/cli-skill)**: leggere la tua telemetria indietro. \ No newline at end of file diff --git a/docs/it/cli/audit.mdx b/docs/it/audit.mdx similarity index 100% rename from docs/it/cli/audit.mdx rename to docs/it/audit.mdx diff --git a/docs/it/cli/backfill.mdx b/docs/it/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/it/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/it/cli/config.mdx b/docs/it/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/it/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/it/cli/flush.mdx b/docs/it/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/it/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/it/cli/harness.mdx b/docs/it/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/it/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/it/cli/migrate.mdx b/docs/it/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/it/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/it/cli/uninstall.mdx b/docs/it/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/it/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/it/cli/update.mdx b/docs/it/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/it/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/it/agenteye/api-keys.mdx b/docs/it/cloud/access.mdx similarity index 96% rename from docs/it/agenteye/api-keys.mdx rename to docs/it/cloud/access.mdx index 64034cf5..469d53e5 100644 --- a/docs/it/agenteye/api-keys.mdx +++ b/docs/it/cloud/access.mdx @@ -1,9 +1,9 @@ --- title: "Chiavi API" -description: "Le chiavi API controllano chi e cosa può raggiungere il tuo server Failproof AI Observability, in modo che un collector possa inviare eventi senza mai acquisire permessi di lettura o amministrazione." +description: "Le chiavi API controllano chi e cosa può raggiungere il tuo server FailproofAI Cloud, in modo che un collector possa inviare eventi senza mai acquisire permessi di lettura o amministrazione." --- -Le chiavi API controllano chi e cosa può raggiungere il tuo server Failproof AI Observability, in modo che un collector possa inviare eventi senza mai acquisire permessi di lettura o amministrazione. Ogni chiave porta uno o più permessi e ogni permesso controlla specifiche rotte del server; concedi solo quelli di cui un job ha bisogno. La maggior parte delle implementazioni crea solo tre tipi di chiave. +Le chiavi API controllano chi e cosa può raggiungere il tuo server FailproofAI Cloud, in modo che un collector possa inviare eventi senza mai acquisire permessi di lettura o amministrazione. Ogni chiave porta uno o più permessi e ogni permesso controlla specifiche rotte del server; concedi solo quelli di cui un job ha bisogno. La maggior parte delle implementazioni crea solo tre tipi di chiave. ## Le 3 chiavi di cui la maggior parte delle implementazioni ha bisogno @@ -81,7 +81,7 @@ Il server applica un catalogo fisso di permessi; ognuno controlla specifiche rot Questi permessi supportano la pagina **Utenti** del dashboard, dove gli ambiti concessi di ogni membro sono mostrati come chip: -![La pagina Utenti: una scheda per utente del dashboard con la sua email, permessi concessi e controlli di modifica/disabilitazione](/agenteye/images/users.png) +![La pagina Utenti: una scheda per utente del dashboard con la sua email, permessi concessi e controlli di modifica/disabilitazione](/cloud/images/users.png) ### Impostazioni operative @@ -90,7 +90,7 @@ Questi permessi supportano la pagina **Utenti** del dashboard, dove gli ambiti c | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | Visualizzazione delle impostazioni operative gestite dal dashboard e dei loro metadati; elenco degli override della finestra di contesto per modello; e risoluzione della finestra effettiva per un modello. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | Modifica delle impostazioni operative e aggiunta, modifica o rimozione degli override della finestra di contesto per modello. I cambiamenti interessano i nuovi eventi senza riavviare il server. | -![La pagina Impostazioni: impostazioni operative gestite dal dashboard come accessi consentiti e durate di sessione/OTP, modificabili senza riavvio](/agenteye/images/settings.png) +![La pagina Impostazioni: impostazioni operative gestite dal dashboard come accessi consentiti e durate di sessione/OTP, modificabili senza riavvio](/cloud/images/settings.png) ### Avvisi e incidenti @@ -246,7 +246,7 @@ La pagina **Chiavi** nel dashboard fornisce un'interfaccia utente per tutte le o Quando crei una chiave dal dashboard non fornisci il segreto; il dashboard genera un segreto forte per te e lo visualizza **una volta** al momento della creazione. Copialo immediatamente e conservalo in modo sicuro; non viene mai più mostrato, esattamente come con una rigenerazione. Puoi comunque selezionare i permessi della chiave direttamente o inizializzarli da un set di permessi (vedi di seguito). -![La pagina Chiavi API: una scheda per chiave che mostra il suo nome, permessi concessi e tempo di creazione, con azioni di rigenerazione e disabilitazione; le chiavi protette come `admin` sono contrassegnate](/agenteye/images/api-keys.png) +![La pagina Chiavi API: una scheda per chiave che mostra il suo nome, permessi concessi e tempo di creazione, con azioni di rigenerazione e disabilitazione; le chiavi protette come `admin` sono contrassegnate](/cloud/images/api-keys.png) --- @@ -275,5 +275,5 @@ Ne hai bisogno solo se stai aggiornando un'istanza esistente; le nuove implement ## Prossimi passi -- [Python SDK](/it/agenteye/python-sdk): come il tuo codice agente si autentica quando invia eventi. -- [Sicurezza](/it/agenteye/security): come funzionano l'accesso, il controllo degli accessi e l'isolamento dei dati per organizzazione. \ No newline at end of file +- [Python SDK](/it/cloud/sdk): come il tuo codice agente si autentica quando invia eventi. +- [Sicurezza](/it/cloud/security): come funzionano l'accesso, il controllo degli accessi e l'isolamento dei dati per organizzazione. \ No newline at end of file diff --git a/docs/it/cloud/agent-skills.mdx b/docs/it/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/it/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
    loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/it/agenteye/alerts.mdx b/docs/it/cloud/alerts.mdx similarity index 75% rename from docs/it/agenteye/alerts.mdx rename to docs/it/cloud/alerts.mdx index e82ea1e2..9f9b9d11 100644 --- a/docs/it/agenteye/alerts.mdx +++ b/docs/it/cloud/alerts.mdx @@ -3,9 +3,9 @@ title: "Avvisi" description: "Scopri nel momento stesso in cui qualcosa supera i tuoi limiti, sul canale che il tuo team già monitora, invece di venire a conoscenza dal cliente." --- -Scopri nel momento stesso in cui qualcosa supera i tuoi limiti, sul canale che il tuo team già monitora, invece di venire a conoscenza dal cliente. Imposta una regola una volta e Failproof AI Observability la controlla secondo una pianificazione, poi ti avvisa via email, Slack, webhook o direttamente nella dashboard. +Scopri nel momento stesso in cui qualcosa supera i tuoi limiti, sul canale che il tuo team già monitora, invece di venire a conoscenza dal cliente. Imposta una regola una volta e FailproofAI Cloud la controlla secondo una pianificazione, poi ti avvisa via email, Slack, webhook o direttamente nella dashboard. -![La pagina Avvisi: una griglia di schede di regole di avviso, ognuna che mostra il suo trigger, la finestra di valutazione, i canali e un badge di gravità info, warning o critical](/agenteye/images/alerts.png) +![La pagina Avvisi: una griglia di schede di regole di avviso, ognuna che mostra il suo trigger, la finestra di valutazione, i canali e un badge di gravità info, warning o critical](/cloud/images/alerts.png) *Ogni regola di avviso a colpo d'occhio: cosa monitora, con quale frequenza, dove avvisa e quanto è urgente.* ## Vieni a conoscenza dei problemi prima dei tuoi utenti @@ -21,9 +21,9 @@ Allega qualsiasi combinazione a una singola regola, e la sua gravità (info, war ## Costruisci la regola in un form, non in JSON -Descrivi cosa significa "rotto" in un form, e Failproof AI Observability scrive la regola sottostante per te. La spec JSON è semplicemente ciò che quel form produce dietro le quinte, quindi puoi leggerla per capire una regola ma raramente la digiti. +Descrivi cosa significa "rotto" in un form, e FailproofAI Cloud scrive la regola sottostante per te. La spec JSON è semplicemente ciò che quel form produce dietro le quinte, quindi puoi leggerla per capire una regola ma raramente la digiti. -![Il form per il nuovo avviso: nome e descrizione, un toggle abilitato e un picker di trigger che offre soglia di metrica, SQL personalizzato, punteggio di valutazione, valutazione composta e condizioni per evento](/agenteye/images/alert-new.png) +![Il form per il nuovo avviso: nome e descrizione, un toggle abilitato e un picker di trigger che offre soglia di metrica, SQL personalizzato, punteggio di valutazione, valutazione composta e condizioni per evento](/cloud/images/alert-new.png) *Scegli un trigger e il form mostra i campi giusti; Salva scrive la regola.* Il percorso semplice è veloce: nominalo, scegli un **trigger** (cosa monitorare), imposta la **soglia e la finestra** (quanto grave, per quanto tempo), allega almeno un **canale**, quindi **Salva** e premi **Test** per attivare una notifica sintetica e confermare che ogni destinazione è collegata. Dietro le quinte questo produce una spec piccola come: @@ -42,7 +42,7 @@ Non sei limitato a un solo tipo di segnale. Scegli il trigger che corrisponde a | **Valutazione composta** | diversi controlli di punteggio si combinano con logica any, all o at-least-N, per cogliere una regressione che si vede solo nei punteggi | | **Per evento** | arriva un singolo evento corrispondente: un agente specifico, un tipo di errore specifico o una sottostringa di messaggio | -Stai già guardando un guasto sulla [pagina Errori](/it/agenteye/error-tracking)? Ogni riga lì ha un pulsante **+ avviso** che apre questo stesso form precompilato per cogliere quel guasto esatto di nuovo, così l'incidente che hai appena triato diventa quello che ti avviserà la prossima volta. +Stai già guardando un guasto sulla [pagina Errori](/it/cloud/errors)? Ogni riga lì ha un pulsante **+ avviso** che apre questo stesso form precompilato per cogliere quel guasto esatto di nuovo, così l'incidente che hai appena triato diventa quello che ti avviserà la prossima volta. **Dove trovarlo:** Gli avvisi si trovano in `//alerts`. La creazione, modifica, eliminazione e test delle regole richiede **`alerts:write`**; `alerts:read` è sufficiente per visualizzare. Il picker dei destinatari elenca i membri della tua organizzazione per nome, così puoi avvisare una persona senza lasciare il form. @@ -52,11 +52,11 @@ Una misurazione errata non dovrebbe svegliarti. Il filtro di rumore **M di N** c ## Cosa succede quando un avviso si attiva -Una violazione apre un **incidente** e avvisa i tuoi canali una volta. Da lì il tuo team lo riconosce, assegna un proprietario, ne discute e lo risolve, tutto su un registro pulito e attribuito. Quel flusso di lavoro di triage ha la sua casa: vedi [Incidenti](/it/agenteye/incidents). +Una violazione apre un **incidente** e avvisa i tuoi canali una volta. Da lì il tuo team lo riconosce, assegna un proprietario, ne discute e lo risolve, tutto su un registro pulito e attribuito. Quel flusso di lavoro di triage ha la sua casa: vedi [Incidenti](/it/cloud/incidents). ## Correlati -- [Incidenti](/it/agenteye/incidents): traccia un avviso che si attiva da aperto a riconosciuto a risolto. -- [Tracciamento degli errori](/it/agenteye/error-tracking): raggruppa i guasti degli agenti e promuovi uno a avviso in un click. -- [Dashboard](/it/agenteye/dashboards): osserva le board condivise da cui provengono le soglie su cui avvisi. -- [CLI e agenti](/it/agenteye/cli-and-agents): crea avvisi e riconosci incidenti dal tuo terminale, o scrivili in CI. \ No newline at end of file +- [Incidenti](/it/cloud/incidents): traccia un avviso che si attiva da aperto a riconosciuto a risolto. +- [Tracciamento degli errori](/it/cloud/errors): raggruppa i guasti degli agenti e promuovi uno a avviso in un click. +- [Dashboard](/it/cloud/dashboards): osserva le board condivise da cui provengono le soglie su cui avvisi. +- [CLI e agenti](/it/cloud/cli): crea avvisi e riconosci incidenti dal tuo terminale, o scrivili in CI. \ No newline at end of file diff --git a/docs/it/agenteye/assistant.mdx b/docs/it/cloud/assistant.mdx similarity index 85% rename from docs/it/agenteye/assistant.mdx rename to docs/it/cloud/assistant.mdx index f9e75433..8836c973 100644 --- a/docs/it/agenteye/assistant.mdx +++ b/docs/it/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "Fai una domanda sui dati del tuo agente in linguaggio naturale e o --- -Fai una domanda sui dati del tuo agente in linguaggio naturale e ottieni una risposta collegata direttamente alle prove. Niente SQL da scrivere, niente dashboard da frugare — l'assistente **Failproof AI Observability** è il modo più veloce per chiunque nel tuo team di ottenere risposte sui tuoi agenti. +Fai una domanda sui dati del tuo agente in linguaggio naturale e ottieni una risposta collegata direttamente alle prove. Niente SQL da scrivere, niente dashboard da frugare — l'assistente **FailproofAI Cloud** è il modo più veloce per chiunque nel tuo team di ottenere risposte sui tuoi agenti. -![L'assistente Failproof AI Observability che risponde a una domanda in linguaggio naturale all'interno del dashboard, mostrando una tabella di Agent Activity dal vivo, una suddivisione dell'utilizzo del modello per agente e considerazioni scritte, con le query eseguite mostrate inline](/agenteye/images/assistant.png) +![L'assistente FailproofAI Cloud che risponde a una domanda in linguaggio naturale all'interno del dashboard, mostrando una tabella di Agent Activity dal vivo, una suddivisione dell'utilizzo del modello per agente e considerazioni scritte, con le query eseguite mostrate inline](/cloud/images/assistant.png) *Fai una domanda in linguaggio naturale e ottieni una risposta costruita dai tuoi dati. Qui scompone quali agenti sono più occupati e quali modelli usano, e mostra le query che ha eseguito per permetterti di verificare ogni numero.* Non c'è niente da imparare. Apri la chat, scrivi quello che vuoi sapere e segui i link che ti restituisce: @@ -36,7 +36,7 @@ Quando una risposta vale la pena conservare, chiedi all'assistente di salvarla. Sulla pagina **Queries** va ancora oltre e diventa un autore SQL: descrivi la query che desideri ("mostra il tasso di errore per agente negli ultimi 7 giorni") e trasmette l'SQL direttamente nell'editor, aprendo una vista di diff così puoi **Accettare** o **Rifiutare** la modifica prima che sia finalizzata. -![La pagina Observability Queries e il suo editor SQL](/agenteye/images/query-lab.png) +![La pagina FailproofAI Cloud Queries e il suo editor SQL](/cloud/images/query-lab.png) *La pagina Queries: questo editor è dove l'assistente trasmette una draft di query, di sola lettura, per te da accettare o rifiutare.* La creazione di SQL chiedendo qui usa il permesso `queries:run`, lo stesso dietro al pulsante **Run** dell'editor. La chat ovunque altro ha bisogno di `agent:use`. @@ -49,7 +49,7 @@ Puoi aprire l'assistente a tutti senza preoccuparti di quello che potrebbe tocca - **Ogni scrittura è in attesa di te.** Le query salvate e i dashboard vengono creati solo dopo il tuo clic esplicito su Approva, e non c'è alcuna impostazione che disattivi questo controllo. - **Non può mai eliminare nulla.** Nessun tool di eliminazione è esposto e l'assistente non possiede permessi di eliminazione. Le eliminazioni rimangono nelle tue mani, nel dashboard. - **Rimane dentro la tua organizzazione.** L'assistente vede solo l'organizzazione che stai visualizzando al momento. -- **Le tue domande rimangono tue.** I prompt e le risposte vivono nel tuo database Observability; l'analisi dei prodotti registra solo i metadati di utilizzo, mai il testo del tuo prompt. +- **Le tue domande rimangono tue.** I prompt e le risposte vivono nel tuo database FailproofAI Cloud; l'analisi dei prodotti registra solo i metadati di utilizzo, mai il testo del tuo prompt. ## Dove trovarlo @@ -57,7 +57,7 @@ L'assistente si trova lungo il bordo destro di ogni pagina sotto la tua organizz ## Correlati -- [CLI e agenti](/it/agenteye/cli-and-agents) -- [Query](/it/agenteye/queries) -- [Dashboard](/it/agenteye/dashboards) -- [Suite di valutazione](/it/agenteye/evaluation-suite) \ No newline at end of file +- [CLI e agenti](/it/cloud/cli) +- [Query](/it/cloud/queries) +- [Dashboard](/it/cloud/dashboards) +- [Suite di valutazione](/it/cloud/evaluators) \ No newline at end of file diff --git a/docs/it/agenteye/audits.mdx b/docs/it/cloud/audits.mdx similarity index 82% rename from docs/it/agenteye/audits.mdx rename to docs/it/cloud/audits.mdx index ed6a5d17..9fc94ada 100644 --- a/docs/it/agenteye/audits.mdx +++ b/docs/it/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "Audit: il tuo analista di affidabilità automatico" -description: "Failproof AI Observability cerca i guasti che non hai mai scritto una regola per gestire e ti consegna un elenco ordinato per priorità e basato su prove di esattamente cosa correggere." +description: "FailproofAI Cloud cerca i guasti che non hai mai scritto una regola per gestire e ti consegna un elenco ordinato per priorità e basato su prove di esattamente cosa correggere." --- -Failproof AI Observability cerca i guasti che non hai mai scritto una regola per gestire e ti consegna un elenco ordinato per priorità e basato su prove di esattamente cosa correggere. È come avere un analista che esamina i tuoi log ogni notte, lasciando sul tuo desk la lista ristretta al mattino. +FailproofAI Cloud cerca i guasti che non hai mai scritto una regola per gestire e ti consegna un elenco ordinato per priorità e basato su prove di esattamente cosa correggere. È come avere un analista che esamina i tuoi log ogni notte, lasciando sul tuo desk la lista ristretta al mattino.
    @@ -12,7 +12,7 @@ Failproof AI Observability cerca i guasti che non hai mai scritto una regola per *Un tour di due minuti: da un'esecuzione programmata a una correzione su cui puoi agire.* -![La pagina Audit: lavori ricorrenti che analizzano le tue sessioni cercando pattern di guasto, ognuno con una pianificazione e sensibilità](/agenteye/images/audits.png) +![La pagina Audit: lavori ricorrenti che analizzano le tue sessioni cercando pattern di guasto, ognuno con una pianificazione e sensibilità](/cloud/images/audits.png) *Ogni audit è un lavoro ricorrente che analizza le tue sessioni e redige raccomandazioni ordinate per priorità e basate su prove.* ## Smetti di indovinare cosa correggere dopo @@ -48,7 +48,7 @@ Gli audit si trovano nel dashboard a **`//audits`** (barra laterale su ## Correlati -- [Alerts](/it/agenteye/alerts): ricevi una notifica nel momento in cui viene superata una soglia che conosci già. -- [Evaluations](/it/agenteye/evaluations): assegna un punteggio a ogni esecuzione così le regressioni di qualità emergono da sole. -- [Error tracking](/it/agenteye/error-tracking): raggruppa e segui gli errori che i tuoi agent generano. -- [Incidents](/it/agenteye/incidents): traccia un problema che un audit scopre fino alla sua correzione. \ No newline at end of file +- [Alerts](/it/cloud/alerts): ricevi una notifica nel momento in cui viene superata una soglia che conosci già. +- [Evaluations](/it/cloud/evaluations): assegna un punteggio a ogni esecuzione così le regressioni di qualità emergono da sole. +- [Error tracking](/it/cloud/errors): raggruppa e segui gli errori che i tuoi agent generano. +- [Incidents](/it/cloud/incidents): traccia un problema che un audit scopre fino alla sua correzione. \ No newline at end of file diff --git a/docs/it/cloud/capture.mdx b/docs/it/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/it/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/it/agenteye/cli-recipes.mdx b/docs/it/cloud/cli-recipes.mdx similarity index 91% rename from docs/it/agenteye/cli-recipes.mdx rename to docs/it/cloud/cli-recipes.mdx index 028ee135..981b121c 100644 --- a/docs/it/agenteye/cli-recipes.mdx +++ b/docs/it/cloud/cli-recipes.mdx @@ -4,9 +4,9 @@ description: "Copia e incolla i pattern di query e le ricette jq che trasformano --- -Estrai i dati di sessione, evento e valutazione (e attiva rivalutazioni) direttamente da uno script o da un agente di codifica, con JSON pulito su stdout che si collega direttamente a `jq`. Queste ricette trasformano i dati di Failproof AI Observability in qualcosa che un utente di terminale o un agente di codifica IA (Claude Code, Cursor) può interrogare e automatizzare, senza navigare nella dashboard. +Estrai i dati di sessione, evento e valutazione (e attiva rivalutazioni) direttamente da uno script o da un agente di codifica, con JSON pulito su stdout che si collega direttamente a `jq`. Queste ricette trasformano i dati di FailproofAI Cloud in qualcosa che un utente di terminale o un agente di codifica IA (Claude Code, Cursor) può interrogare e automatizzare, senza navigare nella dashboard. -I pattern sottostanti sono pronti per il copia-incolla per la CLI di Failproof AI Observability (`agenteye`). Per l'installazione, l'autenticazione e l'elenco completo delle opzioni, vedi [CLI](/it/agenteye/cli); esegui `agenteye -h` o `agenteye -h` per l'aiuto integrato. +I pattern sottostanti sono pronti per il copia-incolla per la CLI di FailproofAI Cloud (`agenteye`). Per l'installazione, l'autenticazione e l'elenco completo delle opzioni, vedi [CLI](/it/cloud/cli); esegui `agenteye -h` o `agenteye -h` per l'aiuto integrato. ## Regole d'oro @@ -173,7 +173,7 @@ Ogni comando `--fields` accetta esattamente i nomi di campo del suo elemento. L' ## Prossimi passi -- [CLI](/it/agenteye/cli): installazione, autenticazione e il riferimento completo delle opzioni per ogni comando. -- [CLI agent skill](/it/agenteye/cli-skill): pacchetto queste ricette come una skill che il tuo agente di codifica può caricare. -- [API keys](/it/agenteye/api-keys): crea e delimita le chiavi con cui la CLI, SDK e collector si autenticano. -- [Python SDK](/it/agenteye/python-sdk): invia eventi in Failproof AI Observability così c'è dati per queste ricette da interrogare. \ No newline at end of file +- [CLI](/it/cloud/cli): installazione, autenticazione e il riferimento completo delle opzioni per ogni comando. +- [CLI agent skill](/it/cloud/agent-skills): pacchetto queste ricette come una skill che il tuo agente di codifica può caricare. +- [API keys](/it/cloud/access): crea e delimita le chiavi con cui la CLI, SDK e collector si autenticano. +- [Python SDK](/it/cloud/sdk): invia eventi in FailproofAI Cloud così c'è dati per queste ricette da interrogare. \ No newline at end of file diff --git a/docs/it/agenteye/cli.mdx b/docs/it/cloud/cli.mdx similarity index 95% rename from docs/it/agenteye/cli.mdx rename to docs/it/cloud/cli.mdx index 3661b257..9631ef51 100644 --- a/docs/it/agenteye/cli.mdx +++ b/docs/it/cloud/cli.mdx @@ -68,7 +68,7 @@ agenteye --version agenteye --help ``` -> **Nota:** L'SDK Python di Failproof AI Observability utilizza anche il nome di distribuzione `agenteye`. L'installazione del CLI con `pipx` o `uv tool` (piuttosto che `pip install` in un virtualenv condiviso) impedisce conflitti tra i due. Un semplice `pip install agenteye` va bene solo se l'SDK non è installato nello stesso ambiente. +> **Nota:** L'SDK Python di FailproofAI Cloud utilizza anche il nome di distribuzione `agenteye`. L'installazione del CLI con `pipx` o `uv tool` (piuttosto che `pip install` in un virtualenv condiviso) impedisce conflitti tra i due. Un semplice `pip install agenteye` va bene solo se l'SDK non è installato nello stesso ambiente. --- @@ -90,7 +90,7 @@ agenteye logout # revoca la sessione e cancella il token archiviato `whoami` non genera mai errori per una sessione mancante o scaduta; invece riporta `logged_in: false`, quindi uno script o agente può controllare lo stato di autenticazione in sicurezza (può comunque uscire con codice diverso da zero se nessuna URL di base è impostata o la dashboard non è raggiungibile). -**Requisiti:** la tua email deve essere autorizzata ad accedere alla dashboard (chiedi all'amministratore di Failproof AI Observability), e la dashboard deve essere raggiungibile al suo URL di base (vedi [Configurazione](#configuration)). Se richiedi un codice e nessuno arriva, probabilmente la tua email non è ancora abilitata per l'accesso alla dashboard. +**Requisiti:** la tua email deve essere autorizzata ad accedere alla dashboard (chiedi all'amministratore di FailproofAI Cloud), e la dashboard deve essere raggiungibile al suo URL di base (vedi [Configurazione](#configuration)). Se richiedi un codice e nessuno arriva, probabilmente la tua email non è ancora abilitata per l'accesso alla dashboard. --- @@ -337,13 +337,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | Autenticato, ma il tuo account manca del permesso richiesto (il messaggio lo nomina) | | 6 | La risorsa richiesta non è stata trovata (ad esempio id di sessione o incidente sconosciuto) | -Questi rendono il CLI sicuro per scripting: un agente di codifica può dirammarsi su un `4` per chiederti di ri-autenticarti, o un `5` per visualizzare il permesso mancante. Vedi [Ricette CLI per agenti](/it/agenteye/cli-recipes) per gestione dei codici di uscita e forme di output JSON. +Questi rendono il CLI sicuro per scripting: un agente di codifica può dirammarsi su un `4` per chiederti di ri-autenticarti, o un `5` per visualizzare il permesso mancante. Vedi [Ricette CLI per agenti](/it/cloud/cli-recipes) per gestione dei codici di uscita e forme di output JSON. --- ## Prossimi passaggi -- **[Ricette CLI per agenti](/it/agenteye/cli-recipes)**: pattern di query copia-incolla, one-liner `jq`, proiezioni `--fields`, gestione dei codici di uscita, e forme di output JSON, scritti per agenti di codifica che guidano il CLI. -- **[Skill agent CLI](/it/agenteye/cli-skill)**: compacchia questo CLI come una *skill* installabile di Claude Code / Codex in modo che un agente di codifica guidi l'osservabilità di Failproof AI da richieste in linguaggio naturale. -- **[Chiavi API](/it/agenteye/api-keys)**: il modello di permessi dietro `keys create --add …`. -- **[Assistente AI](/it/agenteye/assistant)**: abilitazione dell'assistente con cui `agent ask` parla. \ No newline at end of file +- **[Ricette CLI per agenti](/it/cloud/cli-recipes)**: pattern di query copia-incolla, one-liner `jq`, proiezioni `--fields`, gestione dei codici di uscita, e forme di output JSON, scritti per agenti di codifica che guidano il CLI. +- **[Skill agent CLI](/it/cloud/agent-skills)**: compacchia questo CLI come una *skill* installabile di Claude Code / Codex in modo che un agente di codifica guidi l'osservabilità di Failproof AI da richieste in linguaggio naturale. +- **[Chiavi API](/it/cloud/access)**: il modello di permessi dietro `keys create --add …`. +- **[Assistente AI](/it/cloud/assistant)**: abilitazione dell'assistente con cui `agent ask` parla. \ No newline at end of file diff --git a/docs/it/cloud/connect.mdx b/docs/it/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/it/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/it/agenteye/dashboards.mdx b/docs/it/cloud/dashboards.mdx similarity index 69% rename from docs/it/agenteye/dashboards.mdx rename to docs/it/cloud/dashboards.mdx index 3bcac2a0..252fb0c6 100644 --- a/docs/it/agenteye/dashboards.mdx +++ b/docs/it/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "Trasforma i tuoi dati live degli agent in un'unica vista condivisa Trasforma i tuoi dati live degli agent in un'unica vista condivisa che tutto il team monitora. Fissa le query che contano come grafici, e tutti vedono gli stessi numeri a colpo d'occhio, senza rieseguire una singola query. -![Un dashboard creato da query salvate: una linea eventi-per-ora, un grafico a barre errori-per-tipo, un grafico ad area di latenza e token-per-modello](/agenteye/images/dashboard-fleet.png) +![Un dashboard creato da query salvate: una linea eventi-per-ora, un grafico a barre errori-per-tipo, un grafico ad area di latenza e token-per-modello](/cloud/images/dashboard-fleet.png) *Una sola board, quattro query salvate: eventi per ora, errori per tipo, latenza e token per modello.* @@ -25,15 +25,15 @@ Troverai i tuoi dashboard su `//dashboards`. ## Fissa le query che hai già salvato -Ogni tile inizia come una query salvata. Costruisci e salva la query che ti interessa nella libreria [Query](/it/agenteye/queries) (preset incorporati più i tuoi, sui tuoi eventi e valutazioni), quindi fissala a un dashboard come il grafico che si adatta ai dati: una **linea** per le tendenze nel tempo, un **grafico a barre** per confrontare categorie, un **grafico ad area** per il volume, o una **torta** per una scomposizione percentuale. +Ogni tile inizia come una query salvata. Costruisci e salva la query che ti interessa nella libreria [Query](/it/cloud/queries) (preset incorporati più i tuoi, sui tuoi eventi e valutazioni), quindi fissala a un dashboard come il grafico che si adatta ai dati: una **linea** per le tendenze nel tempo, un **grafico a barre** per confrontare categorie, un **grafico ad area** per il volume, o una **torta** per una scomposizione percentuale. Poiché una tile è solo la tua query salvata resa come grafico, non c'è nulla da sincronizzare manualmente. Aggiorna la query una volta e ogni dashboard che la utilizza si aggiorna automaticamente. ## Monitora la qualità, non solo il volume -Il volume ti dice che gli agent sono occupati. La qualità ti dice che stanno effettivamente svolgendo il lavoro. Punta un dashboard ai tuoi [punteggi di valutazione](/it/agenteye/evaluations) e ottieni una board che traccia quanto bene stanno andando le esecuzioni nel tempo, così una regressione di qualità appare come un calo su un grafico invece di una sorpresa da un cliente. +Il volume ti dice che gli agent sono occupati. La qualità ti dice che stanno effettivamente svolgendo il lavoro. Punta un dashboard ai tuoi [punteggi di valutazione](/it/cloud/evaluations) e ottieni una board che traccia quanto bene stanno andando le esecuzioni nel tempo, così una regressione di qualità appare come un calo su un grafico invece di una sorpresa da un cliente. -![Un dashboard focalizzato sulla qualità costruito da query di valutazione salvate](/agenteye/images/dashboard-quality.png) +![Un dashboard focalizzato sulla qualità costruito da query di valutazione salvate](/cloud/images/dashboard-quality.png) *Una board di qualità mantiene i tuoi punteggi di valutazione in primo piano, proprio accanto ai numeri operativi.* @@ -41,6 +41,6 @@ Tieni una board di operazioni e una board di qualità affiancate e il tuo team h ## Correlati -- [Query](/it/agenteye/queries): costruisci e salva le query che diventano le tue tile. -- [Valutazioni](/it/agenteye/evaluations): valuta le tue esecuzioni così puoi tracciare la qualità nel tempo. -- [Avvisi](/it/agenteye/alerts): trasforma una soglia su una qualsiasi di queste metriche in un alert. \ No newline at end of file +- [Query](/it/cloud/queries): costruisci e salva le query che diventano le tue tile. +- [Valutazioni](/it/cloud/evaluations): valuta le tue esecuzioni così puoi tracciare la qualità nel tempo. +- [Avvisi](/it/cloud/alerts): trasforma una soglia su una qualsiasi di queste metriche in un alert. \ No newline at end of file diff --git a/docs/it/agenteye/error-tracking.mdx b/docs/it/cloud/errors.mdx similarity index 67% rename from docs/it/agenteye/error-tracking.mdx rename to docs/it/cloud/errors.mdx index 4fc1c271..481fb203 100644 --- a/docs/it/agenteye/error-tracking.mdx +++ b/docs/it/cloud/errors.mdx @@ -6,14 +6,14 @@ description: "Visualizza tutti gli errori prodotti dai tuoi agenti in un unico p Visualizza tutti gli errori prodotti dai tuoi agenti in un unico posto, raggruppati in modo che un picco caotico venga letto come un unico problema. Hai un percorso con un solo clic da "qualcosa è rosso" all'esatto run che ha causato il problema, senza scorrere un feed in tempo reale per trovarlo. -![La pagina Errori: un istogramma degli errori nel tempo sopra righe di errore rosse raggruppate, ciascuna con un pulsante "+ alert" con un solo clic](/agenteye/images/errors.png) +![La pagina Errori: un istogramma degli errori nel tempo sopra righe di errore rosse raggruppate, ciascuna con un pulsante "+ alert" con un solo clic](/cloud/images/errors.png) *La pagina Errori: un istogramma degli errori nel tempo, con gli errori ripetuti compressi in una riga per incidente.* ## Ogni errore, già raccolto per te Quando un agente si interrompe, non dovresti doversi scorrere un flusso di eventi in tempo reale sperando di catturare le righe rosse prima che scompaiano. La pagina **Errors** fa la raccolta per te. Riunisce tutto ciò che il dashboard mostrebbe in rosso in un'unica superficie di triage, in modo che la prima cosa che vedi sia cosa sta fallendo, non dove cercare. -E cattura più dei casi ovvi. Accanto agli eventi `error` espliciti, Failproof AI Observability evidenzia anche i fallimenti silenziosi: qualsiasi `tool_result`, `hook_completed` o `agent_end` il cui payload contiene un errore appare qui. Uno strumento che ha restituito un errore, o un hook che è uscito male, non sfugge più semplicemente perché nulla ha lanciato un'eccezione rumorosa. +E cattura più dei casi ovvi. Accanto agli eventi `error` espliciti, FailproofAI Cloud evidenzia anche i fallimenti silenziosi: qualsiasi `tool_result`, `hook_completed` o `agent_end` il cui payload contiene un errore appare qui. Uno strumento che ha restituito un errore, o un hook che è uscito male, non sfugge più semplicemente perché nulla ha lanciato un'eccezione rumorosa. Nella parte superiore, un istogramma traccia gli errori nel tempo. Un'occhiata ti dice se si tratta di un flusso costante di fondo o di un picco iniziato pochi minuti fa, così sai subito se devi smettere quello che stai facendo. @@ -23,19 +23,19 @@ Come ogni superficie observe, la pagina Errors è limitata alla tua organizzazio Una singola dipendenza interrotta può attivare lo stesso errore centinaia di volte al minuto. Lasciato così com'è, è una parete di linee quasi identiche che nasconde l'unica cosa che devi effettivamente vedere. -Failproof AI Observability comprime i fallimenti ripetuti che condividono la stessa sessione e tipo di errore in un'unica riga. Un picco viene letto come un incidente. Finisci per contare i problemi, non le righe di log, e il segnale che conta rimane in primo piano invece di essere annegato dal suo stesso volume. +FailproofAI Cloud comprime i fallimenti ripetuti che condividono la stessa sessione e tipo di errore in un'unica riga. Un picco viene letto come un incidente. Finisci per contare i problemi, non le righe di log, e il segnale che conta rimane in primo piano invece di essere annegato dal suo stesso volume. ## Da "qualcosa è rosso" all'evento esatto Fai clic su qualsiasi riga per arrivare direttamente all'interno della sessione di quel run, posizionato sull'evento esatto che ha fallito. Nessuna copia di ID sessione, nessuno scorrimento per cercare il momento in cui è andato male: arrivi direttamente lì, con il grafico di esecuzione completo a un'occhiata di distanza in modo da poter vedere cosa ha fatto l'agente nei momenti prima che si interrompesse. -Se hai `alerts:write`, ogni riga ha anche un pulsante **+ alert**. Fai clic e Observability apre una nuova regola di avviso già compilata per catturare lo stesso errore di nuovo. L'incidente che hai appena esaminato diventa quello che ti avviserà la prossima volta, invece di sorprenderti due volte. +Se hai `alerts:write`, ogni riga ha anche un pulsante **+ alert**. Fai clic e FailproofAI Cloud apre una nuova regola di avviso già compilata per catturare lo stesso errore di nuovo. L'incidente che hai appena esaminato diventa quello che ti avviserà la prossima volta, invece di sorprenderti due volte. **Dove trovarlo:** la pagina **Errors** si trova nella sezione observe del dashboard, a `//errors`. ## Correlati -- [Alerts](/it/agenteye/alerts): trasforma qualsiasi errore in una regola di paging. -- [Incidents](/it/agenteye/incidents): monitora un avviso attivo da apertura a risoluzione. -- [Sessions](/it/agenteye/sessions): apri il run completo dietro qualsiasi errore. -- [Audits](/it/agenteye/audits): lascia che Observability trovi i pattern di errore nei tuoi run per te. \ No newline at end of file +- [Alerts](/it/cloud/alerts): trasforma qualsiasi errore in una regola di paging. +- [Incidents](/it/cloud/incidents): monitora un avviso attivo da apertura a risoluzione. +- [Sessions](/it/cloud/sessions): apri il run completo dietro qualsiasi errore. +- [Audits](/it/cloud/audits): lascia che FailproofAI Cloud trovi i pattern di errore nei tuoi run per te. \ No newline at end of file diff --git a/docs/it/agenteye/evaluations.mdx b/docs/it/cloud/evaluations.mdx similarity index 67% rename from docs/it/agenteye/evaluations.mdx rename to docs/it/cloud/evaluations.mdx index 138f4607..a76408b4 100644 --- a/docs/it/agenteye/evaluations.mdx +++ b/docs/it/cloud/evaluations.mdx @@ -4,15 +4,15 @@ description: "I problemi di qualità ti trovano adesso, invece di scoprirli da u --- -I problemi di qualità ti trovano adesso, invece di scoprirli da un reclamo utente. Connetti il tuo servizio di scoring una volta e Failproof AI Observability valuta automaticamente ogni esecuzione completata, così un calo di utilità o un picco di allucinazioni emerge da solo, prima che un cliente lo noti. +I problemi di qualità ti trovano adesso, invece di scoprirli da un reclamo utente. Connetti il tuo servizio di scoring una volta e FailproofAI Cloud valuta automaticamente ogni esecuzione completata, così un calo di utilità o un picco di allucinazioni emerge da solo, prima che un cliente lo noti. -![La griglia Sessioni con una colonna di score: ogni esecuzione ha un badge di stato di valutazione e badge con codice colore per utilità, fattualità ed efficienza dello strumento](/agenteye/images/sessions-list.png) +![La griglia Sessioni con una colonna di score: ogni esecuzione ha un badge di stato di valutazione e badge con codice colore per utilità, fattualità ed efficienza dello strumento](/cloud/images/sessions-list.png) *Ogni esecuzione nella griglia di sessioni porta i suoi score; i badge rossi, ambra e verdi fanno risaltare le esecuzioni deboli senza dover aprire un singolo transcript.* ## Smetti di campionare le esecuzioni manualmente -Prima facevi controlli spot su una manciata di esecuzioni e speravi che il resto andasse bene. Adesso ogni sessione completata viene valutata nel momento in cui finisce, secondo le dimensioni che contano per te: utilità, efficienza dello strumento, fattualità, sicurezza, quello che è il tuo standard di qualità. Tu definisci le chiavi di score; Failproof AI Observability memorizza, registra le tendenze e visualizza tutto quello che il tuo evaluator rimanda indietro. Nessuna esecuzione sfugge senza essere valutata, e smetti di scoprire una regressione da un ticket di supporto. +Prima facevi controlli spot su una manciata di esecuzioni e speravi che il resto andasse bene. Adesso ogni sessione completata viene valutata nel momento in cui finisce, secondo le dimensioni che contano per te: utilità, efficienza dello strumento, fattualità, sicurezza, quello che è il tuo standard di qualità. Tu definisci le chiavi di score; FailproofAI Cloud memorizza, registra le tendenze e visualizza tutto quello che il tuo evaluator rimanda indietro. Nessuna esecuzione sfugge senza essere valutata, e smetti di scoprire una regressione da un ticket di supporto. Gli score compaiono sulla griglia di sessioni su **`//sessions`** (sidebar → *observe* → *sessions*), un cluster di badge per riga. Vuoi solo le esecuzioni che non hanno raggiunto l'obiettivo? Filtra la griglia per range di score, ad esempio utilità sotto 0.5, e accedi esattamente alle esecuzioni che vale la pena leggere. La visualizzazione degli score richiede il permesso `evaluations:read`. @@ -20,7 +20,7 @@ Gli score compaiono sulla griglia di sessioni su **`//sessions`** (sid Un numero ti dice che un'esecuzione era debole; la pagina della sessione ti dice perché. Apri qualsiasi esecuzione e la barra laterale destra inizia con il riassunto principale, poi mostra una barra per ogni dimensione con il ragionamento del tuo evaluator sotto ciascuna, così passi da "questo ha ottenuto 0.4 sulla fattualità" all'affermazione esatta sbagliata in pochi secondi. -![La barra laterale destra di una sessione: il riassunto della valutazione in alto, poi barre di score per dimensione ciascuna con una riga di ragionamento, accanto alla completa timeline degli eventi](/agenteye/images/session-detail.png) +![La barra laterale destra di una sessione: il riassunto della valutazione in alto, poi barre di score per dimensione ciascuna con una riga di ragionamento, accanto alla completa timeline degli eventi](/cloud/images/session-detail.png) *La vista dei dettagli della sessione: riassunto, barre di score per dimensione e il ragionamento dietro ogni score, proprio accanto alla timeline degli eventi dell'esecuzione.* @@ -30,7 +30,7 @@ Hai distribuito un evaluator più intelligente, o stai guardando un'esecuzione c Un'esecuzione con basso score è rumore; una coorte intera che scivola è un segnale. Le dashboard salvate trasformano i tuoi score in una tendenza che puoi osservare a colpo d'occhio: utilità media questa settimana rispetto alla scorsa, per agent, per ambiente. -![Una dashboard di qualità: barre di score medio per dimensione dell'evaluator accanto a una tendenza nel tempo](/agenteye/images/dashboard-quality.png) +![Una dashboard di qualità: barre di score medio per dimensione dell'evaluator accanto a una tendenza nel tempo](/cloud/images/dashboard-quality.png) *Una dashboard di qualità salvata registra le tendenze delle chiavi di score che presenti, così una deriva lenta è ovvia molto prima che diventi un incidente.* @@ -38,14 +38,14 @@ Le dashboard si trovano su **`//dashboards`** (sidebar → *analyze* ## Connetti un evaluator una volta -Lo scoring è opt-in e rimane completamente disattivato finché non punti Failproof AI Observability a uno scorer. Avvii un piccolo servizio HTTP (Observability fornisce un riferimento funzionante che puoi copiare), imposti due valori sul tuo server e da allora ogni esecuzione viene valutata per te. La guida completa, il contratto di scoring e l'SDK si trovano nella guida approfondita. +Lo scoring è opt-in e rimane completamente disattivato finché non punti FailproofAI Cloud a uno scorer. Avvii un piccolo servizio HTTP (FailproofAI Cloud fornisce un riferimento funzionante che puoi copiare), imposti due valori sul tuo server e da allora ogni esecuzione viene valutata per te. La guida completa, il contratto di scoring e l'SDK si trovano nella guida approfondita. -Non sei sicuro di quali dimensioni vale la pena valutare in primo luogo? L'[agent skill evaluator](/it/agenteye/evaluator-skill) fa in modo che il tuo agent di codifica lo scopra sulle tue stesse sessioni, poi costruisci e distribuisci il servizio. +Non sei sicuro di quali dimensioni vale la pena valutare in primo luogo? L'[agent skill evaluator](/it/cloud/agent-skills) fa in modo che il tuo agent di codifica lo scopra sulle tue stesse sessioni, poi costruisci e distribuisci il servizio. ## Correlati -- [Evaluation suite](/it/agenteye/evaluation-suite): connetti il tuo evaluator, il contratto di scoring e l'SDK. -- [Evaluator agent skill](/it/agenteye/evaluator-skill): lascia che un agent di codifica scelga le tue dimensioni di score e costruisca l'evaluator. -- [Sessions](/it/agenteye/sessions): la griglia run-by-run dove compaiono gli score. -- [Dashboards](/it/agenteye/dashboards): salva e condividi le tendenze di qualità nella tua organizzazione. -- [Audits](/it/agenteye/audits): l'altra funzione di qualità automatica di Observability, per investigazioni tra sessioni. \ No newline at end of file +- [Evaluation suite](/it/cloud/evaluators): connetti il tuo evaluator, il contratto di scoring e l'SDK. +- [Evaluator agent skill](/it/cloud/agent-skills): lascia che un agent di codifica scelga le tue dimensioni di score e costruisca l'evaluator. +- [Sessions](/it/cloud/sessions): la griglia run-by-run dove compaiono gli score. +- [Dashboards](/it/cloud/dashboards): salva e condividi le tendenze di qualità nella tua organizzazione. +- [Audits](/it/cloud/audits): l'altra funzione di qualità automatica di FailproofAI Cloud, per investigazioni tra sessioni. \ No newline at end of file diff --git a/docs/it/agenteye/evaluation-suite.mdx b/docs/it/cloud/evaluators.mdx similarity index 83% rename from docs/it/agenteye/evaluation-suite.mdx rename to docs/it/cloud/evaluators.mdx index 2d4c900c..bb298be5 100644 --- a/docs/it/agenteye/evaluation-suite.mdx +++ b/docs/it/cloud/evaluators.mdx @@ -1,19 +1,19 @@ --- title: "Suite di valutazione" -description: "Failproof AI Observability può valutare automaticamente ogni esecuzione di agent completata per la qualità: tu fornisci un piccolo servizio di scoring e Observability gestisce il resto." +description: "FailproofAI Cloud può valutare automaticamente ogni esecuzione di agent completata per la qualità: tu fornisci un piccolo servizio di scoring e FailproofAI Cloud gestisce il resto." --- -Failproof AI Observability può valutare automaticamente ogni esecuzione di agent completata per la qualità: tu fornisci un piccolo servizio di scoring e Observability gestisce il resto. Usalo per tracciare le dimensioni che ti interessano (utilità, efficienza degli strumenti, fattualità, sicurezza; scegli tu), rilevare regressioni in anticipo e confrontare agent o ambienti a colpo d'occhio. Lo scoring è facoltativo: la pipeline non fa nulla finché non imposti `EVALUATOR_ENDPOINT` sul server. +FailproofAI Cloud può valutare automaticamente ogni esecuzione di agent completata per la qualità: tu fornisci un piccolo servizio di scoring e FailproofAI Cloud gestisce il resto. Usalo per tracciare le dimensioni che ti interessano (utilità, efficienza degli strumenti, fattualità, sicurezza; scegli tu), rilevare regressioni in anticipo e confrontare agent o ambienti a colpo d'occhio. Lo scoring è facoltativo: la pipeline non fa nulla finché non imposti `EVALUATOR_ENDPOINT` sul server. -> **Nota:** Tu definisci le dimensioni del punteggio. Il tuo valutatore può restituire qualsiasi chiave numerica desideri; Observability memorizza, tende alla tendenza e visualizza tutto quello che invii. +> **Nota:** Tu definisci le dimensioni del punteggio. Il tuo valutatore può restituire qualsiasi chiave numerica desideri; FailproofAI Cloud memorizza, tende alla tendenza e visualizza tutto quello che invii. ## In sintesi -1. **Scrivi uno scorer.** Crea un piccolo servizio HTTP che legge una trascrizione della sessione e restituisce i punteggi. Observability fornisce un riferimento funzionante che puoi copiare. Vedi [Scrivere un valutatore con l'SDK](#writing-an-evaluator-with-the-sdk). -2. **Punta Observability su di esso.** Imposta `EVALUATOR_ENDPOINT` (e un `EVALUATOR_TOKEN` condiviso) sul processo server. +1. **Scrivi uno scorer.** Crea un piccolo servizio HTTP che legge una trascrizione della sessione e restituisce i punteggi. FailproofAI Cloud fornisce un riferimento funzionante che puoi copiare. Vedi [Scrivere un valutatore con l'SDK](#writing-an-evaluator-with-the-sdk). +2. **Punta FailproofAI Cloud su di esso.** Imposta `EVALUATOR_ENDPOINT` (e un `EVALUATOR_TOKEN` condiviso) sul processo server. 3. **Guarda i punteggi arrivare.** Ogni sessione completata viene valutata automaticamente; i risultati appaiono nella pagina dei dettagli della sessione, nella griglia delle sessioni e nei dashboard salvati. -![Una vista dettaglio della sessione con il riepilogo della valutazione, barre dei punteggi per dimensione e testo di ragionamento nella barra laterale destra](/agenteye/images/session-detail.png) +![Una vista dettaglio della sessione con il riepilogo della valutazione, barre dei punteggi per dimensione e testo di ragionamento nella barra laterale destra](/cloud/images/session-detail.png) *Una volta configurato un valutatore, ogni esecuzione completata viene valutata e i risultati appaiono nella barra laterale destra della sessione: il riepilogo in alto, poi barre dei punteggi per dimensione con ragionamento.* @@ -23,7 +23,7 @@ Failproof AI Observability può valutare automaticamente ogni esecuzione di agen ```mermaid flowchart LR - ING["ingest /events
    agent_end"] --> SRV["Observability server"] + ING["ingest /events
    agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -31,14 +31,14 @@ flowchart LR SRV --> RES["evaluations
    terminal results"] ``` -Quando l'SDK di Observability emette un evento `agent_end` per una sessione, il server pianifica una valutazione. Quindi invia un POST della trascrizione completa degli eventi al tuo servizio di valutazione, che può: +Quando l'SDK di FailproofAI Cloud emette un evento `agent_end` per una sessione, il server pianifica una valutazione. Quindi invia un POST della trascrizione completa degli eventi al tuo servizio di valutazione, che può: - **Restituire il risultato inline** con `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`. Il risultato viene aggiunto alla timeline di valutazione della sessione. `reasoning` e `summary` sono facoltativi. -- **Rimandare** con `{"status":"pending", "job_id":"abc-123"}`. Observability poi chiama `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` finché il tuo valutatore non restituisce `{"status":"done", ...}` o `{"status":"error", "error":"..."}`. +- **Rimandare** con `{"status":"pending", "job_id":"abc-123"}`. FailproofAI Cloud poi chiama `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` finché il tuo valutatore non restituisce `{"status":"done", ...}` o `{"status":"error", "error":"..."}`. - La cadenza di polling è per job: una risposta `pending` può includere `next_poll_secs` per sovrascrivere; altrimenti Observability usa il valore `default_poll_interval_secs` da `GET /config`; altrimenti il server ricade su `EVALUATOR_POLLING_INTERVAL_SECS` (default 10s). Tutti i valori sono limitati a [1s, 1h]. + La cadenza di polling è per job: una risposta `pending` può includere `next_poll_secs` per sovrascrivere; altrimenti FailproofAI Cloud usa il valore `default_poll_interval_secs` da `GET /config`; altrimenti il server ricade su `EVALUATOR_POLLING_INTERVAL_SECS` (default 10s). Tutti i valori sono limitati a [1s, 1h]. -Anche le sessioni che non emettono mai `agent_end` (ad esempio, un processo agent che si è bloccato) possono essere rilevate: il `GET /config` del valutatore può restituire `{"inactivity_timeout_secs": 1800}`, e Observability valuterà qualsiasi sessione rimasta inattiva per quel tempo. Imposta il campo a `null` oppure omettilo per disabilitare questo fallback. +Anche le sessioni che non emettono mai `agent_end` (ad esempio, un processo agent che si è bloccato) possono essere rilevate: il `GET /config` del valutatore può restituire `{"inactivity_timeout_secs": 1800}`, e FailproofAI Cloud valuterà qualsiasi sessione rimasta inattiva per quel tempo. Imposta il campo a `null` oppure omettilo per disabilitare questo fallback. La pipeline è completamente non operativa quando `EVALUATOR_ENDPOINT` non è impostato. @@ -46,7 +46,7 @@ Una sessione può accumulare **più valutazioni terminali nel tempo**: ogni even Il fallback di inattività si riattiva anche nelle sessioni riprese: se arrivano nuovi eventi dopo una precedente valutazione terminale e la sessione poi rimane inattiva oltre `inactivity_timeout_secs`, una nuova valutazione viene messa in coda. -I guasti transitori (5xx, 429, timeout, errori di rete) vengono ritentati con backoff esponenziale fino a `EVALUATOR_MAX_ATTEMPTS`; le risposte 4xx sono terminali. Observability è sicuro da eseguire con più istanze di server scalate orizzontalmente; il lavoro è partizionato in modo che la stessa sessione non venga mai inviata due volte contemporaneamente. +I guasti transitori (5xx, 429, timeout, errori di rete) vengono ritentati con backoff esponenziale fino a `EVALUATOR_MAX_ATTEMPTS`; le risposte 4xx sono terminali. FailproofAI Cloud è sicuro da eseguire con più istanze di server scalate orizzontalmente; il lavoro è partizionato in modo che la stessa sessione non venga mai inviata due volte contemporaneamente. --- @@ -54,7 +54,7 @@ I guasti transitori (5xx, 429, timeout, errori di rete) vengono ritentati con ba Ogni rotta autenticata usa **autenticazione bearer token**. Lo stesso valore deve essere configurato su entrambi i lati: -- Server Observability: variabile di ambiente `EVALUATOR_TOKEN` +- Server FailproofAI Cloud: variabile di ambiente `EVALUATOR_TOKEN` - Servizio di valutazione: configurato allo stesso modo (l'SDK `agenteye-evaluator` legge `EVALUATOR_TOKEN` per convenzione) Se `EVALUATOR_TOKEN` non è impostato, il server non invia l'header `Authorization`; il valutatore può quindi accettare richieste anonime, il che va bene per una rete interna ma è sconsigliato su internet pubblico. @@ -125,7 +125,7 @@ Il server tratta qualsiasi altro body 2xx come un errore di protocollo e registr Non devi implementare il contratto HTTP a mano. Il pacchetto Python `agenteye-evaluator` ti fornisce un wrapper FastAPI tipizzato che gestisce l'autenticazione, il routing e le forme di richiesta/risposta per te. -Failproof AI Observability fornisce anche un **valutatore di riferimento funzionante** che valuta `helpfulness`, `tool_efficiency` e `factuality` dalla forma della trascrizione. Copialo come punto di partenza e sostituisci la tua logica: un giudice LLM, un motore di regole, qualsiasi cosa si adatti al tuo standard di qualità. +FailproofAI Cloud fornisce anche un **valutatore di riferimento funzionante** che valuta `helpfulness`, `tool_efficiency` e `factuality` dalla forma della trascrizione. Copialo come punto di partenza e sostituisci la tua logica: un giudice LLM, un motore di regole, qualsiasi cosa si adatti al tuo standard di qualità. Valutatore minimo praticabile: @@ -148,7 +148,7 @@ def run(req: EvalRequest) -> EvalResponse: L'istanza `app` gira sotto qualsiasi server ASGI, così `uvicorn module:app` lo avvia. -Per i valutatori che devono rimandare lavoro costoso, restituisci `JobPending` e registra un gestore `@app.job_lookup`; il server Observability polling `GET /evaluate/{job_id}` finché non restituisci uno stato terminale o il cap `EVALUATOR_MAX_POLL_DURATION_SECS` (default 1 h) trascorre. +Per i valutatori che devono rimandare lavoro costoso, restituisci `JobPending` e registra un gestore `@app.job_lookup`; il server FailproofAI Cloud polling `GET /evaluate/{job_id}` finché non restituisci uno stato terminale o il cap `EVALUATOR_MAX_POLL_DURATION_SECS` (default 1 h) trascorre. Il riferimento API completo, il modello asincrono e lo schema degli eventi sono documentati nel README dell'SDK `agenteye-evaluator`. @@ -156,7 +156,7 @@ Il riferimento API completo, il modello asincrono e lo schema degli eventi sono ## Eseguire il tuo valutatore -Il valutatore è **il tuo servizio** — Failproof AI Observability non fornisce un valutatore predefinito, quindi lo crei e lo esegui dove esegui i tuoi servizi. Viene eseguito sotto qualsiasi server ASGI (ad esempio `uvicorn my_evaluator:app`); servi le rotte `/health`, `/config` e `/evaluate` dal [contratto HTTP](#http-contract), poi punta il server su di esso (vedi [Configurare il server](#configuring-the-server)). +Il valutatore è **il tuo servizio** — FailproofAI Cloud non fornisce un valutatore predefinito, quindi lo crei e lo esegui dove esegui i tuoi servizi. Viene eseguito sotto qualsiasi server ASGI (ad esempio `uvicorn my_evaluator:app`); servi le rotte `/health`, `/config` e `/evaluate` dal [contratto HTTP](#http-contract), poi punta il server su di esso (vedi [Configurare il server](#configuring-the-server)). Una volta che il valutatore è raggiungibile, `GET /health` restituisce `{"status":"ok"}`. Dopo che un agent viene eseguito end-to-end, `GET /evaluations` sul server restituisce una riga con `status: "done"` e i punteggi prodotti dal tuo valutatore. @@ -254,7 +254,7 @@ L'admin bootstrap (`ADMIN_KEY`, `ADMIN_EMAIL`) riceve automaticamente questi. - **`/sessions`**: griglia di sessione filtrabile; la colonna dei punteggi mostra lo stato di valutazione e i punteggi di ogni sessione a colpo d'occhio. - **`/dashboards`**: viste di salute eval salvate (vedi [Dashboard](#dashboards) sotto). -![La griglia di sessioni con pillole di stato di valutazione per sessione e badge di punteggio codificati per colore (helpfulness, factuality, tool_efficiency, safety, coherence)](/agenteye/images/sessions-list.png) +![La griglia di sessioni con pillole di stato di valutazione per sessione e badge di punteggio codificati per colore (helpfulness, factuality, tool_efficiency, safety, coherence)](/cloud/images/sessions-list.png) *La griglia di sessioni mostra lo stato di valutazione e i punteggi di ogni esecuzione a colpo d'occhio; i badge rosso/ambra/verde rendono i punteggi bassi evidenti.* @@ -271,7 +271,7 @@ Ogni dashboard fissa: Ogni card mostra il numero di sessioni corrispondenti, una disaggregazione done/error/timeout, la media di ogni punteggio presentato e un piccolo sparkline di tendenza. Aprire un dashboard mostra i pannelli a dimensione intera; **open in sessions** ti porta alla pagina delle sessioni prefiltrrata esattamente a quella sezione. Le metriche sono calcolate lato server su tutto il set di corrispondenza (via `GET /evaluations/aggregate`), così i numeri sono esatti piuttosto che campionati. -![Un dashboard di salute eval con barre di punteggio medio per dimensione del valutatore, una disaggregazione tool ok-vs-error, top tools e una tendenza events-per-hour](/agenteye/images/dashboard-quality.png) +![Un dashboard di salute eval con barre di punteggio medio per dimensione del valutatore, una disaggregazione tool ok-vs-error, top tools e una tendenza events-per-hour](/cloud/images/dashboard-quality.png) **Permessi:** visualizzare richiede sia `dashboards:read` che `evaluations:read`; creare e modificare richiede `dashboards:write`; eliminare richiede `dashboards:delete`. L'admin bootstrap riceve tutti questi automaticamente. @@ -293,7 +293,7 @@ Ogni card mostra il numero di sessioni corrispondenti, una disaggregazione done/ ## Prossimi passi -- [Skill agent valutatore](/it/agenteye/evaluator-skill): fai progettare a un agent di codifica le tue dimensioni in base a sessioni reali e costruisci questo servizio per te. -- [Python SDK](/it/agenteye/python-sdk): emetti gli eventi `agent_end` che attivano lo scoring. -- [Chiavi API](/it/agenteye/api-keys): i permessi `evaluations:read` e `evaluations:trigger`. -- [Audit](/it/agenteye/audits): l'altra funzione di qualità automatizzata di Observability, per la revisione basata su policy. \ No newline at end of file +- [Skill agent valutatore](/it/cloud/agent-skills): fai progettare a un agent di codifica le tue dimensioni in base a sessioni reali e costruisci questo servizio per te. +- [Python SDK](/it/cloud/sdk): emetti gli eventi `agent_end` che attivano lo scoring. +- [Chiavi API](/it/cloud/access): i permessi `evaluations:read` e `evaluations:trigger`. +- [Audit](/it/cloud/audits): l'altra funzione di qualità automatizzata di FailproofAI Cloud, per la revisione basata su policy. \ No newline at end of file diff --git a/docs/it/agenteye/event-stream.mdx b/docs/it/cloud/event-stream.mdx similarity index 78% rename from docs/it/agenteye/event-stream.mdx rename to docs/it/cloud/event-stream.mdx index ebf3f2d6..1ab56845 100644 --- a/docs/it/agenteye/event-stream.mdx +++ b/docs/it/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "Nel momento in cui il tuo agent fa qualcosa, lo vedi." Nel momento in cui il tuo agent fa qualcosa, lo vedi. L'Event Stream è il tuo polso in tempo reale su ogni agent in produzione: niente attese, niente grep sui log, niente supposizioni su quello che è appena successo. -![L'Event Stream dal vivo: righe di eventi codificate per colore che scorrono in tempo reale, filtrabili per ambiente, agent, sessione, tipo di evento e testo libero](/agenteye/images/events-stream.png) +![L'Event Stream dal vivo: righe di eventi codificate per colore che scorrono in tempo reale, filtrabili per ambiente, agent, sessione, tipo di evento e testo libero](/cloud/images/events-stream.png) *Ogni evento da ogni agent della tua organizzazione, i più recenti per primi, aggiornati mentre accadono.* @@ -37,14 +37,14 @@ La ricerca in testo libero va dritto a un messaggio, un nome di strumento o un I L'Event Stream è la home della tua organizzazione. Accedi e è la prima superficie su cui atterri, su `//`, quindi il triage inizia dal momento in cui arrivi. -Dietro, i tuoi agent emettono eventi tramite l'SDK, il collector li spedisce al tuo server Failproof AI Observability, e il flusso li traccia mentre arrivano nell'infrastruttura che controlli. Quando vuoi la vista aggregata invece della traccia grezza, gli eventi di ogni esecuzione si comprimono in una singola riga su Sessions, a un clic di distanza. +Dietro, i tuoi agent emettono eventi tramite l'SDK, il collector li spedisce al tuo server FailproofAI Cloud, e il flusso li traccia mentre arrivano nell'infrastruttura che controlli. Quando vuoi la vista aggregata invece della traccia grezza, gli eventi di ogni esecuzione si comprimono in una singola riga su Sessions, a un clic di distanza. Questa è la fonte di verità grezza su cui si costruiscono tutte le altre superfici di osservazione, quindi quando un numero sembra sbagliato altrove, il flusso è dove confermi quello che è effettivamente accaduto. ## Correlati -- [Sessions](/it/agenteye/sessions): gli stessi eventi aggregati in una riga per esecuzione, con un grafico di esecuzione in stile git. -- [Telemetry](/it/agenteye/telemetry): quello che i tuoi agent inviano e come gli eventi raggiungono il flusso. -- [Error tracking](/it/agenteye/error-tracking): una singola superficie di triage per tutto quello che è andato male. -- [Alerts](/it/agenteye/alerts): trasforma qualsiasi soglia in una regola di paging. -- [CLI and agents](/it/agenteye/cli-and-agents): lo stesso flusso dal vivo dal tuo terminale. \ No newline at end of file +- [Sessions](/it/cloud/sessions): gli stessi eventi aggregati in una riga per esecuzione, con un grafico di esecuzione in stile git. +- [Telemetry](/it/cloud/performance): quello che i tuoi agent inviano e come gli eventi raggiungono il flusso. +- [Error tracking](/it/cloud/errors): una singola superficie di triage per tutto quello che è andato male. +- [Alerts](/it/cloud/alerts): trasforma qualsiasi soglia in una regola di paging. +- [CLI and agents](/it/cloud/cli): lo stesso flusso dal vivo dal tuo terminale. \ No newline at end of file diff --git a/docs/it/cloud/fleet.mdx b/docs/it/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/it/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/it/agenteye/incidents.mdx b/docs/it/cloud/incidents.mdx similarity index 84% rename from docs/it/agenteye/incidents.mdx rename to docs/it/cloud/incidents.mdx index d0cea46d..d239d5dc 100644 --- a/docs/it/agenteye/incidents.mdx +++ b/docs/it/cloud/incidents.mdx @@ -5,7 +5,7 @@ description: "Quando scatta un alert, tutti vedono che l'incident è aperto, chi Quando scatta un alert, la prima domanda è sempre "chi se ne occupa?". Gli Incidents rispondono a questa domanda: nel momento in cui qualcosa viene rilevato, tutti possono vedere che l'incident è aperto, chi lo gestisce, e esattamente cosa è successo finora, con un registro pulito e attribuito che puoi usare direttamente in una post-mortem. -![La inbox degli Incidents: card di incident collegati agli alert e aperti manualmente, raggruppati per stato, ciascuno con un badge di severità e un assegnatario](/agenteye/images/incidents.png) +![La inbox degli Incidents: card di incident collegati agli alert e aperti manualmente, raggruppati per stato, ciascuno con un badge di severità e un assegnatario](/cloud/images/incidents.png) *La inbox raggruppa gli incident aperti per stato e filtra per severità e assegnatario, così vedi cosa ha bisogno di un intervento umano adesso.* ## Sapere chi se ne occupa, a colpo d'occhio @@ -16,10 +16,10 @@ Niente più "qualcuno sta guardando questo?" in un thread di chat. Una rilevazio Quando l'incident è finito, hai già il rapporto. Apri un incident qualsiasi e ottieni l'evidenza della rilevazione, i suoi assegnatari e sottoscrittori, un thread di commenti per coordinare sul posto, e una timeline di attività in sola aggiunta. -![Una vista dei dettagli dell'incident: l'alert principale e il riepilogo della rilevazione, assegnatari e sottoscrittori, una timeline di attività attribuita, e un thread di commenti](/agenteye/images/incident-detail.png) +![Una vista dei dettagli dell'incident: l'alert principale e il riepilogo della rilevazione, assegnatari e sottoscrittori, una timeline di attività attribuita, e un thread di commenti](/cloud/images/incident-detail.png) *Tutto ciò che è accaduto, in ordine, ogni riga firmata da chi l'ha fatto.* -Ogni azione (aperto, riconosciuto, risolto, e così via) viene scritta in quella timeline e non viene mai modificata. Ogni entry è attribuita: all'operatore che l'ha eseguita, via email, o a **automated** per tutto ciò che Failproof AI Observability ha fatto da solo, come aprire l'incident sulla rilevazione. Nulla è anonimo e nulla va perso, quindi la post-mortem più o meno si scrive da sola. +Ogni azione (aperto, riconosciuto, risolto, e così via) viene scritta in quella timeline e non viene mai modificata. Ogni entry è attribuita: all'operatore che l'ha eseguita, via email, o a **automated** per tutto ciò che FailproofAI Cloud ha fatto da solo, come aprire l'incident sulla rilevazione. Nulla è anonimo e nulla va perso, quindi la post-mortem più o meno si scrive da sola. ## Come si muove un incident @@ -44,6 +44,6 @@ Gli Incidents si trovano a `//incidents`. La visualizzazione richiede ## Correlati -- [Alerts](/it/agenteye/alerts): le regole che aprono questi incident quando una soglia viene superata. -- [Error tracking](/it/agenteye/error-tracking): vedi ogni errore in un unico posto e promuovi uno a alert. -- [Audits](/it/agenteye/audits): l'analista programmato che trova i guasti che nessuna regola stava controllando. \ No newline at end of file +- [Alerts](/it/cloud/alerts): le regole che aprono questi incident quando una soglia viene superata. +- [Error tracking](/it/cloud/errors): vedi ogni errore in un unico posto e promuovi uno a alert. +- [Audits](/it/cloud/audits): l'analista programmato che trova i guasti che nessuna regola stava controllando. \ No newline at end of file diff --git a/docs/it/cloud/managed-policies.mdx b/docs/it/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/it/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/it/agenteye/overview.mdx b/docs/it/cloud/overview.mdx similarity index 51% rename from docs/it/agenteye/overview.mdx rename to docs/it/cloud/overview.mdx index 7fcb341e..41cb8be4 100644 --- a/docs/it/agenteye/overview.mdx +++ b/docs/it/cloud/overview.mdx @@ -1,15 +1,15 @@ --- title: "Failproof AI: Osserva gli Agenti per Individuare i Fallimenti" -description: "Failproof AI Observability è una piattaforma self-hosted per osservare, valutare e migliorare i tuoi agenti AI in produzione." +description: "FailproofAI Cloud è una piattaforma self-hosted per osservare, valutare e migliorare i tuoi agenti AI in produzione." --- -Failproof AI Observability è una piattaforma self-hosted per osservare, valutare e migliorare i tuoi agenti AI in produzione. Registra tutto quello che fanno i tuoi agenti (ogni chiamata a strumento, richiesta ai modelli, hook e errore), assegna un punteggio alla qualità di ogni esecuzione e mette in evidenza i fallimenti che non sapevi di dovere cercare, il tutto in una dashboard che esegui direttamente nella tua infrastruttura. +FailproofAI Cloud è una piattaforma self-hosted per osservare, valutare e migliorare i tuoi agenti AI in produzione. Registra tutto quello che fanno i tuoi agenti (ogni chiamata a strumento, richiesta ai modelli, hook e errore), assegna un punteggio alla qualità di ogni esecuzione e mette in evidenza i fallimenti che non sapevi di dovere cercare, il tutto in una dashboard che esegui direttamente nella tua infrastruttura. -Se distribuisci agenti AI e sei stanco di indovinare perché un'esecuzione è andata male, questa è la pagina giusta da cui iniziare. Spiega cosa Failproof AI Observability ti offre e come i vari componenti si incastrano insieme, prima di installare qualsiasi cosa. +Se distribuisci agenti AI e sei stanco di indovinare perché un'esecuzione è andata male, questa è la pagina giusta da cui iniziare. Spiega cosa FailproofAI Cloud ti offre e come i vari componenti si incastrano insieme, prima di installare qualsiasi cosa. -> **Failproof AI Observability è un prodotto enterprise di Failproof AI.** Vuoi vederlo in azione? Richiedi una demo: invia un'email a [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +> **FailproofAI Cloud è un prodotto enterprise di Failproof AI.** Vuoi vederlo in azione? Richiedi una demo: invia un'email a [nikita@befailproof.ai](mailto:nikita@befailproof.ai). -![Una sessione di Failproof AI Observability disegnata come un grafo di esecuzione in stile git accanto alla sua timeline degli eventi, con una ripartizione per esecuzione di strumenti, modelli e hook nella colonna di destra](/agenteye/images/session-detail.png) +![Una sessione di FailproofAI Cloud disegnata come un grafo di esecuzione in stile git accanto alla sua timeline degli eventi, con una ripartizione per esecuzione di strumenti, modelli e hook nella colonna di destra](/cloud/images/session-detail.png) *Ogni esecuzione dell'agente è disegnata come un grafo di esecuzione in stile git (sinistra) accanto alla sua timeline degli eventi. I sub-agenti paralleli ottengono ciascuno la loro corsia; la colonna di destra suddivide gli strumenti, i modelli, gli hook e la spesa di token per l'esecuzione.* @@ -29,52 +29,52 @@ Due brevi video mostrano le due cose che i team cercano per primi: tracciare un'
    -*Failproof Audit: lascia che Failproof AI Observability esamini i tuoi log tra le sessioni e ti dica cosa sistemare.* +*Failproof Audit: lascia che FailproofAI Cloud esamini i tuoi log tra le sessioni e ti dica cosa sistemare.* --- ## Perché i team lo usano - **Vedi cosa ha effettivamente fatto il tuo agente.** Ogni esecuzione diventa un grafo di esecuzione leggibile in stile git: quali strumenti hanno girato in parallelo, quali sub-agenti si sono ramificati, dove si è fermato e quanto ha speso. -- **Rileva le regressioni di qualità automaticamente.** Connetti un piccolo servizio di scoring e Failproof AI Observability assegna un punteggio a ogni esecuzione completata, in modo che un calo di utilità o un picco di allucinazioni si noti da solo. +- **Rileva le regressioni di qualità automaticamente.** Connetti un piccolo servizio di scoring e FailproofAI Cloud assegna un punteggio a ogni esecuzione completata, in modo che un calo di utilità o un picco di allucinazioni si noti da solo. - **Trova i fallimenti per cui non hai scritto una regola.** Gli audit ricorrenti analizzano i tuoi log tra le sessioni alla ricerca di cluster di errori, outlier di latenza, punteggi bassi ed esecuzioni bloccate, quindi ti consegnano scoperte classificate e supportate da prove. - **Ricevi notifiche quando conta davvero.** Le regole di soglia si attivano sulla base di tasso di errore, latenza, costo o punteggi degli evaluator e aprono incident che puoi riconoscere, assegnare e risolvere. - **Fai domande in linguaggio naturale.** Un assistente AI all'interno della dashboard risponde a domande come "come sta andando la qualità in produzione questa settimana?" sui tuoi dati. Qualsiasi modifica effettuata è sottoposta ad approvazione. -- **Mantieni i tuoi dati.** Failproof AI Observability è self-hosted: gli eventi, i prompt e l'analisi rimangono nell'infrastruttura che controlli. +- **Mantieni i tuoi dati.** FailproofAI Cloud è self-hosted: gli eventi, i prompt e l'analisi rimangono nell'infrastruttura che controlli. --- ## Cosa ottieni -Failproof AI Observability è organizzato attorno a tre concetti (**osserva**, **analizza** e **amministra**), rispecchiati nella barra laterale sinistra della dashboard. +FailproofAI Cloud è organizzato attorno a tre concetti (**osserva**, **analizza** e **amministra**), rispecchiati nella barra laterale sinistra della dashboard. **Osserva** (la verità grezza di cosa è successo): -- **[Flusso di eventi](/it/agenteye/event-stream)**: il trail live, passo dopo passo, di ogni esecuzione (chiamate a strumenti, chiamate ai modelli, hook, errori). -- **[Sessioni](/it/agenteye/sessions)**: quegli eventi consolidati in una riga per esecuzione, ognuno pronto per essere assegnato un punteggio, con un grafo di esecuzione in stile git. -- **[Metriche di performance](/it/agenteye/telemetry)**: heatmap di latenza per superficie e vitali p50/p95/p99 per modelli, strumenti e hook, in modo che un picco di coda risalti dalla mediana. -- **[Tracciamento degli errori](/it/agenteye/error-tracking)**: una superficie di triage unica per tutto ciò che è andato storto, a un clic da un alert che si attiva. +- **[Flusso di eventi](/it/cloud/event-stream)**: il trail live, passo dopo passo, di ogni esecuzione (chiamate a strumenti, chiamate ai modelli, hook, errori). +- **[Sessioni](/it/cloud/sessions)**: quegli eventi consolidati in una riga per esecuzione, ognuno pronto per essere assegnato un punteggio, con un grafo di esecuzione in stile git. +- **[Metriche di performance](/it/cloud/performance)**: heatmap di latenza per superficie e vitali p50/p95/p99 per modelli, strumenti e hook, in modo che un picco di coda risalti dalla mediana. +- **[Tracciamento degli errori](/it/cloud/errors)**: una superficie di triage unica per tutto ciò che è andato storto, a un clic da un alert che si attiva. -![La pagina strumenti di osservazione: una heatmap di latenza, una banda percentile e una barra di distribuzione degli strumenti su 24 intervalli di tempo](/agenteye/images/tools.png) +![La pagina strumenti di osservazione: una heatmap di latenza, una banda percentile e una barra di distribuzione degli strumenti su 24 intervalli di tempo](/cloud/images/tools.png) *Ogni superficie di osservazione associa una sparkline e vitali p50/p95/p99 con una heatmap di latenza e una banda percentile. Mostrato qui: Strumenti.* **Analizza** (trasforma l'attività in risposte): -- **[Query](/it/agenteye/queries)** e **[dashboard](/it/agenteye/dashboards)**: SQL salvate sui tuoi eventi e valutazioni, rappresentate graficamente in dashboard condivise scoped all'organizzazione. -- **[Valutazioni](/it/agenteye/evaluations)**: punteggi di qualità prodotti dal tuo servizio di valutazione, con motivazioni per ogni punteggio. -- **[Audit](/it/agenteye/audits)**: indagini ricorrenti che rivelano pattern di fallimento tra le sessioni. -- **[Avvisi](/it/agenteye/alerts)** e **[incident](/it/agenteye/incidents)**: regole di soglia che ti notificano, più un flusso di lavoro per gli incident per triarli. +- **[Query](/it/cloud/queries)** e **[dashboard](/it/cloud/dashboards)**: SQL salvate sui tuoi eventi e valutazioni, rappresentate graficamente in dashboard condivise scoped all'organizzazione. +- **[Valutazioni](/it/cloud/evaluations)**: punteggi di qualità prodotti dal tuo servizio di valutazione, con motivazioni per ogni punteggio. +- **[Audit](/it/cloud/audits)**: indagini ricorrenti che rivelano pattern di fallimento tra le sessioni. +- **[Avvisi](/it/cloud/alerts)** e **[incident](/it/cloud/incidents)**: regole di soglia che ti notificano, più un flusso di lavoro per gli incident per triarli. **Interfacce** (accedi ai tuoi dati come preferisci): -- **[CLI](/it/agenteye/cli-and-agents)**: gestisci l'intera distribuzione dal terminale o da uno script, e lascia che un agente di codifica lo faccia per te in linguaggio naturale. -- **[Assistente AI](/it/agenteye/assistant)**: fai domande sui tuoi agenti in linguaggio naturale, direttamente all'interno della dashboard. -- **API REST**: tutto quello che fa la dashboard e la CLI è supportato da un'API REST che puoi chiamare direttamente con una [chiave API](/it/agenteye/api-keys) scoped — ingesta eventi, interroga sessioni e valutazioni, e gestisci dashboard, avvisi, audit, utenti e chiavi, in modo da poter integrare Failproof AI Observability nel tuo tooling. +- **[CLI](/it/cloud/cli)**: gestisci l'intera distribuzione dal terminale o da uno script, e lascia che un agente di codifica lo faccia per te in linguaggio naturale. +- **[Assistente AI](/it/cloud/assistant)**: fai domande sui tuoi agenti in linguaggio naturale, direttamente all'interno della dashboard. +- **API REST**: tutto quello che fa la dashboard e la CLI è supportato da un'API REST che puoi chiamare direttamente con una [chiave API](/it/cloud/access) scoped — ingesta eventi, interroga sessioni e valutazioni, e gestisci dashboard, avvisi, audit, utenti e chiavi, in modo da poter integrare FailproofAI Cloud nel tuo tooling. **Amministra** (gestiscilo per il tuo team): -- **[Chiavi API](/it/agenteye/api-keys)**: token scoped per il collector, la dashboard e l'assistente. +- **[Chiavi API](/it/cloud/access)**: token scoped per il collector, la dashboard e l'assistente. - **Utenti**: accesso passwordless basato su email con allowlist. - **Impostazioni**: configurazione per organizzazione, inclusi override della finestra di contesto dei modelli. @@ -90,18 +90,18 @@ I dati fluiscono in una direzione, dal codice del tuo agente alla dashboard: il - **Dashboard**: dove esplori tutto. - **Servizi opzionali**: un servizio di scoring (valutazioni) e un servizio assistente AI (la chat all'interno della dashboard). -Per il vocabolario utilizzato in tutta la documentazione (*event, session, evaluation, audit, finding, incident*), vedi [Concetti](/it/agenteye/concepts). +Per il vocabolario utilizzato in tutta la documentazione (*event, session, evaluation, audit, finding, incident*), vedi [Concetti](/it/concepts). --- -## Ottenere Failproof AI Observability +## Ottenere FailproofAI Cloud -Failproof AI Observability è un prodotto enterprise di Failproof AI, e funziona insieme a Failproof AI Enforcement — il prodotto di policy e guardrail — sotto il marchio Failproof AI. Funziona interamente nel tuo ambiente. Se non hai ancora accesso ai pacchetti, richiedi una demo e ti faremo partire: invia un'email a [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +FailproofAI Cloud è un prodotto enterprise di Failproof AI, e funziona insieme a FailproofAI guardrails — il prodotto di policy e guardrail — sotto il marchio Failproof AI. Funziona interamente nel tuo ambiente. Se non hai ancora accesso ai pacchetti, richiedi una demo e ti faremo partire: invia un'email a [nikita@befailproof.ai](mailto:nikita@befailproof.ai). --- ## Passaggi successivi -- [Concetti](/it/agenteye/concepts): il vocabolario di Failproof AI Observability in un'unica pagina. -- [Observability](/it/agenteye/observability): segui quello che fanno i tuoi agenti, esecuzione per esecuzione. -- [Sicurezza](/it/agenteye/security): come Failproof AI Observability mantiene i tuoi dati isolati e sotto il tuo controllo. \ No newline at end of file +- [Concetti](/it/concepts): il vocabolario di FailproofAI Cloud in un'unica pagina. +- [FailproofAI Cloud](/it/cloud/overview): segui quello che fanno i tuoi agenti, esecuzione per esecuzione. +- [Sicurezza](/it/cloud/security): come FailproofAI Cloud mantiene i tuoi dati isolati e sotto il tuo controllo. \ No newline at end of file diff --git a/docs/it/agenteye/telemetry.mdx b/docs/it/cloud/performance.mdx similarity index 81% rename from docs/it/agenteye/telemetry.mdx rename to docs/it/cloud/performance.mdx index a55b17a5..4cd1725f 100644 --- a/docs/it/agenteye/telemetry.mdx +++ b/docs/it/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "Vedi l'istante in cui i tuoi modelli, strumenti o hook rallentano Vedi l'istante in cui i tuoi modelli, strumenti o hook rallentano o fanno lievitare i costi, e intercetta un picco di latenza coda prima che i tuoi utenti lo avvertano. Tre pagine dedicate trasformano i tempi grezzi in p50, p95 e p99 leggibili a colpo d'occhio. -![La pagina Models che mostra una mappa di calore della latenza, una banda percentile e figure di token, costo e finestra di contesto per modello](/agenteye/images/models.png) +![La pagina Models che mostra una mappa di calore della latenza, una banda percentile e figure di token, costo e finestra di contesto per modello](/cloud/images/models.png) *La pagina Models: una mappa di calore della latenza, una banda percentile e token per modello, costo stimato e riempimento della finestra di contesto.* ## Smetti di lasciare che le medie nascondano le tue peggiori esecuzioni @@ -24,13 +24,13 @@ Un mirino di hover condiviso collega la mappa di calore e la banda, così un pic La pagina Models (mostrata in alto) risponde alle due domande che una fattura pone sempre: quale modello e quanto costa. In aggiunta alla vista di latenza condivisa, aggiunge **consumo di token per modello**, **costo stimato** e **riempimento della finestra di contesto**, così la crescita della prompt incontrollata e una compattazione imminente sono visibili prima di sorprenderti. -Failproof AI Observability riconosce automaticamente gli ID dei modelli comuni. Se una finestra sembra scorretta o esegui un modello privato tuo, correggilo o aggiungine uno in **Settings**, in **model context windows**, e le letture del riempimento seguiranno. +FailproofAI Cloud riconosce automaticamente gli ID dei modelli comuni. Se una finestra sembra scorretta o esegui un modello privato tuo, correggilo o aggiungine uno in **Settings**, in **model context windows**, e le letture del riempimento seguiranno. ## Tools: distingui il lento dal rotto Una chiamata di strumento può essere lenta, oppure può stare fallendo in silenzio, e vuoi sapere quale sia in secondi, non dopo aver scavato nei log. -![La pagina Tools che mostra la mappa di calore della latenza condivisa e la banda percentile accanto a un dettaglio di successo e fallimento e una barra di distribuzione degli strumenti](/agenteye/images/tools.png) +![La pagina Tools che mostra la mappa di calore della latenza condivisa e la banda percentile accanto a un dettaglio di successo e fallimento e una barra di distribuzione degli strumenti](/cloud/images/tools.png) *La pagina Tools: la stessa mappa di calore e banda percentile, più un dettaglio di successo e fallimento e una barra di distribuzione degli strumenti.* Accanto alla vista di latenza condivisa, la pagina Tools aggiunge un **dettaglio di successo e fallimento** e una **barra di distribuzione degli strumenti**, così vedi a colpo d'occhio quali strumenti usi di più e quali stanno consumando il tuo budget di errori. @@ -39,14 +39,14 @@ Accanto alla vista di latenza condivisa, la pagina Tools aggiunge un **dettaglio Quando un hook del ciclo di vita fa rallentare un'esecuzione, "gli hook sono lenti" non è qualcosa su cui puoi agire. La pagina Hooks ti porta a quello che conta. -![La pagina Hooks che mostra la latenza suddivisa per nome dell'hook e evento trigger sulla mappa di calore e banda percentile condivise](/agenteye/images/hooks.png) +![La pagina Hooks che mostra la latenza suddivisa per nome dell'hook e evento trigger sulla mappa di calore e banda percentile condivise](/cloud/images/hooks.png) *La pagina Hooks: latenza suddivisa per nome dell'hook e evento trigger.* Sulla stessa mappa di calore della latenza e banda percentile, la pagina Hooks suddivide l'attività per **nome dell'hook** e **evento trigger**, così arrivi all'hook singolo e all'evento singolo che hanno bisogno di attenzione. ## Correlati -- [Event stream](/it/agenteye/event-stream): il percorso codificato a colori live di ogni evento. -- [Sessions](/it/agenteye/sessions): raggruppa gli eventi in una riga per esecuzione e apri il suo grafico di esecuzione. -- [Error tracking](/it/agenteye/error-tracking): una superficie di triage per tutto quello che il dashboard dipinge di rosso. -- [Dashboards](/it/agenteye/dashboards): viste riepilogative sulla tua flotta. \ No newline at end of file +- [Event stream](/it/cloud/event-stream): il percorso codificato a colori live di ogni evento. +- [Sessions](/it/cloud/sessions): raggruppa gli eventi in una riga per esecuzione e apri il suo grafico di esecuzione. +- [Error tracking](/it/cloud/errors): una superficie di triage per tutto quello che il dashboard dipinge di rosso. +- [Dashboards](/it/cloud/dashboards): viste riepilogative sulla tua flotta. \ No newline at end of file diff --git a/docs/it/agenteye/queries.mdx b/docs/it/cloud/queries.mdx similarity index 78% rename from docs/it/agenteye/queries.mdx rename to docs/it/cloud/queries.mdx index c99b38bc..83ebf86e 100644 --- a/docs/it/agenteye/queries.mdx +++ b/docs/it/cloud/queries.mdx @@ -3,9 +3,9 @@ title: "Query" description: "Poni qualsiasi domanda sui dati del tuo agente e ottieni una risposta in pochi secondi." --- -Poni qualsiasi domanda sui dati del tuo agente e ottieni una risposta in pochi secondi. Failproof AI Observability ti offre una libreria di query salvate e pronte all'uso sui tuoi eventi e valutazioni, così puoi partire da un esempio funzionante invece di un editor SQL vuoto. +Poni qualsiasi domanda sui dati del tuo agente e ottieni una risposta in pochi secondi. FailproofAI Cloud ti offre una libreria di query salvate e pronte all'uso sui tuoi eventi e valutazioni, così puoi partire da un esempio funzionante invece di un editor SQL vuoto. -![La libreria delle query salvate: una griglia di query riutilizzabili, sia preset built-in che personalizzati](/agenteye/images/queries.png) +![La libreria delle query salvate: una griglia di query riutilizzabili, sia preset built-in che personalizzati](/cloud/images/queries.png) *La tua libreria di query salvate in `//queries`: i preset built-in accanto alle query che il tuo team ha salvato.* @@ -21,7 +21,7 @@ Trovalo in `//queries`. Apri qualsiasi query e arriverà nel compositore SQL, dove puoi modificarla e vedere la risposta immediatamente: nessuna esportazione, nessun andata e ritorno, nessuna attesa di qualcun altro. -![Il compositore di query SQL che esegue una query salvata, con una barra laterale dello schema e una griglia di risultati live](/agenteye/images/query-lab.png) +![Il compositore di query SQL che esegue una query salvata, con una barra laterale dello schema e una griglia di risultati live](/cloud/images/query-lab.png) *Il compositore SQL: la tua query a sinistra, una barra laterale dello schema per non dimenticare mai un nome di colonna, e una griglia di risultati live sotto.* @@ -42,14 +42,14 @@ agenteye query list # le stesse query salvate, dal tuo terminale agenteye query run errs --arg prod # eseguine una e stampa le righe (aggiungi --json per usarla in pipe) ``` - Vedi [CLI e agenti](/it/agenteye/cli-and-agents) per l'insieme completo di comandi. + Vedi [CLI e agenti](/it/cloud/cli) per l'insieme completo di comandi. -- **Dall'assistente AI.** Non sei sicuro di come formulare l'SQL? Chiedi all'[assistente AI](/it/agenteye/assistant) nel dashboard in inglese naturale e ti farà uno schema della query e la salverà nella tua libreria per te. +- **Dall'assistente AI.** Non sei sicuro di come formulare l'SQL? Chiedi all'[assistente AI](/it/cloud/assistant) nel dashboard in inglese naturale e ti farà uno schema della query e la salverà nella tua libreria per te. L'esecuzione di una query salvata è controllata dal permesso `queries:run`, mantenuto separato dai permessi per creare o eliminare query, così puoi concedere accesso in lettura senza permettere a tutti di riscrivere la libreria. ## Correlati -- [Dashboard](/it/agenteye/dashboards): fissa i risultati delle query in grafici condivisi a livello organizzativo. -- [Assistente AI](/it/agenteye/assistant): poni domande in inglese naturale e ottieni una query in cambio. -- [CLI e agenti](/it/agenteye/cli-and-agents): esegui e salva le stesse query dal tuo terminale. \ No newline at end of file +- [Dashboard](/it/cloud/dashboards): fissa i risultati delle query in grafici condivisi a livello organizzativo. +- [Assistente AI](/it/cloud/assistant): poni domande in inglese naturale e ottieni una query in cambio. +- [CLI e agenti](/it/cloud/cli): esegui e salva le stesse query dal tuo terminale. \ No newline at end of file diff --git a/docs/it/agenteye/python-sdk.mdx b/docs/it/cloud/sdk.mdx similarity index 92% rename from docs/it/agenteye/python-sdk.mdx rename to docs/it/cloud/sdk.mdx index c36a2466..51085dc7 100644 --- a/docs/it/agenteye/python-sdk.mdx +++ b/docs/it/cloud/sdk.mdx @@ -5,11 +5,11 @@ description: "Vedi esattamente cosa hanno fatto i tuoi agenti AI in produzione: --- -Vedi esattamente cosa hanno fatto i tuoi agenti AI in produzione: ogni esecuzione dell'agente, chiamata di strumento, richiesta del modello, hook e intervento umano. L'SDK Python per l'Observability di Failproof AI registra questa traccia dall'interno del codice del tuo agente, così puoi eseguire il debug, audit e valutazione di ciò che è accaduto. Usalo ogni volta che vuoi che Failproof AI Observability osservi i tuoi agenti. +Vedi esattamente cosa hanno fatto i tuoi agenti AI in produzione: ogni esecuzione dell'agente, chiamata di strumento, richiesta del modello, hook e intervento umano. L'SDK Python per l'FailproofAI Cloud di Failproof AI registra questa traccia dall'interno del codice del tuo agente, così puoi eseguire il debug, audit e valutazione di ciò che è accaduto. Usalo ogni volta che vuoi che FailproofAI Cloud osservi i tuoi agenti. Sotto il cofano, l'SDK scrive eventi strutturati in file JSONL locali e il daemon collector li preleva e li invia automaticamente alla piattaforma. Non devi gestire tu stesso questi file. -> **Consiglio:** Nuovo a Failproof AI Observability? Questa pagina è il riferimento completo degli eventi SDK. +> **Consiglio:** Nuovo a FailproofAI Cloud? Questa pagina è il riferimento completo degli eventi SDK.
    @@ -27,7 +27,7 @@ Una volta installato, confermalo: python -c "import agenteye; print(agenteye.__version__)" ``` -Preferisci lasciare che un agente di codifica gestisca l'intera integrazione? L'[Agent Skill Python SDK](/it/agenteye/python-sdk-skill) conosce il percorso di installazione, pianifica i punti di strumentazione, li scrive e verifica che gli eventi arrivino. +Preferisci lasciare che un agente di codifica gestisca l'intera integrazione? L'[Agent Skill Python SDK](/it/cloud/agent-skills) conosce il percorso di installazione, pianifica i punti di strumentazione, li scrive e verifica che gli eventi arrivino. --- @@ -61,7 +61,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### Strumentazione di una chiamata reale -In pratica racchiudi il tuo codice agente esistente. Circonda una chiamata al modello con `model_request` prima e `model_response` dopo, in modo che i due eventi abbracciamo la richiesta reale e Failproof AI Observability possa abbinarli: +In pratica racchiudi il tuo codice agente esistente. Circonda una chiamata al modello con `model_request` prima e `model_response` dopo, in modo che i due eventi abbracciamo la richiesta reale e FailproofAI Cloud possa abbinarli: ```python import anthropic @@ -100,7 +100,7 @@ Racchiudi le chiamate ai strumenti allo stesso modo con `tool_use` e `tool_resul Ecco come appaiono questi eventi una volta raggiunto il dashboard, codificati per colore per tipo e filtrabili per ambiente, agente e sessione: -![Lo stream live degli Events, codificato per colore per tipo di evento e filtrabile per ambiente, agente e sessione](/agenteye/images/events-stream.png) +![Lo stream live degli Events, codificato per colore per tipo di evento e filtrabile per ambiente, agente e sessione](/cloud/images/events-stream.png) --- @@ -157,9 +157,9 @@ Questo rende la redazione tua scelta e tua responsabilità. Se un prompt o paylo ## Riferimento degli eventi -La maggior parte degli eventi viene in coppie start/end che condividono un ID di correlazione: `tool_use` e `tool_result` condividono un `tool_call_id`, `hook_triggered` e `hook_completed` condividono un `hook_id`, e `human_wait` e `human_input` condividono un `input_id`. Emetti l'evento di inizio, fai il lavoro, poi emetti l'evento di fine con lo stesso ID. Failproof AI Observability abbina la coppia e calcola `duration_ms` per te, così non passi mai `duration_ms` da solo. +La maggior parte degli eventi viene in coppie start/end che condividono un ID di correlazione: `tool_use` e `tool_result` condividono un `tool_call_id`, `hook_triggered` e `hook_completed` condividono un `hook_id`, e `human_wait` e `human_input` condividono un `input_id`. Emetti l'evento di inizio, fai il lavoro, poi emetti l'evento di fine con lo stesso ID. FailproofAI Cloud abbina la coppia e calcola `duration_ms` per te, così non passi mai `duration_ms` da solo. -![Un grafo di esecuzione in stile git di una sessione accanto alla sua timeline degli eventi, ricostruito da eventi appaiati, con il pannello di breakdown strumento/modello/hook](/agenteye/images/session-detail.png) +![Un grafo di esecuzione in stile git di una sessione accanto alla sua timeline degli eventi, ricostruito da eventi appaiati, con il pannello di breakdown strumento/modello/hook](/cloud/images/session-detail.png) Tutti i metodi degli eventi richiedono questi due campi: @@ -433,5 +433,5 @@ Ogni file viene scritto atomicamente: l'SDK scrive in un file temporaneo e poi l ## Prossimi step -- [Event stream](/it/agenteye/event-stream): guarda questi eventi arrivare in tempo reale, codificati per colore e filtrabili per ambiente, agente e sessione. -- [Sessions](/it/agenteye/sessions): vedi come gli eventi appaiati ricostruiscono ogni esecuzione dell'agente come un grafo di esecuzione e timeline. \ No newline at end of file +- [Event stream](/it/cloud/event-stream): guarda questi eventi arrivare in tempo reale, codificati per colore e filtrabili per ambiente, agente e sessione. +- [Sessions](/it/cloud/sessions): vedi come gli eventi appaiati ricostruiscono ogni esecuzione dell'agente come un grafo di esecuzione e timeline. \ No newline at end of file diff --git a/docs/it/agenteye/security.mdx b/docs/it/cloud/security.mdx similarity index 51% rename from docs/it/agenteye/security.mdx rename to docs/it/cloud/security.mdx index d1409f0a..e0ed4373 100644 --- a/docs/it/agenteye/security.mdx +++ b/docs/it/cloud/security.mdx @@ -1,22 +1,22 @@ --- title: "Sicurezza" -description: "Failproof AI Observability è costruito per stare vicino ai tuoi agenti in produzione, il che significa che vede i tuoi prompt, gli input degli strumenti e gli output." +description: "FailproofAI Cloud è costruito per stare vicino ai tuoi agenti in produzione, il che significa che vede i tuoi prompt, gli input degli strumenti e gli output." --- -Failproof AI Observability è costruito per stare vicino ai tuoi agenti in produzione, il che significa che vede i tuoi prompt, gli input degli strumenti e gli output. Questa pagina spiega come mantiene questi dati isolati, controllati e nelle tue mani. Se stai valutando Failproof AI Observability per una revisione di sicurezza, inizia da qui. +FailproofAI Cloud è costruito per stare vicino ai tuoi agenti in produzione, il che significa che vede i tuoi prompt, gli input degli strumenti e gli output. Questa pagina spiega come mantiene questi dati isolati, controllati e nelle tue mani. Se stai valutando FailproofAI Cloud per una revisione di sicurezza, inizia da qui. --- ## I tuoi dati rimangono nel tuo ambiente -Failproof AI Observability è self-hosted. Gli eventi, i prompt, le risposte del modello e l'analitca sono memorizzati nei tuoi database, nel tuo ambiente. Nessun dato viene inviato a un servizio SaaS di terze parti per l'archiviazione, e i tuoi dati rimangono nel tuo account cloud. +FailproofAI Cloud è self-hosted. Gli eventi, i prompt, le risposte del modello e l'analitca sono memorizzati nei tuoi database, nel tuo ambiente. Nessun dato viene inviato a un servizio SaaS di terze parti per l'archiviazione, e i tuoi dati rimangono nel tuo account cloud. --- ## Isolamento dei tenant -Un'istanza di Failproof AI Observability può ospitare molte organizzazioni, ognuna isolata a livello di storage — applicato dal database, non solo dall'interfaccia utente: +Un'istanza di FailproofAI Cloud può ospitare molte organizzazioni, ognuna isolata a livello di storage — applicato dal database, non solo dall'interfaccia utente: - I dati operativi di un'organizzazione (utenti, chiavi, dashboard, query salvate) sono vincolati a quell'organizzazione, e le letture cross-org sono bloccate dal database stesso. - Ogni evento acquisito è contrassegnato con l'organizzazione proprietaria, quindi gli eventi di un'organizzazione non possono mai essere letti da un'altra. @@ -27,9 +27,9 @@ Ogni rotta della dashboard è vincolata sotto uno slug dell'organizzazione (`//…`). Sessions si trova sotto **Observe** nella barra laterale sinistra, accanto a Events, con i filtri di intervallo di date, ambiente, agent e sessione nella parte superiore dell'elenco. Ogni riga è a un clic dal suo grafico di esecuzione completo. -Per attivare i badge di punteggio e il filtraggio per intervallo di punteggio, collega un valutatore: vedi [Evaluations](/it/agenteye/evaluations). +Per attivare i badge di punteggio e il filtraggio per intervallo di punteggio, collega un valutatore: vedi [Evaluations](/it/cloud/evaluations). --- ## Correlati -- [Event stream](/it/agenteye/event-stream): il percorso grezzo e per-passo da cui ogni sessione è stata raggruppata. -- [Evaluations](/it/agenteye/evaluations): collega un valutatore in modo che ogni esecuzione ottenga un badge di punteggio per cui puoi filtrare. -- [Telemetry](/it/agenteye/telemetry): come le esecuzioni vanno dal tuo agent in queste sessioni. \ No newline at end of file +- [Event stream](/it/cloud/event-stream): il percorso grezzo e per-passo da cui ogni sessione è stata raggruppata. +- [Evaluations](/it/cloud/evaluations): collega un valutatore in modo che ogni esecuzione ottenga un badge di punteggio per cui puoi filtrare. +- [Telemetry](/it/cloud/performance): come le esecuzioni vanno dal tuo agent in queste sessioni. \ No newline at end of file diff --git a/docs/it/concepts.mdx b/docs/it/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/it/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/it/daemon.mdx b/docs/it/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/it/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/it/dashboard.mdx b/docs/it/dashboard.mdx index fd644b56..63129c53 100644 --- a/docs/it/dashboard.mdx +++ b/docs/it/dashboard.mdx @@ -69,7 +69,7 @@ Un rapporto guidato dalla personalità su come il tuo agente si è effettivament 4. **How to improve** — elenco di righe calmo, uno per policy prescritta: nome policy in bianco, descrizione di una riga, comando di installazione + pulsante di copia sul lato destro. L'intestazione della sezione legge `enable all N → projected · ` (il punteggio che raggiungeresti con ogni correzione applicata), e il suo pulsante `[install all]` copia il comando combinato `failproofai policy add a b c …` per ogni policy prescritta. 5. **Come back better** — due carte una accanto all'altra. Sinistra: imposta un promemoria (selettore di cadenza `3d` / `7d` / `14d` / `30d`; persiste tramite `/api/auth/reminder` una volta autenticato). Destra: sblocca i vantaggi di failproof — `invite a friend` apre un modale che accetta un elenco di email di amici separati da virgola/spazio/newline (max 10 per invio), le invia tramite POST a `/api/audit/invite`, che inoltrano a `POST /v0/invite` del server api. Il server api invia un'email per destinatario da `invite@failproof.ai` con il mittente in Cc e `Reply-To` impostato, quindi il destinatario vede chi lo ha invitato e il mittente riceve una copia nella sua inbox. Gli utenti anonimi vengono instradati prima attraverso `AuthDialog` in modo che l'email del mittente sia nota prima che gli inviti vengano inviati. L'adempimento dei diritti / vantaggi è un seguito. -Guidato dal runtime `failproofai audit` — vedi [Audit CLI](/it/cli/audit) per il motore di scansione sottostante, i flag supportati e gli invarianti di cache per transcript. La dashboard memorizza nella cache il risultato più recente su `~/.failproofai/audit-dashboard.json` (modo `0600`, slot singolo, i nuovi run sovrascrivono) quindi le revisioni sono istantanee; **sia la cache per transcript che il risultato complessivo vengono rifiutati in lettura una volta che sono più vecchi di 7 giorni** quindi la dashboard non serve mai silenziosamente un risultato di una settimana — passato il TTL `/audit` cade nel suo stato vuoto e chiede un'esecuzione nuova. Facendo clic su `[ re-audit now ]` vicino al fondo del rapporto si invia `/api/audit/run` con `noCache: true` — la riesecuzione dell'audit bypassa la cache per transcript e ripete la scansione di ogni transcript da zero invece di restituire silenziosamente il risultato memorizzato — e la dashboard esegue il polling di `/api/audit/status` a 1Hz fino al termine dell'esecuzione; una striscia di progresso rosa appiccicatizia si fissa all'inizio del viewport durante l'esecuzione con un timer di tempo trascorso, e il risultato nuovo si scambia al suo posto al successo (nessun ricaricamento della pagina intera; un riesame fallito dell'audit lascia intatto il rapporto precedente). In caso di errore la striscia diventa rossa con copia codificata da `RerunError.kind` (`timeout` / `network` / `post_failed`). Lo stato vuoto (nessuna cache o scaduta) e lo stato zero-sessioni (cache esiste ma la scansione non ha trovato alcun transcript) sono visualizzati separatamente. +Guidato dal runtime `failproofai audit` — vedi [Audit CLI](/it/audit) per il motore di scansione sottostante, i flag supportati e gli invarianti di cache per transcript. La dashboard memorizza nella cache il risultato più recente su `~/.failproofai/audit-dashboard.json` (modo `0600`, slot singolo, i nuovi run sovrascrivono) quindi le revisioni sono istantanee; **sia la cache per transcript che il risultato complessivo vengono rifiutati in lettura una volta che sono più vecchi di 7 giorni** quindi la dashboard non serve mai silenziosamente un risultato di una settimana — passato il TTL `/audit` cade nel suo stato vuoto e chiede un'esecuzione nuova. Facendo clic su `[ re-audit now ]` vicino al fondo del rapporto si invia `/api/audit/run` con `noCache: true` — la riesecuzione dell'audit bypassa la cache per transcript e ripete la scansione di ogni transcript da zero invece di restituire silenziosamente il risultato memorizzato — e la dashboard esegue il polling di `/api/audit/status` a 1Hz fino al termine dell'esecuzione; una striscia di progresso rosa appiccicatizia si fissa all'inizio del viewport durante l'esecuzione con un timer di tempo trascorso, e il risultato nuovo si scambia al suo posto al successo (nessun ricaricamento della pagina intera; un riesame fallito dell'audit lascia intatto il rapporto precedente). In caso di errore la striscia diventa rossa con copia codificata da `RerunError.kind` (`timeout` / `network` / `post_failed`). Lo stato vuoto (nessuna cache o scaduta) e lo stato zero-sessioni (cache esiste ma la scansione non ha trovato alcun transcript) sono visualizzati separatamente. ### Policy diff --git a/docs/it/architecture.mdx b/docs/it/how-it-works.mdx similarity index 100% rename from docs/it/architecture.mdx rename to docs/it/how-it-works.mdx diff --git a/docs/it/introduction.mdx b/docs/it/introduction.mdx index f5c57577..4b1895a6 100644 --- a/docs/it/introduction.mdx +++ b/docs/it/introduction.mdx @@ -55,4 +55,4 @@ failproofai policies --install # abilita le politiche (oppure salta — `failp failproofai # avvia la dashboard ``` -Consulta la guida [Inizia](/it/getting-started) per la procedura dettagliata completa. \ No newline at end of file +Consulta la guida [Inizia](/it/quickstart) per la procedura dettagliata completa. \ No newline at end of file diff --git a/docs/it/policies.mdx b/docs/it/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/it/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/it/getting-started.mdx b/docs/it/quickstart.mdx similarity index 100% rename from docs/it/getting-started.mdx rename to docs/it/quickstart.mdx diff --git a/docs/it/reference/files.mdx b/docs/it/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/it/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/ja/agent-support.mdx b/docs/ja/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/ja/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/ja/agenteye/cli-and-agents.mdx b/docs/ja/agenteye/cli-and-agents.mdx deleted file mode 100644 index 738af318..00000000 --- a/docs/ja/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "Failproof AI Observabilityのデプロイ全体を、コマンド一つで。" ---- - - -Failproof AI Observabilityのデプロイ全体を、コマンド一つで。ターミナルを離れずに本番環境の確認、APIキーの発行、インシデントの承認が可能。CIへのスクリプト組み込みや、コーディングエージェントへの自然言語による指示にも対応しています。 - -```bash -pipx install agenteye -agenteye login --email you@example.com # 6桁のコードがメールで届きます -agenteye --json sessions --since 24h # 過去1日のエージェント実行一覧（新しい順） -``` - -*`agenteye` CLIはダッシュボードと通信します。これはサーバーにイベントを送信するコレクターとは別のツールです。* - -## デプロイ全体を、コマンド一つで - -簡単な確認のためにタブを行き来するのはもう終わりにしましょう。`agenteye` CLIは単一のバイナリからデータの参照と組織の管理を行えるため、ダッシュボードをクリックして回っていた作業が1行のコマンドになります。再実行、エイリアス登録、ランブックへの貼り付けも自由自在です。4つの機能領域を提供します： - -- **データの参照：** `sessions`、`events`、`evals`、`errors`を時間・エージェント・環境でフィルタリング。 -- **組織の管理：** `keys`、`users`、`settings`、`alerts`、`incidents`。 -- **分析の実行：** 保存済みSQLとイベントデータに対するアドホックな `query` ランナー。 -- **アシスタントへの質問：** `agent ask` でダッシュボード上のものと同じ読み取り専用アナリストに問い合わせ。 - -`pipx` で一度インストールし、メールで届く6桁のコードでサインインすれば準備完了です。セッションは約1日持続します。期限切れになったら `agenteye login` を再実行してください。ブラウザを開かずに本番環境の確認、キーの発行、発火中のインシデントのトリアージが行えます： - -```bash -agenteye errors --since 24h --aggregate # エラータイプ別にグループ化して何が壊れているかを確認 -agenteye incidents list --state firing # 現在発火中のインシデントを確認 -agenteye keys create ci --add events:add # イベント送信専用のキーを発行（シークレットは一度だけ表示） -``` - -一つ覚えておくべき習慣があります：`--json` のようなグローバルオプションはコマンドの前に置きます。`agenteye --json sessions` が正しく、`agenteye sessions --json` は正しくありません。 - -## スクリプト化してCIに組み込む - -すべてのコマンドは `--json` に対応しており、それがすべてを変えます。クリーンなJSONがstdoutに出力され、人間向けのステータスや警告はstderrに出力されるため、`--json` でキャプチャした出力は余計な行を取り除く必要なくそのまま `jq` にパイプできます。これにより、CLIはプロンプトで使う場合にも、出力をパースするコーディングエージェントにとっても等しく使いやすいツールになっています： - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -無人実行を前提に設計されています。ターミナルが接続されていない場合は確認プロンプトが自動的にスキップされるため、パイプラインで処理が止まることはありません。また、すべてのコマンドは意味のある終了コードを返します：`0` 成功、`4` 未ログイン、`5` 権限不足（メッセージに権限名が明示されます。例：`alerts:write`）、`3` ダッシュボードに接続不可。スクリプトは `4` を受け取って再認証を行ったり、`5` を受け取って管理者に何を依頼すべきかを正確に把握したりと、失敗の原因が不明なまま処理を終了せずに対処できます。 - -## コーディングエージェントに自然言語で操作させる - -さらに言えば、これらのフラグをすべて覚える必要はないはずです。**CLIスキル**は `agenteye-cli` という小さなAgent Skillフォルダで、Claude CodeやCodexのようなコーディングエージェントに自然言語のリクエストからCLIを操作する方法を教えます。「今日、何か壊れているものはある？」と聞けば、エージェントが適切なコマンドを選んであなたの代わりに実行し、結果を文章で答えてくれます。 - -Claude Codeの場合、`agenteye-cli` フォルダを `~/.claude/skills/` に置くだけで自動的に検出されます。Failproof AI Observabilityがそのフォルダを提供します。スキルはすでにインストール済みのCLIを操作するだけなので、追加でインストールするものはありません。メールコードによるログインはエージェントが代行できないため、先にご自身でログインしておいてください。 - -エージェントはあなたのログイン権限でCLIを実行するため、読み取りも書き込みも含め、あなたが許可されているすべての操作が可能です：キーの作成、設定の変更、インシデントの解決など。エージェントに対してCLIの「本当によろしいですか？」プロンプトは表示されないため、スキルは変更を行う前に正確なコマンドを提示してあなたの承認を待つよう設計されています。確認ステップはあなた自身です。 - -```text -you session run-001 が失敗した原因は？ - -agent Running: agenteye --json events --session-id run-001 --all - checkout-agentが3回目のツール呼び出しでTimeoutErrorが発生しました。 -``` - -読み取りはすぐに実行され、書き込みはすべてあなたの確認を待ちます： - -```text -you CIにイベントの送信だけできるキーを発行して。 - -agent APIキーを作成します。以下のコマンドを実行します： - agenteye keys create ci --add events:add - 続行してよいですか？ - -you yes - -agent 完了しました。events:addのみの権限で「ci」キーを作成しました。シークレットは一度しか表示されないため、今すぐ保存してください。 -``` - -## 関連情報 - -- [CLIリファレンス](/ja/agenteye/cli)：すべてのコマンド、フラグ、JSONの形式。 -- [エージェント向けCLIレシピ](/ja/agenteye/cli-recipes)：コピー＆ペーストで使える `jq` パターンと終了コードの処理方法。 -- [CLIエージェントスキル](/ja/agenteye/cli-skill)：`agenteye-cli` スキルのインストールと実行方法。 -- [AIアシスタント](/ja/agenteye/assistant)：`agent ask` が接続するダッシュボード内のアナリスト。 \ No newline at end of file diff --git a/docs/ja/agenteye/cli-skill.mdx b/docs/ja/agenteye/cli-skill.mdx deleted file mode 100644 index 7ff1d45a..00000000 --- a/docs/ja/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Failproof AI Observability CLI エージェントスキル" -description: "コーディングエージェントに「今日何か壊れてる？」と聞くだけで、ライブの Failproof AI Observability データから回答を得られます。コマンドを暗記する必要はありません。" ---- - - -コーディングエージェントに *「今日何か壊れてる？」* と聞くだけで、ライブの Failproof AI Observability データから回答を得られます。コマンドを暗記する必要はありません。**Failproof AI Observability CLI スキル**（`agenteye-cli`）は *エージェントスキル* です。これは、Claude Code や Codex などのコーディングエージェントがオンデマンドで読み込む小さな指示フォルダです。このスキルにより、エージェントは *「CI にイベントのプッシュだけできるキーを作って」* や *「発火中のインシデントを Ack して私にアサインして」* といった平易な英語のリクエストを通じて、[`agenteye` CLI](/ja/agenteye/cli) を使って Observability のデプロイを操作できるようになります。 - -これは**サービスでも独立したバイナリでもありません**。デプロイするものは何もありません。すでにインストール済みの CLI の上で動作し、エージェントが `agenteye --json …` を呼び出してクリーンな JSON をパースし、散文で回答します。エージェントができることはすべて、同じコマンドを入力すれば自分でもできます。 - ---- - -## 他の Failproof AI Observability インターフェースとの関係 - -Failproof AI Observability では、同じデータとコントロールに到達する方法が4つあります。それぞれ補完し合う関係です： - -| インターフェース | 内容 | 実行場所 | 使いどころ | -|---|---|---|---| -| **[CLI](/ja/agenteye/cli)** | `agenteye` のコマンド・フラグリファレンス | ターミナル | 特定のコマンドを実行またはスクリプト化したいとき | -| **[CLI レシピ](/ja/agenteye/cli-recipes)** | コピペ可能な `jq`/パイプラインパターン | ターミナル / スクリプト | CLI を自動化に組み込みたいとき | -| **CLI スキル**（このドキュメント） | CLI への自然言語フロントドア | ワークステーション上のコーディングエージェント | コマンドを選ばずに *ただ聞く* だけにしたいとき | -| **[Evaluator スキル](/ja/agenteye/evaluator-skill)** | スコアリングサービスを設計・構築するための兄弟スキル | ワークステーション上のコーディングエージェント | eval スコアを *読む* のではなく *生成* したいとき | -| **[Python SDK スキル](/ja/agenteye/python-sdk-skill)** | エージェントがテレメトリを送出できるようにする兄弟スキル | ワークステーション上のコーディングエージェント | このスキルが読み取るイベントをエージェントに *生成* させたいとき | -| **[ダッシュボード内 AI アシスタント](/ja/agenteye/assistant)** | ダッシュボードに埋め込まれたチャット | サーバーサイド（ダッシュボード内） | データに対するダッシュボード内 Q&A を使いたいとき | - -スキル自体は独自の権限を持ちません。あなたの言葉を CLI コールに変換し、あなたとして実行するだけです： - -```mermaid -flowchart TD - YOU["あなた: 「発火中のインシデントを Ack して」"] --> AGENT["コーディングエージェント (Claude Code / Codex)
    agenteye-cli スキルを読み込む"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|認証済み CLI セッション| API["Observability ダッシュボード API"] -``` - -### ダッシュボード内 AI アシスタントとの違い：重要な区別 - -これらは影響範囲が大きく異なる2つの別ツールです： - -- **ダッシュボード内 AI アシスタント**（[AI アシスタント](/ja/agenteye/assistant)）はダッシュボードに埋め込まれたチャットで、エージェントサービスによってバックアップされています。**読み取り専用＋承認ゲート付き作成**：保存クエリやダッシュボードの下書きを作成できますが、書き込みはすべてあなたの明示的なクリック承認を求めて停止し、削除は行いません。`agent:use` 権限でゲートされており、閲覧中の組織のデータのみを参照します。 -- **CLI スキル**は *あなたの* ワークステーション上の *あなたの* コーディングエージェント内で動作し、**あなた**として `agenteye` CLI を操作します。API キーの作成・ローテーション・無効化、組織設定の変更、インシデントの解決、保存クエリの削除など、**ミューテーションを含む CLI の全機能**を実行できます。制限はあなたの CLI ログインの権限のみです。これらのコマンドを手動で実行するのと同じくらい慎重に扱ってください。 - ---- - -## 前提条件 - -1. **`agenteye` CLI がインストール済み**で `PATH` に通っていること（[CLI](/ja/agenteye/cli) リファレンス参照：`pipx install agenteye`）。 -2. **ダッシュボード URL** が設定されていること（`AGENTEYE_DASHBOARD_URL`、またはエージェントが `--base-url` を渡す）。 -3. **ログイン済みセッション**：事前に `agenteye login` を実行しておくこと。スキルはメールで送られるワンタイムコードによるログインを**完了できません**。セッションが存在しないか期限切れの場合（CLI 終了コード `4`）、`agenteye login` を実行するよう案内します。 - ---- - -## 入手方法 - -このスキルは Failproof AI の公開スキルコレクションで公開されています： - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -一切ゲートはありません。リポジトリは公開されており、スキルは独自の認証情報を必要としません。**公開**の `agenteye` CLI をあなたのダッシュボードに対して、あなたがログインしたセッションを使って動かすだけだからです。誰かに許可を求める必要はありません。 - -スキルは独自のフォルダとして提供されており、`pipx install agenteye` パッケージには**含まれていません**。そこを探さないようにしてください。 - -## スキルのインストール - -最も手軽な方法は [`skills`](https://skills.sh) CLI です。フォルダを取得し、エージェントが参照する場所に配置します： - -```bash -# Claude Code、このプロジェクトのみ -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# すべてのプロジェクト（~/.claude/skills/ にインストール） -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex の場合 -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -インストール後は他のスキルと同様に管理できます： - -```bash -npx skills list -a claude-code # インストール済みスキルの確認 -npx skills update agenteye-cli # 最新バージョンに更新 -npx skills remove agenteye-cli # 削除 -``` - -手動でインストールしたい場合も可能です。エージェントスキルは `SKILL.md`（およびオプションの参照ファイル）を含むフォルダに過ぎないので、コピーするだけで機能します： - -- **Claude Code**：`agenteye-cli/` フォルダを `~/.claude/skills/`（すべてのプロジェクト）または `<リポジトリ>/.claude/skills/`（そのリポジトリのみ）に配置します。Claude Code が自動検出します。`/skills` リストで確認するか、スキルの説明に合致する質問をするだけで確認できます。 -- **Codex（OpenAI）**：Codex は同じ `SKILL.md` を読み込みます。バンドルされている `agents/openai.yaml` で `allow_implicit_invocation: true` が設定されているため、タスクが一致すると Codex が自動でスキルを選択します。明示的に呼び出す場合は `$agenteye-cli` を使用してください。 - ---- - -## 安全性：エージェントが CLI を実行するときミューテーションは確認を求めません - -> **警告：** エージェントに変更を加えさせる前に必ずお読みください。 - -`agenteye` CLI は通常、破壊的な操作の前に *「本当によろしいですか？」* と尋ねます。しかし、**ターミナルに接続されていない場合（コーディングエージェントが実行する方法はまさにこれです）は確認を自動スキップし、`--json` もスキップします。** そのため、エージェントに対して安全確認プロンプトは**表示されません**。 - -スキルはこれを補うように設計されています。実行するコマンドを明示し、**状態変更の前に明示的な OK を求める**よう指示されています。この規律を守ってください。エージェントを通じて Failproof AI Observability を操作するとき、*あなた自身*が確認ステップです。注意すべき状態変更コマンドは以下のとおりです： - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- 書き込み系の `incidents` サブコマンド：`ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -**Observe** 系（`events`、`sessions`、`evals`、`errors`、`list`、`whoami`、`orgs list/current/perms`）はすべて読み取り専用で、何も変更しません。 - -エージェントは**あなた**として動作するため、あなたのログインが許可していることしかできません。権限は**組織ごと**に解決されます（[API キー](/ja/agenteye/api-keys)を参照）。権限のないコマンドは終了コード `5` と該当する権限名を返すため、エージェントは不透明に失敗するのではなく、管理者に何を依頼すればよいかを正確に伝えられます。 - ---- - -## 何を聞けるか - -実際のやり取りの例です。まず読み取り、次に OK を待つ変更： - -```text -you ▸ Is anything broken in the last day? - -agent ▸ Running: agenteye --json errors --since 24h --aggregate - 3 error types in the last 24h. The biggest is TimeoutError - (18 hits, all on the checkout-agent), then RateLimited (5). - Want the failing sessions for TimeoutError? - -you ▸ Give CI a key that can only push events. - -agent ▸ This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you ▸ yes - -agent ▸ Done. Key "ci" created with events:add only. - The secret is shown only once, so store it now. I can't reprint it. -``` - -スキルは各平易な英語の意図を適切な `agenteye` コマンドにマッピングします。有効な値を先に確認し（`list `、`whoami`）、推測せず、変更前に正確なコマンドを提示します。その他の例： - -- *「過去24時間で何か壊れている・失敗しているものはある？」* → `errors --since 24h --aggregate`、その後詳細。 -- *「セッション `run-001` はなぜ失敗した？」* → `events --session-id run-001 --all` + `evals --session-id run-001`。 -- *「今週の品質トレンドは？」* → `evals --aggregate --since 7d`、その後低スコアの実行を詳しく調査。 -- *「CI にイベントのプッシュだけできるキーを作って。」* → `keys create ci --add events:add`（コマンドを提示し、作成してワンタイムシークレットを取得）。 -- *「誰がアクセス権を持っている？Dana を読み取り専用にして。」* → `users list` → `users update dana@… --permission-set read-only`（あなたに確認後）。 -- *「発火中のインシデントを Ack して私にアサインして。」* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`。 - -これらの背後にある正確なコマンド、フラグ、JSON の形式については、[CLI](/ja/agenteye/cli) リファレンスと[エージェント向け CLI レシピ](/ja/agenteye/cli-recipes)を参照してください。 - ---- - -## 次のステップ - -- **[CLI](/ja/agenteye/cli)**：`agenteye` のコマンドとフラグの完全リファレンス。 -- **[エージェント向け CLI レシピ](/ja/agenteye/cli-recipes)**：コピペ可能な `jq` パターンと終了コードの処理。 -- **[Evaluator エージェントスキル](/ja/agenteye/evaluator-skill)**：`agenteye evals` が読み取るスコアを生成する評価器を構築するための兄弟スキル。 -- **[Python SDK エージェントスキル](/ja/agenteye/python-sdk-skill)**：`agenteye` が読み取るテレメトリを送出するようにエージェントを計装する兄弟スキル。 -- **[AI アシスタント](/ja/agenteye/assistant)**：ダッシュボード内アシスタント（このターミナルスキルとは別物です）。 -- **[API キー](/ja/agenteye/api-keys)**：スキルが実行できる内容を制限する組織ごとの権限モデル。 \ No newline at end of file diff --git a/docs/ja/agenteye/codex-capture.mdx b/docs/ja/agenteye/codex-capture.mdx deleted file mode 100644 index 52d7a617..00000000 --- a/docs/ja/agenteye/codex-capture.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "Codex セッションキャプチャ" -description: "チームのローカル OpenAI Codex セッションを通常のセッションおよびイベントとして AgentEye に取り込みます — Codex の使い方を変える必要はありません。" ---- - -エンジニアたちはすでに毎日 OpenAI Codex を使っています。Codex セッションキャプチャは、それらのコーディングセッションを通常のセッションおよびイベントとして AgentEye に取り込むことで、他の観測対象と並べて検索・再生・評価できるようにします。これは [Python SDK](/ja/agenteye/python-sdk) を補完する機能です。SDK は自分で書いたエージェントを計装するのに対し、こちらはチームがすでに行っている Codex の作業をキャプチャします — 使い方を変える必要は一切ありません。 - -小さなバックグラウンドコレクターが Codex のローカルセッショントランスクリプトを書き込みと同時に読み取り、AgentEye に送信します。1 台のマシンに 1 つのコレクターを置くだけで、すべてのローカル Codex サーフェスを一括でキャプチャできます — サーフェスごとのセットアップは不要です。 - -同じコレクターで他のエージェントもキャプチャできます — [OpenClaw](/ja/agenteye/openclaw-capture) や [Hermes](/ja/agenteye/hermes-capture) をご覧ください。使用しているものをそれぞれ有効にしてください。1 つのコレクターで複数を同時にキャプチャできます。 - ---- - -## キャプチャされる内容 - -**ローカル**で動作するすべての Codex サーフェスは同じオンディスクのセッショントランスクリプトを生成し、コレクターはそれらをすべて取得します。 - -- Codex **CLI** および `codex exec` -- **VS Code / IDE 拡張機能** -- セッションをローカルで実行している場合の**デスクトップアプリ** - -各 Codex セッションは AgentEye の[セッション](/ja/agenteye/sessions)になり、ユーザーとアシスタントのメッセージ、推論、ツール呼び出し、ツールの結果、トークン使用量が対応する[イベント](/ja/agenteye/event-stream)になります。各セッションの発生元（CLI、IDE、またはデスクトップ）も記録されるため、区別することができます。 - -> **クラウドセッションはキャプチャされません。** デスクトップアプリはセッションをますます Codex クラウドで実行するようになっており、マシン上にはメタデータのみが保存されます — ローカルに読み取るトランスクリプトは存在しません。ローカルで実行されたセッションのみがキャプチャされます。 - ---- - -## 有効にする方法 - -キャプチャは有効にするまでオフになっています。`events:add` 権限を持つ API キー（[API キー](/ja/agenteye/api-keys)を参照）でコレクターをインストールし、Codex キャプチャを有効にします。 - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -これにより、コレクターのインストール、バックグラウンドサービスとしての登録、およびキャプチャの開始が行われます。動作を確認するには次のコマンドを実行してください。 - -```bash -agenteye-collector health -``` - -初回実行時に既存の Codex セッションが一括でバックフィルされ、その後の新しいアクティビティは数秒以内にストリーミングされます。Codex 自身のファイルは読み取り専用です — 変更・移動・削除は一切されません。また、再起動をまたいでも各セッションは正確に 1 回だけ送信されます。 - ---- - -## 表示される場所 - -キャプチャされたセッションは **Sessions** に表示され、そのイベントは他の観測対象エージェントと同様に **Events** ストリームに表示されます — そのため、[セッションリプレイ](/ja/agenteye/sessions)、[検索](/ja/agenteye/queries)、[評価](/ja/agenteye/evaluations)、[アラート](/ja/agenteye/alerts)もすべて利用できます。Codex エージェントでフィルタリングすると、それだけを表示できます。 - ---- - -## プライバシー - -Codex トランスクリプトにはセッション全体が含まれます — コマンド出力、ファイルの内容、Codex が読み書きしたものすべてを含み、シークレットが含まれることもあります。キャプチャされたセッションはそのまま送信されるため、AgentEye にその内容を集約することが適切なマシンおよびチームに限ってキャプチャを有効にしてください。また、コレクターには `events:add` のみにスコープされたキーを使用してください。データがどのように隔離されて保管されるかについては、[セキュリティ](/ja/agenteye/security)をご覧ください。 \ No newline at end of file diff --git a/docs/ja/agenteye/concepts.mdx b/docs/ja/agenteye/concepts.mdx deleted file mode 100644 index 34635b35..00000000 --- a/docs/ja/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "概念" -description: "Failproof AI Observability の用語集 — イベント、セッション、評価、監査、ファインディング、インシデントをひとつの場所で定義します。" ---- - - -このページでは、Failproof AI Observability が使用する用語を定義します。他のガイドで見慣れない用語があれば、ここで確認できます。最初から通読する必要はありません。ざっと目を通すか、確認したい用語が出てきたときに参照してください。 - ---- - -## データモデル - -**イベント** -データの最小単位です。1 つのイベントは、エージェントが実行した単一のステップを記録します。`tool_use`、`model_request`、`hook_completed`、`error` などがあります。エージェントは [Python SDK](/ja/agenteye/python-sdk) を通じてイベントを送信し、**Events** ページにリアルタイムで表示されます。 - -**セッション** -`session_id` によって識別される、1 回のエージェント実行です。セッションは同じ ID を持つすべてのイベントをまとめたもので、**Sessions** ページの 1 行として表示され、詳細ページでは実行グラフとして描画されます。セッションは通常 `agent_start` で始まり、`agent_end` で終わります。 - -**エージェント** -`agent_id` によって識別される、実行内の名前付きアクターです。1 回の実行に複数のエージェントが関与することがあります。たとえば、サマライザーのサブエージェントを生成するプランナーなどです。サブエージェントは `parent_id` を持ち、これによって Failproof AI Observability は実行グラフ上でそれぞれ独立したレーンに描画できます。 - -**環境** -実行が行われた場所を示すラベルです。`production`、`staging`、`dev` などがあります。SDK の設定時に一度だけ設定します。ダッシュボードのほぼすべてのページで環境によるフィルタリングが可能です。 - -**コンテキストウィンドウ使用率** -レスポンスがモデルのコンテキストウィンドウを消費した割合です。Failproof AI Observability は認識しているモデルの `model_response` イベントにこの値を付与するため、プロンプトの増大や差し迫ったコンパクションをイベントストリーム上で直接確認できます。 - ---- - -## 品質 - -**評価（Evaluation）** -完了したセッションに対して、あなたが実行するスコアリングサービスが生成する品質スコアです。評価はオプトイン方式です。評価器を接続するまで、セッションは記録されますがスコアリングは行われません。各評価には複数の名前付きスコア（例：`helpfulness`、`factuality`、`tool_efficiency`）を含めることができ、それぞれに短い根拠メモが付きます。[Evaluation suite](/ja/agenteye/evaluation-suite) を参照してください。 - -**スコアキー** -評価器が報告する 1 つの評価軸の名前です（例：`helpfulness`）。アラートと監査は、特定のスコアキーを時系列で監視できます。 - -**評価器（Evaluator）** -あなたのスコアリングサービスです。Failproof AI Observability は完了した実行のトランスクリプトをこのサービスに POST し、返されたスコアを保存します。デフォルトの評価器は提供されません。スコアリングのロジックはあなた自身が実装します。 - ---- - -## 障害の発見と修正 - -**フック（Hook）** -エージェントフレームワークがステップの前後に実行するガードレールまたは副作用です。コンテンツの安全チェック、PII のマスキング、予算ガードなどが該当します。フックは `outcome`（allow、deny、modify）を持つ `hook_triggered` / `hook_completed` イベントを送信し、専用のオブザーブページを持ちます。 - -**アラートルール** -エラー率、p95 レイテンシ、トークンコスト、または評価器のスコアなどのメトリクスが設定したしきい値を超えたときに発火するルールです。ルールが発火すると、インシデントが作成され、設定したチャンネル（メール、Slack、webhook、ダッシュボード内）に通知が送られます。[Alerts](/ja/agenteye/alerts) を参照してください。 - -**インシデント** -アラートルールが発火したときに作成されるオープンな問題です。インシデントにはライフサイクル（承認、割り当て、解決）があり、すべての操作を記録するアクティビティタイムラインを持ちます。手動で作成することもできます。 - -**監査（Audit）** -まだルールを定義していない障害パターンをセッション横断でログから探り出す、定期的な調査です（毎時から毎週まで設定可能）。エラーのクラスター、低スコア、レイテンシの外れ値、ツール呼び出しのループ、完了しなかった実行などを検出します。アラートがすでに把握しているメトリクスを監視するのに対し、監査は次に注目すべき点を教えてくれます。[Audits](/ja/agenteye/audits) を参照してください。 - -**ファインディング（Finding）** -監査実行から得られる、優先度付きかつ証拠に基づいた結果の 1 件です。ファインディングはパターンを名付け、その背後にある正確なセッションにリンクし、トリアージのライフサイクル（承認、解決、ミュート、却下）を持ちます。Failproof AI Observability は実行をまたいでファインディングを重複排除するため、既知のパターンは積み重なるのではなく更新されます。 - -**AI アシスタント** -ダッシュボード内のチャット機能で、あなたのデータを基にエージェントに関する質問に平易な言葉で回答します。デフォルトでは読み取り専用です。アシスタントが作成するもの（保存済みクエリ、ダッシュボードなど）は承認が必要であり、削除操作は一切できません。[AI assistant](/ja/agenteye/assistant) を参照してください。 - ---- - -## 実行環境 - -**組織（テナント）** -独立したワークスペースです。1 つの Failproof AI Observability インスタンスで複数の組織をホストでき、それぞれが独自のユーザー、キー、データを持ちます。すべてのダッシュボード URL は組織のスラッグ（`//…`）にスコープされます。 - -**コレクター** -`agenteye-collector` は、各エージェントマシン上で動作する軽量なデーモンです。SDK がディスクに書き込んだイベントをバッチ処理し、サーバーに送信します。 - -**API キー** -クライアントをサーバーに対して認証するためのスコープ付きトークンです。キーには細かい権限が設定されます（例：コレクター用の `events:add`、ダッシュボードキー用の読み取り専用スコープ）。[API keys](/ja/agenteye/api-keys) を参照してください。 - -**サーバー** -インジェストおよび API サービスです。イベントを受信し、運用状態をデータベースに保存し、ダッシュボードと CLI を提供します。 - -**ダッシュボード** -Web UI です。すべてのページは組織にスコープされ、サーバーの API を通じてデータを読み取ります。 - ---- - -## 次のステップ - -- [Overview](/ja/agenteye/overview): これらのコンポーネントがどのように組み合わさるかを説明します。 -- [Observability](/ja/agenteye/observability): オブザーブのサーフェス（Events、Sessions、Models、Tools、Hooks、Errors）について説明します。 \ No newline at end of file diff --git a/docs/ja/agenteye/evaluator-skill.mdx b/docs/ja/agenteye/evaluator-skill.mdx deleted file mode 100644 index 3466dea6..00000000 --- a/docs/ja/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: "Failproof AI オブザーバビリティ 評価エージェントスキル" -description: "「エージェントの品質が不安定かもしれない」という状態から、コーディングエージェントが設計と実装の両方を担いながら、スコアリングサービスをデプロイするところまで到達できます。" ---- - - -「エージェントの品質が不安定かもしれない」という状態から、コーディングエージェントが設計と実装の両方を担いながら、スコアリングサービスをデプロイするところまで到達できます。**Failproof AI オブザーバビリティ 評価スキル**（`agenteye-evaluator`）は*エージェントスキル*です。Claude Code や Codex などのコーディングエージェントがオンデマンドで読み込む、小さな命令のフォルダーです。このスキルは、エージェントが*あなたの*エージェントにとって追跡すべき品質軸を判断し、それをスコアリングする[評価サービス](/ja/agenteye/evaluation-suite)を作成・テスト・デプロイする方法を教えます。 - -これはホスト型のスコアラーでも、アップロード先のレジストリでも、プラグインシステムでもありません。評価サービスは[Evaluation suite](/ja/agenteye/evaluation-suite)ガイドに記載のとおり、あくまでご自身のインフラ上で動作するHTTPサービスとして、あなた自身のものとして維持されます。このスキルは、エージェントがそれをうまく構築できるよう教えるだけです。スキルが行うことは、同じコードを自分で書けばすべて自分でも実現できます。 - ---- - -## 難しいのは、何をスコアリングするかを決めること - -SDKのサーフェスは小さく、デコレーターとふたつのモデルだけです。エージェントは[コントラクト](/ja/agenteye/evaluation-suite#http-contract)だけからでもそれを書くことができます。評価システムが失敗するのはそこではありません。失敗の原因は、間違ったものをスコアリングすることです。そして間違ったものをスコアリングする評価システムは、ないよりも悪い結果をもたらします。誰もが無視することを覚えてしまうダッシュボードを生み出すからです。 - -だから、スキルの大部分はコードが存在する前の段階にあります。スキルはエージェントにあなたへのインタビューをさせます（「うまくいったセッションを説明してください。次に、うまくいかなかったものを」）。そして[`agenteye` CLI](/ja/agenteye/cli)を通じて実際のセッションを取得し、最初から最後まで読み込みます。この２つの側面は通常一致せず、そのギャップこそが重要です。あなたが測定したいと意図していることと、実際のトランスクリプトがサポートできることの差です。ある軸が残るのは、イベントから**算出可能**で、かつ**識別力がある**場合のみです。良いセッションでも悪いセッションでも0.9のスコアになるなら、何も教えてくれないため除外されます。 - -返ってくるのは、コードが一行も書かれる前に、あなたが承認するための理由付きの2〜4軸の提案です。 - -```mermaid -flowchart TD - YOU["あなた: 'サポートボットの評価を作りたい'"] --> AGENT["コーディングエージェント（Claude Code / Codex）
    agenteye-evaluatorスキルを読み込む"] - AGENT -->|"インタビュー: 良い状態と悪い状態とは？"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["実際のセッション
    実際に起きていること"] - DATA --> DIMS["2〜4軸、あなたが承認"] - DIMS --> SVC["あなたの評価サービス
    agenteye-evaluator SDK"] - SVC --> SCORES["スコアがダッシュボードと
    agenteye evalsに表示される"] -``` - ---- - -## 他の評価コンポーネントとの関係 - -スコアリングに関するドキュメントは4つあり、順番に引き継ぎ合います。 - -| ページ | 内容 | 参照するタイミング | -|---|---|---| -| **[Evaluations](/ja/agenteye/evaluations)** | 機能：セッショングリッドのスコア、ダッシュボード、再評価 | 自動スコアリングで何が得られるか知りたいとき | -| **[Evaluation suite](/ja/agenteye/evaluation-suite)** | HTTPコントラクト、SDK、サーバー環境変数 | 評価サービスを自分で実装またはデバッグするとき | -| **評価スキル**（このドキュメント） | スコアラーの設計と構築のための自然言語インターフェース | 「evalを作りたい」から動作するサービスまで進めたいとき | -| **[CLIスキル](/ja/agenteye/cli-skill)** | `agenteye` CLIへの自然言語インターフェース | すでにあるスコアを*読み取りたい*とき | -| **[Python SDKスキル](/ja/agenteye/python-sdk-skill)** | エージェントのインストルメント化への自然言語インターフェース | エージェントがまだセッションを出力していない — スコアリング対象がない | - -### CLIスキルとの違い：構築 vs. 読み取り - -ふたつのスキルは意図的に重複しないよう設計されており、両方インストールするのが通常の構成です。エージェントはあなたの質問内容に応じてどちらを使うか選択します。 - -- **`agenteye-evaluator`**（このドキュメント）はスコアを*生成する*ものを構築します。初めてスコアが出るところでその役割を終えます。 -- **[`agenteye-cli`](/ja/agenteye/cli-skill)**はすでに存在するスコアを読み取ります（`agenteye evals`）。「今週、品質は下がったか？」がその問いであり、このスキルの問いではありません。 - ---- - -## 前提条件 - -1. **`agenteye` CLIのインストールとログイン**（`pipx install agenteye`、その後`agenteye login`）。スキルはこれを2回使います。設計の元となる実際のセッションを取得するときと、最後にスコアが届いたことを確認するときです。ログインには`events:read`と、最終確認のための`evaluations:read`が必要です。CLIスキルと同様に、メールで届くワンタイムコードを使ったログインを代行することは**できません**。 -2. **評価サービスを置く場所。** サービスはイメージとしてビルドされ、常駐するサービスとして実行されます。そのため、一時的なファイルではなく、正式なリポジトリが必要です。評価サービスはスコアリング対象のエージェントとは別のリポジトリに置かれることが多く、スキルは既存のリポジトリを探し、新しくスキャフォールドする前に確認を求めます。 -3. **`agenteye-evaluator` SDKホイール** — エージェントが`pip`コマンドを打ち始める前に次のセクションを読んでください。 - ---- - -## 入手先 - -スキルはFailproof AIの公開スキルコレクションで公開されています。 - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -リポジトリは公開されており、スキル自体に認証情報は不要です。ログインしたセッションで`agenteye` CLIを操作し、*あなたの*リポジトリにコードを書くだけです。スキルは独自のフォルダーとして配布されており、`pipx install agenteye`パッケージには含まれていません。そちらで探さないようにしてください。 - -## スキルのインストール - -最も手軽な方法は[`skills`](https://skills.sh) CLIを使うことです。フォルダーを取得し、エージェントが参照する場所に配置します。 - -```bash -# Claude Code、このプロジェクトのみ -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# すべてのプロジェクト（~/.claude/skills/ にインストール） -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codexの場合 -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -インストール後は、他のスキルと同様に管理できます。 - -```bash -npx skills list -a claude-code # インストール済みを確認 -npx skills update agenteye-evaluator # 最新版を取得 -npx skills remove agenteye-evaluator # 削除 -``` - -手動でインストールする場合は、エージェントスキルは`SKILL.md`（とオプションの参照ファイル）を含むフォルダーにすぎないため、コピーするだけでも動作します。 - -- **Claude Code**：`agenteye-evaluator/`フォルダーを`~/.claude/skills/`（全プロジェクト共通）または`/.claude/skills/`（そのリポジトリのみ）に置いてください。Claude Codeは自動で認識します。`/skills`リストで確認するか、evalについて質問してみてください。 -- **Codex（OpenAI）**：Codexも同じ`SKILL.md`を読み取ります。同梱の`agents/openai.yaml`に`allow_implicit_invocation: true`が設定されているため、タスクが一致するとCodexが自動でスキルを選択します。明示的に呼び出す場合は`$agenteye-evaluator`と指定してください。 - ---- - -## SDKは公開PyPIにありません - -> **警告：** エージェントにSDKをインストールさせる前にこのセクションを読んでください。 - -スキルは公開されていますが、それが使用するSDKは公開されていません。`agenteye-evaluator`はプライベートのリリース成果物としてのみ配布されており、`agenteye`と異なり、**公開PyPIではパッケージ名が未取得**です。そのため、`pip install agenteye-evaluator`と単純に実行すると、第三者のパッケージが本番のトランスクリプトを読み取るサービスに取り込まれる可能性があります。これはタイポの問題ではなく、サプライチェーンのリスクです。 - -スキルはこれを認識しており、代わりにインストールの優先順位を順番に確認します。AgentEyeのリポジトリ内であればモノレポのソース、そうでなければGitHub Releases（アクセス権が必要）からのプライベートリリースホイール、いずれも利用できない場合は、**その場での代替策をとるのではなく、Failproof AIの担当者にホイールを問い合わせるよう案内して停止します**。 - -したがって、エージェントが公開PyPIから`pip install agenteye-evaluator`を単純に提案してきた場合、それはスキルが読み込まれていないサインです。その場で止めて、スキルが正しくインストールされているか確認してください。 - ---- - -## 何を依頼できるか - -実際のやり取りは、漠然とした依頼から始まり、コードではなく承認済みの設計で終わります。 - -```text -あなた ▸ サポートボットの評価を作りたいんだけど。品質が不安定な気がする。 - -エージェント ▸ 何かを書く前に：うまくいったセッションと、 - うまくいかなかったセッションをそれぞれ説明してください。 - -あなた ▸ 良い例：ドキュメントから回答できている。悪い例：でたらめなことを言う、 - または諦めて「サポートに連絡してください」と言う。 - -エージェント ▸ 実行中：agenteye --json sessions --status error --since 7d - 5つのセッションを読み込み中、うち2つはあなたが問題と判断するもの。 - - あなたの言葉から「正確性」と「回避」が見えます。トランスクリプトは - 回避については同意（5件中2件がサポートへの引き継ぎで終わっており、 - model_response.contentで明確に確認できます）が、正確性については - イベントにグラウンドトゥルースが含まれていないため判定できません。 - - そこで、次の3つをスコアリングすることを提案します： - - deflection ルールベース、コスト低、2つのセッションを識別可能 - - tool_efficiency ルールベース、run-004のリトライループを検出 - - groundedness LLMジャッジ：すべての主張が取得したドキュメントにあるか？ - - 「正確性」にはグラウンドトゥルースが必要ですが存在しません。 - 「groundedness」がトランスクリプトで実際にサポートできる最も近い指標です。 - この3つの名前でよいですか？後で変更すると履歴が分裂します。 - -あなた ▸ はい - -エージェント ▸ このリポジトリに評価サービスが見当たりません。 - ここにスキャフォールドしますか、それとも別の場所にありますか？ -``` - -ここからエージェントは、まずルールベースの軸を書きます（無料、即時、決定論的）。そして、単純な評価システムをクラッシュさせる空のセッションや未完了のセッションを含む、実際にキャプチャされたセッションに対してテストします。主観的な軸にのみLLMジャッジを使います。[ディスパッチャーの制限](/ja/agenteye/evaluation-suite#configuring-the-server)（リクエストタイムアウト30秒、デプロイ全体で同時8リクエスト）を把握しているため、ジャッジが確実に収まらない場合は、5倍のコストでキャンセル・リトライが繰り返されるのを避けるため、`JobPending`で非同期処理を選択します。 - -そしてデプロイし、2つのサーバー環境変数を設定し、`agenteye --json evals --session-id `でスコアが実際に届いたことを確認します。スコアが届くことだけが唯一の証明です。 - ---- - -## 注意すべき点 - -- **軸の名前はほぼ永続的です。** スコアのキーは任意の文字列であり、プラットフォームは送信された値をそのままトレンド表示します。つまり、後から誰かが悪い選択を修正することはありません。後から名前を変更すると履歴が分裂します。古いセッションは古いキーを保持し、トレンドが壊れます。だからこそスキルはコードを書く前に明示的な承認を求めます。そのプロンプトを真剣に受け止めてください。 -- **フィクスチャーは実際の本番トランスクリプトです。** 実際のセッションを元に設計するということは、それらをディスクに取得することを意味し、顧客データが含まれている可能性があります。スキルはgitにコミットする前に確認を求めます。不安な場合は`fixtures/`をリポジトリから除外し、各開発者が自分でセッションを取得するようにしてください。 -- **エージェントはすべてのトランスクリプトを読み取るサービスを作成・デプロイします。** CLIログインの権限の範囲内であなたとして動作しますが、本番データに触れる他のコードと同様に、評価サービスをレビューしてください。 - ---- - -## 次のステップ - -- **[Evaluation suite](/ja/agenteye/evaluation-suite)**：スキルが設定するHTTPコントラクト、SDK、サーバー環境変数。 -- **[Evaluations](/ja/agenteye/evaluations)**：スコアが届いた後に表示される場所。 -- **[CLIスキル](/ja/agenteye/cli-skill)**：スコアラーを構築するのではなく結果を読み取るための、姉妹スキル。 -- **[CLI](/ja/agenteye/cli)**：スキルが設計の元となるセッションデータのコマンドリファレンス。 \ No newline at end of file diff --git a/docs/ja/agenteye/hermes-capture.mdx b/docs/ja/agenteye/hermes-capture.mdx deleted file mode 100644 index 1f3a8117..00000000 --- a/docs/ja/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Hermesセッションキャプチャ" -description: "チームのHermesゲートウェイセッション（Slack、Telegram、CLI、スケジュール実行）をAgentEyeに通常のセッションおよびイベントとして取り込みます。" ---- - -[Hermes](https://hermes-agent.nousresearch.com)は、チームがすでに使っている場所（Slack、Telegram、CLI、スケジュール実行）からの問い合わせに応答します。HermesセッションキャプチャはそのすべてをAgentEyeに通常のセッションおよびイベントとして取り込むため、チームが毎日対話するアシスタントも、自分たちで作成したエージェントと同様に可観測性を持てます。 - -小さなバックグラウンドコレクターが、書き込まれているHermesのローカルセッションストアを読み取り、セッションをAgentEyeに送信します。[Codex](/ja/agenteye/codex-capture)や[OpenClaw](/ja/agenteye/openclaw-capture)のキャプチャと同じ仕組みで動作し、1つのコレクターで複数を同時にキャプチャできます。 - ---- - -## キャプチャされる内容 - -マシン上のすべてのHermesセッションが、どのチャンネルから来たものであっても、キャプチャされます。各セッションはAgentEyeの[セッション](/ja/agenteye/sessions)となり、ユーザーとアシスタントのメッセージ、ツール呼び出し、ツール結果が対応する[イベント](/ja/agenteye/event-stream)になります。 - -セッションが開始されたチャンネル（Slack、Telegram、CLI、またはスケジュール実行）はセッションに記録されるため、区別したり特定のチャンネルでフィルタリングしたりできます。あわせて、セッションが実行されたモデル、開始元のチャットとユーザー、セッションが別のセッションを生成した場合はその親セッションへのリンクも記録されます。 - -セッションは、まだ何も発言されていなくても、Hermesが開始した時点で表示されます。また、あるターンの返答とそのツール呼び出しは、実際に発生した順序に保たれます。セッション終了時には、終了した理由、コスト、使用トークン数も取得できます。 - ---- - -## 有効にする - -キャプチャは有効化するまでオフです。`events:add`権限を持つAPIキーを使ってコレクターをインストールし（[APIキー](/ja/agenteye/api-keys)を参照）、Hermesキャプチャをオンにします。 - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -これにより、コレクターのインストール、バックグラウンドサービスへの登録、キャプチャの開始が行われます。正常に動作しているか確認するには： - -```bash -agenteye-collector health -``` - -同じマシンで複数のエージェントをキャプチャしますか？各エージェントのフラグを同じコマンドに追加してください。例：`--hermes-enabled --codex-enabled` - -初回実行時には既存のHermesセッションが一度バックフィルされ、その後の新しいアクティビティは数秒以内にストリーミングされます。Hermesのデータは読み取り専用で、変更や削除は一切行われません。また、再起動をまたいでも各メッセージは1回だけ送信されます。 - -`health`コマンドは、コレクターがキャプチャしたすべてのデータが実際にAgentEyeに届いているかどうかも報告します。バッチを送信できなかった場合は破棄せず保持して再試行し、未送信のデータがある間はチェックが「unhealthy」と報告します。つまり「healthy」はプロセスが生きているだけでなく、データが届いていることを意味します。 - ---- - -## 表示される場所 - -キャプチャされたセッションは**Sessions**に表示され、そのイベントは**Events**ストリームに表示されます。他のエージェントと同様に扱われるため、[セッションリプレイ](/ja/agenteye/sessions)、[検索](/ja/agenteye/queries)、[評価](/ja/agenteye/evaluations)、[アラート](/ja/agenteye/alerts)がすべて利用できます。Hermesエージェントでフィルタリングすると、そのセッションのみを表示できます。 - ---- - -## プライバシー - -Hermesセッションには、コマンド出力、ファイルの内容、エージェントが読み書きしたすべての内容を含む完全なトランスクリプトが含まれており、シークレット情報が含まれる場合があります。キャプチャされたセッションはそのまま送信されるため、AgentEyeにそのコンテンツを集約することが適切な環境でのみキャプチャを有効にしてください。また、コレクターには`events:add`のみにスコープされたキーを付与してください。データの分離方法については[セキュリティ](/ja/agenteye/security)を参照してください。 \ No newline at end of file diff --git a/docs/ja/agenteye/observability.mdx b/docs/ja/agenteye/observability.mdx deleted file mode 100644 index d09021ea..00000000 --- a/docs/ja/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "オブザーブ" -description: "オブザーブ画面では、エージェントのリアルタイムの動作を監視し、個々の実行を詳しく調べることができます。" ---- - - -オブザーブ画面では、エージェントのリアルタイムの動作を監視し、個々の実行を詳しく調べることができます。すべての情報はライブで表示され、組織単位にスコープされており、日付範囲・環境・エージェント・セッションでフィルタリングできます。「何かおかしい」と感じてから数秒で該当の実行を特定できます。 - -![タイプ別に色分けされ、環境・エージェント・セッションでフィルタリング可能なライブイベントストリーム](/agenteye/images/events-stream.png) - -4つの画面があり、それぞれ専用のページを持っています。 - -- **[イベントストリーム](/ja/agenteye/event-stream)**: すべてのエージェントにわたる全実行のステップごとのライブログで、最新のものから順に表示されます。組織のホーム画面であり、トリアージの出発点です。 -- **[セッションと実行グラフ](/ja/agenteye/sessions)**: それらのイベントを1実行1行にまとめたビューと、各実行の展開をgit風に可視化したグラフです。 -- **[パフォーマンスメトリクス](/ja/agenteye/telemetry)**: モデル・ツール・フックのレイテンシヒートマップとp50/p95/p99のバイタル。テールスパイクがメディアンから際立って見えます。 -- **[エラートラッキング](/ja/agenteye/error-tracking)**: 発生したすべての問題を一元管理するトリアージ画面。発火中のアラートから問題の実行まで1クリックで到達できます。 - -## 関連情報 - -- [評価](/ja/agenteye/evaluations): すべての実行を品質スコアで評価します。 -- [アラート](/ja/agenteye/alerts): 任意のしきい値をページングルールに変換します。 -- [監査](/ja/agenteye/audits): Failproof AI Observability がセッション全体にわたる障害パターンを自動検出します。 -- [CLIとエージェント](/ja/agenteye/cli-and-agents): ターミナルから同じオブザーバビリティを利用できます。 \ No newline at end of file diff --git a/docs/ja/agenteye/openclaw-capture.mdx b/docs/ja/agenteye/openclaw-capture.mdx deleted file mode 100644 index 60c5b66e..00000000 --- a/docs/ja/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "OpenClaw セッションキャプチャ" -description: "チームのローカル OpenClaw セッションを通常のセッションやイベントとして AgentEye に取り込めます — OpenClaw の実行方法は変更不要です。" ---- - -チームが [OpenClaw](https://docs.openclaw.ai) を利用している場合、OpenClaw セッションキャプチャを使うと、それらのセッションを通常のセッションやイベントとして AgentEye に取り込めます。これにより、他のエージェントの記録と並べて検索・再生・評価が可能になります。[Python SDK](/ja/agenteye/python-sdk) との補完関係にあり、SDK が自分で書いたエージェントを計装するのに対し、こちらはチームがすでに行っている OpenClaw の作業を — 実行方法を一切変えずに — キャプチャします。 - -小さなバックグラウンドコレクターが OpenClaw のローカルセッショントランスクリプトを書き込まれた順に読み取り、AgentEye に送信します。[Codex キャプチャ](/ja/agenteye/codex-capture) と同じ仕組みで動作し、1 つのコレクターで両方を同時にキャプチャできます。 - ---- - -## キャプチャされる内容 - -マシンの OpenClaw 設定に含まれるすべてのエージェントが、そのマシンのコレクターによってキャプチャされます — エージェントごとのセットアップは不要です。 - -各 OpenClaw セッションは AgentEye の[セッション](/ja/agenteye/sessions)となり、ユーザー・アシスタントのメッセージ、ツール呼び出し、ツール結果が対応する[イベント](/ja/agenteye/event-stream)になります。 - ---- - -## 有効にする方法 - -キャプチャは有効化するまでオフのままです。`events:add` 権限を持つ API キー（[API キー](/ja/agenteye/api-keys)を参照）を使ってコレクターをインストールし、OpenClaw キャプチャを有効にします。 - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -これでコレクターがインストールされ、バックグラウンドサービスとして登録され、キャプチャが開始されます。動作を確認するには次のコマンドを実行します。 - -```bash -agenteye-collector health -``` - -同一マシンで複数のエージェントをキャプチャする場合は、各フラグを同じコマンドに追加してください。例: `--openclaw-enabled --codex-enabled`。 - -初回実行時、既存の OpenClaw セッションが一度バックフィルされ、その後の新しいアクティビティは数秒以内にストリーミングされます。OpenClaw 自身のファイルは読み取り専用で、変更・移動・削除は一切行われません。また、再起動をまたいでも各セッションはちょうど 1 回だけ送信されます。 - ---- - -## 表示場所 - -キャプチャされたセッションは **Sessions** に表示され、そのイベントは **Events** ストリームに表示されます。他のエージェントと同様に扱われるため、[セッションリプレイ](/ja/agenteye/sessions)、[検索](/ja/agenteye/queries)、[評価](/ja/agenteye/evaluations)、[アラート](/ja/agenteye/alerts)がすべて利用できます。OpenClaw エージェントでフィルタリングすることで、そのセッションだけを表示できます。 - ---- - -## プライバシー - -OpenClaw のトランスクリプトにはセッションの全内容が含まれます。コマンドの出力、ファイルの内容、エージェントが読み書きしたあらゆる情報が含まれ、シークレット情報が含まれる場合もあります。キャプチャされたセッションはそのまま送信されるため、AgentEye にそのコンテンツを集約することが適切なマシンおよびチームに対してのみキャプチャを有効にしてください。また、コレクターには `events:add` のみにスコープを絞ったキーを使用してください。データがどのように分離して保管されるかについては、[セキュリティ](/ja/agenteye/security)を参照してください。 \ No newline at end of file diff --git a/docs/ja/agenteye/python-sdk-skill.mdx b/docs/ja/agenteye/python-sdk-skill.mdx deleted file mode 100644 index 16c57094..00000000 --- a/docs/ja/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,131 +0,0 @@ ---- -title: "Failproof AI Observability Python SDK エージェントスキル" -description: "計装されていないエージェントから可視化できるイベントへ — コーディングエージェントが計装ポイントを見つけ、実装し、正常に動作することを確認します。" ---- - -コーディングエージェントに *「このエージェントに Failproof AI Observability を追加して」* と伝えるだけで、エージェントがループを読み込み、計装箇所を特定し、コードを書き、ジョブ完了と宣言する前にイベントを検証してくれます。 - -**Python SDK スキル**（`agenteye-python-sdk`）は *エージェントスキル* です。Claude Code や Codex などのコーディングエージェントが、タスクに合致した際にオンデマンドで読み込む指示ファイルのフォルダです。このスキルは [Python SDK](/ja/agenteye/python-sdk) の使い方をエージェントに教えるものであり、ライブラリではなく、SDK の動作自体には何も変更を加えません。 - -## 計装は書きやすい分、静かに間違えやすい - -SDK はシンプルです。イベントメソッドは 13 個、すべてキーワード専用です。コーディングエージェントは [Python SDK](/ja/agenteye/python-sdk) リファレンスを読めば、もっともらしい計装を 1 分で生成できます。 - -問題は、この SDK は間違えてもエラーを投げず、誤った計装は正しい計装とまったく同じように見えることです。ダッシュボードを開いて空っぽだと気づくまで分かりません。実際に時間を浪費させるミスはすべて「沈黙」の形をしています。 - -| ミスの内容 | 見え方 | -|---|---| -| `agent_start` がない | すべてのイベントは記録される。セッションはゼロ。 | -| 環境が設定されていない | すべて正常に動作し、`dev` 環境として記録される。 | -| `outcome="failure"` | 実行結果は成功表示になる — カウントされるのは `failed`、`error`、`timeout`、`rejected` のみ。 | -| フィールド名のタイポ | 受理されて新しいフィールドとして保存される。 | -| スレッドプールからイベントを送出 | 無言でドロップされる。 | - -これらはいずれもエラーを投げません。テストでも検出されません。スキルにはそれぞれのケースが、検出のためのチェックとともにコントラクトとして明記されています。 - -## スキルの動作手順 - -このスキルは、注意深いエンジニアが行うのと同じ 3 ステップを実行します。 - -1. **計画する。** エージェントのループを読み込み、あなたにしか答えられない 2 つの問いを立てます。「1 回の実行とは何か（`session_id`）」と「識別可能なアクターは誰か（`agent_id`）」です。コードを書く前にこれを合意します。後から変更すると履歴が分断され、トレンドが壊れるからです。 -2. **実装する。** すべての呼び出し箇所に渡すのではなく、1 回の実行につき 1 度だけアイデンティティをバインドし、並行処理に安全な設計を選択します。単純な近道では、並行する 2 つの実行が 1 つのセッションに無言で混入してしまうため、この選択が重要です。 -3. **検証する。** エージェントを実行し、生成されたイベントファイルを読み込んで、`agent_start` が存在するか、環境が正しいか、1 回の実行が 1 つのセッションを生成しているかを確認します。 - -この 3 ステップ目こそ、みんなが省略するステップです。SDK はイベントをローカルファイルに書き込むので、サーバーも API キーもネットワークも不要で、ラップトップ上で完全な統合を証明できます。だからこそスキルはこのステップを必ず実行します。 - -## 他のスキルとの関係 - -3 つのスキルが明確に役割分担しています。 - -| スキル | 使うタイミング | 対象 | -|---|---|---| -| **Python SDK スキル**（このページ） | エージェントにテレメトリを *送出* させたいとき — 「オブザーバビリティを追加して」「エージェントが表示されない」 | エージェントのリポジトリにコードを書く。何も読み込まない。 | -| **[Evaluator スキル](/ja/agenteye/evaluator-skill)** | 実行結果を *スコアリング* したいとき — 「何を計測すべきか？」 | リポジトリにコードを書く。テレメトリを読み込む。 | -| **[CLI スキル](/ja/agenteye/cli-skill)** | 何が起きたかを *読み取りたい*、またはデプロイを操作したいとき | あなたの代わりに CLI を操作する（変更を含む） | - -この順番で連携します。このスキルでイベントを流し、Evaluator でスコアリングし、CLI で読み返します。エージェントがセッションを送出するまで、評価するものも読み取るものも存在しないため、ゼロから始めるならここからスタートしてください。 - -## 前提条件 - -1. **Python 3.10 以上** と計装したいエージェントのコードベース。 -2. **SDK。** パブリックのパッケージインデックスではなく、プライベートの wheel としてお客様に配布されます。入手方法とインストール方法はオンボーディング時にご案内します。スキルはインストールパスを把握しており、見つからない場合は推測せずに確認します。 -3. **それだけ。** ダッシュボードへのログイン、API キー、ネットワーク接続は不要です。SDK が書き込んだイベントファイルを使って検証するため、オフラインで作業を完了し、証明できます。 - -## 入手方法 - -スキルはパブリックの [`FailproofAI/skills`](https://github.com/FailproofAI/skills) コレクションにあります。 - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -`-g` を追加するとカレントプロジェクトだけでなくすべてのプロジェクトにインストールされます。シンボリックリンクを使用しない環境では `--copy` を指定してください。Codex の場合は `-a codex` を渡してください。 - -## 手動インストール - -エージェントスキルは `SKILL.md` と参照ファイルを含むフォルダです。インストーラーを使わない場合は以下の手順で設定できます。 - -- **Claude Code**: `agenteye-python-sdk/` フォルダを `~/.claude/skills/`（すべてのプロジェクト）または `/.claude/skills/`（そのリポジトリのみ）にコピーします。Claude Code は自動的に検出します — `/skills` リストで確認するか、それに合致する質問をしてみてください。 -- **Codex**: Codex も同じ `SKILL.md` を読み込みます。同梱の `agents/openai.yaml` に `allow_implicit_invocation: true` が設定されているため、タスクに合致すると自動選択されます。明示的に呼び出す場合は `$agenteye-python-sdk` を使用してください。 - -**計装したいコードを含むリポジトリ内** でエージェントを実行してください — スキルは何かを提案する前にエージェントのループを読み込みます。 - -## セッションの例 - -```text -you ▸ Add Failproof AI Observability to this agent. - -agent ▸ Read your loop — one tool dispatcher in tools.py, one Anthropic - wrapper in llm.py. Two questions before I write anything. - - What counts as one run? You handle requests concurrently, so I'd - use the request id you already have in ctx.request_id — then a - session here and a request in your logs are the same string. - - And who are the actors? I see a planner path and a researcher - path. Label them separately, or is that one agent to you? - -you ▸ Request id is right. They're one agent, call it "assistant". - -agent ▸ Done. Wrapped the dispatcher and the LLM client; agent_start and - agent_end bracket handle_request, including the error path. Used - a context-local for identity — your dispatcher runs tools in a - thread pool, and the simpler approach would have mixed concurrent - runs into one session without complaining. - - Verified against a real run: 11 events across 4 files — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, one session - per request, durations paired. - - One thing I did NOT do: your worker pool in batch.py submits - straight to the executor, so events from there would be dropped. - Want me to fix those too? -``` - -注目すべきパターン：提案する前にコードを読み込み、あなたにしか答えられない質問だけを行い、既存の ID を再利用し、スレッドプールを確認した上で並行処理に安全な設計を選択し、成功を宣言する代わりに **実際のイベントを読み込んで検証し**、静かに失敗することが分かっていた箇所をフラグとして報告しています。 - -## 使えるプロンプトの例 - -- *「エージェントがダッシュボードに表示されないのはなぜ？」* → 段階的に確認します。イベントが書き込まれているか、`agent_start` があるか、環境が正しいか、コレクターが同じ場所を読んでいるか。 -- *「すべてが dev 環境として記録される。」* → 環境が一度も設定されていないか、後の呼び出しでリセットされています。 -- *「トークントラッキングを追加して。」* → LLM ラッパーを見つけて、モデル、停止理由、使用量を記録します。 -- *「サブエージェントも計装して。」* → 1 つのセッション、異なるエージェントラベル、親の下にネスト。 -- *「計装のテストを書いて。」* → SDK を一時ディレクトリに向けて、書き込まれたイベントをアサートします。 - -## 注意点 - -**検証ステップを省略しないこと。** このスキルが価値を持つのは最後のステップ、つまりエージェントを実行して実際のイベントを読み返すことにあります。計装を書いて終わりにしたエージェントは、作業の簡単な半分しか終えていません。静かに失敗する半分が残っています。 - -**コードの前に名前を決めること。** `session_id` と `agent_id` は、すべての画面でグルーピングの軸になります。後からリネームすると履歴が分断されます。古い実行は古いラベルのままになり、トレンドが壊れます。スキルが確認しますので、少し時間をかけて答える価値があります。 - -**エージェントがパブリックのインデックスから SDK をインストールしようとしている場合、スキルが読み込まれていません。** SDK はプライベートで配布されています。そのような提案は、コーディングエージェントがスキルに従わずに推測していることを示す確実なサインです。その場で止めて、スキルがインストールされているかを確認してください。 - -それ以外の影響範囲は小さく、ワーキングディレクトリにコードを書き込み、指定した場所にイベントファイルを書き込むだけです。デプロイから何かを読み取ることも、デプロイに変更を加えることもありません。 - -## 次のステップ - -- **[Python SDK](/ja/agenteye/python-sdk)**: このスキルが自動化する処理の背後にある完全なイベントリファレンス — すべてのイベントタイプとフィールド。 -- **[Sessions](/ja/agenteye/sessions)**: イベントが記録された後、計装によって生成されるもの。 -- **[Evaluator エージェントスキル](/ja/agenteye/evaluator-skill)**: 実行が記録されたら次のステップ — スコアリング。 -- **[CLI エージェントスキル](/ja/agenteye/cli-skill)**: テレメトリの読み返し。 \ No newline at end of file diff --git a/docs/ja/agenteye/security.mdx b/docs/ja/agenteye/security.mdx deleted file mode 100644 index 3d9eb465..00000000 --- a/docs/ja/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "セキュリティ" -description: "Failproof AI Observabilityはプロダクション環境のエージェントの近くに配置されるため、プロンプト、ツールの入力、出力を参照します。" ---- - - -Failproof AI Observabilityはプロダクション環境のエージェントの近くに配置されるため、プロンプト、ツールの入力、出力を参照します。このページでは、データの隔離・制御・管理方法について説明します。セキュリティレビューのためにFailproof AI Observabilityを評価している場合は、まずここをお読みください。 - ---- - -## データはあなたの環境に留まる - -Failproof AI Observabilityはセルフホスト型です。イベント、プロンプト、モデルのレスポンス、アナリティクスはすべて、あなた自身のデータベースおよび環境に保存されます。サードパーティのSaaSにデータが送信されることはなく、データは常にあなた自身のクラウドアカウント内に留まります。 - ---- - -## テナント分離 - -1つのFailproof AI Observabilityインスタンスで複数の組織をホストできます。各組織はストレージ層で分離されており、UIではなくデータベース自体によって強制されます。 - -- 組織の運用データ（ユーザー、APIキー、ダッシュボード、保存済みクエリ）はその組織にスコープされており、組織をまたいだ読み取りはデータベース自体によってブロックされます。 -- 取り込まれたすべてのイベントには所有組織のスタンプが押されるため、ある組織のイベントを別の組織が読み取ることはできません。 - -すべてのダッシュボードルートは組織スラグ（`//…`）の配下にスコープされています。 - ---- - -## サインイン - -Failproof AI Observabilityはパスワードレスのメールベースサインインを採用しています。フィッシングやリークの対象となるパスワードは存在しません。ユーザーがワンタイムコード（またはワンクリックマジックリンク）をリクエストすると、それがメールで送信され、短時間で失効します。サインインは**許可リスト**によって制御されており、あなたが許可したメールアドレス（またはドメイン）のみが認証できます。 - -![Failproof AI Observabilityのサインイン画面。メールアドレスに使い捨てコードを送信します](/agenteye/images/login.png) - ---- - -## APIキーによるスコープ付きアクセス - -すべてのクライアントは、きめ細かな最小権限を持つAPIキーで認証します。コレクターには`events:add`のみが必要です。ダッシュボードやアシスタント用のキーは読み取り専用にできます。破壊的な操作（削除、再生成）は、明示的に付与を選択する別個の権限です。 - -![APIキーページ：各キーの権限付与が読み取り・書き込み・破壊的スコープごとに色分けされています](/agenteye/images/api-keys.png) - -管理者のブートストラップキーはセットアップ用に保持し、その他の用途には権限を絞ったキーを発行してください。詳しくは[APIキー](/ja/agenteye/api-keys)をご覧ください。 - ---- - -## 読み取り専用・承認ゲート付きアシスタント - -ダッシュボード内の[AIアシスタント](/ja/agenteye/assistant)はデータに関する質問に回答しますが、設計上の制約があります。 - -- **デフォルトで読み取り専用**：実行されるSQLはガードを通過し、`SELECT`/`WITH`クエリのみ、単一ステートメント、行数上限付きで許可されます。 -- アシスタントが作成するもの（保存済みクエリ、ダッシュボードなど）はすべて**承認ゲート付き**：書き込みが行われる前に、あなたがすべての内容を確認・承認します。 -- アシスタントは**削除を行うことができません**。 - -そのため、チームメンバーが「今週最もエラーが多かったエージェントはどれか？」と質問して結果を活用できる一方、アシスタントが自律的にデータを変更・削除することはありません。 - ---- - -## 転送中のセキュリティ - -すべてのトラフィックはHTTPSで通信されます。TLSはあなた自身の証明書で終端するため、コレクターからサーバーへの通信、およびブラウザからサーバーへの通信は転送中に暗号化されます。 - ---- - -## 次のステップ - -- [概要](/ja/agenteye/overview)：Failproof AI Observabilityの全体像 -- [APIキー](/ja/agenteye/api-keys)：コレクター、ダッシュボード、アシスタントへのアクセスのスコープ設定 -- [オブザーバビリティ](/ja/agenteye/observability)：Failproof AI Observabilityがエージェントから収集する情報 \ No newline at end of file diff --git a/docs/ja/cli/audit.mdx b/docs/ja/audit.mdx similarity index 100% rename from docs/ja/cli/audit.mdx rename to docs/ja/audit.mdx diff --git a/docs/ja/cli/backfill.mdx b/docs/ja/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/ja/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/ja/cli/config.mdx b/docs/ja/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/ja/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/ja/cli/flush.mdx b/docs/ja/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/ja/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/ja/cli/harness.mdx b/docs/ja/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/ja/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/ja/cli/migrate.mdx b/docs/ja/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/ja/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/ja/cli/uninstall.mdx b/docs/ja/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/ja/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/ja/cli/update.mdx b/docs/ja/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/ja/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/ja/agenteye/api-keys.mdx b/docs/ja/cloud/access.mdx similarity index 95% rename from docs/ja/agenteye/api-keys.mdx rename to docs/ja/cloud/access.mdx index 5bcce8de..dbe61928 100644 --- a/docs/ja/agenteye/api-keys.mdx +++ b/docs/ja/cloud/access.mdx @@ -1,10 +1,10 @@ --- title: "APIキー" -description: "APIキーはFailproof AI Observabilityサーバーへのアクセスを制御し、コレクターが読み取り権限や管理者権限を持つことなくイベントを送信できるようにします。" +description: "APIキーはFailproofAI Cloudサーバーへのアクセスを制御し、コレクターが読み取り権限や管理者権限を持つことなくイベントを送信できるようにします。" --- -APIキーはFailproof AI Observabilityサーバーへのアクセスを制御し、コレクターが読み取り権限や管理者権限を持つことなくイベントを送信できるようにします。各キーには1つ以上のパーミッションが付与されており、各パーミッションは特定のサーバールートへのアクセスを制限します。必要な最小限のパーミッションのみを付与してください。ほとんどのデプロイメントでは、3種類のキーを作成するだけで十分です。 +APIキーはFailproofAI Cloudサーバーへのアクセスを制御し、コレクターが読み取り権限や管理者権限を持つことなくイベントを送信できるようにします。各キーには1つ以上のパーミッションが付与されており、各パーミッションは特定のサーバールートへのアクセスを制限します。必要な最小限のパーミッションのみを付与してください。ほとんどのデプロイメントでは、3種類のキーを作成するだけで十分です。 ## ほとんどのデプロイメントで必要な3つのキー @@ -82,7 +82,7 @@ APIキーはFailproof AI Observabilityサーバーへのアクセスを制御し これらのパーミッションはダッシュボードの**Users**ページを支援しており、各メンバーに付与されたスコープがチップとして表示されます: -![Usersページ: 各ダッシュボードユーザーのカード（メール、付与されたパーミッション、編集/無効化コントロール）](/agenteye/images/users.png) +![Usersページ: 各ダッシュボードユーザーのカード（メール、付与されたパーミッション、編集/無効化コントロール）](/cloud/images/users.png) ### 運用設定 @@ -91,7 +91,7 @@ APIキーはFailproof AI Observabilityサーバーへのアクセスを制御し | `settings:read` | `GET /settings`、`GET /settings/schema`、`GET /settings/model-context-windows`、`GET /settings/model-context-windows/resolve` | ダッシュボード管理の運用設定とそのメタデータの表示、モデルごとのコンテキストウィンドウオーバーライドの一覧表示、モデルの有効なウィンドウの解決。 | | `settings:write` | `PUT /settings/:key`、`PUT /settings/model-context-windows`、`DELETE /settings/model-context-windows` | 運用設定の編集と、モデルごとのコンテキストウィンドウオーバーライドの追加/変更/削除。変更はサーバーを再起動せずに新しいイベントに反映されます。 | -![Settingsページ: 許可されたサインインやセッション/OTP有効期間などのダッシュボード管理の運用設定（再起動なしで編集可能）](/agenteye/images/settings.png) +![Settingsページ: 許可されたサインインやセッション/OTP有効期間などのダッシュボード管理の運用設定（再起動なしで編集可能）](/cloud/images/settings.png) ### アラートとインシデント @@ -247,7 +247,7 @@ curl -s -X POST http://your-server/keys//regenerate \ ダッシュボードからキーを作成する場合、シークレットを入力する必要はありません。ダッシュボードが強力なシークレットを生成し、作成時に**一度だけ**表示します。すぐにコピーして安全に保管してください。再生成の場合と同様、二度と表示されません。パーミッションを直接選択することも、パーミッションセットからシードすることもできます（以下を参照）。 -![APIキーページ: 各キーのカード（名前、付与されたパーミッション、作成日時）と再生成・無効化アクション。`admin`などの保護されたキーはマーク付き](/agenteye/images/api-keys.png) +![APIキーページ: 各キーのカード（名前、付与されたパーミッション、作成日時）と再生成・無効化アクション。`admin`などの保護されたキーはマーク付き](/cloud/images/api-keys.png) --- @@ -276,5 +276,5 @@ curl -s -X POST http://your-server/keys//regenerate \ ## 次のステップ -- [Python SDK](/ja/agenteye/python-sdk): エージェントコードがイベント送信時にどのように認証するか。 -- [Security](/ja/agenteye/security): サインイン、アクセス制御、オーガナイゼーションごとのデータ分離の仕組み。 \ No newline at end of file +- [Python SDK](/ja/cloud/sdk): エージェントコードがイベント送信時にどのように認証するか。 +- [Security](/ja/cloud/security): サインイン、アクセス制御、オーガナイゼーションごとのデータ分離の仕組み。 \ No newline at end of file diff --git a/docs/ja/cloud/agent-skills.mdx b/docs/ja/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/ja/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
    loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/ja/agenteye/alerts.mdx b/docs/ja/cloud/alerts.mdx similarity index 75% rename from docs/ja/agenteye/alerts.mdx rename to docs/ja/cloud/alerts.mdx index 846f2dca..3d0bda78 100644 --- a/docs/ja/agenteye/alerts.mdx +++ b/docs/ja/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "チームがすでに使っているチャンネルで、問題が --- -チームがすでに使っているチャンネルで、問題が閾値を超えた瞬間に通知を受け取りましょう。顧客からの報告で気づく前に。ルールを一度設定するだけで、Failproof AI Observabilityがスケジュールに従ってチェックし、メール・Slack・webhook、またはダッシュボード上で通知します。 +チームがすでに使っているチャンネルで、問題が閾値を超えた瞬間に通知を受け取りましょう。顧客からの報告で気づく前に。ルールを一度設定するだけで、FailproofAI Cloudがスケジュールに従ってチェックし、メール・Slack・webhook、またはダッシュボード上で通知します。 -![アラートページ：アラートルールのカードグリッド。それぞれトリガー、評価ウィンドウ、チャンネル、およびinfo・warning・criticalの重大度バッジを表示している](/agenteye/images/alerts.png) +![アラートページ：アラートルールのカードグリッド。それぞれトリガー、評価ウィンドウ、チャンネル、およびinfo・warning・criticalの重大度バッジを表示している](/cloud/images/alerts.png) *すべてのアラートルールを一目で確認：監視対象、確認頻度、通知先、緊急度。* ## ユーザーより先に問題を把握する @@ -22,9 +22,9 @@ description: "チームがすでに使っているチャンネルで、問題が ## ルールはJSONでなくフォームで作る -「壊れている」状態をフォームで記述すると、Failproof AI Observabilityが基盤となるルールを自動生成します。JSONの仕様はあくまでフォームが裏で生成するものなので、ルールを理解するために読むことはあっても、直接入力することはほとんどありません。 +「壊れている」状態をフォームで記述すると、FailproofAI Cloudが基盤となるルールを自動生成します。JSONの仕様はあくまでフォームが裏で生成するものなので、ルールを理解するために読むことはあっても、直接入力することはほとんどありません。 -![新規アラートフォーム：名前と説明、有効化トグル、およびmetric threshold・custom SQL・evaluation score・compound eval・per-eventの条件を選べるトリガーピッカー](/agenteye/images/alert-new.png) +![新規アラートフォーム：名前と説明、有効化トグル、およびmetric threshold・custom SQL・evaluation score・compound eval・per-eventの条件を選べるトリガーピッカー](/cloud/images/alert-new.png) *トリガーを選ぶとフォームが適切なフィールドに切り替わります。保存するとルールが書き込まれます。* 基本的な流れはシンプルです：名前を入力し、**トリガー**（監視対象）を選び、**閾値とウィンドウ**（どの程度悪化したら、どの期間で）を設定し、**チャンネル**を少なくとも1つ追加して、**保存**します。その後 **テスト** を実行して仮の通知を送信し、すべての送信先が正しく設定されていることを確認しましょう。裏ではこのような小さなスペックが生成されます： @@ -43,7 +43,7 @@ description: "チームがすでに使っているチャンネルで、問題が | **Compound eval** | 複数のスコアチェックをany・all・at-least-Nロジックで組み合わせ、スコア全体にまたがる回帰を検出したいとき | | **Per event** | 特定のエージェント・特定のエラー種別・メッセージの部分文字列など、条件に一致する単一イベントが発生したとき | -[エラーページ](/ja/agenteye/error-tracking)で障害を確認している最中ですか？各行には **+ alert** ボタンがあり、クリックするとその障害を再発時にキャッチするための内容があらかじめ入力されたフォームが開きます。今トリアージしたインシデントが、次回自動で通知されるようになります。 +[エラーページ](/ja/cloud/errors)で障害を確認している最中ですか？各行には **+ alert** ボタンがあり、クリックするとその障害を再発時にキャッチするための内容があらかじめ入力されたフォームが開きます。今トリアージしたインシデントが、次回自動で通知されるようになります。 **場所：** アラートは `//alerts` にあります。ルールの作成・編集・削除・テストには **`alerts:write`** 権限が必要です。閲覧だけなら `alerts:read` で十分です。通知先ピッカーには組織のメンバーが名前で表示されるため、フォームを離れずに特定の担当者に通知できます。 @@ -53,11 +53,11 @@ description: "チームがすでに使っているチャンネルで、問題が ## アラートが発火したときの動作 -閾値超過が発生すると**インシデント**が開かれ、チャンネルへの通知が1回送信されます。その後チームは確認・担当者のアサイン・議論・解決を行い、すべてがクリーンな記録として残ります。このトリアージワークフローには専用の場所があります：[インシデント](/ja/agenteye/incidents)をご覧ください。 +閾値超過が発生すると**インシデント**が開かれ、チャンネルへの通知が1回送信されます。その後チームは確認・担当者のアサイン・議論・解決を行い、すべてがクリーンな記録として残ります。このトリアージワークフローには専用の場所があります：[インシデント](/ja/cloud/incidents)をご覧ください。 ## 関連情報 -- [インシデント](/ja/agenteye/incidents)：発火したアラートをオープンから確認済み・解決済みまで追跡する。 -- [エラートラッキング](/ja/agenteye/error-tracking)：エージェントの障害をグループ化し、ワンクリックでアラートに昇格させる。 -- [ダッシュボード](/ja/agenteye/dashboards)：アラートの閾値の基となる共有ボードを監視する。 -- [CLIとエージェント](/ja/agenteye/cli-and-agents)：ターミナルからアラートの作成やインシデントの確認を行う、またはCIにスクリプトとして組み込む。 \ No newline at end of file +- [インシデント](/ja/cloud/incidents)：発火したアラートをオープンから確認済み・解決済みまで追跡する。 +- [エラートラッキング](/ja/cloud/errors)：エージェントの障害をグループ化し、ワンクリックでアラートに昇格させる。 +- [ダッシュボード](/ja/cloud/dashboards)：アラートの閾値の基となる共有ボードを監視する。 +- [CLIとエージェント](/ja/cloud/cli)：ターミナルからアラートの作成やインシデントの確認を行う、またはCIにスクリプトとして組み込む。 \ No newline at end of file diff --git a/docs/ja/agenteye/assistant.mdx b/docs/ja/cloud/assistant.mdx similarity index 86% rename from docs/ja/agenteye/assistant.mdx rename to docs/ja/cloud/assistant.mdx index f8baf981..59a0a919 100644 --- a/docs/ja/agenteye/assistant.mdx +++ b/docs/ja/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "エージェントのデータに自然な言葉で質問すると --- -エージェントのデータに自然な言葉で質問すると、証拠へ直接リンクした回答が得られます。SQLを書く必要も、ダッシュボードを掘り下げる必要もありません。**Failproof AI Observability**アシスタントは、チームの誰もがエージェントに関する答えをすばやく得るための最短の方法です。 +エージェントのデータに自然な言葉で質問すると、証拠へ直接リンクした回答が得られます。SQLを書く必要も、ダッシュボードを掘り下げる必要もありません。**FailproofAI Cloud**アシスタントは、チームの誰もがエージェントに関する答えをすばやく得るための最短の方法です。 -![Failproof AI Observabilityアシスタントがダッシュボード内で自然言語の質問に回答している画面。ライブのエージェントアクティビティテーブル、エージェントごとのモデル使用状況の内訳、テキストによる要点が表示され、実行したクエリもインラインで示されている](/agenteye/images/assistant.png) +![FailproofAI Cloudアシスタントがダッシュボード内で自然言語の質問に回答している画面。ライブのエージェントアクティビティテーブル、エージェントごとのモデル使用状況の内訳、テキストによる要点が表示され、実行したクエリもインラインで示されている](/cloud/images/assistant.png) *自然な言葉で質問すると、自分のデータから構築された回答が得られます。ここでは、最もアクティブなエージェントとそれらが使用するモデルを分析し、すべての数値を確認できるよう実行したクエリも表示されます。* 学習コストはゼロです。チャットを開いて知りたいことを入力するだけで、返ってきたリンクをたどれます。 @@ -36,7 +36,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a **クエリ**ページではさらに一歩進んで、SQLの作成者として機能します。欲しいクエリを説明すると（「過去7日間のエージェント別エラー率を表示して」）、エディタに直接SQLをストリーミングし、変更を反映する前に**承認**または**却下**できるdiffビューを開きます。 -![ObservabilityのクエリページとそのSQLエディタ](/agenteye/images/query-lab.png) +![ObservabilityのクエリページとそのSQLエディタ](/cloud/images/query-lab.png) *クエリページ：アシスタントが下書きの読み取り専用クエリをストリーミングし、承認または却下できるエディタです。* ここで質問してSQLを作成する際には`queries:run`権限が使用されます。これはエディタの**実行**ボタンと同じ権限です。他のすべての場所でのチャットには`agent:use`が必要です。 @@ -57,7 +57,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a ## 関連情報 -- [CLIとエージェント](/ja/agenteye/cli-and-agents) -- [クエリ](/ja/agenteye/queries) -- [ダッシュボード](/ja/agenteye/dashboards) -- [評価スイート](/ja/agenteye/evaluation-suite) \ No newline at end of file +- [CLIとエージェント](/ja/cloud/cli) +- [クエリ](/ja/cloud/queries) +- [ダッシュボード](/ja/cloud/dashboards) +- [評価スイート](/ja/cloud/evaluators) \ No newline at end of file diff --git a/docs/ja/agenteye/audits.mdx b/docs/ja/cloud/audits.mdx similarity index 82% rename from docs/ja/agenteye/audits.mdx rename to docs/ja/cloud/audits.mdx index 8607ec5a..9c753a58 100644 --- a/docs/ja/agenteye/audits.mdx +++ b/docs/ja/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "監査：自動信頼性アナリスト" -description: "Failproof AI Observabilityは、ルールを書いていなかった障害を自動的に発見し、優先順位付きで根拠のある「修正すべき項目リスト」を提供します。" +description: "FailproofAI Cloudは、ルールを書いていなかった障害を自動的に発見し、優先順位付きで根拠のある「修正すべき項目リスト」を提供します。" --- -Failproof AI Observabilityは、ルールを書いていなかった障害を自動的に発見し、優先順位付きで根拠のある「修正すべき項目リスト」を提供します。まるで専任のアナリストが毎晩ログを精査し、翌朝には簡潔なリストをデスクに残してくれるようなものです。 +FailproofAI Cloudは、ルールを書いていなかった障害を自動的に発見し、優先順位付きで根拠のある「修正すべき項目リスト」を提供します。まるで専任のアナリストが毎晩ログを精査し、翌朝には簡潔なリストをデスクに残してくれるようなものです。
    @@ -12,7 +12,7 @@ Failproof AI Observabilityは、ルールを書いていなかった障害を自 *2分間のツアー：スケジュール実行から実際に対処できる修正案まで。* -![監査ページ：セッション内の障害パターンをスキャンする定期ジョブ。スケジュールと感度設定付き](/agenteye/images/audits.png) +![監査ページ：セッション内の障害パターンをスキャンする定期ジョブ。スケジュールと感度設定付き](/cloud/images/audits.png) *各監査は定期的に実行されるジョブで、セッションを分析して優先順位付きの根拠ある推奨事項をまとめます。* ## 次に何を修正すべきか、推測をやめよう @@ -48,7 +48,7 @@ Failproof AI Observabilityは、ルールを書いていなかった障害を自 ## 関連情報 -- [アラート](/ja/agenteye/alerts)：既知のしきい値を超えた瞬間に通知を受け取る。 -- [評価](/ja/agenteye/evaluations)：すべての実行をスコアリングして品質の低下を自動的に検出する。 -- [エラートラッキング](/ja/agenteye/error-tracking)：エージェントがスローするエラーをグループ化して追跡する。 -- [インシデント](/ja/agenteye/incidents)：監査で発見した問題を修正完了まで追跡する。 \ No newline at end of file +- [アラート](/ja/cloud/alerts)：既知のしきい値を超えた瞬間に通知を受け取る。 +- [評価](/ja/cloud/evaluations)：すべての実行をスコアリングして品質の低下を自動的に検出する。 +- [エラートラッキング](/ja/cloud/errors)：エージェントがスローするエラーをグループ化して追跡する。 +- [インシデント](/ja/cloud/incidents)：監査で発見した問題を修正完了まで追跡する。 \ No newline at end of file diff --git a/docs/ja/cloud/capture.mdx b/docs/ja/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/ja/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/ja/agenteye/cli-recipes.mdx b/docs/ja/cloud/cli-recipes.mdx similarity index 90% rename from docs/ja/agenteye/cli-recipes.mdx rename to docs/ja/cloud/cli-recipes.mdx index b84952aa..852a82fc 100644 --- a/docs/ja/agenteye/cli-recipes.mdx +++ b/docs/ja/cloud/cli-recipes.mdx @@ -4,9 +4,9 @@ description: "セッション・イベント・評価データをスクリプト --- -セッション・イベント・評価データをスクリプトやコーディングエージェントから直接取得（および再評価のトリガー）できます。stdout にクリーンな JSON を出力するため、そのまま `jq` にパイプ可能です。これらのレシピは、Failproof AI Observability のデータを、ダッシュボードをクリックせずにターミナルユーザーや AI コーディングエージェント（Claude Code、Cursor）がクエリ・自動化できる形に変換します。 +セッション・イベント・評価データをスクリプトやコーディングエージェントから直接取得（および再評価のトリガー）できます。stdout にクリーンな JSON を出力するため、そのまま `jq` にパイプ可能です。これらのレシピは、FailproofAI Cloud のデータを、ダッシュボードをクリックせずにターミナルユーザーや AI コーディングエージェント（Claude Code、Cursor）がクエリ・自動化できる形に変換します。 -以下のパターンは、Failproof AI Observability CLI（`agenteye`）ですぐにコピペして使えます。インストール・認証・全オプションの一覧は [CLI](/ja/agenteye/cli) を参照してください。組み込みヘルプは `agenteye -h` または `agenteye -h` で確認できます。 +以下のパターンは、FailproofAI Cloud CLI（`agenteye`）ですぐにコピペして使えます。インストール・認証・全オプションの一覧は [CLI](/ja/cloud/cli) を参照してください。組み込みヘルプは `agenteye -h` または `agenteye -h` で確認できます。 ## 基本ルール @@ -173,7 +173,7 @@ esac ## 次のステップ -- [CLI](/ja/agenteye/cli): インストール・認証・全コマンドのオプションリファレンス。 -- [CLI エージェントスキル](/ja/agenteye/cli-skill): これらのレシピをコーディングエージェントが読み込めるスキルとしてパッケージ化する方法。 -- [API キー](/ja/agenteye/api-keys): CLI・SDK・コレクターが認証に使うキーの作成とスコープ設定。 -- [Python SDK](/ja/agenteye/python-sdk): Failproof AI Observability にイベントを送信して、これらのレシピがクエリできるデータを用意する方法。 \ No newline at end of file +- [CLI](/ja/cloud/cli): インストール・認証・全コマンドのオプションリファレンス。 +- [CLI エージェントスキル](/ja/cloud/agent-skills): これらのレシピをコーディングエージェントが読み込めるスキルとしてパッケージ化する方法。 +- [API キー](/ja/cloud/access): CLI・SDK・コレクターが認証に使うキーの作成とスコープ設定。 +- [Python SDK](/ja/cloud/sdk): FailproofAI Cloud にイベントを送信して、これらのレシピがクエリできるデータを用意する方法。 \ No newline at end of file diff --git a/docs/ja/agenteye/cli.mdx b/docs/ja/cloud/cli.mdx similarity index 91% rename from docs/ja/agenteye/cli.mdx rename to docs/ja/cloud/cli.mdx index 1d49be65..fb44ba73 100644 --- a/docs/ja/agenteye/cli.mdx +++ b/docs/ja/cloud/cli.mdx @@ -1,10 +1,10 @@ --- title: "CLI" -description: "ターミナルまたはスクリプトから Failproof AI Observability の全機能を操作できます。ダッシュボードへのアクセスは不要です。" +description: "ターミナルまたはスクリプトから FailproofAI Cloud の全機能を操作できます。ダッシュボードへのアクセスは不要です。" --- -ターミナルまたはスクリプトから Failproof AI Observability の全機能を操作できます。ダッシュボードへのアクセスは不要です。`agenteye` CLI はデータ（セッション、イベントログ、評価）の照会と、組織管理（API キー、ユーザー、設定、アラート、インシデント、保存済みクエリ）を行います。チェックの自動化、Observability を CI に組み込む場合、またはコーディングエージェントが本番環境を検査する場合に役立ちます。すべてのコマンドは `--json` フラグに対応しているため、プロンプトでの手動操作でも、コーディングエージェント（Claude Code、Cursor）がシェルから呼び出して結果をパースする場合でも、同様に利用できます。 +ターミナルまたはスクリプトから FailproofAI Cloud の全機能を操作できます。ダッシュボードへのアクセスは不要です。`agenteye` CLI はデータ（セッション、イベントログ、評価）の照会と、組織管理（API キー、ユーザー、設定、アラート、インシデント、保存済みクエリ）を行います。チェックの自動化、FailproofAI Cloud を CI に組み込む場合、またはコーディングエージェントが本番環境を検査する場合に役立ちます。すべてのコマンドは `--json` フラグに対応しているため、プロンプトでの手動操作でも、コーディングエージェント（Claude Code、Cursor）がシェルから呼び出して結果をパースする場合でも、同様に利用できます。 1 つのバイナリで以下が可能です: @@ -69,7 +69,7 @@ agenteye --version agenteye --help ``` -> **注意:** Failproof AI Observability の Python SDK も `agenteye` という配布名を使用しています。`pipx` または `uv tool` でインストール（共有 virtualenv への `pip install` ではなく）することで、両者の競合を避けられます。SDK が同一環境にインストールされていない場合に限り、`pip install agenteye` のみでも問題ありません。 +> **注意:** FailproofAI Cloud の Python SDK も `agenteye` という配布名を使用しています。`pipx` または `uv tool` でインストール（共有 virtualenv への `pip install` ではなく）することで、両者の競合を避けられます。SDK が同一環境にインストールされていない場合に限り、`pip install agenteye` のみでも問題ありません。 --- @@ -91,7 +91,7 @@ agenteye logout # セッションを無効化し、保存済みトークン `whoami` はセッションが存在しない場合や期限切れでもエラーになりません。代わりに `logged_in: false` を返すため、スクリプトやエージェントが安全に認証状態を確認できます（ベース URL が設定されていない場合やダッシュボードに到達できない場合は非ゼロで終了することがあります）。 -**要件:** ダッシュボードへのサインインが許可されたメールアドレスであること（Failproof AI Observability 管理者に確認してください）、およびダッシュボードがベース URL で到達可能であること（[設定](#configuration)を参照）。コードをリクエストしても届かない場合、そのメールアドレスはまだダッシュボードアクセスが有効になっていない可能性があります。 +**要件:** ダッシュボードへのサインインが許可されたメールアドレスであること（FailproofAI Cloud 管理者に確認してください）、およびダッシュボードがベース URL で到達可能であること（[設定](#configuration)を参照）。コードをリクエストしても届かない場合、そのメールアドレスはまだダッシュボードアクセスが有効になっていない可能性があります。 --- @@ -338,13 +338,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | 認証済みだが必要な権限がない（メッセージに権限名が表示されます） | | 6 | 指定されたリソースが見つからない（例: 不明なセッションまたはインシデント ID） | -これらにより CLI を安全にスクリプト化できます: コーディングエージェントは `4` で再認証を促したり、`5` で不足している権限を通知したりできます。終了コードの処理パターンと JSON 出力の形式については、[エージェント向け CLI レシピ](/ja/agenteye/cli-recipes)を参照してください。 +これらにより CLI を安全にスクリプト化できます: コーディングエージェントは `4` で再認証を促したり、`5` で不足している権限を通知したりできます。終了コードの処理パターンと JSON 出力の形式については、[エージェント向け CLI レシピ](/ja/cloud/cli-recipes)を参照してください。 --- ## 次のステップ -- **[エージェント向け CLI レシピ](/ja/agenteye/cli-recipes)**: コピー&ペーストで使えるクエリパターン、`jq` ワンライナー、`--fields` プロジェクション、終了コードの処理、JSON 出力の形式。CLI を操作するコーディングエージェント向けに書かれています。 -- **[CLI エージェントスキル](/ja/agenteye/cli-skill)**: この CLI を Claude Code / Codex のインストール可能な*スキル*としてパッケージ化し、コーディングエージェントが平易な英語のリクエストから Failproof AI Observability を操作できるようにします。 -- **[API キー](/ja/agenteye/api-keys)**: `keys create --add …` の背後にある権限モデル。 -- **[AI アシスタント](/ja/agenteye/assistant)**: `agent ask` が利用するアシスタントの有効化。 \ No newline at end of file +- **[エージェント向け CLI レシピ](/ja/cloud/cli-recipes)**: コピー&ペーストで使えるクエリパターン、`jq` ワンライナー、`--fields` プロジェクション、終了コードの処理、JSON 出力の形式。CLI を操作するコーディングエージェント向けに書かれています。 +- **[CLI エージェントスキル](/ja/cloud/agent-skills)**: この CLI を Claude Code / Codex のインストール可能な*スキル*としてパッケージ化し、コーディングエージェントが平易な英語のリクエストから FailproofAI Cloud を操作できるようにします。 +- **[API キー](/ja/cloud/access)**: `keys create --add …` の背後にある権限モデル。 +- **[AI アシスタント](/ja/cloud/assistant)**: `agent ask` が利用するアシスタントの有効化。 \ No newline at end of file diff --git a/docs/ja/cloud/connect.mdx b/docs/ja/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/ja/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/ja/agenteye/dashboards.mdx b/docs/ja/cloud/dashboards.mdx similarity index 69% rename from docs/ja/agenteye/dashboards.mdx rename to docs/ja/cloud/dashboards.mdx index f092069a..619c20a9 100644 --- a/docs/ja/agenteye/dashboards.mdx +++ b/docs/ja/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "ライブエージェントデータをチーム全員が確認で ライブエージェントデータをチーム全員が確認できる共有ビューに変換します。重要なクエリをチャートとして固定しておけば、誰でも一目で同じ数値を確認できます。クエリを再実行する必要はありません。 -![保存済みクエリから構築されたダッシュボード：時間あたりのイベント数の折れ線グラフ、エラータイプ別の棒グラフ、レイテンシのエリアチャート、モデル別トークン数](/agenteye/images/dashboard-fleet.png) +![保存済みクエリから構築されたダッシュボード：時間あたりのイベント数の折れ線グラフ、エラータイプ別の棒グラフ、レイテンシのエリアチャート、モデル別トークン数](/cloud/images/dashboard-fleet.png) *1枚のボードに4つの保存済みクエリ：時間あたりのイベント数、エラータイプ別、レイテンシ、モデル別トークン数。* @@ -25,15 +25,15 @@ description: "ライブエージェントデータをチーム全員が確認で ## 保存済みクエリをピンする -すべてのタイルは保存済みクエリから始まります。[クエリ](/ja/agenteye/queries)ライブラリ（組み込みプリセットと独自クエリ、イベントおよび評価データに対応）で目的のクエリを作成・保存し、データに合ったチャートとしてダッシュボードにピンします。時系列のトレンドには**折れ線**、カテゴリの比較には**棒**、ボリュームには**エリア**、割合の内訳には**円**グラフを選べます。 +すべてのタイルは保存済みクエリから始まります。[クエリ](/ja/cloud/queries)ライブラリ（組み込みプリセットと独自クエリ、イベントおよび評価データに対応）で目的のクエリを作成・保存し、データに合ったチャートとしてダッシュボードにピンします。時系列のトレンドには**折れ線**、カテゴリの比較には**棒**、ボリュームには**エリア**、割合の内訳には**円**グラフを選べます。 タイルは保存済みクエリをチャートとして表示しているだけなので、手動で同期する必要はありません。クエリを一度更新すれば、それを使用するすべてのダッシュボードも自動的に更新されます。 ## 量だけでなく品質も監視する -量はエージェントが動いているかどうかを示します。品質はエージェントが実際に仕事をこなしているかどうかを示します。[評価スコア](/ja/agenteye/evaluations)をダッシュボードに表示すれば、実行の品質を時系列で追跡できます。品質の低下はチャートの落ち込みとして現れるため、ユーザーから突然クレームが来るより前に気づくことができます。 +量はエージェントが動いているかどうかを示します。品質はエージェントが実際に仕事をこなしているかどうかを示します。[評価スコア](/ja/cloud/evaluations)をダッシュボードに表示すれば、実行の品質を時系列で追跡できます。品質の低下はチャートの落ち込みとして現れるため、ユーザーから突然クレームが来るより前に気づくことができます。 -![保存済み評価クエリから構築された品質重視のダッシュボード](/agenteye/images/dashboard-quality.png) +![保存済み評価クエリから構築された品質重視のダッシュボード](/cloud/images/dashboard-quality.png) *品質ボードは、オペレーションの数値と並べて評価スコアを前面に表示します。* @@ -41,6 +41,6 @@ description: "ライブエージェントデータをチーム全員が確認で ## 関連ページ -- [クエリ](/ja/agenteye/queries)：タイルの元となるクエリを作成・保存する。 -- [評価](/ja/agenteye/evaluations)：実行にスコアを付けて品質を時系列でチャート化する。 -- [アラート](/ja/agenteye/alerts)：これらのメトリクスのしきい値を超えたときに通知を受け取る。 \ No newline at end of file +- [クエリ](/ja/cloud/queries)：タイルの元となるクエリを作成・保存する。 +- [評価](/ja/cloud/evaluations)：実行にスコアを付けて品質を時系列でチャート化する。 +- [アラート](/ja/cloud/alerts)：これらのメトリクスのしきい値を超えたときに通知を受け取る。 \ No newline at end of file diff --git a/docs/ja/agenteye/error-tracking.mdx b/docs/ja/cloud/errors.mdx similarity index 68% rename from docs/ja/agenteye/error-tracking.mdx rename to docs/ja/cloud/errors.mdx index 75e19839..188490f5 100644 --- a/docs/ja/agenteye/error-tracking.mdx +++ b/docs/ja/cloud/errors.mdx @@ -6,14 +6,14 @@ description: "エージェントが発生させたすべての障害を一カ所 エージェントが発生させたすべての障害を一カ所で確認できます。大量のエラーが発生しても、1つの問題としてグループ化されます。「何かが赤くなっている」という状態から、問題のある実行を特定するまで、ライブフィードをスクロールすることなくワンクリックで辿り着けます。 -![Errorsページ：上部に時系列の障害ヒストグラム、下部にグループ化された赤いエラー行が並び、それぞれに「+ alert」ボタンがある](/agenteye/images/errors.png) +![Errorsページ：上部に時系列の障害ヒストグラム、下部にグループ化された赤いエラー行が並び、それぞれに「+ alert」ボタンがある](/cloud/images/errors.png) *Errorsページ：時系列の障害ヒストグラムと、繰り返し発生した障害を1行にまとめたインシデント一覧。* ## すべての障害を自動収集 エージェントが壊れたとき、ライブイベントストリームをスクロールして赤い行を見逃さないように監視し続ける必要はありません。**Errors** ページがその収集作業を代わりに行います。ダッシュボードで赤く表示されるすべての情報を1つのトリアージ画面にまとめるため、最初に目にするのは「何が壊れているか」であり、「どこを探すべきか」ではありません。 -また、明らかな障害だけでなく、静かな失敗も捕捉します。明示的な `error` イベントに加え、Failproof AI Observability は `tool_result`、`hook_completed`、`agent_end` のペイロードに失敗が含まれている場合もここに表示します。エラーを返したツールや異常終了したフックも、大きな例外がスローされなかったからといって見逃されることはありません。 +また、明らかな障害だけでなく、静かな失敗も捕捉します。明示的な `error` イベントに加え、FailproofAI Cloud は `tool_result`、`hook_completed`、`agent_end` のペイロードに失敗が含まれている場合もここに表示します。エラーを返したツールや異常終了したフックも、大きな例外がスローされなかったからといって見逃されることはありません。 ページ上部のヒストグラムは、エラーを時系列でプロットします。一目で、これが断続的な背景ノイズなのか、数分前から始まったスパイクなのかが分かるため、すぐに対応の優先度を判断できます。 @@ -23,19 +23,19 @@ description: "エージェントが発生させたすべての障害を一カ所 依存関係が1つ壊れるだけで、同じエラーが1分間に何百回も発火することがあります。そのままでは、ほぼ同一の行が壁のように並び、本当に見るべき情報が埋もれてしまいます。 -Failproof AI Observability は、同じセッションとエラータイプを共有する繰り返しの障害を1行に折りたたみます。大量のエラーが1件のインシデントとして表示されます。ログ行ではなく問題の数を数えられるようになり、重要なシグナルが大量のノイズに埋もれることなく上位に留まります。 +FailproofAI Cloud は、同じセッションとエラータイプを共有する繰り返しの障害を1行に折りたたみます。大量のエラーが1件のインシデントとして表示されます。ログ行ではなく問題の数を数えられるようになり、重要なシグナルが大量のノイズに埋もれることなく上位に留まります。 ## 「何かが赤い」から正確なイベントへ 任意の行をクリックすると、そのランのセッション内に直接ジャンプし、失敗した正確なイベントの位置が表示されます。セッション ID をコピーしたり、問題が起きた瞬間を探してスクロールしたりする必要はありません。エージェントが壊れる直前に何をしていたかが分かる完全な実行グラフが一目で確認できる状態で、その場所に直接到達します。 -`alerts:write` 権限を持っている場合、各行には **+ alert** ボタンも表示されます。クリックすると Observability が新しいアラートルールを開き、同じ障害を再度検知するための設定があらかじめ入力された状態になっています。トリアージしたばかりのインシデントが、次回は二度目のサプライズではなく、通知として届くようになります。 +`alerts:write` 権限を持っている場合、各行には **+ alert** ボタンも表示されます。クリックすると FailproofAI Cloud が新しいアラートルールを開き、同じ障害を再度検知するための設定があらかじめ入力された状態になっています。トリアージしたばかりのインシデントが、次回は二度目のサプライズではなく、通知として届くようになります。 **場所：** **Errors** ページはダッシュボードのオブザーブセクションにあり、`//errors` でアクセスできます。 ## 関連ページ -- [Alerts](/ja/agenteye/alerts)：任意の障害をページングルールに変換します。 -- [Incidents](/ja/agenteye/incidents)：発火したアラートをオープンからリゾルブまで追跡します。 -- [Sessions](/ja/agenteye/sessions)：エラーの背後にある完全な実行を開きます。 -- [Audits](/ja/agenteye/audits)：Observability がすべての実行にわたる障害パターンを自動検出します。 \ No newline at end of file +- [Alerts](/ja/cloud/alerts)：任意の障害をページングルールに変換します。 +- [Incidents](/ja/cloud/incidents)：発火したアラートをオープンからリゾルブまで追跡します。 +- [Sessions](/ja/cloud/sessions)：エラーの背後にある完全な実行を開きます。 +- [Audits](/ja/cloud/audits)：FailproofAI Cloud がすべての実行にわたる障害パターンを自動検出します。 \ No newline at end of file diff --git a/docs/ja/agenteye/evaluations.mdx b/docs/ja/cloud/evaluations.mdx similarity index 68% rename from docs/ja/agenteye/evaluations.mdx rename to docs/ja/cloud/evaluations.mdx index cf7008e7..82671592 100644 --- a/docs/ja/agenteye/evaluations.mdx +++ b/docs/ja/cloud/evaluations.mdx @@ -4,15 +4,15 @@ description: "品質の問題が自然と見つかるようになります。ユ --- -品質の問題が自然と見つかるようになります。ユーザーのクレームで初めて気づくことはなくなります。スコアリングサービスを一度接続するだけで、Failproof AI Observability がすべての完了済み実行を自動的に採点します。応答の有用性の低下やハルシネーションの急増を、顧客が気づく前に自動で検出します。 +品質の問題が自然と見つかるようになります。ユーザーのクレームで初めて気づくことはなくなります。スコアリングサービスを一度接続するだけで、FailproofAI Cloud がすべての完了済み実行を自動的に採点します。応答の有用性の低下やハルシネーションの急増を、顧客が気づく前に自動で検出します。 -![スコア列付きのセッショングリッド: 各実行に評価ステータスのバッジと、有用性・事実性・ツール効率を色分けしたバッジが表示されている](/agenteye/images/sessions-list.png) +![スコア列付きのセッショングリッド: 各実行に評価ステータスのバッジと、有用性・事実性・ツール効率を色分けしたバッジが表示されている](/cloud/images/sessions-list.png) *セッショングリッドのすべての実行にスコアが付いています。赤・黄・緑のバッジにより、トランスクリプトを一つも開かずに問題のある実行が一目でわかります。* ## 手作業によるサンプリングをやめる -これまでは一部の実行だけをスポットチェックして、残りは問題ないと祈るしかありませんでした。今後は、完了したすべてのセッションが終了した瞬間にスコアリングされます。対象ディメンションは、有用性・ツール効率・事実性・安全性など、あなたが重視する品質基準に合わせて設定できます。スコアのキーはあなたが定義し、Failproof AI Observability は評価器が返すあらゆるデータを保存・傾向分析・表示します。採点漏れは一切なく、サポートチケットで回帰を知ることもなくなります。 +これまでは一部の実行だけをスポットチェックして、残りは問題ないと祈るしかありませんでした。今後は、完了したすべてのセッションが終了した瞬間にスコアリングされます。対象ディメンションは、有用性・ツール効率・事実性・安全性など、あなたが重視する品質基準に合わせて設定できます。スコアのキーはあなたが定義し、FailproofAI Cloud は評価器が返すあらゆるデータを保存・傾向分析・表示します。採点漏れは一切なく、サポートチケットで回帰を知ることもなくなります。 スコアは **`//sessions`**（サイドバー → *observe* → *sessions*）のセッショングリッドに表示され、各行にバッジのクラスターが付きます。スコアが低い実行だけを確認したい場合は、スコア範囲でグリッドをフィルタリングしてください。たとえば有用性が 0.5 未満のように絞り込めば、確認すべき実行だけを取り出せます。スコアの閲覧には `evaluations:read` 権限が必要です。 @@ -20,7 +20,7 @@ description: "品質の問題が自然と見つかるようになります。ユ 数値は実行の問題を示しますが、セッションページはその理由を教えてくれます。任意の実行を開くと、右パネルに概要サマリーが表示され、その下に各ディメンションのスコアバーと評価器が生成した根拠が示されます。「事実性が 0.4 だった」という状態から、どの主張が誤っていたかまで、数秒で確認できます。 -![セッションの右パネル: 上部に評価サマリー、続いて各ディメンションのスコアバーと根拠の一行説明、隣にはイベントタイムライン全体が表示されている](/agenteye/images/session-detail.png) +![セッションの右パネル: 上部に評価サマリー、続いて各ディメンションのスコアバーと根拠の一行説明、隣にはイベントタイムライン全体が表示されている](/cloud/images/session-detail.png) *セッション詳細ビュー: サマリー、ディメンション別スコアバー、各スコアの根拠が実行のイベントタイムラインの隣に表示されます。* @@ -30,7 +30,7 @@ description: "品質の問題が自然と見つかるようになります。ユ 1 件の低スコアはノイズに過ぎませんが、コホート全体の低下はシグナルです。保存済みダッシュボードを使えば、スコアをひと目で確認できるトレンドに変換できます。エージェント別・環境別に、今週と先週の平均有用性を比較するといった使い方も可能です。 -![品質ダッシュボード: 評価ディメンションごとの平均スコアバーと経時的なトレンド](/agenteye/images/dashboard-quality.png) +![品質ダッシュボード: 評価ディメンションごとの平均スコアバーと経時的なトレンド](/cloud/images/dashboard-quality.png) *保存済みの品質ダッシュボードは注目するスコアキーのトレンドを表示するため、インシデントになる前の緩やかな低下を早期に発見できます。* @@ -38,14 +38,14 @@ description: "品質の問題が自然と見つかるようになります。ユ ## 評価器を一度接続する -スコアリングはオプトイン方式で、Failproof AI Observability にスコアラーを指定するまでは完全にオフになっています。小さな HTTP サービスを一つ立ち上げ（Observability にはコピーして使える実用的なリファレンス実装が付属しています）、サーバーに 2 つの値を設定するだけで、以降のすべての実行が自動的に採点されます。詳細なウォークスルー・スコアリングの仕様・SDK は詳細ガイドに記載されています。 +スコアリングはオプトイン方式で、FailproofAI Cloud にスコアラーを指定するまでは完全にオフになっています。小さな HTTP サービスを一つ立ち上げ（FailproofAI Cloud にはコピーして使える実用的なリファレンス実装が付属しています）、サーバーに 2 つの値を設定するだけで、以降のすべての実行が自動的に採点されます。詳細なウォークスルー・スコアリングの仕様・SDK は詳細ガイドに記載されています。 -どのディメンションを採点すべきか迷っている場合は、[evaluator agent skill](/ja/agenteye/evaluator-skill) を使えば、コーディングエージェントが実際のセッションをもとに最適なスコアディメンションを見つけ出し、サービスを構築・デプロイしてくれます。 +どのディメンションを採点すべきか迷っている場合は、[evaluator agent skill](/ja/cloud/agent-skills) を使えば、コーディングエージェントが実際のセッションをもとに最適なスコアディメンションを見つけ出し、サービスを構築・デプロイしてくれます。 ## 関連情報 -- [Evaluation suite](/ja/agenteye/evaluation-suite): 評価器の接続、スコアリングの仕様、SDK について。 -- [Evaluator agent skill](/ja/agenteye/evaluator-skill): コーディングエージェントにスコアのディメンション選定と評価器の構築を任せる。 -- [Sessions](/ja/agenteye/sessions): スコアが表示される実行単位のグリッド。 -- [Dashboards](/ja/agenteye/dashboards): 組織全体の品質トレンドを保存・共有する。 -- [Audits](/ja/agenteye/audits): セッションをまたいだ調査に対応する、Observability のもう一つの自動品質機能。 \ No newline at end of file +- [Evaluation suite](/ja/cloud/evaluators): 評価器の接続、スコアリングの仕様、SDK について。 +- [Evaluator agent skill](/ja/cloud/agent-skills): コーディングエージェントにスコアのディメンション選定と評価器の構築を任せる。 +- [Sessions](/ja/cloud/sessions): スコアが表示される実行単位のグリッド。 +- [Dashboards](/ja/cloud/dashboards): 組織全体の品質トレンドを保存・共有する。 +- [Audits](/ja/cloud/audits): セッションをまたいだ調査に対応する、FailproofAI Cloud のもう一つの自動品質機能。 \ No newline at end of file diff --git a/docs/ja/agenteye/evaluation-suite.mdx b/docs/ja/cloud/evaluators.mdx similarity index 82% rename from docs/ja/agenteye/evaluation-suite.mdx rename to docs/ja/cloud/evaluators.mdx index 9c5e9ed2..d7be85f7 100644 --- a/docs/ja/agenteye/evaluation-suite.mdx +++ b/docs/ja/cloud/evaluators.mdx @@ -1,20 +1,20 @@ --- title: "評価スイート" -description: "Failproof AI Observability は、完了したすべてのエージェント実行を自動的に品質スコアリングできます。小さなスコアリングサービスを用意するだけで、あとは Observability が処理します。" +description: "FailproofAI Cloud は、完了したすべてのエージェント実行を自動的に品質スコアリングできます。小さなスコアリングサービスを用意するだけで、あとは FailproofAI Cloud が処理します。" --- -Failproof AI Observability は、完了したすべてのエージェント実行を自動的に品質スコアリングできます。小さなスコアリングサービスを用意するだけで、あとは Observability が処理します。追跡したい指標（有用性、ツール効率、事実性、安全性など、選択は自由）を管理し、品質低下を早期に検知し、エージェントや環境を一目で比較できます。スコアリングはオプトイン式です。サーバーに `EVALUATOR_ENDPOINT` を設定するまでパイプラインは何もしません。 +FailproofAI Cloud は、完了したすべてのエージェント実行を自動的に品質スコアリングできます。小さなスコアリングサービスを用意するだけで、あとは FailproofAI Cloud が処理します。追跡したい指標（有用性、ツール効率、事実性、安全性など、選択は自由）を管理し、品質低下を早期に検知し、エージェントや環境を一目で比較できます。スコアリングはオプトイン式です。サーバーに `EVALUATOR_ENDPOINT` を設定するまでパイプラインは何もしません。 -> **注意:** スコアの次元はご自身が定義します。評価器はお好きな数値キーを返せます。Observability は送り返された内容をそのまま保存・トレンド表示・ダッシュボード表示します。 +> **注意:** スコアの次元はご自身が定義します。評価器はお好きな数値キーを返せます。FailproofAI Cloud は送り返された内容をそのまま保存・トレンド表示・ダッシュボード表示します。 ## 概要 -1. **スコアラーを作成する。** セッションのトランスクリプトを読み込んでスコアを返す小さな HTTP サービスを立ち上げます。Observability には動作するリファレンス実装が含まれているのでコピーして使えます。[SDK を使った評価器の作成](#writing-an-evaluator-with-the-sdk) を参照してください。 -2. **Observability にエンドポイントを設定する。** サーバープロセスに `EVALUATOR_ENDPOINT`（および共有の `EVALUATOR_TOKEN`）を設定します。 +1. **スコアラーを作成する。** セッションのトランスクリプトを読み込んでスコアを返す小さな HTTP サービスを立ち上げます。FailproofAI Cloud には動作するリファレンス実装が含まれているのでコピーして使えます。[SDK を使った評価器の作成](#writing-an-evaluator-with-the-sdk) を参照してください。 +2. **FailproofAI Cloud にエンドポイントを設定する。** サーバープロセスに `EVALUATOR_ENDPOINT`（および共有の `EVALUATOR_TOKEN`）を設定します。 3. **スコアを確認する。** 完了したセッションはすべて自動的にスコアリングされ、セッション詳細ページ・セッション一覧グリッド・保存済みダッシュボードに結果が表示されます。 -![評価サマリー、次元別スコアバー、右ペインの推論テキストを含むセッション詳細ビュー](/agenteye/images/session-detail.png) +![評価サマリー、次元別スコアバー、右ペインの推論テキストを含むセッション詳細ビュー](/cloud/images/session-detail.png) *評価器を設定すると、完了した各実行がスコアリングされ、結果がセッションの右ペインに表示されます。上部にサマリー、続いて各次元のスコアバーと推論テキストが表示されます。* @@ -24,7 +24,7 @@ Failproof AI Observability は、完了したすべてのエージェント実 ```mermaid flowchart LR - ING["ingest /events
    agent_end"] --> SRV["Observability server"] + ING["ingest /events
    agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -32,14 +32,14 @@ flowchart LR SRV --> RES["evaluations
    terminal results"] ``` -Observability SDK がセッションの `agent_end` イベントを送出すると、サーバーは評価をスケジュールします。次に、完全なイベントトランスクリプトを評価器サービスに POST します。評価器は次のどちらかを行えます。 +FailproofAI Cloud SDK がセッションの `agent_end` イベントを送出すると、サーバーは評価をスケジュールします。次に、完全なイベントトランスクリプトを評価器サービスに POST します。評価器は次のどちらかを行えます。 - **インラインで結果を返す:** `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}` を返します。結果はセッションの評価タイムラインに追記されます。`reasoning` と `summary` はオプションです。 -- **処理を遅延させる:** `{"status":"pending", "job_id":"abc-123"}` を返します。Observability は評価器が `{"status":"done", ...}` または `{"status":"error", "error":"..."}` を返すまで `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` をポーリングします。 +- **処理を遅延させる:** `{"status":"pending", "job_id":"abc-123"}` を返します。FailproofAI Cloud は評価器が `{"status":"done", ...}` または `{"status":"error", "error":"..."}` を返すまで `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` をポーリングします。 - ポーリング間隔はジョブごとに設定できます。`pending` レスポンスに `next_poll_secs` を含めると間隔を上書きできます。省略した場合、Observability は `GET /config` の `default_poll_interval_secs` を使用し、それもなければ `EVALUATOR_POLLING_INTERVAL_SECS`（デフォルト 10 秒）にフォールバックします。すべての値は [1 秒、1 時間] にクランプされます。 + ポーリング間隔はジョブごとに設定できます。`pending` レスポンスに `next_poll_secs` を含めると間隔を上書きできます。省略した場合、FailproofAI Cloud は `GET /config` の `default_poll_interval_secs` を使用し、それもなければ `EVALUATOR_POLLING_INTERVAL_SECS`（デフォルト 10 秒）にフォールバックします。すべての値は [1 秒、1 時間] にクランプされます。 -`agent_end` を送出しないセッション（クラッシュしたエージェントプロセスなど）もピックアップできます。評価器の `GET /config` が `{"inactivity_timeout_secs": 1800}` を返すと、Observability はその時間アイドル状態になったセッションを評価します。このフォールバックを無効にするには、フィールドを `null` に設定するか省略してください。 +`agent_end` を送出しないセッション（クラッシュしたエージェントプロセスなど）もピックアップできます。評価器の `GET /config` が `{"inactivity_timeout_secs": 1800}` を返すと、FailproofAI Cloud はその時間アイドル状態になったセッションを評価します。このフォールバックを無効にするには、フィールドを `null` に設定するか省略してください。 `EVALUATOR_ENDPOINT` が未設定の場合、パイプラインは完全に no-op になります。 @@ -47,7 +47,7 @@ Observability SDK がセッションの `agent_end` イベントを送出する アイドル状態フォールバックは再開されたセッションでも再び動作します。以前の終端評価後に新しいイベントが届き、その後セッションが `inactivity_timeout_secs` を超えてアイドル状態になった場合、新しい評価がエンキューされます。 -一時的な障害（5xx、429、タイムアウト、ネットワークエラー）は `EVALUATOR_MAX_ATTEMPTS` に達するまで指数バックオフで再試行されます。4xx レスポンスは終端扱いです。Observability は水平スケールされた複数のサーバーインスタンスで安全に実行できます。同じセッションが同時に2回ディスパッチされないようにワークが分割されます。 +一時的な障害（5xx、429、タイムアウト、ネットワークエラー）は `EVALUATOR_MAX_ATTEMPTS` に達するまで指数バックオフで再試行されます。4xx レスポンスは終端扱いです。FailproofAI Cloud は水平スケールされた複数のサーバーインスタンスで安全に実行できます。同じセッションが同時に2回ディスパッチされないようにワークが分割されます。 --- @@ -55,7 +55,7 @@ Observability SDK がセッションの `agent_end` イベントを送出する 認証が必要なすべてのルートは**ベアラートークン認証**を使用します。両側で同じ値を設定する必要があります。 -- Observability サーバー: 環境変数 `EVALUATOR_TOKEN` +- FailproofAI Cloud サーバー: 環境変数 `EVALUATOR_TOKEN` - 評価器サービス: 同じ方法で設定（`agenteye-evaluator` SDK は慣例として `EVALUATOR_TOKEN` を読み込みます） `EVALUATOR_TOKEN` が未設定の場合、サーバーは `Authorization` ヘッダーを送信しません。評価器は匿名リクエストを受け付けることができますが、内部ネットワーク専用であれば問題ありませんが、公開インターネット上では非推奨です。 @@ -126,7 +126,7 @@ Observability SDK がセッションの `agent_end` イベントを送出する HTTP コントラクトを手動で実装する必要はありません。`agenteye-evaluator` Python パッケージは、認証・ルーティング・リクエスト/レスポンス形式を処理する型付き FastAPI ラッパーを提供します。 -Failproof AI Observability には、トランスクリプトの形状から `helpfulness`、`tool_efficiency`、`factuality` をスコアリングする**動作するリファレンス評価器**も含まれています。出発点としてコピーし、独自のロジック（LLM ジャッジ、ルールエンジンなど、品質基準に合ったもの）に置き換えてください。 +FailproofAI Cloud には、トランスクリプトの形状から `helpfulness`、`tool_efficiency`、`factuality` をスコアリングする**動作するリファレンス評価器**も含まれています。出発点としてコピーし、独自のロジック（LLM ジャッジ、ルールエンジンなど、品質基準に合ったもの）に置き換えてください。 最小限の評価器: @@ -149,7 +149,7 @@ def run(req: EvalRequest) -> EvalResponse: `app` インスタンスはあらゆる ASGI サーバーで動作するため、`uvicorn module:app` で起動できます。 -重い処理を遅延させる必要がある評価器では、代わりに `JobPending` を返し、`@app.job_lookup` ハンドラーを登録してください。Observability サーバーは評価器が終端ステータスを返すか、`EVALUATOR_MAX_POLL_DURATION_SECS` の上限（デフォルト 1 時間）に達するまで `GET /evaluate/{job_id}` をポーリングします。 +重い処理を遅延させる必要がある評価器では、代わりに `JobPending` を返し、`@app.job_lookup` ハンドラーを登録してください。FailproofAI Cloud サーバーは評価器が終端ステータスを返すか、`EVALUATOR_MAX_POLL_DURATION_SECS` の上限（デフォルト 1 時間）に達するまで `GET /evaluate/{job_id}` をポーリングします。 完全な API リファレンス、非同期パターン、イベントスキーマは `agenteye-evaluator` SDK の README に記載されています。 @@ -157,7 +157,7 @@ def run(req: EvalRequest) -> EvalResponse: ## 評価器の実行 -評価器は**ご自身のサービス**です。Failproof AI Observability はデフォルトの評価器を提供しないため、ご自身のサービスを実行している場所でビルドして実行してください。任意の ASGI サーバー（例: `uvicorn my_evaluator:app`）で動作します。[HTTP コントラクト](#http-contract) の `/health`、`/config`、`/evaluate` ルートを提供し、サーバーからアクセスできるように設定してください（[サーバーの設定](#configuring-the-server) を参照）。 +評価器は**ご自身のサービス**です。FailproofAI Cloud はデフォルトの評価器を提供しないため、ご自身のサービスを実行している場所でビルドして実行してください。任意の ASGI サーバー（例: `uvicorn my_evaluator:app`）で動作します。[HTTP コントラクト](#http-contract) の `/health`、`/config`、`/evaluate` ルートを提供し、サーバーからアクセスできるように設定してください（[サーバーの設定](#configuring-the-server) を参照）。 評価器に到達できるようになると、`GET /health` が `{"status":"ok"}` を返します。エージェントがエンドツーエンドで実行された後、サーバーの `GET /evaluations` は `status: "done"` と評価器が生成したスコアを含む行を返します。 @@ -255,7 +255,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. - **`/sessions`**: フィルタリング可能なセッション一覧グリッド。スコア列で各セッションの評価ステータスとスコアを一目で確認できます。 - **`/dashboards`**: 保存済みの評価ヘルスビュー（以下の[ダッシュボード](#dashboards)を参照）。 -![セッションごとの評価ステータスバッジとカラーコードのスコアバッジ（helpfulness、factuality、tool_efficiency、safety、coherence）が表示されたセッション一覧グリッド](/agenteye/images/sessions-list.png) +![セッションごとの評価ステータスバッジとカラーコードのスコアバッジ（helpfulness、factuality、tool_efficiency、safety、coherence）が表示されたセッション一覧グリッド](/cloud/images/sessions-list.png) *セッション一覧グリッドでは各実行の評価ステータスとスコアを一目で確認できます。赤/黄/緑のバッジで低スコアをすぐに発見できます。* @@ -272,7 +272,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. 各カードにはマッチするセッション数、done/error/timeout の内訳、各注目スコアの平均、小さなトレンドスパークラインが表示されます。ダッシュボードを開くとフルサイズのパネルが表示されます。**セッションで開く**をクリックすると、そのスライスに絞り込まれた状態でセッションページが開きます。メトリクスはサーバーサイドでマッチするセット全体にわたって計算されます（`GET /evaluations/aggregate` 経由）。そのため数値はサンプリングではなく正確です。 -![評価器の次元ごとの平均スコアバー、ツールの成功/エラー内訳、上位ツール、1 時間あたりのイベント数トレンドを含む評価ヘルスダッシュボード](/agenteye/images/dashboard-quality.png) +![評価器の次元ごとの平均スコアバー、ツールの成功/エラー内訳、上位ツール、1 時間あたりのイベント数トレンドを含む評価ヘルスダッシュボード](/cloud/images/dashboard-quality.png) **権限:** 閲覧には `dashboards:read` と `evaluations:read` の両方が必要です。作成と編集には `dashboards:write`、削除には `dashboards:delete` が必要です。ブートストラップ管理者はこれらすべてを自動的に受け取ります。 @@ -294,7 +294,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. ## 次のステップ -- [評価器エージェントスキル](/ja/agenteye/evaluator-skill): コーディングエージェントに、実際のセッションに対して次元を設計し、このサービスを構築させる。 -- [Python SDK](/ja/agenteye/python-sdk): スコアリングをトリガーする `agent_end` イベントを送出する。 -- [API キー](/ja/agenteye/api-keys): `evaluations:read` と `evaluations:trigger` 権限。 -- [監査](/ja/agenteye/audits): Observability のもう一つの自動品質機能、ポリシーベースのレビュー。 \ No newline at end of file +- [評価器エージェントスキル](/ja/cloud/agent-skills): コーディングエージェントに、実際のセッションに対して次元を設計し、このサービスを構築させる。 +- [Python SDK](/ja/cloud/sdk): スコアリングをトリガーする `agent_end` イベントを送出する。 +- [API キー](/ja/cloud/access): `evaluations:read` と `evaluations:trigger` 権限。 +- [監査](/ja/cloud/audits): FailproofAI Cloud のもう一つの自動品質機能、ポリシーベースのレビュー。 \ No newline at end of file diff --git a/docs/ja/agenteye/event-stream.mdx b/docs/ja/cloud/event-stream.mdx similarity index 80% rename from docs/ja/agenteye/event-stream.mdx rename to docs/ja/cloud/event-stream.mdx index b13131c8..89b35287 100644 --- a/docs/ja/agenteye/event-stream.mdx +++ b/docs/ja/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "エージェントが何かをした瞬間、それがすぐに見 エージェントが何かをした瞬間、それがすぐに見える。イベントストリームは、本番環境で動くすべてのエージェントをリアルタイムで把握するための窓口です。待ち時間なし、ログのgrepなし、何が起きたのかを推測する必要もありません。 -![ライブのイベントストリーム：色分けされたイベント行がリアルタイムで流れ、環境・エージェント・セッション・イベントタイプ・フリーテキストでフィルタリング可能](/agenteye/images/events-stream.png) +![ライブのイベントストリーム：色分けされたイベント行がリアルタイムで流れ、環境・エージェント・セッション・イベントタイプ・フリーテキストでフィルタリング可能](/cloud/images/events-stream.png) *組織内のすべてのエージェントからのすべてのイベントが、最新のものから順に、発生と同時に更新される。* @@ -37,14 +37,14 @@ description: "エージェントが何かをした瞬間、それがすぐに見 イベントストリームは組織のホーム画面です。サインインすると最初に表示されるのがこの画面で、`//` でアクセスできます。到着した瞬間からトリアージを開始できます。 -その裏では、エージェントがSDKを通じてイベントを送信し、コレクターがそれをFailproof AI Observabilityサーバーに転送し、ストリームが自分たちで管理するインフラにイベントが届くたびにリアルタイムで表示します。生のトレイルではなく集計されたビューが必要な場合は、各実行のイベントがSessions上で1行にまとめられており、1クリックで確認できます。 +その裏では、エージェントがSDKを通じてイベントを送信し、コレクターがそれをFailproofAI Cloudサーバーに転送し、ストリームが自分たちで管理するインフラにイベントが届くたびにリアルタイムで表示します。生のトレイルではなく集計されたビューが必要な場合は、各実行のイベントがSessions上で1行にまとめられており、1クリックで確認できます。 これはすべての観察用サーフェスが基盤とする生の情報源です。他の場所で数値がおかしいと感じたときは、このストリームで実際に何が起きたかを確認してください。 ## 関連情報 -- [Sessions](/ja/agenteye/sessions)：同じイベントを実行ごとに1行にまとめ、gitスタイルの実行グラフで表示。 -- [Telemetry](/ja/agenteye/telemetry)：エージェントが送信する内容と、イベントがストリームに到達するまでの仕組み。 -- [Error tracking](/ja/agenteye/error-tracking)：問題が発生したすべての事象を一元管理するトリアージ画面。 -- [Alerts](/ja/agenteye/alerts)：任意のしきい値をアラートルールに変換。 -- [CLI and agents](/ja/agenteye/cli-and-agents)：ターミナルから同じライブトレイルを確認。 \ No newline at end of file +- [Sessions](/ja/cloud/sessions)：同じイベントを実行ごとに1行にまとめ、gitスタイルの実行グラフで表示。 +- [Telemetry](/ja/cloud/performance)：エージェントが送信する内容と、イベントがストリームに到達するまでの仕組み。 +- [Error tracking](/ja/cloud/errors)：問題が発生したすべての事象を一元管理するトリアージ画面。 +- [Alerts](/ja/cloud/alerts)：任意のしきい値をアラートルールに変換。 +- [CLI and agents](/ja/cloud/cli)：ターミナルから同じライブトレイルを確認。 \ No newline at end of file diff --git a/docs/ja/cloud/fleet.mdx b/docs/ja/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/ja/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/ja/agenteye/incidents.mdx b/docs/ja/cloud/incidents.mdx similarity index 86% rename from docs/ja/agenteye/incidents.mdx rename to docs/ja/cloud/incidents.mdx index 568fd552..0419627c 100644 --- a/docs/ja/agenteye/incidents.mdx +++ b/docs/ja/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "アラートが発火すると、誰もがインシデントの状 アラートが発火したとき、最初に浮かぶ疑問は常に「誰が対応しているのか?」です。インシデントはその答えを提供します。何かが閾値を超えた瞬間に、全員がインシデントの発生、担当者、そしてこれまでの経緯を正確に把握できます。また、ポストモーテムにそのまま活用できる、クリーンで帰属情報付きの記録が残ります。 -![インシデントの受信トレイ: アラートに紐づいたインシデントと手動で作成されたインシデントのカードが、状態ごとにグループ化され、それぞれに重大度バッジと担当者が表示されている](/agenteye/images/incidents.png) +![インシデントの受信トレイ: アラートに紐づいたインシデントと手動で作成されたインシデントのカードが、状態ごとにグループ化され、それぞれに重大度バッジと担当者が表示されている](/cloud/images/incidents.png) *受信トレイはオープンなインシデントを状態別にグループ化し、重大度や担当者でフィルタリングできるため、今すぐ人が対応すべきものを一目で確認できます。* ## 誰が担当しているか、一目でわかる @@ -17,10 +17,10 @@ description: "アラートが発火すると、誰もがインシデントの状 インシデントが解決したとき、ドキュメントはすでに出来上がっています。任意のインシデントを開くと、閾値超過の証拠、担当者とサブスクライバー、その場での連携用コメントスレッド、そして追記のみ可能なアクティビティタイムラインが表示されます。 -![インシデントの詳細ビュー: 親アラートと閾値超過のサマリー、担当者とサブスクライバー、帰属情報付きのアクティビティタイムライン、コメントスレッド](/agenteye/images/incident-detail.png) +![インシデントの詳細ビュー: 親アラートと閾値超過のサマリー、担当者とサブスクライバー、帰属情報付きのアクティビティタイムライン、コメントスレッド](/cloud/images/incident-detail.png) *起きたことすべてが時系列で並び、各行には実行した担当者の名前が付いています。* -すべてのアクション（作成、宣言、解決など）はタイムラインに書き込まれ、後から編集されることはありません。各エントリには帰属情報が付きます。アクションを実行したオペレーターのメールアドレス、または Failproof AI Observability が自律的に行った処理（閾値超過時のインシデント作成など）の場合は **automated** と表示されます。匿名のものも、失われるものも一切ありません。ポストモーテムはほぼ自動的に出来上がります。 +すべてのアクション（作成、宣言、解決など）はタイムラインに書き込まれ、後から編集されることはありません。各エントリには帰属情報が付きます。アクションを実行したオペレーターのメールアドレス、または FailproofAI Cloud が自律的に行った処理（閾値超過時のインシデント作成など）の場合は **automated** と表示されます。匿名のものも、失われるものも一切ありません。ポストモーテムはほぼ自動的に出来上がります。 ## インシデントの状態遷移 @@ -45,6 +45,6 @@ stateDiagram-v2 ## 関連項目 -- [アラート](/ja/agenteye/alerts): 閾値を超えたときにインシデントを作成するルール。 -- [エラートラッキング](/ja/agenteye/error-tracking): すべての障害を一か所で確認し、アラートに昇格させる。 -- [監査](/ja/agenteye/audits): どのルールも監視していなかった障害を発見する、スケジュール済みアナリスト。 \ No newline at end of file +- [アラート](/ja/cloud/alerts): 閾値を超えたときにインシデントを作成するルール。 +- [エラートラッキング](/ja/cloud/errors): すべての障害を一か所で確認し、アラートに昇格させる。 +- [監査](/ja/cloud/audits): どのルールも監視していなかった障害を発見する、スケジュール済みアナリスト。 \ No newline at end of file diff --git a/docs/ja/cloud/managed-policies.mdx b/docs/ja/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/ja/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/ja/agenteye/overview.mdx b/docs/ja/cloud/overview.mdx similarity index 51% rename from docs/ja/agenteye/overview.mdx rename to docs/ja/cloud/overview.mdx index dc235f77..021b6d53 100644 --- a/docs/ja/agenteye/overview.mdx +++ b/docs/ja/cloud/overview.mdx @@ -1,16 +1,16 @@ --- title: "Failproof AI: エージェントの障害を観測する" -description: "Failproof AI Observability は、本番環境のAIエージェントを観測・評価・改善するためのセルフホスト型プラットフォームです。" +description: "FailproofAI Cloud は、本番環境のAIエージェントを観測・評価・改善するためのセルフホスト型プラットフォームです。" --- -Failproof AI Observability は、本番環境のAIエージェントを観測・評価・改善するためのセルフホスト型プラットフォームです。エージェントのあらゆる動作（ツール呼び出し、モデルリクエスト、フック、エラー）を記録し、各実行の品質をスコアリングして、気づかなかった障害を洗い出します。これらすべてを、自社インフラ内で稼働するダッシュボードで確認できます。 +FailproofAI Cloud は、本番環境のAIエージェントを観測・評価・改善するためのセルフホスト型プラットフォームです。エージェントのあらゆる動作（ツール呼び出し、モデルリクエスト、フック、エラー）を記録し、各実行の品質をスコアリングして、気づかなかった障害を洗い出します。これらすべてを、自社インフラ内で稼働するダッシュボードで確認できます。 -AIエージェントをリリースしていて、実行が失敗した原因の推測に疲れているなら、まずこのページから始めてください。インストールの前に、Failproof AI Observability が提供するものと各要素の関係を説明します。 +AIエージェントをリリースしていて、実行が失敗した原因の推測に疲れているなら、まずこのページから始めてください。インストールの前に、FailproofAI Cloud が提供するものと各要素の関係を説明します。 -> **Failproof AI Observability は Failproof AI のエンタープライズ製品です。** 実際の動作を見たいですか？デモをリクエストしてください: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) までメールをお送りください。 +> **FailproofAI Cloud は Failproof AI のエンタープライズ製品です。** 実際の動作を見たいですか？デモをリクエストしてください: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) までメールをお送りください。 -![Failproof AI Observability のセッション画面。Gitスタイルの実行グラフとイベントタイムラインを並べて表示し、右側のパネルにツール・モデル・フックの実行ごとの内訳を示している](/agenteye/images/session-detail.png) +![FailproofAI Cloud のセッション画面。Gitスタイルの実行グラフとイベントタイムラインを並べて表示し、右側のパネルにツール・モデル・フックの実行ごとの内訳を示している](/cloud/images/session-detail.png) *各エージェント実行はGitスタイルの実行グラフ（左）とイベントタイムラインとして表示されます。並列サブエージェントはそれぞれ独自のレーンを持ち、右パネルには実行ごとのツール・モデル・フック・トークン消費の内訳が表示されます。* @@ -30,52 +30,52 @@ AIエージェントをリリースしていて、実行が失敗した原因の
    -*Failproof Audit: Failproof AI Observability がセッションをまたいでログを解析し、修正すべき箇所を教えてくれます。* +*Failproof Audit: FailproofAI Cloud がセッションをまたいでログを解析し、修正すべき箇所を教えてくれます。* --- ## チームが使う理由 - **エージェントが実際に何をしたかを把握できる。** すべての実行は読みやすいGitスタイルの実行グラフになります。どのツールが並列で動いたか、どのサブエージェントが分岐したか、どこで止まったか、何にコストがかかったかが一目でわかります。 -- **品質の低下を自動で検出できる。** 小規模なスコアリングサービスを接続すると、Failproof AI Observability がすべての完了済み実行をスコアリングし、有用性の低下やハルシネーションの増加を自動的に検出します。 +- **品質の低下を自動で検出できる。** 小規模なスコアリングサービスを接続すると、FailproofAI Cloud がすべての完了済み実行をスコアリングし、有用性の低下やハルシネーションの増加を自動的に検出します。 - **ルールを書いていない障害も発見できる。** 定期監査がセッションをまたいでログを解析し、エラーのクラスター、レイテンシの外れ値、低スコア、スタックした実行を検出して、根拠付きのランク付きファインディングを提供します。 - **重要なときに通知を受け取れる。** エラーレート、レイテンシ、コスト、評価スコアに対してしきい値ルールを設定でき、確認・割り当て・解決ができるインシデントを発生させます。 - **自然言語で質問できる。** ダッシュボード内のAIアシスタントに「今週の本番環境の品質トレンドは?」と自分のデータに基づいて質問できます。アシスタントが行う変更はすべて承認が必要です。 -- **データを自社で管理できる。** Failproof AI Observability はセルフホスト型のため、イベント、プロンプト、分析データはすべて自社管理のインフラ内に留まります。 +- **データを自社で管理できる。** FailproofAI Cloud はセルフホスト型のため、イベント、プロンプト、分析データはすべて自社管理のインフラ内に留まります。 --- ## 提供機能 -Failproof AI Observability は3つのコンセプト（**observe（観測）**、**analyze（分析）**、**admin（管理）**）を中心に構成されており、ダッシュボードの左サイドバーに反映されています。 +FailproofAI Cloud は3つのコンセプト（**observe（観測）**、**analyze（分析）**、**admin（管理）**）を中心に構成されており、ダッシュボードの左サイドバーに反映されています。 **Observe**（実際に起きたことの記録）: -- **[イベントストリーム](/ja/agenteye/event-stream)**: すべての実行のステップごとのリアルタイムトレイル（ツール呼び出し、モデル呼び出し、フック、エラー）。 -- **[セッション](/ja/agenteye/sessions)**: それらのイベントを1実行1行にまとめたもの。各実行はスコアリング可能で、Gitスタイルの実行グラフが付属。 -- **[パフォーマンスメトリクス](/ja/agenteye/telemetry)**: モデル・ツール・フックのサーフェスごとのレイテンシヒートマップとp50/p95/p99バイタル。テールスパイクが中央値から際立って見えます。 -- **[エラートラッキング](/ja/agenteye/error-tracking)**: 発生したすべての問題を1つのトリアージ画面で確認でき、発火したアラートからワンクリックでアクセス可能。 +- **[イベントストリーム](/ja/cloud/event-stream)**: すべての実行のステップごとのリアルタイムトレイル（ツール呼び出し、モデル呼び出し、フック、エラー）。 +- **[セッション](/ja/cloud/sessions)**: それらのイベントを1実行1行にまとめたもの。各実行はスコアリング可能で、Gitスタイルの実行グラフが付属。 +- **[パフォーマンスメトリクス](/ja/cloud/performance)**: モデル・ツール・フックのサーフェスごとのレイテンシヒートマップとp50/p95/p99バイタル。テールスパイクが中央値から際立って見えます。 +- **[エラートラッキング](/ja/cloud/errors)**: 発生したすべての問題を1つのトリアージ画面で確認でき、発火したアラートからワンクリックでアクセス可能。 -![Toolsの観測ページ: レイテンシヒートマップ、パーセンタイルバンド、24の時間ビンにわたるツール分布バー](/agenteye/images/tools.png) +![Toolsの観測ページ: レイテンシヒートマップ、パーセンタイルバンド、24の時間ビンにわたるツール分布バー](/cloud/images/tools.png) *各観測サーフェスにはスパークラインとp50/p95/p99バイタル、レイテンシヒートマップ、パーセンタイルバンドが表示されます。表示例: ツール。* **Analyze**（活動を洞察に変える）: -- **[クエリ](/ja/agenteye/queries)** と **[ダッシュボード](/ja/agenteye/dashboards)**: イベントと評価に対して保存済みSQLを実行し、組織スコープの共有ダッシュボードにグラフ化。 -- **[評価](/ja/agenteye/evaluations)**: 独自の評価サービスが生成する品質スコア。スコアごとの理由付きで表示。 -- **[監査](/ja/agenteye/audits)**: セッションをまたいで障害パターンを検出する定期調査。 -- **[アラート](/ja/agenteye/alerts)** と **[インシデント](/ja/agenteye/incidents)**: 通知を発するしきい値ルールと、トリアージのためのインシデントワークフロー。 +- **[クエリ](/ja/cloud/queries)** と **[ダッシュボード](/ja/cloud/dashboards)**: イベントと評価に対して保存済みSQLを実行し、組織スコープの共有ダッシュボードにグラフ化。 +- **[評価](/ja/cloud/evaluations)**: 独自の評価サービスが生成する品質スコア。スコアごとの理由付きで表示。 +- **[監査](/ja/cloud/audits)**: セッションをまたいで障害パターンを検出する定期調査。 +- **[アラート](/ja/cloud/alerts)** と **[インシデント](/ja/cloud/incidents)**: 通知を発するしきい値ルールと、トリアージのためのインシデントワークフロー。 **Interfaces**（自分のやり方でデータにアクセス）: -- **[CLI](/ja/agenteye/cli-and-agents)**: ターミナルやスクリプトからデプロイ全体を操作でき、コーディングエージェントに自然言語で任せることも可能。 -- **[AIアシスタント](/ja/agenteye/assistant)**: ダッシュボード内から自然言語でエージェントに関する質問ができます。 -- **REST API**: ダッシュボードとCLIで行えることはすべてREST APIで実行可能です。スコープ付きの[APIキー](/ja/agenteye/api-keys)で直接呼び出せ、イベントの取り込み、セッションと評価のクエリ、ダッシュボード・アラート・監査・ユーザー・キーの管理が可能です。Failproof AI Observability を自社ツールと連携させられます。 +- **[CLI](/ja/cloud/cli)**: ターミナルやスクリプトからデプロイ全体を操作でき、コーディングエージェントに自然言語で任せることも可能。 +- **[AIアシスタント](/ja/cloud/assistant)**: ダッシュボード内から自然言語でエージェントに関する質問ができます。 +- **REST API**: ダッシュボードとCLIで行えることはすべてREST APIで実行可能です。スコープ付きの[APIキー](/ja/cloud/access)で直接呼び出せ、イベントの取り込み、セッションと評価のクエリ、ダッシュボード・アラート・監査・ユーザー・キーの管理が可能です。FailproofAI Cloud を自社ツールと連携させられます。 **Admin**（チームのための運用）: -- **[APIキー](/ja/agenteye/api-keys)**: コレクター・ダッシュボード・アシスタント向けのスコープ付きトークン。 +- **[APIキー](/ja/cloud/access)**: コレクター・ダッシュボード・アシスタント向けのスコープ付きトークン。 - **ユーザー**: パスワードレスのメールベース認証と許可リスト。 - **設定**: モデルのコンテキストウィンドウオーバーライドを含む組織ごとの設定。 @@ -91,18 +91,18 @@ Failproof AI Observability は3つのコンセプト（**observe（観測）** - **ダッシュボード**: すべてを探索できる場所。 - **オプションサービス**: スコアリングサービス（評価）とAIアシスタントサービス（ダッシュボード内チャット）。 -ドキュメント全体で使用される用語（*event、session、evaluation、audit、finding、incident*）については、[コンセプト](/ja/agenteye/concepts)を参照してください。 +ドキュメント全体で使用される用語（*event、session、evaluation、audit、finding、incident*）については、[コンセプト](/ja/concepts)を参照してください。 --- -## Failproof AI Observability の入手方法 +## FailproofAI Cloud の入手方法 -Failproof AI Observability は Failproof AI のエンタープライズ製品で、ポリシーとガードレール製品であるFailproof AI Enforcementと連携して Failproof AI ブランドのもとで動作します。完全に自社環境内で稼働します。パッケージへのアクセス権をまだお持ちでない場合は、デモをリクエストしてください。セットアップをお手伝いします: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) までメールをお送りください。 +FailproofAI Cloud は Failproof AI のエンタープライズ製品で、ポリシーとガードレール製品であるFailproofAI guardrailsと連携して Failproof AI ブランドのもとで動作します。完全に自社環境内で稼働します。パッケージへのアクセス権をまだお持ちでない場合は、デモをリクエストしてください。セットアップをお手伝いします: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) までメールをお送りください。 --- ## 次のステップ -- [コンセプト](/ja/agenteye/concepts): Failproof AI Observability の用語を一か所にまとめて解説。 -- [オブザーバビリティ](/ja/agenteye/observability): エージェントの動作を実行ごとに追跡する。 -- [セキュリティ](/ja/agenteye/security): Failproof AI Observability がデータをどのように隔離し、管理下に置くか。 \ No newline at end of file +- [コンセプト](/ja/concepts): FailproofAI Cloud の用語を一か所にまとめて解説。 +- [オブザーバビリティ](/ja/cloud/overview): エージェントの動作を実行ごとに追跡する。 +- [セキュリティ](/ja/cloud/security): FailproofAI Cloud がデータをどのように隔離し、管理下に置くか。 \ No newline at end of file diff --git a/docs/ja/agenteye/telemetry.mdx b/docs/ja/cloud/performance.mdx similarity index 81% rename from docs/ja/agenteye/telemetry.mdx rename to docs/ja/cloud/performance.mdx index 20ac7d29..54886490 100644 --- a/docs/ja/agenteye/telemetry.mdx +++ b/docs/ja/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "モデル、ツール、またはhookが遅くなったりコス モデル、ツール、またはhookが遅くなったりコストが膨らんだ瞬間を即座に把握し、ユーザーが気づく前にテールレイテンシのスパイクを検出できます。3つの専用ページが生のタイミングデータをp50、p95、p99に変換し、一目で読み取ることができます。 -![レイテンシヒートマップ、パーセンタイルバンド、モデルごとのトークン数・コスト・コンテキストウィンドウ使用率を表示するModelsページ](/agenteye/images/models.png) +![レイテンシヒートマップ、パーセンタイルバンド、モデルごとのトークン数・コスト・コンテキストウィンドウ使用率を表示するModelsページ](/cloud/images/models.png) *Modelsページ：レイテンシヒートマップ、パーセンタイルバンド、モデルごとのトークン数・推定コスト・コンテキストウィンドウ使用率。* ## 平均値に最悪の実行を隠させない @@ -24,13 +24,13 @@ description: "モデル、ツール、またはhookが遅くなったりコス Modelsページ（上図）は、請求書が常に提起する2つの問いに答えます：どのモデルで、いくらか。共有レイテンシビューに加えて、**モデルごとのトークン消費量**、**推定コスト**、**コンテキストウィンドウ使用率**が追加されるため、プロンプトの急激な肥大化や差し迫ったコンパクションを驚かされる前に把握できます。 -Failproof AI Observabilityは一般的なモデルIDを自動的に認識します。ウィンドウサイズが正しくない場合や独自のプライベートモデルを使用している場合は、**Settings** の **model context windows** で修正または追加してください。使用率の表示もそれに従って更新されます。 +FailproofAI Cloudは一般的なモデルIDを自動的に認識します。ウィンドウサイズが正しくない場合や独自のプライベートモデルを使用している場合は、**Settings** の **model context windows** で修正または追加してください。使用率の表示もそれに従って更新されます。 ## Tools：遅いものと壊れているものを見分ける ツール呼び出しは遅いこともあれば、静かに失敗していることもあります。どちらなのかを、ログを掘り返してからではなく、数秒で知る必要があります。 -![共有レイテンシヒートマップとパーセンタイルバンドの横に成功・失敗の内訳とツール分布バーを表示するToolsページ](/agenteye/images/tools.png) +![共有レイテンシヒートマップとパーセンタイルバンドの横に成功・失敗の内訳とツール分布バーを表示するToolsページ](/cloud/images/tools.png) *Toolsページ：同じヒートマップとパーセンタイルバンド、さらに成功・失敗の内訳とツール分布バー。* 共有レイテンシビューに加えて、Toolsページには**成功・失敗の内訳**と**ツール分布バー**が追加されます。これにより、最も頻繁に使われているツールと、エラーバジェットを消費しているツールが一目でわかります。 @@ -39,14 +39,14 @@ Failproof AI Observabilityは一般的なモデルIDを自動的に認識しま ライフサイクルhookが実行を遅らせているとき、「hookが遅い」というだけでは対処できません。Hooksページは問題のある一つにたどり着く手助けをします。 -![共有ヒートマップとパーセンタイルバンドの上にhook名とトリガーイベントごとにレイテンシを分解表示するHooksページ](/agenteye/images/hooks.png) +![共有ヒートマップとパーセンタイルバンドの上にhook名とトリガーイベントごとにレイテンシを分解表示するHooksページ](/cloud/images/hooks.png) *Hooksページ：hook名とトリガーイベントごとに分解されたレイテンシ。* 同じレイテンシヒートマップとパーセンタイルバンドの上で、Hooksページは**hook名**と**トリガーイベント**ごとにアクティビティを分解します。これにより、注意が必要な単一のhookと単一のイベントに直接たどり着けます。 ## 関連項目 -- [イベントストリーム](/ja/agenteye/event-stream)：すべてのイベントのリアルタイム・カラーコード付きトレイル。 -- [セッション](/ja/agenteye/sessions)：イベントを実行ごとに1行にまとめ、実行グラフを開く。 -- [エラートラッキング](/ja/agenteye/error-tracking)：ダッシュボードが赤く表示するすべての問題を一元的にトリアージするサーフェス。 -- [ダッシュボード](/ja/agenteye/dashboards)：フリート全体のロールアップビュー。 \ No newline at end of file +- [イベントストリーム](/ja/cloud/event-stream)：すべてのイベントのリアルタイム・カラーコード付きトレイル。 +- [セッション](/ja/cloud/sessions)：イベントを実行ごとに1行にまとめ、実行グラフを開く。 +- [エラートラッキング](/ja/cloud/errors)：ダッシュボードが赤く表示するすべての問題を一元的にトリアージするサーフェス。 +- [ダッシュボード](/ja/cloud/dashboards)：フリート全体のロールアップビュー。 \ No newline at end of file diff --git a/docs/ja/agenteye/queries.mdx b/docs/ja/cloud/queries.mdx similarity index 85% rename from docs/ja/agenteye/queries.mdx rename to docs/ja/cloud/queries.mdx index 37093763..7c447469 100644 --- a/docs/ja/agenteye/queries.mdx +++ b/docs/ja/cloud/queries.mdx @@ -6,7 +6,7 @@ description: "エージェントデータに関するあらゆる質問を投げ エージェントデータに関するあらゆる質問を投げかけ、数秒で答えを得られます。Failproof AI のオブザーバビリティ機能は、イベントや評価に対してすぐに実行できる保存済みクエリのライブラリを提供しているため、空のSQLエディタではなく実際に動くサンプルからスタートできます。 -![保存済みクエリライブラリ: 組み込みプリセットとカスタムクエリが並んだグリッド表示](/agenteye/images/queries.png) +![保存済みクエリライブラリ: 組み込みプリセットとカスタムクエリが並んだグリッド表示](/cloud/images/queries.png) *`//queries` の保存済みクエリライブラリ: 組み込みプリセットとチームが保存したクエリが並んで表示されます。* @@ -22,7 +22,7 @@ description: "エージェントデータに関するあらゆる質問を投げ クエリを開くとSQLコンポーザーに読み込まれ、その場で調整して即座に結果を確認できます。エクスポートも往復も、他の誰かを待つ必要もありません。 -![保存済みクエリを実行中のSQLクエリコンポーザー。スキーマサイドバーとライブ結果グリッドが表示されている](/agenteye/images/query-lab.png) +![保存済みクエリを実行中のSQLクエリコンポーザー。スキーマサイドバーとライブ結果グリッドが表示されている](/cloud/images/query-lab.png) *SQLコンポーザー: 左側にクエリ、列名を調べる手間を省くスキーマサイドバー、そして下部にライブ結果グリッド。* @@ -43,14 +43,14 @@ agenteye query list # the same saved queries, from your terminal agenteye query run errs --arg prod # run one and print the rows (add --json to pipe it) ``` - フルコマンドセットは [CLI and agents](/ja/agenteye/cli-and-agents) を参照してください。 + フルコマンドセットは [CLI and agents](/ja/cloud/cli) を参照してください。 -- **AIアシスタントから。** SQLの書き方が分からない場合は、ダッシュボード内の [AIアシスタント](/ja/agenteye/assistant) に平易な言葉で質問するだけで、クエリを下書きしてライブラリに保存してくれます。 +- **AIアシスタントから。** SQLの書き方が分からない場合は、ダッシュボード内の [AIアシスタント](/ja/cloud/assistant) に平易な言葉で質問するだけで、クエリを下書きしてライブラリに保存してくれます。 保存済みクエリの実行は `queries:run` 権限によって管理されており、クエリの作成・削除に必要な権限とは分離されています。そのため、ライブラリの書き換えを許可することなく読み取りアクセスのみを付与できます。 ## 関連情報 -- [ダッシュボード](/ja/agenteye/dashboards): クエリ結果をorg全体で共有するグラフにピン留めする。 -- [AIアシスタント](/ja/agenteye/assistant): 平易な言葉で質問し、クエリを取得する。 -- [CLI and agents](/ja/agenteye/cli-and-agents): ターミナルから同じクエリを実行・保存する。 \ No newline at end of file +- [ダッシュボード](/ja/cloud/dashboards): クエリ結果をorg全体で共有するグラフにピン留めする。 +- [AIアシスタント](/ja/cloud/assistant): 平易な言葉で質問し、クエリを取得する。 +- [CLI and agents](/ja/cloud/cli): ターミナルから同じクエリを実行・保存する。 \ No newline at end of file diff --git a/docs/ja/agenteye/python-sdk.mdx b/docs/ja/cloud/sdk.mdx similarity index 91% rename from docs/ja/agenteye/python-sdk.mdx rename to docs/ja/cloud/sdk.mdx index faf7ccd6..9f0e5c07 100644 --- a/docs/ja/agenteye/python-sdk.mdx +++ b/docs/ja/cloud/sdk.mdx @@ -4,11 +4,11 @@ description: "AIエージェントが本番環境で何をしたかを正確に --- -AIエージェントが本番環境で何をしたかを正確に把握する: すべてのエージェント実行、ツール呼び出し、モデルリクエスト、フック、および人間の介入。Failproof AI Observability Python SDKは、エージェントコードの内側からその実行履歴を記録し、何が起きたかをデバッグ・監査・評価できるようにします。Failproof AI Observabilityでエージェントを観測したい場合にご利用ください。 +AIエージェントが本番環境で何をしたかを正確に把握する: すべてのエージェント実行、ツール呼び出し、モデルリクエスト、フック、および人間の介入。FailproofAI Cloud Python SDKは、エージェントコードの内側からその実行履歴を記録し、何が起きたかをデバッグ・監査・評価できるようにします。FailproofAI Cloudでエージェントを観測したい場合にご利用ください。 内部では、SDKが構造化イベントをローカルのJSONLファイルに書き込み、コレクターデーモンがそれらを自動的に収集してプラットフォームに送信します。これらのファイルを自分で管理する必要はありません。 -> **ヒント:** Failproof AI Observabilityを初めてお使いですか？このページはSDKイベントの完全なリファレンスです。 +> **ヒント:** FailproofAI Cloudを初めてお使いですか？このページはSDKイベントの完全なリファレンスです。
    @@ -26,7 +26,7 @@ SDKは公開パッケージインデックスではなく、プライベート python -c "import agenteye; print(agenteye.__version__)" ``` -コーディングエージェントに統合作業をすべて任せたい場合は、[Python SDK Agent Skill](/ja/agenteye/python-sdk-skill) をご利用ください。インストールパスを把握し、計装ポイントを計画・実装して、イベントが正しく届いているか検証します。 +コーディングエージェントに統合作業をすべて任せたい場合は、[Python SDK Agent Skill](/ja/cloud/agent-skills) をご利用ください。インストールパスを把握し、計装ポイントを計画・実装して、イベントが正しく届いているか検証します。 --- @@ -60,7 +60,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### 実際の呼び出しへの計装 -実際には、既存のエージェントコードをラップします。モデル呼び出しの前に `model_request`、後に `model_response` を配置することで、2つのイベントが実際のリクエストをまたぎ、Failproof AI Observabilityがペアとして関連付けられるようになります: +実際には、既存のエージェントコードをラップします。モデル呼び出しの前に `model_request`、後に `model_response` を配置することで、2つのイベントが実際のリクエストをまたぎ、FailproofAI Cloudがペアとして関連付けられるようになります: ```python import anthropic @@ -99,7 +99,7 @@ agenteye.event.model_response( ダッシュボードに届いたイベントは、タイプ別に色分けされ、環境・エージェント・セッションでフィルタリングできます: -![ライブイベントストリーム。イベントタイプ別に色分けされ、環境・エージェント・セッションでフィルタリング可能](/agenteye/images/events-stream.png) +![ライブイベントストリーム。イベントタイプ別に色分けされ、環境・エージェント・セッションでフィルタリング可能](/cloud/images/events-stream.png) --- @@ -153,9 +153,9 @@ SDKは明示的に渡したフィールドのみを記録します。プロン ## イベントリファレンス -ほとんどのイベントは相関IDを共有する開始/終了ペアで構成されています: `tool_use` と `tool_result` は `tool_call_id` を共有し、`hook_triggered` と `hook_completed` は `hook_id` を共有し、`human_wait` と `human_input` は `input_id` を共有します。開始イベントを発行し、処理を実行してから、同じIDで終了イベントを発行してください。Failproof AI Observabilityがペアを照合し `duration_ms` を自動計算するため、`duration_ms` を自分で渡す必要はありません。 +ほとんどのイベントは相関IDを共有する開始/終了ペアで構成されています: `tool_use` と `tool_result` は `tool_call_id` を共有し、`hook_triggered` と `hook_completed` は `hook_id` を共有し、`human_wait` と `human_input` は `input_id` を共有します。開始イベントを発行し、処理を実行してから、同じIDで終了イベントを発行してください。FailproofAI Cloudがペアを照合し `duration_ms` を自動計算するため、`duration_ms` を自分で渡す必要はありません。 -![セッションのgit形式の実行グラフとイベントタイムライン。ペアイベントから再構築され、ツール・モデル・フックの内訳パネルを表示](/agenteye/images/session-detail.png) +![セッションのgit形式の実行グラフとイベントタイムライン。ペアイベントから再構築され、ツール・モデル・フックの内訳パネルを表示](/cloud/images/session-detail.png) すべてのイベントメソッドに以下の2フィールドが必須です: @@ -429,5 +429,5 @@ agenteye.event.tool_use( ## 次のステップ -- [イベントストリーム](/ja/agenteye/event-stream): これらのイベントがリアルタイムで届く様子を、タイプ別の色分けと環境・エージェント・セッションによるフィルタリングで確認できます。 -- [セッション](/ja/agenteye/sessions): ペアイベントが各エージェント実行を実行グラフとタイムラインとしてどのように再構築するかを確認できます。 \ No newline at end of file +- [イベントストリーム](/ja/cloud/event-stream): これらのイベントがリアルタイムで届く様子を、タイプ別の色分けと環境・エージェント・セッションによるフィルタリングで確認できます。 +- [セッション](/ja/cloud/sessions): ペアイベントが各エージェント実行を実行グラフとタイムラインとしてどのように再構築するかを確認できます。 \ No newline at end of file diff --git a/docs/ja/cloud/security.mdx b/docs/ja/cloud/security.mdx new file mode 100644 index 00000000..8caefd93 --- /dev/null +++ b/docs/ja/cloud/security.mdx @@ -0,0 +1,68 @@ +--- +title: "セキュリティ" +description: "FailproofAI Cloudはプロダクション環境のエージェントの近くに配置されるため、プロンプト、ツールの入力、出力を参照します。" +--- + + +FailproofAI Cloudはプロダクション環境のエージェントの近くに配置されるため、プロンプト、ツールの入力、出力を参照します。このページでは、データの隔離・制御・管理方法について説明します。セキュリティレビューのためにFailproofAI Cloudを評価している場合は、まずここをお読みください。 + +--- + +## データはあなたの環境に留まる + +FailproofAI Cloudはセルフホスト型です。イベント、プロンプト、モデルのレスポンス、アナリティクスはすべて、あなた自身のデータベースおよび環境に保存されます。サードパーティのSaaSにデータが送信されることはなく、データは常にあなた自身のクラウドアカウント内に留まります。 + +--- + +## テナント分離 + +1つのFailproofAI Cloudインスタンスで複数の組織をホストできます。各組織はストレージ層で分離されており、UIではなくデータベース自体によって強制されます。 + +- 組織の運用データ（ユーザー、APIキー、ダッシュボード、保存済みクエリ）はその組織にスコープされており、組織をまたいだ読み取りはデータベース自体によってブロックされます。 +- 取り込まれたすべてのイベントには所有組織のスタンプが押されるため、ある組織のイベントを別の組織が読み取ることはできません。 + +すべてのダッシュボードルートは組織スラグ（`//…`）の配下にスコープされています。 + +--- + +## サインイン + +FailproofAI Cloudはパスワードレスのメールベースサインインを採用しています。フィッシングやリークの対象となるパスワードは存在しません。ユーザーがワンタイムコード（またはワンクリックマジックリンク）をリクエストすると、それがメールで送信され、短時間で失効します。サインインは**許可リスト**によって制御されており、あなたが許可したメールアドレス（またはドメイン）のみが認証できます。 + +![FailproofAI Cloudのサインイン画面。メールアドレスに使い捨てコードを送信します](/cloud/images/login.png) + +--- + +## APIキーによるスコープ付きアクセス + +すべてのクライアントは、きめ細かな最小権限を持つAPIキーで認証します。コレクターには`events:add`のみが必要です。ダッシュボードやアシスタント用のキーは読み取り専用にできます。破壊的な操作（削除、再生成）は、明示的に付与を選択する別個の権限です。 + +![APIキーページ：各キーの権限付与が読み取り・書き込み・破壊的スコープごとに色分けされています](/cloud/images/api-keys.png) + +管理者のブートストラップキーはセットアップ用に保持し、その他の用途には権限を絞ったキーを発行してください。詳しくは[APIキー](/ja/cloud/access)をご覧ください。 + +--- + +## 読み取り専用・承認ゲート付きアシスタント + +ダッシュボード内の[AIアシスタント](/ja/cloud/assistant)はデータに関する質問に回答しますが、設計上の制約があります。 + +- **デフォルトで読み取り専用**：実行されるSQLはガードを通過し、`SELECT`/`WITH`クエリのみ、単一ステートメント、行数上限付きで許可されます。 +- アシスタントが作成するもの（保存済みクエリ、ダッシュボードなど）はすべて**承認ゲート付き**：書き込みが行われる前に、あなたがすべての内容を確認・承認します。 +- アシスタントは**削除を行うことができません**。 + +そのため、チームメンバーが「今週最もエラーが多かったエージェントはどれか？」と質問して結果を活用できる一方、アシスタントが自律的にデータを変更・削除することはありません。 + +--- + +## 転送中のセキュリティ + +すべてのトラフィックはHTTPSで通信されます。TLSはあなた自身の証明書で終端するため、コレクターからサーバーへの通信、およびブラウザからサーバーへの通信は転送中に暗号化されます。 + +--- + +## 次のステップ + +- [概要](/ja/cloud/overview)：FailproofAI Cloudの全体像 +- [APIキー](/ja/cloud/access)：コレクター、ダッシュボード、アシスタントへのアクセスのスコープ設定 +- [オブザーバビリティ](/ja/cloud/overview)：FailproofAI Cloudがエージェントから収集する情報 \ No newline at end of file diff --git a/docs/ja/agenteye/sessions.mdx b/docs/ja/cloud/sessions.mdx similarity index 74% rename from docs/ja/agenteye/sessions.mdx rename to docs/ja/cloud/sessions.mdx index 1a9ac69f..c00f7892 100644 --- a/docs/ja/agenteye/sessions.mdx +++ b/docs/ja/cloud/sessions.mdx @@ -4,9 +4,9 @@ description: "1回の実行で発生したすべてのイベントを1行にま --- -実行が失敗した原因を推測するのはもう終わりです。Failproof AI Observability は、1回の実行で発生したすべてのイベントを読みやすい1行にまとめ、実行全体を git スタイルの図として数秒で把握できるように描画します。エージェントが何をどの順番で行ったか、ステップごとに正確に確認できます。 +実行が失敗した原因を推測するのはもう終わりです。FailproofAI Cloud は、1回の実行で発生したすべてのイベントを読みやすい1行にまとめ、実行全体を git スタイルの図として数秒で把握できるように描画します。エージェントが何をどの順番で行ったか、ステップごとに正確に確認できます。 -![セッション一覧：環境やエージェントをまたいで1実行1行で表示され、ステータスのバッジと評価スコアのバッジが付く](/agenteye/images/sessions-list.png) +![セッション一覧：環境やエージェントをまたいで1実行1行で表示され、ステータスのバッジと評価スコアのバッジが付く](/cloud/images/sessions-list.png) *1実行1行：ステータスのバッジで実行の結果が一目でわかり、評価器を接続するとスコアバッジも表示されます。* @@ -30,7 +30,7 @@ description: "1回の実行で発生したすべてのイベントを1行にま ## 実行全体を図として読む -![セッションの git スタイルの実行グラフとイベントタイムラインが並び、右側にはツール・モデル・フックの内訳パネルが表示される](/agenteye/images/session-detail.png) +![セッションの git スタイルの実行グラフとイベントタイムラインが並び、右側にはツール・モデル・フックの内訳パネルが表示される](/cloud/images/session-detail.png) *実行グラフ（左）はイベントタイムラインの隣に表示され、右側のパネルには実行で使用されたツール・モデル・フックおよびトークン消費量の内訳が表示されます。* @@ -38,7 +38,7 @@ description: "1回の実行で発生したすべてのイベントを1行にま 右側のパネルでは実行単位の内訳を確認できます。使用されたツールとモデル、発火したフック、トークン消費量が表示されます。「この実行のコストはなぜこんなに高いのか」「遅いツールはどれか」という疑問への答えが、その原因となったグラフのすぐ隣に置かれています。 -個々のイベントはアドレス指定が可能なため、「セッションの3分の2くらいのところ」という曖昧な説明ではなく、特定の瞬間へのリンクを共有できます。任意のイベントからリンクをコピーするか、[監査](/ja/agenteye/audits)の検出結果やエラーのリンクをたどると、そのイベントが選択・スクロールされた状態でセッションが開きます。非常に長い実行でも同様に機能します。タイムラインはブラウザへの負荷を考慮して一定範囲のウィンドウを読み込みますが、そのウィンドウ外を指すリンクでも、先頭に戻されることなく対象のイベントを見つけます。イベントが保持期間を過ぎている場合は、何も選択されないまま終わるのではなく、その旨がページに表示されます。 +個々のイベントはアドレス指定が可能なため、「セッションの3分の2くらいのところ」という曖昧な説明ではなく、特定の瞬間へのリンクを共有できます。任意のイベントからリンクをコピーするか、[監査](/ja/cloud/audits)の検出結果やエラーのリンクをたどると、そのイベントが選択・スクロールされた状態でセッションが開きます。非常に長い実行でも同様に機能します。タイムラインはブラウザへの負荷を考慮して一定範囲のウィンドウを読み込みますが、そのウィンドウ外を指すリンクでも、先頭に戻されることなく対象のイベントを見つけます。イベントが保持期間を過ぎている場合は、何も選択されないまま終わるのではなく、その旨がページに表示されます。 --- @@ -46,12 +46,12 @@ description: "1回の実行で発生したすべてのイベントを1行にま すべてのダッシュボードページは組織単位 (`//…`) でスコープされています。セッションは左サイドバーの **Observe** にあり、Events の隣に配置されています。リストの上部には日付範囲・環境・エージェント・セッションのフィルターが並んでいます。各行を1クリックで完全な実行グラフにアクセスできます。 -スコアバッジとスコア範囲によるフィルタリングを有効にするには、評価器を接続してください。詳細は [評価](/ja/agenteye/evaluations) を参照してください。 +スコアバッジとスコア範囲によるフィルタリングを有効にするには、評価器を接続してください。詳細は [評価](/ja/cloud/evaluations) を参照してください。 --- ## 関連情報 -- [イベントストリーム](/ja/agenteye/event-stream)：各セッションの元となる、ステップごとの生の履歴。 -- [評価](/ja/agenteye/evaluations)：評価器を接続して、各実行にフィルタリング可能なスコアバッジを付与する。 -- [テレメトリ](/ja/agenteye/telemetry)：エージェントの実行がこれらのセッションに取り込まれるまでの仕組み。 \ No newline at end of file +- [イベントストリーム](/ja/cloud/event-stream)：各セッションの元となる、ステップごとの生の履歴。 +- [評価](/ja/cloud/evaluations)：評価器を接続して、各実行にフィルタリング可能なスコアバッジを付与する。 +- [テレメトリ](/ja/cloud/performance)：エージェントの実行がこれらのセッションに取り込まれるまでの仕組み。 \ No newline at end of file diff --git a/docs/ja/concepts.mdx b/docs/ja/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/ja/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/ja/daemon.mdx b/docs/ja/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/ja/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/ja/dashboard.mdx b/docs/ja/dashboard.mdx index 6924e82c..da1c1868 100644 --- a/docs/ja/dashboard.mdx +++ b/docs/ja/dashboard.mdx @@ -69,7 +69,7 @@ Hermes と OpenClaw はユーザースコープでグループ化に使える作 4. **改善方法** — 推奨ポリシーごとの落ち着いた行リスト：白文字のポリシー名、1行の説明、右側にインストールコマンドとコピーボタン。セクションヘッダーには `enable all N → projected · `（すべての修正を適用した場合に到達するスコア）と表示され、`[install all]` ボタンは推奨されるすべてのポリシーの `failproofai policy add a b c …` コマンドをまとめてコピーします。 5. **次回に備えて** — 横並びの2枚のカード。左：リマインダーの設定（`3d` / `7d` / `14d` / `30d` のケイデンスピッカー。認証後に `/api/auth/reminder` を通じて永続化）。右：failproof 特典のアンロック — `invite a friend` はモーダルを開き、カンマ・スペース・改行で区切られた友人のメールアドレスリスト（1回の送信で最大10件）を受け取り、`/api/audit/invite` に POST します。これが api-server の `POST /v0/invite` に転送されます。api-server は `invite@failproof.ai` から受信者1人につき1通のメールを送信し、送信者を Cc に含めて `Reply-To` を設定します。これにより受信者は誰が招待したかがわかり、送信者も自分の受信トレイにコピーを受け取ります。匿名ユーザーは招待送信前に送信者のメールアドレスを確認するため、最初に `AuthDialog` にルーティングされます。資格・特典の付与は今後の対応となります。 -`failproofai audit` ランタイムによって駆動されます — 基盤となるスキャンエンジン、サポートされるフラグ、トランスクリプトごとのキャッシュの不変条件については [Audit CLI](/ja/cli/audit) を参照してください。ダッシュボードは最新の結果を `~/.failproofai/audit-dashboard.json`（モード `0600`、シングルスロット、新しい実行で上書き）にキャッシュするため、再訪問は即座に表示されます。**トランスクリプトごとのキャッシュと全体結果のキャッシュはどちらも、7日を超えた時点で読み取り時に破棄されます**。これによりダッシュボードが1週間前の古い結果を暗黙的に提供することはありません — TTL を過ぎると `/audit` は空の状態にフォールスルーし、新しい実行を促します。レポート下部の `[ re-audit now ]` をクリックすると `noCache: true` で `/api/audit/run` に POST されます — 再監査はトランスクリプトごとのキャッシュをバイパスし、キャッシュ済み結果を暗黙的に返すのではなく、すべてのトランスクリプトをゼロから再スキャンします — ダッシュボードは実行が完了するまで 1Hz で `/api/audit/status` をポーリングします。実行中は経過タイマーとともにピンクのプログレスストリップがビューポートの上部に固定表示され、成功時には新しい結果がページの全体リロードなしにその場で差し替えられます。再監査に失敗した場合、ストリップは赤に変わり、`RerunError.kind`（`timeout` / `network` / `post_failed`）に応じたコピーが表示され、前のレポートはそのまま維持されます。空の状態（キャッシュなしまたは期限切れ）とセッションゼロの状態（キャッシュは存在するがスキャンでトランスクリプトが見つからなかった）は個別に表示されます。 +`failproofai audit` ランタイムによって駆動されます — 基盤となるスキャンエンジン、サポートされるフラグ、トランスクリプトごとのキャッシュの不変条件については [Audit CLI](/ja/audit) を参照してください。ダッシュボードは最新の結果を `~/.failproofai/audit-dashboard.json`（モード `0600`、シングルスロット、新しい実行で上書き）にキャッシュするため、再訪問は即座に表示されます。**トランスクリプトごとのキャッシュと全体結果のキャッシュはどちらも、7日を超えた時点で読み取り時に破棄されます**。これによりダッシュボードが1週間前の古い結果を暗黙的に提供することはありません — TTL を過ぎると `/audit` は空の状態にフォールスルーし、新しい実行を促します。レポート下部の `[ re-audit now ]` をクリックすると `noCache: true` で `/api/audit/run` に POST されます — 再監査はトランスクリプトごとのキャッシュをバイパスし、キャッシュ済み結果を暗黙的に返すのではなく、すべてのトランスクリプトをゼロから再スキャンします — ダッシュボードは実行が完了するまで 1Hz で `/api/audit/status` をポーリングします。実行中は経過タイマーとともにピンクのプログレスストリップがビューポートの上部に固定表示され、成功時には新しい結果がページの全体リロードなしにその場で差し替えられます。再監査に失敗した場合、ストリップは赤に変わり、`RerunError.kind`（`timeout` / `network` / `post_failed`）に応じたコピーが表示され、前のレポートはそのまま維持されます。空の状態（キャッシュなしまたは期限切れ）とセッションゼロの状態（キャッシュは存在するがスキャンでトランスクリプトが見つからなかった）は個別に表示されます。 ### ポリシー diff --git a/docs/ja/architecture.mdx b/docs/ja/how-it-works.mdx similarity index 100% rename from docs/ja/architecture.mdx rename to docs/ja/how-it-works.mdx diff --git a/docs/ja/introduction.mdx b/docs/ja/introduction.mdx index 69e06712..ebc4cd7c 100644 --- a/docs/ja/introduction.mdx +++ b/docs/ja/introduction.mdx @@ -54,4 +54,4 @@ failproofai policies --install # enable policies (or skip — `failproofai` wi failproofai # launch the dashboard ``` -詳細な手順については、[はじめ方](/ja/getting-started)ガイドをご覧ください。 \ No newline at end of file +詳細な手順については、[はじめ方](/ja/quickstart)ガイドをご覧ください。 \ No newline at end of file diff --git a/docs/ja/policies.mdx b/docs/ja/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/ja/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/ja/getting-started.mdx b/docs/ja/quickstart.mdx similarity index 100% rename from docs/ja/getting-started.mdx rename to docs/ja/quickstart.mdx diff --git a/docs/ja/reference/files.mdx b/docs/ja/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/ja/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/ko/agent-support.mdx b/docs/ko/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/ko/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/ko/agenteye/cli-and-agents.mdx b/docs/ko/agenteye/cli-and-agents.mdx deleted file mode 100644 index 3a26ee42..00000000 --- a/docs/ko/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "Failproof AI Observability 배포 전체를 명령어 하나로 관리하세요." ---- - - -Failproof AI Observability 배포 전체를 명령어 하나로 관리하세요. 터미널을 벗어나지 않고 프로덕션을 점검하고, API 키를 발급하고, 인시던트를 확인할 수 있습니다. 이를 CI에 스크립트로 작성하거나, 코딩 에이전트가 일상 언어로 대신 처리하도록 할 수도 있습니다. - -```bash -pipx install agenteye -agenteye login --email you@example.com # 6자리 코드가 이메일로 전송됩니다 -agenteye --json sessions --since 24h # 최근 하루 동안의 모든 에이전트 실행, 최신순 정렬 -``` - -*`agenteye` CLI는 대시보드와 통신합니다. 이벤트를 서버로 전송하는 콜렉터와는 별개의 도구입니다.* - -## 배포 전체를 명령어 하나로 - -간단한 질문에 답하려고 탭을 여러 개 열어둘 필요가 없습니다. `agenteye` CLI는 단일 바이너리로 데이터를 읽고 조직을 관리합니다. 대시보드를 클릭해야 했던 작업이 이제 한 줄 명령어로 해결됩니다. 다시 실행하거나, 별칭으로 등록하거나, 런북에 붙여 넣을 수 있습니다. 네 가지 기능을 제공합니다: - -- **데이터 조회:** `sessions`, `events`, `evals`, `errors`를 시간, 에이전트, 환경별로 필터링합니다. -- **조직 관리:** `keys`, `users`, `settings`, `alerts`, `incidents`를 관리합니다. -- **분석 실행:** 저장된 SQL과 이벤트 데이터에 대한 임시 `query` 실행기를 사용합니다. -- **어시스턴트 질의:** `agent ask`로 대시보드에서 대화하는 것과 동일한 읽기 전용 분석가에게 질문합니다. - -`pipx`로 한 번 설치하고, 이메일로 전송된 6자리 코드로 로그인하면 준비 완료입니다. 세션은 약 하루 동안 유지되며, 만료되면 `agenteye login`을 다시 실행하세요. 브라우저를 열지 않고도 프로덕션 점검, 키 발급, 인시던트 트리아지 등을 처리할 수 있습니다: - -```bash -agenteye errors --since 24h --aggregate # 오류 유형별로 그룹화된 장애 현황 -agenteye incidents list --state firing # 현재 발생 중인 인시던트 -agenteye keys create ci --add events:add # 이벤트 푸시만 가능한 키 (비밀값은 한 번만 표시) -``` - -한 가지 알아둘 사항: `--json`과 같은 전역 옵션은 명령어 앞에 위치합니다. `agenteye --json sessions`가 올바른 형식이며, `agenteye sessions --json`은 올바르지 않습니다. - -## 스크립트 작성 및 CI 연동 - -모든 명령어에 `--json`을 사용할 수 있으며, 이것이 모든 것을 바꿉니다. 정제된 JSON은 stdout으로 출력되고, 사람을 위한 상태 메시지와 경고는 stderr로 출력됩니다. 따라서 `--json`으로 캡처한 결과를 불필요한 줄 없이 바로 `jq`에 파이프할 수 있습니다. 이 덕분에 CLI는 프롬프트에서 직접 사용하는 경우와 출력을 파싱하는 코딩 에이전트 모두에게 동일하게 유용합니다: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -무인 실행을 위해 설계되었습니다. 터미널이 연결되어 있지 않으면 확인 프롬프트가 자동으로 건너뛰어져 파이프라인이 중단되지 않습니다. 모든 명령어는 의미 있는 종료 코드를 반환합니다: `0` 성공, `4` 미로그인, `5` 권한 부족 (메시지에 해당 권한 명시, 예: `alerts:write`), `3` 대시보드 연결 불가. 스크립트에서 `4`를 감지하면 재인증을 수행하고, `5`를 감지하면 관리자에게 정확히 어떤 권한이 필요한지 알 수 있어 오류 원인을 모른 채 실패하는 상황을 방지합니다. - -## 코딩 에이전트가 일상 언어로 처리하도록 - -더 나아가, 이 플래그들을 직접 기억할 필요조차 없습니다. **CLI 스킬**은 `agenteye-cli`라는 작은 Agent Skill 폴더로, Claude Code나 Codex 같은 코딩 에이전트가 일상 언어 요청으로 CLI를 사용할 수 있도록 가르쳐 줍니다. "오늘 뭔가 문제가 있나요?"라고 물으면 에이전트가 적절한 명령어를 선택해 실행하고 결과를 설명해 줍니다. - -Claude Code의 경우, `agenteye-cli` 폴더를 `~/.claude/skills/`에 넣으면 자동으로 인식됩니다. Failproof AI Observability가 해당 폴더를 제공하며, 이미 설치된 CLI를 활용하는 것이므로 추가 설치가 필요하지 않습니다. 단, 로그인은 직접 먼저 해야 합니다. 스킬은 이메일 코드 로그인을 대신 완료할 수 없습니다. - -에이전트는 사용자 권한으로 CLI를 실행하므로, 로그인이 허용하는 모든 작업(읽기와 쓰기 모두)이 가능합니다: 키 생성, 설정 변경, 인시던트 해결. CLI의 "정말 하시겠습니까?" 프롬프트는 에이전트에게는 표시되지 않으므로, 스킬은 변경 작업 전에 정확한 명령어를 명시하고 사용자의 승인을 기다리도록 작성되어 있습니다. 사용자가 직접 확인 단계가 됩니다. - -```text -you Why did session run-001 fail? - -agent Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -읽기 작업은 즉시 실행되고, 쓰기 작업은 매번 사용자 확인을 기다립니다: - -```text -you Give CI a key that can only push events. - -agent This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you yes - -agent Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## 관련 문서 - -- [CLI 레퍼런스](/ko/agenteye/cli): 모든 명령어, 플래그, JSON 구조. -- [에이전트를 위한 CLI 레시피](/ko/agenteye/cli-recipes): 복사해서 쓸 수 있는 `jq` 패턴과 종료 코드 처리. -- [CLI 에이전트 스킬](/ko/agenteye/cli-skill): `agenteye-cli` 스킬 설치 및 실행. -- [AI 어시스턴트](/ko/agenteye/assistant): `agent ask`가 연결되는 대시보드 내 분석가. \ No newline at end of file diff --git a/docs/ko/agenteye/cli-skill.mdx b/docs/ko/agenteye/cli-skill.mdx deleted file mode 100644 index 33b87795..00000000 --- a/docs/ko/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Failproof AI Observability CLI 에이전트 스킬" -description: "코딩 에이전트에게 '오늘 뭔가 고장났나요?'라고 물어보면, 명령어를 외울 필요 없이 실시간 Failproof AI Observability 데이터를 바탕으로 답을 받을 수 있습니다." ---- - - -코딩 에이전트에게 *"오늘 뭔가 고장났나요?"* 라고 물어보면, 명령어를 외울 필요 없이 실시간 Failproof AI Observability 데이터를 바탕으로 답을 받을 수 있습니다. **Failproof AI Observability CLI 스킬** (`agenteye-cli`)은 *에이전트 스킬*입니다. Claude Code나 Codex 같은 코딩 에이전트가 필요할 때 불러오는 작은 인스트럭션 폴더로, *"CI에 이벤트만 푸시할 수 있는 키를 만들어줘"* 나 *"발생 중인 인시던트를 ack하고 나한테 할당해줘"* 같은 자연어 요청을 통해 [`agenteye` CLI](/ko/agenteye/cli)로 Observability 배포 환경을 조작하는 방법을 에이전트에게 가르쳐줍니다. - -이것은 **서비스나 별도의 바이너리가 아닙니다**. 배포할 것이 없습니다. 이미 설치된 CLI 위에서 동작하며, 에이전트가 `agenteye --json …`을 실행하고 깔끔한 JSON을 파싱한 뒤 산문 형태로 답변해줍니다. 에이전트가 할 수 있는 모든 것은 여러분이 직접 같은 명령어를 입력해도 할 수 있는 것들입니다. - ---- - -## 다른 Failproof AI Observability 인터페이스와의 관계 - -Failproof AI Observability는 동일한 데이터와 컨트롤에 접근할 수 있는 네 가지 방법을 제공합니다. 이들은 서로 보완적입니다. - -| 인터페이스 | 설명 | 실행 위치 | 사용 시점 | -|---|---|---|---| -| **[CLI](/ko/agenteye/cli)** | `agenteye` 명령어/플래그 레퍼런스 | 터미널 | 특정 명령어를 직접 실행하거나 스크립트로 만들 때 | -| **[CLI 레시피](/ko/agenteye/cli-recipes)** | `jq`/파이프라인 패턴 복붙 모음 | 터미널 / 스크립트 | CLI를 자동화에 연결할 때 | -| **CLI 스킬** (이 문서) | CLI에 자연어로 접근하는 진입점 | 워크스테이션의 코딩 에이전트 | 그냥 물어보고 에이전트가 명령어를 선택하게 하고 싶을 때 | -| **[Evaluator 스킬](/ko/agenteye/evaluator-skill)** | 스코어링 서비스를 설계하고 구축하는 형제 스킬 | 워크스테이션의 코딩 에이전트 | eval 점수를 *읽는* 게 아니라 *생성*하고 싶을 때 | -| **[Python SDK 스킬](/ko/agenteye/python-sdk-skill)** | 에이전트가 텔레메트리를 내보내도록 계측하는 형제 스킬 | 워크스테이션의 코딩 에이전트 | 이 스킬이 읽는 이벤트를 에이전트가 *생성*하게 하고 싶을 때 | -| **[대시보드 내 AI 어시스턴트](/ko/agenteye/assistant)** | 대시보드에 내장된 채팅 | 서버 사이드 (대시보드 내) | 대시보드에서 데이터를 Q&A 방식으로 조회하고 싶을 때 | - -스킬 자체에는 아무런 권한이 없습니다. 여러분의 말을 CLI 호출로 변환해줄 뿐이며, 호출은 여러분 권한으로 실행됩니다. - -```mermaid -flowchart TD - YOU["you: 'ack the firing incident'"] --> AGENT["coding agent (Claude Code / Codex)
    loads the agenteye-cli skill"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|your authenticated CLI session| API["Observability dashboard API"] -``` - -### 대시보드 내 AI 어시스턴트와의 차이: 중요한 구분 - -이 둘은 영향 범위가 매우 다른 별개의 도구입니다. - -- **대시보드 내 AI 어시스턴트** ([AI 어시스턴트](/ko/agenteye/assistant))는 에이전트 서비스를 기반으로 대시보드에 내장된 채팅입니다. **읽기 전용 + 승인 게이트 방식의 저작**: 저장된 쿼리와 대시보드를 초안으로 작성할 수 있지만, 모든 쓰기 작업은 사용자의 명시적 클릭 승인이 필요하며 절대 삭제하지 않습니다. `agent:use` 권한으로 게이트되어 있으며, 현재 보고 있는 조직의 데이터만 접근할 수 있습니다. -- **CLI 스킬**은 *여러분의* 워크스테이션에서 *여러분의* 코딩 에이전트 안에서 실행되며, `agenteye` CLI를 **여러분** 권한으로 구동합니다. API 키 생성/교체/비활성화, 조직 설정 변경, 인시던트 해결, 저장된 쿼리 삭제 등 **뮤테이션을 포함한 CLI의 전체 기능**을 수행할 수 있으며, CLI 로그인의 권한 범위 내에서만 제한됩니다. 해당 명령어를 직접 입력하는 것과 동일한 수준의 주의를 기울여 다루세요. - ---- - -## 사전 요구사항 - -1. **`agenteye` CLI 설치** 및 `PATH` 등록 ([CLI](/ko/agenteye/cli) 레퍼런스 참고: `pipx install agenteye`) -2. **대시보드 URL 설정** (`AGENTEYE_DASHBOARD_URL` 환경 변수 또는 에이전트가 `--base-url` 전달) -3. **로그인된 세션**: 먼저 직접 `agenteye login`을 실행하세요. 스킬은 이메일로 전송되는 일회용 코드 로그인을 대신 완료할 수 **없습니다**. 세션이 없거나 만료된 경우 (CLI 종료 코드 `4`) `agenteye login`을 실행하라고 안내합니다. - ---- - -## 다운로드 위치 - -스킬은 Failproof AI의 공개 스킬 컬렉션에 게시되어 있습니다. - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -접근에 아무런 제한이 없습니다. 저장소는 공개되어 있고, 스킬은 자체 자격 증명이 필요 없습니다. *여러분이* 로그인한 세션을 사용해 **공개** `agenteye` CLI를 *여러분의* 대시보드에 연결할 뿐이기 때문입니다. 별도로 요청할 필요가 없습니다. - -스킬은 별도 폴더로 제공되며, `pipx install agenteye` 패키지 **내부에 포함되어 있지 않으므로** 거기서 찾지 마세요. - -## 스킬 설치 - -가장 빠른 방법은 [`skills`](https://skills.sh) CLI를 사용하는 것입니다. 폴더를 가져와서 에이전트가 찾는 위치에 저장해줍니다. - -```bash -# Claude Code, 현재 프로젝트에만 적용 -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# 모든 프로젝트에 적용 (~/.claude/skills/에 설치) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex 사용 시 -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -이후 다른 스킬과 동일하게 관리할 수 있습니다. - -```bash -npx skills list -a claude-code # 설치된 스킬 목록 -npx skills update agenteye-cli # 최신 버전으로 업데이트 -npx skills remove agenteye-cli # 제거 -``` - -직접 설치하고 싶으신가요? 에이전트 스킬은 `SKILL.md`(와 선택적 레퍼런스 파일)가 들어 있는 폴더일 뿐이므로, 복사해서 사용해도 됩니다. - -- **Claude Code**: `agenteye-cli/` 폴더를 `~/.claude/skills/`(모든 프로젝트) 또는 `/.claude/skills/`(해당 저장소만)에 넣으세요. Claude Code가 자동으로 감지합니다. `/skills` 목록으로 확인하거나, 설명과 일치하는 질문을 직접 해보세요. -- **Codex (OpenAI)**: Codex도 동일한 `SKILL.md`를 읽습니다. 번들로 제공되는 `agents/openai.yaml`에 `allow_implicit_invocation: true`가 설정되어 있어 작업이 일치하면 Codex가 자동으로 스킬을 선택합니다. 명시적으로 호출하려면 `$agenteye-cli`를 사용하세요. - ---- - -## 안전 주의사항: 에이전트가 CLI를 실행할 때 뮤테이션은 확인 프롬프트가 나타나지 않습니다 - -> **경고:** 에이전트가 변경 작업을 수행하도록 허용하기 전에 반드시 읽으세요. - -`agenteye` CLI는 일반적으로 파괴적 작업 전에 *"정말 하시겠습니까?"* 라고 물어봅니다. 그러나 **터미널에 연결되어 있지 않을 때(코딩 에이전트가 실행하는 방식이 정확히 이 경우입니다)는 해당 확인을 자동으로 건너뛰며, `--json`도 마찬가지로 건너뜁니다.** 따라서 에이전트에게는 안전 프롬프트가 **표시되지 않습니다**. - -스킬은 이를 보완하도록 작성되어 있습니다. 실행할 정확한 명령어를 명시하고, 상태를 변경하기 전에 반드시 명시적으로 **OK를 받도록** 지시받았습니다. 이 규칙을 지켜주세요. 에이전트를 통해 Failproof AI Observability를 조작할 때는, *여러분이* 확인 단계입니다. 주의해야 할 상태 변경 명령어들: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- 쓰기 `incidents` 서브커맨드: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -**Observe** 하위의 모든 것(`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`)은 읽기 전용이며 아무것도 변경하지 않습니다. - -에이전트는 **여러분** 권한으로 동작하므로, 여러분의 로그인이 허용된 작업만 수행할 수 있습니다. 권한은 **조직별로** 확인됩니다 ([API 키](/ko/agenteye/api-keys) 참고). 권한이 없는 명령어는 정확한 권한 이름과 함께 종료 코드 `5`를 반환하므로, 에이전트가 불명확하게 실패하는 대신 어드민에게 무엇을 요청해야 하는지 정확히 알려줄 수 있습니다. - ---- - -## 요청 예시 - -실제 대화 흐름의 예시입니다. 읽기 작업과 확인을 기다리는 변경 작업이 포함되어 있습니다. - -```text -you ▸ Is anything broken in the last day? - -agent ▸ Running: agenteye --json errors --since 24h --aggregate - 3 error types in the last 24h. The biggest is TimeoutError - (18 hits, all on the checkout-agent), then RateLimited (5). - Want the failing sessions for TimeoutError? - -you ▸ Give CI a key that can only push events. - -agent ▸ This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you ▸ yes - -agent ▸ Done. Key "ci" created with events:add only. - The secret is shown only once, so store it now. I can't reprint it. -``` - -스킬은 자연어 의도를 적절한 `agenteye` 명령어로 매핑하고, 추측하지 않기 위해 먼저 유효한 값을 조회(`list `, `whoami`)한 뒤, 변경 전에 정확한 명령어를 명시합니다. 더 많은 예시: - -- *"최근 24시간 동안 고장났거나 실패한 게 있나요?"* → `errors --since 24h --aggregate` 후 세부 분석 -- *"세션 `run-001`이 왜 실패했나요?"* → `events --session-id run-001 --all` + `evals --session-id run-001` -- *"이번 주 품질 트렌드는 어떤가요?"* → `evals --aggregate --since 7d` 후 낮은 점수의 실행 드릴다운 -- *"CI에 이벤트만 푸시할 수 있는 키를 만들어줘."* → `keys create ci --add events:add` (명령어를 명시한 뒤 생성하고 일회성 시크릿 캡처) -- *"누가 접근 권한이 있나요? Dana를 읽기 전용으로 변경해줘."* → `users list` → 여러분에게 확인 후 `users update dana@… --permission-set read-only` -- *"발생 중인 인시던트를 ack하고 나한테 할당해줘."* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…` - -이 작업들 뒤에 있는 정확한 명령어, 플래그, JSON 형태는 [CLI](/ko/agenteye/cli) 레퍼런스와 [에이전트를 위한 CLI 레시피](/ko/agenteye/cli-recipes)를 참고하세요. - ---- - -## 다음 단계 - -- **[CLI](/ko/agenteye/cli)**: `agenteye`의 전체 명령어 및 플래그 레퍼런스 -- **[에이전트를 위한 CLI 레시피](/ko/agenteye/cli-recipes)**: `jq` 패턴 복붙 모음 및 종료 코드 처리 -- **[Evaluator 에이전트 스킬](/ko/agenteye/evaluator-skill)**: `agenteye evals`가 읽는 점수를 생성하는 evaluator를 구축하는 형제 스킬 -- **[Python SDK 에이전트 스킬](/ko/agenteye/python-sdk-skill)**: `agenteye`가 읽는 텔레메트리를 에이전트가 내보내도록 계측하는 형제 스킬 -- **[AI 어시스턴트](/ko/agenteye/assistant)**: 대시보드 내 어시스턴트 (이 터미널 스킬과 혼동하지 마세요) -- **[API 키](/ko/agenteye/api-keys)**: 스킬이 수행할 수 있는 작업 범위를 결정하는 조직별 권한 모델 \ No newline at end of file diff --git a/docs/ko/agenteye/codex-capture.mdx b/docs/ko/agenteye/codex-capture.mdx deleted file mode 100644 index eef114ab..00000000 --- a/docs/ko/agenteye/codex-capture.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "Codex 세션 캡처" -description: "팀의 로컬 OpenAI Codex 세션을 AgentEye에 일반 세션 및 이벤트로 수집합니다 — Codex 실행 방식을 변경할 필요가 없습니다." ---- - -엔지니어들은 이미 매일 OpenAI Codex를 사용하고 있습니다. Codex 세션 캡처는 해당 코딩 세션을 AgentEye에 일반 세션 및 이벤트로 가져와, 다른 모든 관측 데이터와 함께 검색하고, 재생하고, 평가할 수 있게 합니다. 이 기능은 [Python SDK](/ko/agenteye/python-sdk)를 보완합니다. SDK는 직접 작성한 에이전트를 계측하는 반면, Codex 세션 캡처는 팀이 이미 사용 중인 Codex 작업을 수집합니다 — 실행 방식을 변경할 필요가 없습니다. - -소형 백그라운드 수집기가 Codex의 로컬 세션 트랜스크립트를 작성되는 즉시 읽어 AgentEye로 전송합니다. 머신당 하나의 수집기로 모든 로컬 Codex 서피스를 한 번에 캡처할 수 있으며, 서피스별 설정은 필요하지 않습니다. - -동일한 수집기로 다른 에이전트도 캡처할 수 있습니다 — [OpenClaw](/ko/agenteye/openclaw-capture) 및 [Hermes](/ko/agenteye/hermes-capture)를 참조하세요. 사용하는 항목마다 활성화하면 단일 수집기로 여러 항목을 동시에 캡처할 수 있습니다. - ---- - -## 캡처 대상 - -**로컬**에서 실행되는 모든 Codex 서피스는 동일한 온디스크 세션 트랜스크립트를 생성하며, 수집기가 이를 모두 수집합니다: - -- Codex **CLI** 및 `codex exec` -- **VS Code / IDE 확장 프로그램** -- **데스크톱 앱** (로컬에서 세션을 실행할 경우) - -각 Codex 세션은 AgentEye [세션](/ko/agenteye/sessions)이 되고, 사용자 및 어시스턴트 메시지, 추론, 도구 호출, 도구 결과, 토큰 사용량은 해당하는 [이벤트](/ko/agenteye/event-stream)가 됩니다. 각 세션이 발생한 서피스(CLI, IDE, 데스크톱)가 기록되므로 구분이 가능합니다. - -> **클라우드 세션은 캡처되지 않습니다.** 데스크톱 앱은 점점 더 많은 세션을 Codex 클라우드에서 실행하며 머신에는 메타데이터만 저장합니다 — 읽을 수 있는 로컬 트랜스크립트가 없습니다. 로컬에서 실행된 세션만 캡처됩니다. - ---- - -## 활성화 방법 - -캡처는 활성화하기 전까지 비활성 상태입니다. `events:add` 권한이 있는 API 키([API keys](/ko/agenteye/api-keys) 참조)로 수집기를 설치하고, Codex 캡처를 활성화합니다: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -이 명령으로 수집기가 설치되고, 백그라운드 서비스로 등록되며, 캡처가 시작됩니다. 실행 여부를 확인합니다: - -```bash -agenteye-collector health -``` - -최초 실행 시 기존 Codex 세션이 한 번 백필되며, 이후 새로운 활동은 몇 초 이내에 스트리밍됩니다. Codex의 파일은 읽기만 할 뿐 수정, 이동, 삭제되지 않으며, 재시작 여부와 관계없이 각 세션은 정확히 한 번만 전송됩니다. - ---- - -## 확인 위치 - -캡처된 세션은 **Sessions**에, 이벤트는 **Events** 스트림에 표시됩니다. 다른 에이전트와 동일하게 표시되므로 [세션 재생](/ko/agenteye/sessions), [검색](/ko/agenteye/queries), [평가](/ko/agenteye/evaluations), [알림](/ko/agenteye/alerts) 모두 사용 가능합니다. Codex 에이전트로 필터링하면 해당 항목만 확인할 수 있습니다. - ---- - -## 개인정보 보호 - -Codex 트랜스크립트에는 명령 출력, 파일 내용, Codex가 읽거나 쓴 모든 내용을 포함한 전체 세션이 담겨 있으며, 시크릿 정보가 포함될 수 있습니다. 캡처된 세션은 그대로 전송되므로, AgentEye에 해당 콘텐츠를 중앙화하는 것이 적절한 머신과 팀에 대해서만 캡처를 활성화하고, 수집기에는 `events:add`만 범위로 지정된 키를 사용하세요. 데이터 격리 방법에 대한 자세한 내용은 [Security](/ko/agenteye/security)를 참조하세요. \ No newline at end of file diff --git a/docs/ko/agenteye/concepts.mdx b/docs/ko/agenteye/concepts.mdx deleted file mode 100644 index 088977cb..00000000 --- a/docs/ko/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "개념" -description: "Failproof AI Observability의 핵심 용어 — 이벤트, 세션, 평가, 감사, 발견 사항, 인시던트 — 를 한곳에서 정의합니다." ---- - - -이 페이지는 Failproof AI Observability에서 사용하는 용어를 정의합니다. 다른 가이드에서 낯선 용어를 만나면 여기에서 확인하세요. 처음부터 끝까지 읽을 필요는 없습니다. 훑어보거나, 특정 용어가 궁금할 때 다시 돌아오세요. - ---- - -## 데이터 모델 - -**이벤트(Event)** -데이터의 최소 단위입니다. 하나의 이벤트는 에이전트가 수행한 단일 단계를 기록합니다: `tool_use`, `model_request`, `hook_completed`, `error` 등. 에이전트는 [Python SDK](/ko/agenteye/python-sdk)를 통해 이벤트를 내보내며, **Events** 페이지에서 실시간으로 확인할 수 있습니다. - -**세션(Session)** -`session_id`로 식별되는 하나의 에이전트 실행 단위입니다. 세션은 동일한 id를 공유하는 모든 이벤트를 묶어 **Sessions** 페이지의 단일 행으로 표시하고, 세부 페이지에서는 실행 그래프로 나타냅니다. 세션은 보통 `agent_start`로 시작하고 `agent_end`로 종료됩니다. - -**에이전트(Agent)** -`agent_id`로 식별되는 실행 내 행위자입니다. 하나의 실행에 여러 에이전트가 관여할 수 있습니다. 예를 들어, 요약 서브 에이전트를 생성하는 플래너 에이전트가 있을 수 있습니다. 서브 에이전트는 `parent_id`를 가지며, Failproof AI Observability는 이를 기반으로 실행 그래프에서 각각의 레인에 표시합니다. - -**환경(Environment)** -실행이 발생한 위치를 나타내는 레이블입니다: `production`, `staging`, `dev`. SDK를 구성할 때 한 번 설정합니다. 거의 모든 대시보드 페이지에서 환경별로 필터링할 수 있습니다. - -**컨텍스트 윈도우 사용률(Context-window fill)** -응답이 모델의 컨텍스트 윈도우를 얼마나 사용했는지를 나타내는 백분율입니다. Failproof AI Observability는 인식된 모델의 `model_response` 이벤트에 이 값을 기록하므로, 프롬프트 증가와 임박한 컴팩션을 이벤트 스트림에서 바로 확인할 수 있습니다. - ---- - -## 품질 - -**평가(Evaluation)** -완료된 세션에 대해 사용자가 운영하는 스코어링 서비스가 생성하는 품질 점수입니다. 평가는 선택 사항입니다. 평가자를 연결하기 전까지는 세션이 기록되지만 점수는 매겨지지 않습니다. 각 평가에는 여러 개의 명명된 점수(예: `helpfulness`, `factuality`, `tool_efficiency`)가 포함될 수 있으며, 각각에 간단한 이유 설명이 붙습니다. [평가 스위트](/ko/agenteye/evaluation-suite)를 참고하세요. - -**점수 키(Score key)** -평가자가 보고하는 하나의 차원 이름으로, 예를 들어 `helpfulness`가 있습니다. 알림과 감사는 특정 점수 키를 시간에 따라 모니터링할 수 있습니다. - -**평가자(Evaluator)** -사용자의 스코어링 서비스입니다. Failproof AI Observability는 완료된 실행의 전사 내용을 평가자에게 POST하고, 반환된 점수를 저장합니다. 기본 평가자는 제공되지 않으며, 스코어링 로직은 사용자가 직접 구현합니다. - ---- - -## 실패 탐지 및 수정 - -**훅(Hook)** -에이전트 프레임워크가 단계 전후로 실행하는 가드레일 또는 부수 효과입니다: 콘텐츠 안전 검사, PII 제거, 예산 제한 등. 훅은 `outcome`(allow, deny, modify)이 포함된 `hook_triggered` / `hook_completed` 이벤트를 내보내며, 별도의 관찰 페이지를 가집니다. - -**알림 규칙(Alert rule)** -지정한 임계값을 지표가 초과할 때 실행되는 규칙입니다: 오류율, p95 레이턴시, 토큰 비용, 또는 평가자 점수 등. 규칙이 실행되면 인시던트가 생성되고 선택한 채널(이메일, Slack, 웹훅, 대시보드 내)로 알림이 전송됩니다. [알림](/ko/agenteye/alerts)을 참고하세요. - -**인시던트(Incident)** -알림 규칙이 실행될 때 생성되는 오픈 이슈입니다. 인시던트는 수명 주기(확인, 할당, 해결)와 모든 작업을 기록하는 활동 타임라인을 가집니다. 수동으로 직접 열 수도 있습니다. - -**감사(Audit)** -규칙을 별도로 정의하지 않은 실패 패턴을 찾기 위해 세션 전체에 걸쳐 로그를 주기적으로(시간별~주간) 분석하는 조사입니다: 오류 클러스터, 낮은 점수, 레이턴시 이상값, 도구 호출 루프, 완료되지 않은 실행 등. 알림이 이미 알고 있는 지표를 감시한다면, 감사는 다음에 무엇을 살펴봐야 할지 알려줍니다. [감사](/ko/agenteye/audits)를 참고하세요. - -**발견 사항(Finding)** -감사 실행에서 나온 순위가 매겨진 증거 기반의 결과입니다. 발견 사항은 패턴을 명명하고, 그 배후의 정확한 세션을 링크하며, 트리아지 수명 주기(확인, 해결, 음소거, 기각)를 가집니다. Failproof AI Observability는 실행이 반복되어도 이미 알려진 패턴은 새로 쌓이지 않고 업데이트되도록 발견 사항을 중복 제거합니다. - -**AI 어시스턴트(The AI assistant)** -사용자 자신의 데이터를 기반으로 에이전트에 대한 질문에 자연어로 답하는 대시보드 내 채팅입니다. 기본적으로 읽기 전용이며, 어시스턴트가 생성하는 것(저장된 쿼리, 대시보드)은 승인 절차를 거쳐야 하고, 삭제는 절대 할 수 없습니다. [AI 어시스턴트](/ko/agenteye/assistant)를 참고하세요. - ---- - -## 운영 - -**조직(Organization, 테넌트)** -격리된 워크스페이스입니다. 하나의 Failproof AI Observability 인스턴스는 각자의 사용자, 키, 데이터를 가진 여러 조직을 호스팅할 수 있습니다. 모든 대시보드 URL은 조직 슬러그(`//…`) 아래에 범위가 지정됩니다. - -**컬렉터(Collector)** -`agenteye-collector`는 각 에이전트 머신에서 실행되는 경량 데몬으로, SDK가 디스크에 기록하는 이벤트를 배치로 묶어 서버로 전송합니다. - -**API 키(API key)** -클라이언트를 서버에 인증하는 범위가 지정된 토큰입니다. 키는 세분화된 권한을 가집니다(예: 컬렉터를 위한 `events:add`, 대시보드 키를 위한 읽기 전용 범위). [API 키](/ko/agenteye/api-keys)를 참고하세요. - -**서버(Server)** -수집 및 API 서비스입니다. 이벤트를 수집하고, 데이터베이스에 운영 상태를 저장하며, 대시보드와 CLI를 제공합니다. - -**대시보드(Dashboard)** -웹 UI입니다. 모든 페이지는 조직 범위로 지정되며 서버의 API를 통해 데이터를 읽습니다. - ---- - -## 다음 단계 - -- [개요](/ko/agenteye/overview): 이 구성 요소들이 어떻게 맞물리는지 확인하세요. -- [Observability](/ko/agenteye/observability): 관찰 화면(Events, Sessions, Models, Tools, Hooks, Errors). \ No newline at end of file diff --git a/docs/ko/agenteye/evaluator-skill.mdx b/docs/ko/agenteye/evaluator-skill.mdx deleted file mode 100644 index 859070d7..00000000 --- a/docs/ko/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: "Failproof AI Observability 평가자 에이전트 스킬" -description: "코딩 에이전트가 설계와 구현을 모두 담당하여, '에이전트 품질이 가끔 떨어지는 것 같다'는 막연한 생각을 실제 배포된 스코어링 서비스로 만들어 드립니다." ---- - - -코딩 에이전트가 설계와 구현을 모두 담당하여, *"에이전트 품질이 가끔 떨어지는 것 같다"* 는 막연한 생각을 실제 배포된 스코어링 서비스로 만들어 드립니다. **Failproof AI Observability 평가자 스킬** (`agenteye-evaluator`)은 *Agent Skill*입니다. Claude Code나 Codex 같은 코딩 에이전트가 필요할 때 불러오는 소규모 지침 폴더로, 에이전트에게 *여러분의* 에이전트에서 추적할 가치가 있는 품질 지표를 파악하고, 그 지표를 평가하는 [평가자 서비스](/ko/agenteye/evaluation-suite)를 작성·테스트·배포하는 방법을 가르칩니다. - -이 스킬은 호스팅된 스코어러도, 업로드 레지스트리도, 플러그인 시스템도 **아닙니다**. 여러분의 평가자는 [Evaluation suite](/ko/agenteye/evaluation-suite) 가이드에 설명된 대로, 여러분의 인프라에서 운영되는 HTTP 서비스로 완전히 여러분의 소유입니다. 이 스킬은 에이전트가 그것을 잘 만들 수 있도록 가르칠 뿐이며, 스킬이 하는 모든 작업은 여러분이 직접 동일한 코드를 작성해서도 할 수 있습니다. - ---- - -## 가장 어려운 부분은 무엇을 평가할지 결정하는 것입니다 - -SDK 인터페이스는 간단합니다 — 데코레이터 하나와 모델 두 개 — 그리고 에이전트는 [계약](/ko/agenteye/evaluation-suite#http-contract)만으로도 이를 작성할 수 있습니다. 평가자가 실패하는 원인은 거기에 있지 않습니다. 실패의 원인은 잘못된 것을 측정하기 때문입니다. 잘못된 것을 측정하는 평가자는 없는 것보다 나쁩니다. 모두가 무시하게 되는 대시보드를 만들어낼 뿐입니다. - -그래서 이 스킬의 대부분은 코드가 작성되기 전 단계에 할애됩니다. 스킬은 에이전트가 여러분을 인터뷰하도록 합니다(*"잘 동작한 실행을 설명해 주세요. 이제 잘못된 실행을 설명해 주세요"*). 그런 다음 [`agenteye` CLI](/ko/agenteye/cli)를 통해 실제 세션을 가져와 처음부터 끝까지 읽습니다. 이 두 가지는 대개 서로 다른 그림을 보여주는데, 그 차이가 핵심입니다: 여러분이 측정하려는 것과 실제 트랜스크립트에서 지원 가능한 것 사이의 간극입니다. 이벤트에서 **계산 가능**하고 **변별력이 있는** 경우에만 지표로 살아남습니다 — 좋은 실행과 나쁜 실행 모두에서 0.9를 기록한다면, 아무것도 알려주지 못하므로 제외됩니다. - -결과물은 2~4개 지표와 그 근거가 담긴 제안서로, 코드 한 줄이 작성되기 전에 여러분의 승인을 받습니다. - -```mermaid -flowchart TD - YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent (Claude Code / Codex)
    loads the agenteye-evaluator skill"] - AGENT -->|"interview: what does good vs bad look like?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["your real sessions
    what actually happens"] - DATA --> DIMS["2-4 dimensions, you sign off"] - DIMS --> SVC["your evaluator service
    agenteye-evaluator SDK"] - SVC --> SCORES["scores land in the dashboard
    and agenteye evals"] -``` - ---- - -## 다른 평가 구성 요소와의 관계 - -평가(scoring)를 다루는 문서는 네 가지이며, 순서대로 서로 연결됩니다: - -| 페이지 | 내용 | 참조 시점 | -|---|---|---| -| **[Evaluations](/ko/agenteye/evaluations)** | 기능: 세션 그리드의 점수, 대시보드, 재평가 | 자동 평가가 무엇을 제공하는지 알고 싶을 때 | -| **[Evaluation suite](/ko/agenteye/evaluation-suite)** | HTTP 계약, SDK, 서버 환경 변수 | 직접 평가자를 구현하거나 디버깅할 때 | -| **평가자 스킬** (이 문서) | 스코어러 설계 *및* 구현을 위한 자연어 진입점 | "평가를 원한다"는 생각에서 실행 중인 서비스까지 가고 싶을 때 | -| **[CLI skill](/ko/agenteye/cli-skill)** | `agenteye` CLI를 위한 자연어 진입점 | 이미 보유한 점수를 *읽고* 싶을 때 | -| **[Python SDK skill](/ko/agenteye/python-sdk-skill)** | 에이전트 계측을 위한 자연어 진입점 | 에이전트가 아직 세션을 내보내지 않아 평가할 대상이 없을 때 | - -### CLI 스킬 대비: 생성 vs 읽기 - -두 스킬은 의도적으로 겹치지 않으며, 둘 다 설치하는 것이 일반적인 구성입니다 — 에이전트는 여러분의 요청에 따라 적절한 스킬을 선택합니다: - -- **`agenteye-evaluator`** (이 문서)는 점수를 *생성하는* 것을 구축합니다. 처음으로 점수가 생성되면 역할이 끝납니다. -- **[`agenteye-cli`](/ko/agenteye/cli-skill)** 는 이미 존재하는 점수를 읽습니다(`agenteye evals`). *"이번 주 품질이 떨어졌나요?"* 는 이 스킬이 답하는 질문이고, 이 문서의 스킬이 답하는 질문이 아닙니다. - ---- - -## 사전 요구 사항 - -1. **`agenteye` CLI가 설치되고 로그인된 상태** (`pipx install agenteye`, 이후 `agenteye login`). 스킬은 두 가지 용도로 CLI를 사용합니다: 설계 기반이 되는 실제 세션 가져오기, 그리고 마지막에 점수가 제대로 생성됐는지 확인하기. 로그인 계정에는 `events:read` 권한이 필요하고, 최종 확인을 위해 `evaluations:read` 권한도 필요합니다. CLI 스킬과 마찬가지로, 이메일로 전송되는 일회용 코드 로그인은 **자동으로 완료할 수 없습니다**. -2. **평가자가 실행될 공간.** 평가자는 이미지로 빌드되어 장기 실행 서비스로 운영되므로, 임시 파일이 아닌 실제 저장소가 필요합니다. 평가자는 평가 대상 에이전트와 별도의 저장소에 운영되는 경우가 많습니다 — 스킬은 기존 저장소를 찾아보고, 새로 생성하기 전에 확인을 요청합니다. -3. **`agenteye-evaluator` SDK 휠** — 에이전트가 `pip` 명령을 입력하기 전에 다음 섹션을 먼저 읽으세요. - ---- - -## 입수 방법 - -이 스킬은 Failproof AI의 공개 스킬 컬렉션에 게시되어 있습니다: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -저장소는 공개되어 있으며 스킬 자체에는 별도의 인증 정보가 필요 없습니다 — 스킬은 여러분이 로그인한 세션으로 `agenteye` CLI를 구동하고 *여러분의* 저장소에 코드를 작성할 뿐입니다. 이 스킬은 별도 폴더로 제공되며 `pipx install agenteye` 패키지에는 포함되어 있지 **않으니**, 그곳에서 찾지 마세요. - -## 스킬 설치 - -가장 빠른 방법은 [`skills`](https://skills.sh) CLI를 사용하는 것입니다. 폴더를 가져와 에이전트가 찾는 위치에 배치해 줍니다: - -```bash -# Claude Code, 이 프로젝트에만 적용 -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# 모든 프로젝트 (~/.claude/skills/에 설치) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codex 사용 시 -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -이후 다른 스킬과 동일하게 관리합니다: - -```bash -npx skills list -a claude-code # 설치된 스킬 목록 -npx skills update agenteye-evaluator # 최신 버전으로 업데이트 -npx skills remove agenteye-evaluator # 제거 -``` - -수동으로 설치하고 싶으신가요? Agent Skill은 `SKILL.md`(및 선택적 참조 파일)가 포함된 폴더에 불과하므로, 복사해서 사용해도 됩니다: - -- **Claude Code**: `agenteye-evaluator/` 폴더를 `~/.claude/skills/`(모든 프로젝트) 또는 `/.claude/skills/`(해당 저장소 전용)에 배치하세요. Claude Code가 자동으로 인식합니다 — `/skills` 목록으로 확인하거나, 평가를 요청해 보세요. -- **Codex (OpenAI)**: Codex도 동일한 `SKILL.md`를 읽습니다. 번들된 `agents/openai.yaml`에는 `allow_implicit_invocation: true`가 설정되어 있어, 작업 내용이 일치하면 Codex가 자동으로 스킬을 선택합니다. 명시적으로 호출하려면 `$agenteye-evaluator`를 사용하세요. - ---- - -## SDK는 공개 PyPI에 없습니다 - -> **경고:** 에이전트가 SDK를 설치하기 전에 반드시 읽으세요. - -스킬은 공개되어 있지만, 스킬이 구동하는 SDK는 그렇지 않습니다. `agenteye-evaluator`는 비공개 릴리스 아티팩트로만 제공되며, `agenteye`와 달리 공개 PyPI에서 **이름이 등록되어 있지 않습니다** — 따라서 `pip install agenteye-evaluator`를 그냥 실행하면 프로덕션 트랜스크립트를 읽는 서비스에 알 수 없는 패키지가 설치될 수 있습니다. 이는 오타의 문제가 아니라 공급망 보안 문제입니다. - -스킬은 이를 알고 있으며, 적용 가능한 첫 번째 단계부터 순서대로 설치를 시도합니다: AgentEye 저장소 내부라면 모노레포 소스, 그렇지 않으면 GitHub Releases의 비공개 릴리스 휠(접근 권한 필요), 둘 다 접근할 수 없으면 **즉흥적으로 처리하는 대신 Failproof AI 담당자에게 휠을 문의하라고 안내하고 중단합니다**. - -따라서 에이전트가 공개 PyPI에서 `pip install agenteye-evaluator`를 바로 제안한다면, 스킬이 로드되지 않은 것입니다. 즉시 중단하고 스킬이 설치되어 있는지 확인하세요. - ---- - -## 사용 가능한 요청 예시 - -실제 진행 과정은 막연한 요청으로 시작해서 코드 작성이 아닌 승인된 설계로 끝납니다: - -```text -you ▸ I want evals for our support bot. I think it's sometimes bad. - -agent ▸ Before I write anything: describe a run that went well, - and one that went badly. - -you ▸ Good: it answers from the docs. Bad: it makes stuff up, - or gives up and says "contact support". - -agent ▸ Running: agenteye --json sessions --status error --since 7d - Reading 5 sessions, including two you'd call bad. - - Your words say factuality and deflection. The transcripts - agree on deflection (2 of 5 end with a support hand-off, - and it's plain in model_response.content), but factuality - needs a ground truth your events don't carry. - - So I'd score three things: - - deflection rules, cheap, separates your two runs - - tool_efficiency rules, catches the retry loop in run-004 - - groundedness LLM judge: is every claim in the docs it retrieved? - - "factuality" would need ground truth we don't have; groundedness - is the closest thing your transcripts can actually support. - OK with those three names? Renaming later splits the history. - -you ▸ yes - -agent ▸ No evaluator in this repo. Should I scaffold one here, or do - you have one elsewhere? -``` - -여기서부터 에이전트는 규칙 기반 지표를 먼저 작성합니다(비용 없음, 즉각적, 결정적). 그런 다음 빈 세션이나 중단된 세션처럼 단순한 평가자를 충돌시킬 수 있는 실제 캡처된 세션에 대해 테스트하고, 주관적인 지표에만 LLM 판정을 사용합니다. 에이전트는 [디스패처의 제한 사항](/ko/agenteye/evaluation-suite#configuring-the-server) — 30초 요청 타임아웃과 배포 전체에서 동시 8건 처리 — 을 알고 있으므로, 판정이 안정적으로 완료되기 어렵다면 비용을 5배로 늘려가며 취소와 재시도를 반복하는 대신 `JobPending`으로 비동기 처리합니다. - -이후 배포를 완료하고, 두 개의 서버 환경 변수를 설정하며, `agenteye --json evals --session-id `로 점수가 실제로 생성됐는지 확인합니다. 점수 생성이 유일한 증거입니다. - ---- - -## 주의 사항 - -- **지표 이름은 사실상 영구적입니다.** 점수 키는 임의 문자열이고 플랫폼은 전송되는 모든 것의 추세를 추적하므로, 잘못된 선택을 사후에 교정할 방법이 없습니다. 나중에 이름을 바꾸면 히스토리가 분리됩니다: 이전 세션에는 이전 키가 유지되어 추세가 끊깁니다. 이것이 스킬이 코드를 작성하기 전에 명시적인 승인을 받는 이유입니다 — 그 프롬프트를 진지하게 받아들이세요. -- **픽스처는 실제 프로덕션 트랜스크립트입니다.** 실제 세션을 기반으로 설계한다는 것은 세션을 디스크로 가져온다는 의미이며, 고객 데이터가 포함될 수 있습니다. 스킬은 git에 커밋하기 전에 확인을 요청합니다. 확신이 없다면 `fixtures/`를 저장소 밖에 두고 각 개발자가 직접 가져오도록 하세요. -- **에이전트가 모든 트랜스크립트를 읽는 서비스를 작성하고 배포합니다.** CLI 로그인 권한 범위 내에서 여러분처럼 행동하지만, 프로덕션 데이터에 접근하는 다른 모든 코드와 동일하게 평가자를 검토하세요. - ---- - -## 다음 단계 - -- **[Evaluation suite](/ko/agenteye/evaluation-suite)**: 스킬이 구성하는 HTTP 계약, SDK, 서버 환경 변수. -- **[Evaluations](/ko/agenteye/evaluations)**: 점수가 생성된 후 표시되는 위치. -- **[CLI skill](/ko/agenteye/cli-skill)**: 스코어러를 구축하는 대신 결과를 읽기 위한 형제 스킬. -- **[CLI](/ko/agenteye/cli)**: 스킬이 설계 기반으로 삼는 세션 데이터의 명령어 참조. \ No newline at end of file diff --git a/docs/ko/agenteye/hermes-capture.mdx b/docs/ko/agenteye/hermes-capture.mdx deleted file mode 100644 index a30bb3ce..00000000 --- a/docs/ko/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Hermes 세션 캡처" -description: "팀의 Hermes 게이트웨이 세션 — Slack, Telegram, CLI, 예약 실행 — 을 일반 세션 및 이벤트로 AgentEye에 가져옵니다." ---- - -[Hermes](https://hermes-agent.nousresearch.com)는 팀원들이 이미 사용하는 어떤 채널에서든 — Slack, Telegram, CLI, 예약 실행 — 응답을 제공합니다. Hermes 세션 캡처는 이 모든 것을 AgentEye에 일반 세션 및 이벤트로 가져오므로, 팀이 매일 대화하는 어시스턴트도 직접 작성한 에이전트만큼 관찰 가능해집니다. - -소형 백그라운드 수집기가 Hermes의 로컬 세션 저장소를 작성 즉시 읽어 AgentEye로 전송합니다. 동작 방식은 [Codex](/ko/agenteye/codex-capture) 및 [OpenClaw](/ko/agenteye/openclaw-capture) 캡처와 동일하며, 하나의 수집기로 여러 에이전트를 동시에 캡처할 수 있습니다. - ---- - -## 캡처 항목 - -머신의 모든 Hermes 세션은 어느 채널에서 시작되었든 캡처됩니다. 각 세션은 AgentEye [세션](/ko/agenteye/sessions)이 되고, 사용자 및 어시스턴트 메시지, 도구 호출, 도구 결과는 해당하는 [이벤트](/ko/agenteye/event-stream)가 됩니다. - -세션이 시작된 채널 — Slack, Telegram, CLI, 예약 실행 — 은 세션에 기록되므로 구분하거나 하나씩 필터링할 수 있습니다. 함께 기록되는 정보로는 세션이 실행된 모델, 세션이 시작된 채팅 및 사용자, 그리고 세션이 다른 세션을 생성한 경우 부모 세션으로의 링크가 있습니다. - -세션은 Hermes가 시작하는 즉시 표시되며, 아직 아무 말도 나누지 않은 상태여도 마찬가지입니다. 한 턴의 응답과 도구 호출은 실제로 발생한 순서대로 유지됩니다. 세션이 종료되면 종료 이유, 비용, 사용된 토큰 수도 함께 확인할 수 있습니다. - ---- - -## 활성화 방법 - -캡처는 활성화하기 전까지 비활성 상태입니다. `events:add` 권한이 있는 API 키([API 키](/ko/agenteye/api-keys) 참조)로 수집기를 설치하고 Hermes 캡처를 활성화합니다. - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -이 명령으로 수집기가 설치되고, 백그라운드 서비스로 등록되며, 캡처가 시작됩니다. 실행 여부를 확인합니다. - -```bash -agenteye-collector health -``` - -동일한 머신에서 여러 에이전트를 캡처하려면? 같은 명령에 각 에이전트의 플래그를 추가하면 됩니다 — 예: `--hermes-enabled --codex-enabled`. - -첫 실행 시 기존 Hermes 세션이 한 번 백필되고, 이후 새로운 활동은 몇 초 이내에 스트리밍됩니다. Hermes의 데이터는 읽기만 할 뿐 수정하거나 삭제하지 않으며, 각 메시지는 재시작이 있더라도 한 번만 전송됩니다. - -`health` 명령은 수집기가 캡처한 모든 데이터가 실제로 AgentEye에 도달했는지도 알려줍니다. 배치 전송에 실패한 경우 삭제되지 않고 보존되어 재시도되며, 미전송 데이터가 남아 있는 동안은 비정상 상태로 보고됩니다 — 따라서 "정상"은 단순히 프로세스가 살아있다는 의미가 아니라 데이터가 실제로 도착했음을 의미합니다. - ---- - -## 확인 위치 - -캡처된 세션은 **Sessions**에, 이벤트는 **Events** 스트림에 표시되며, 다른 에이전트와 동일하게 취급됩니다 — 따라서 [세션 재생](/ko/agenteye/sessions), [검색](/ko/agenteye/queries), [평가](/ko/agenteye/evaluations), [알림](/ko/agenteye/alerts) 모두 이 세션에 적용됩니다. Hermes 에이전트로 필터링하면 해당 세션만 볼 수 있습니다. - ---- - -## 개인정보 보호 - -Hermes 세션에는 전체 대화 내용 — 명령 출력, 파일 내용, 에이전트가 읽거나 쓴 모든 것 — 이 포함되며, 비밀 정보가 담길 수 있습니다. 캡처된 세션은 있는 그대로 전송되므로, AgentEye에 해당 콘텐츠를 중앙화하는 것이 적절한 환경에서만 캡처를 활성화하고, 수집기에는 `events:add` 권한만 있는 키를 부여하세요. 데이터가 격리되어 보관되는 방식은 [보안](/ko/agenteye/security)을 참조하세요. \ No newline at end of file diff --git a/docs/ko/agenteye/observability.mdx b/docs/ko/agenteye/observability.mdx deleted file mode 100644 index 2349fb37..00000000 --- a/docs/ko/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "관찰" -description: "관찰 화면은 에이전트가 지금 무엇을 하고 있는지 실시간으로 확인하고 특정 실행을 상세히 살펴볼 수 있는 공간입니다." ---- - - -관찰 화면은 에이전트가 지금 무엇을 하고 있는지 실시간으로 확인하고 특정 실행을 상세히 살펴볼 수 있는 공간입니다. 이곳의 모든 정보는 실시간이며 조직 범위로 제공되고, 날짜 범위·환경·에이전트·세션 기준으로 필터링할 수 있어 "뭔가 이상한데"라는 느낌에서 정확한 실행 기록까지 수초 안에 도달할 수 있습니다. - -![환경, 에이전트, 세션 기준으로 필터링 가능하고 유형별로 색상이 구분된 실시간 이벤트 스트림](/agenteye/images/events-stream.png) - -각각의 페이지로 구성된 네 가지 화면: - -- **[이벤트 스트림](/ko/agenteye/event-stream)**: 모든 에이전트의 모든 실행에 대한 실시간 단계별 기록으로, 최신순으로 정렬됩니다. 조직의 홈 화면이자 트리아지의 첫 번째 출발점입니다. -- **[세션 및 실행 그래프](/ko/agenteye/sessions)**: 이벤트를 실행 단위 한 행으로 집계하고, 각 실행이 어떻게 전개되었는지를 git 스타일의 그림으로 보여줍니다. -- **[성능 메트릭](/ko/agenteye/telemetry)**: 모델, 도구, 훅에 대한 지연 시간 히트맵과 p50/p95/p99 지표를 제공하여 꼬리 스파이크가 중앙값과 어떻게 다른지 한눈에 파악할 수 있습니다. -- **[오류 추적](/ko/agenteye/error-tracking)**: 문제가 발생한 모든 항목을 한 화면에서 트리아지하고, 알림 발생에서 해당 실행까지 클릭 한 번으로 이동합니다. - -## 관련 항목 - -- [평가](/ko/agenteye/evaluations): 모든 실행의 품질을 점수화합니다. -- [알림](/ko/agenteye/alerts): 임의의 임계값을 호출 규칙으로 전환합니다. -- [감사](/ko/agenteye/audits): Failproof AI Observability가 자동으로 세션 전반의 실패 패턴을 찾아드립니다. -- [CLI 및 에이전트](/ko/agenteye/cli-and-agents): 터미널에서도 동일한 관찰 기능을 사용할 수 있습니다. \ No newline at end of file diff --git a/docs/ko/agenteye/openclaw-capture.mdx b/docs/ko/agenteye/openclaw-capture.mdx deleted file mode 100644 index b5d0c156..00000000 --- a/docs/ko/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "OpenClaw 세션 캡처" -description: "팀의 로컬 OpenClaw 세션을 AgentEye로 가져와 일반 세션 및 이벤트로 확인하세요 — OpenClaw 실행 방식은 전혀 변경할 필요가 없습니다." ---- - -팀이 [OpenClaw](https://docs.openclaw.ai)를 사용하고 있다면, OpenClaw 세션 캡처를 통해 해당 세션들을 AgentEye의 일반 세션 및 이벤트로 가져올 수 있습니다. 이를 통해 다른 관찰 데이터와 함께 검색, 재생, 평가가 가능합니다. 이 기능은 [Python SDK](/ko/agenteye/python-sdk)를 보완합니다. SDK는 직접 작성한 에이전트를 계측하는 반면, 이 기능은 팀이 이미 수행하는 OpenClaw 작업을 캡처합니다 — 실행 방식은 전혀 바꿀 필요가 없습니다. - -소규모 백그라운드 수집기가 OpenClaw의 로컬 세션 트랜스크립트를 작성 즉시 읽어 AgentEye로 전송합니다. [Codex 캡처](/ko/agenteye/codex-capture)와 동일한 방식으로 동작하며, 하나의 수집기로 두 가지를 동시에 캡처할 수 있습니다. - ---- - -## 캡처되는 내용 - -머신의 OpenClaw 설정에 구성된 모든 에이전트는 해당 머신의 수집기에 의해 캡처됩니다 — 에이전트별 별도 설정은 필요하지 않습니다. - -각 OpenClaw 세션은 AgentEye [세션](/ko/agenteye/sessions)이 되고, 사용자 및 어시스턴트 메시지, 툴 호출, 툴 결과는 대응하는 [이벤트](/ko/agenteye/event-stream)가 됩니다. - ---- - -## 활성화 방법 - -캡처는 기본적으로 비활성화 상태입니다. `events:add` 권한이 있는 API 키([API 키](/ko/agenteye/api-keys) 참고)로 수집기를 설치하고 OpenClaw 캡처를 활성화하세요: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -이 명령은 수집기를 설치하고, 백그라운드 서비스로 등록하며, 캡처를 시작합니다. 정상 동작 여부를 확인하려면: - -```bash -agenteye-collector health -``` - -동일한 머신에서 여러 에이전트를 캡처하려면? 각 에이전트의 플래그를 동일한 명령에 추가하면 됩니다 — 예: `--openclaw-enabled --codex-enabled`. - -최초 실행 시 기존 OpenClaw 세션이 한 번 백필되고, 이후 새 활동은 수 초 내에 스트리밍됩니다. OpenClaw의 파일은 읽기만 할 뿐 — 수정, 이동, 삭제는 절대 하지 않습니다 — 각 세션은 재시작이 있더라도 정확히 한 번만 전송됩니다. - ---- - -## 확인 위치 - -캡처된 세션은 **Sessions**에 표시되고, 이벤트는 **Events** 스트림에 표시됩니다 — 다른 에이전트 관찰 데이터와 동일합니다. 따라서 [세션 재생](/ko/agenteye/sessions), [검색](/ko/agenteye/queries), [평가](/ko/agenteye/evaluations), [알림](/ko/agenteye/alerts) 모두 해당 데이터에 적용됩니다. OpenClaw 에이전트로 필터링하면 해당 세션만 볼 수 있습니다. - ---- - -## 개인정보 보호 - -OpenClaw 트랜스크립트에는 커맨드 출력, 파일 내용, 에이전트가 읽거나 쓴 모든 내용을 포함한 전체 세션이 담겨 있으며, 비밀 정보가 포함될 수 있습니다. 캡처된 세션은 있는 그대로 전송되므로, AgentEye에 해당 내용을 중앙화하는 것이 적절한 머신 및 팀에 대해서만 캡처를 활성화하고, 수집기에는 `events:add` 권한만 가진 키를 사용하세요. 데이터 격리 방식에 대해서는 [보안](/ko/agenteye/security)을 참고하세요. \ No newline at end of file diff --git a/docs/ko/agenteye/python-sdk-skill.mdx b/docs/ko/agenteye/python-sdk-skill.mdx deleted file mode 100644 index cc960cb3..00000000 --- a/docs/ko/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,130 +0,0 @@ ---- -title: "Failproof AI Observability Python SDK Agent Skill" -description: "계측되지 않은 에이전트에서 가시적인 이벤트로: 코딩 에이전트가 계측 지점을 찾아내고, 작성하고, 정상적으로 반영됐음을 검증합니다." ---- - -코딩 에이전트에게 *"이 에이전트에 Failproof AI Observability를 추가해줘"* 라고 말하면, 에이전트가 루프를 읽고, 계측 위치를 파악하고, 코드를 작성하고, 작업을 완료로 선언하기 전에 이벤트를 검증합니다. - -**Python SDK 스킬** (`agenteye-python-sdk`)은 *Agent Skill*입니다. 이는 Claude Code나 Codex 같은 코딩 에이전트가 작업이 일치할 때 온디맨드로 로드하는 명령어 폴더입니다. 이 스킬은 에이전트에게 [Python SDK](/ko/agenteye/python-sdk) 사용법을 가르쳐줍니다. 라이브러리가 아니며 SDK의 작동 방식을 변경하지 않습니다. - -## 계측은 작성하기 쉽지만 조용히 잘못되기도 쉽습니다 - -SDK는 작습니다: 키워드 전용 인수를 사용하는 이벤트 메서드 13개가 전부입니다. 코딩 에이전트는 [Python SDK](/ko/agenteye/python-sdk) 레퍼런스를 읽고 1분 안에 그럴듯한 계측 코드를 만들어낼 수 있습니다. - -문제는 이 SDK가 잘못 사용해도 오류를 발생시키지 않는다는 점이며, 잘못된 계측은 대시보드를 열었을 때 비어 있다는 걸 발견하기 전까지는 올바른 계측과 완전히 똑같아 보입니다. 실제로 시간을 낭비하게 만드는 실수들은 모두 침묵입니다: - -| 실수 | 보이는 것 | -|---|---| -| `agent_start` 없음 | 모든 이벤트가 기록됨. 세션은 0. | -| 환경 설정 안 됨 | 모든 것이 동작하지만 `dev`에 기록됨. | -| `outcome="failure"` | 실행이 성공으로 표시됨 — `failed`, `error`, `timeout`, `rejected`만 집계됨. | -| 오타가 있는 필드 이름 | 새 필드로 저장됨. | -| 스레드 풀에서 이벤트 발행 | 조용히 드롭됨. | - -이 중 어떤 것도 오류를 발생시키지 않습니다. 어떤 것도 테스트에서 나타나지 않습니다. 모두 스킬에 포함되어 있으며, 이를 잡아내는 검사와 함께 명세로 기술되어 있습니다. - -## 작동 방식, 순서대로 - -스킬은 신중한 엔지니어라면 수행할 세 단계를 동일하게 실행합니다: - -1. **계획.** 에이전트 루프를 읽고, 오직 개발자만 답할 수 있는 두 가지 질문을 합니다: 하나의 실행이 무엇인지(`session_id`), 그리고 구별 가능한 행위자가 누구인지(`agent_id`). 코드를 작성하기 전에 이 사항들을 합의합니다. 나중에 변경하면 히스토리가 분리되고 트렌드가 깨지기 때문입니다. -2. **작성.** 모든 호출 위치에 identity를 전달하는 대신 실행당 한 번만 바인딩하고, 동시성에 안전한 방식을 선택합니다. 이는 중요한 세부 사항으로, 명백해 보이는 지름길은 두 개의 겹치는 실행을 하나의 세션에 조용히 섞어버립니다. -3. **검증.** 에이전트를 실행하고 결과 이벤트 파일을 읽어 `agent_start`가 존재하는지, 환경이 올바른지, 하나의 실행이 하나의 세션을 생성했는지 확인합니다. - -세 번째 단계가 사람들이 건너뛰는 단계입니다. SDK는 이벤트를 로컬 파일에 기록하므로, 완전한 통합은 서버, API 키, 네트워크 없이 노트북에서 검증할 수 있습니다. 바로 이것이 스킬이 이 단계를 고집하는 이유입니다. - -## 다른 스킬들과의 관계 - -세 가지 스킬, 명확한 역할 분담: - -| 스킬 | 사용 시점 | 영향 범위 | -|---|---|---| -| **Python SDK 스킬** (이 페이지) | 에이전트가 텔레메트리를 *발행*하게 하려고 할 때 — "observability 추가", "내 에이전트가 왜 안 보이지?" | 에이전트 레포에 코드를 작성. 아무것도 읽지 않음. | -| **[Evaluator 스킬](/ko/agenteye/evaluator-skill)** | 실행을 *평가*하려고 할 때 — "무엇을 측정해야 하지?" | 레포에 코드 작성, 텔레메트리 읽기 | -| **[CLI 스킬](/ko/agenteye/cli-skill)** | 발생한 일을 *조회*하거나 배포를 운영하려고 할 때 | 사용자 권한으로 CLI 실행, 변경 포함 | - -이 순서로 연결됩니다: 이 스킬이 이벤트를 흐르게 하고, evaluator가 점수를 매기고, CLI가 결과를 읽습니다. 에이전트가 세션을 발행하기 전까지는 평가할 것도, 읽을 것도 없습니다. 처음부터 시작한다면 여기서 시작하세요. - -## 사전 요구 사항 - -1. **Python 3.10+** 및 계측하려는 에이전트 코드베이스. -2. **SDK.** 공개 패키지 인덱스가 아닌 프라이빗 wheel로 고객에게 배포됩니다. 온보딩 과정에서 SDK를 획득하고 설치하는 방법을 안내합니다. 스킬은 설치 경로를 알고 있으며, 찾을 수 없는 경우 추측 대신 직접 물어봅니다. -3. **그 외 없음.** 대시보드 로그인, API 키, 네트워크가 필요 없습니다. 스킬은 SDK가 기록하는 이벤트 파일을 기준으로 검증하므로 오프라인에서도 작업을 완료하고 증명할 수 있습니다. - -## 스킬 가져오는 방법 - -스킬은 공개 [`FailproofAI/skills`](https://github.com/FailproofAI/skills) 컬렉션에 있습니다: - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -현재 프로젝트 대신 모든 프로젝트에 설치하려면 `-g`를 추가하고, 환경이 심볼릭 링크를 지원하지 않으면 `--copy`를 사용하세요. Codex의 경우 `-a codex`를 전달하세요. - -## 수동 설치 - -Agent Skills는 `SKILL.md`와 참조 파일들을 포함하는 폴더입니다. 설치 도구를 사용하지 않으려면: - -- **Claude Code**: `agenteye-python-sdk/` 폴더를 `~/.claude/skills/`(모든 프로젝트) 또는 `/.claude/skills/`(해당 레포만)에 복사합니다. Claude Code가 자동으로 인식합니다 — `/skills` 목록을 확인하거나, 일치하는 내용을 질문해보세요. -- **Codex**: Codex는 동일한 `SKILL.md`를 읽습니다. 번들된 `agents/openai.yaml`이 `allow_implicit_invocation: true`로 설정되어 있어 작업이 일치하면 자동 선택됩니다. 그렇지 않으면 `$agenteye-python-sdk`로 호출하세요. - -계측하려는 코드가 있는 **레포지토리에서** 에이전트를 실행하세요 — 스킬은 무엇을 제안하기 전에 에이전트 루프를 먼저 읽습니다. - -## 세션 예시 - -```text -you ▸ 이 에이전트에 Failproof AI Observability를 추가해줘. - -agent ▸ 루프를 확인했습니다 — tools.py에 도구 디스패처 하나, llm.py에 - Anthropic 래퍼 하나. 코드 작성 전에 두 가지 질문이 있습니다. - - 하나의 실행이란 무엇인가요? 요청을 동시에 처리하므로 ctx.request_id에 - 이미 있는 요청 id를 사용하는 게 좋을 것 같습니다 — 그러면 여기서의 - 세션과 로그의 요청이 동일한 문자열이 됩니다. - - 그리고 행위자는 누구인가요? planner 경로와 researcher 경로가 보입니다. - 별도로 레이블링할까요, 아니면 하나의 에이전트로 볼까요? - -you ▸ 요청 id가 맞아. 하나의 에이전트야, "assistant"로 불러. - -agent ▸ 완료했습니다. 디스패처와 LLM 클라이언트를 래핑했고, agent_start와 - agent_end가 에러 경로를 포함해 handle_request를 감쌉니다. - Identity에는 컨텍스트 로컬을 사용했습니다 — 디스패처가 도구를 스레드 - 풀에서 실행하는데, 더 간단한 방법을 쓰면 동시 실행 결과가 하나의 세션에 - 섞여도 아무 경고가 없었을 겁니다. - - 실제 실행으로 검증했습니다: 4개 파일에 걸쳐 11개 이벤트 — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, 요청당 하나의 - 세션, 지속 시간 짝 맞춤. - - 하지 않은 것 하나: batch.py의 워커 풀이 executor에 직접 제출하므로 - 거기서 발행된 이벤트는 드롭됩니다. 그것도 수정할까요? -``` - -주목할 패턴: 제안하기 전에 코드를 먼저 읽었고, 개발자만 답할 수 있는 질문만 했으며, 이미 갖고 있던 id를 재사용하고, 스레드 풀을 확인했기 때문에 동시성에 안전한 방식을 선택했고, 성공을 선언하는 대신 **실제 이벤트를 읽어 검증**한 뒤 — 조용히 실패할 것을 알고 있는 곳을 표시했습니다. - -## 질문할 수 있는 것들 - -- *"내 에이전트가 왜 대시보드에 안 보이지?"* → 단계적으로 확인합니다: 이벤트가 기록되고 있는지, `agent_start`가 있는지, 환경이 올바른지, 수집기가 같은 위치를 읽고 있는지. -- *"모든 것이 dev에 기록되고 있어."* → 환경이 설정되지 않았거나, 이후 호출에서 재설정되었습니다. -- *"토큰 추적을 추가해줘."* → LLM 래퍼를 찾아 모델, 중지 이유, 사용량을 기록합니다. -- *"서브 에이전트도 계측해줘."* → 하나의 세션, 구별되는 에이전트 레이블, 부모 아래 중첩됩니다. -- *"계측 테스트를 작성해줘."* → SDK가 임시 디렉터리를 가리키게 하고 기록된 이벤트를 어서트합니다. - -## 주의할 점 - -**검증 단계를 실행하게 하세요.** 이 스킬을 가치 있게 만드는 단계는 마지막 단계입니다 — 에이전트를 실행하고 이벤트를 다시 읽는 것. 계측을 작성하고 멈추는 에이전트는 쉬운 절반만 한 것이며, 조용히 실패하는 절반이 나머지입니다. - -**코드 전에 이름을 합의하세요.** `session_id`와 `agent_id`는 모든 화면이 그룹화하는 기준 축입니다. 나중에 이름을 바꾸면 히스토리가 분리됩니다: 이전 실행은 옛 레이블을 유지하고 트렌드가 깨집니다. 스킬이 질문할 것이고, 답변은 잠깐의 생각을 충분히 투자할 가치가 있습니다. - -**에이전트가 공개 인덱스에서 SDK를 설치하자고 제안한다면, 스킬이 로드되지 않은 것입니다.** SDK는 프라이빗으로 배포됩니다. 그 제안은 코딩 에이전트가 스킬을 따르지 않고 추측하고 있다는 확실한 신호입니다 — 거기서 멈추고 스킬이 설치됐는지 확인하세요. - -그 외에는 영향 범위가 작습니다: 작업 디렉터리에 코드를 작성하고 지정한 위치에 이벤트 파일을 작성합니다. 배포에서는 아무것도 읽지 않고 변경하지도 않습니다. - -## 다음 단계 - -- **[Python SDK](/ko/agenteye/python-sdk)**: 이 스킬이 자동화하는 것의 배경이 되는 완전한 이벤트 레퍼런스 — 모든 이벤트 타입과 필드. -- **[Sessions](/ko/agenteye/sessions)**: 이벤트가 기록된 후 계측이 생성하는 것. -- **[Evaluator Agent Skill](/ko/agenteye/evaluator-skill)**: 실행이 기록되기 시작한 후 다음 단계 — 평가. -- **[CLI Agent Skill](/ko/agenteye/cli-skill)**: 텔레메트리 결과 조회. \ No newline at end of file diff --git a/docs/ko/agenteye/security.mdx b/docs/ko/agenteye/security.mdx deleted file mode 100644 index 76a950c7..00000000 --- a/docs/ko/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "보안" -description: "Failproof AI Observability는 프로덕션 에이전트 가까이에서 동작하도록 설계되어 있으며, 프롬프트, 도구 입력값, 출력값을 모두 처리합니다." ---- - - -Failproof AI Observability는 프로덕션 에이전트 가까이에서 동작하도록 설계되어 있으며, 프롬프트, 도구 입력값, 출력값을 모두 처리합니다. 이 페이지에서는 해당 데이터를 격리하고, 통제하며, 여러분의 손에 유지하는 방법을 설명합니다. 보안 검토를 위해 Failproof AI Observability를 평가 중이라면 여기서 시작하세요. - ---- - -## 데이터는 여러분의 환경에 보관됩니다 - -Failproof AI Observability는 자체 호스팅 방식입니다. 이벤트, 프롬프트, 모델 응답, 분석 데이터는 모두 여러분 자신의 환경에 있는 데이터베이스에 저장됩니다. 서드파티 SaaS에 데이터가 전송되거나 저장되지 않으며, 모든 데이터는 여러분의 클라우드 계정 내에 유지됩니다. - ---- - -## 테넌트 격리 - -하나의 Failproof AI Observability 인스턴스에서 여러 조직을 호스팅할 수 있으며, 각 조직은 스토리지 계층에서 격리됩니다. 이 격리는 UI가 아닌 데이터베이스 수준에서 강제됩니다. - -- 조직의 운영 데이터(사용자, 키, 대시보드, 저장된 쿼리)는 해당 조직 범위로 한정되며, 조직 간 데이터 읽기는 데이터베이스 자체에서 차단됩니다. -- 수집된 모든 이벤트에는 소유 조직 정보가 기록되므로, 한 조직의 이벤트를 다른 조직에서 절대 읽을 수 없습니다. - -모든 대시보드 라우트는 org 슬러그(`//…`) 하위에 범위가 지정됩니다. - ---- - -## 로그인 - -Failproof AI Observability는 비밀번호 없는 이메일 기반 로그인을 사용합니다. 피싱하거나 유출될 비밀번호 자체가 없습니다. 사용자가 일회용 코드(또는 원클릭 매직 링크)를 요청하면 이메일로 전송되며, 짧은 시간 내에 만료됩니다. 로그인은 **허용 목록**으로 제한됩니다. 여러분이 허용한 이메일 주소(또는 도메인)만 인증할 수 있습니다. - -![이메일로 일회용 코드를 전송하는 Failproof AI Observability 로그인 화면](/agenteye/images/login.png) - ---- - -## API 키를 이용한 범위 기반 접근 제어 - -모든 클라이언트는 세분화된 최소 권한을 가진 API 키로 인증합니다. 수집기는 `events:add` 권한만 필요하고, 대시보드 또는 어시스턴트 키는 읽기 전용으로 설정할 수 있습니다. 삭제, 재생성과 같은 파괴적인 작업은 별도의 권한으로 관리하며, 여러분이 직접 부여 여부를 결정합니다. - -![각 키의 권한 부여 현황을 읽기, 쓰기, 파괴적 범위별로 색상 구분하여 표시하는 API 키 페이지](/agenteye/images/api-keys.png) - -관리자 부트스트랩 키는 설정용으로만 보관하고, 그 외 모든 용도에는 제한된 키를 발급하세요. [API 키](/ko/agenteye/api-keys) 문서를 참고하세요. - ---- - -## 읽기 전용, 승인 기반 어시스턴트 - -대시보드 내 [AI 어시스턴트](/ko/agenteye/assistant)는 여러분의 데이터를 기반으로 질문에 답변하지만, 설계상 다음과 같은 제약이 있습니다. - -- **기본적으로 읽기 전용**입니다. 어시스턴트의 SQL은 `SELECT`/`WITH` 쿼리만 허용하고, 단일 구문으로 제한되며, 행 수 상한이 적용되는 가드를 통해 실행됩니다. -- 어시스턴트가 생성하는 모든 것(저장된 쿼리, 대시보드)은 **승인 기반**으로 처리됩니다. 모든 쓰기 작업은 실행 전에 여러분이 검토하고 승인해야 합니다. -- 어시스턴트는 **절대 삭제할 수 없습니다**. - -따라서 팀원이 "이번 주에 가장 많이 오류가 발생한 에이전트는 무엇인가요?"라고 묻고 결과를 활용하더라도, 어시스턴트가 스스로 데이터를 변경하거나 삭제하는 것은 불가능합니다. - ---- - -## 전송 중 보안 - -모든 트래픽은 HTTPS를 통해 전송됩니다. 여러분이 직접 인증서로 TLS를 종료하므로, 수집기-서버 간 및 브라우저-서버 간 트래픽은 전송 중 암호화됩니다. - ---- - -## 다음 단계 - -- [개요](/ko/agenteye/overview): Failproof AI Observability의 전체 구조를 확인하세요. -- [API 키](/ko/agenteye/api-keys): 수집기, 대시보드, 어시스턴트에 대한 접근 범위를 설정하세요. -- [관찰 가능성](/ko/agenteye/observability): Failproof AI Observability가 에이전트에서 수집하는 정보를 확인하세요. \ No newline at end of file diff --git a/docs/ko/cli/audit.mdx b/docs/ko/audit.mdx similarity index 100% rename from docs/ko/cli/audit.mdx rename to docs/ko/audit.mdx diff --git a/docs/ko/cli/backfill.mdx b/docs/ko/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/ko/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/ko/cli/config.mdx b/docs/ko/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/ko/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/ko/cli/flush.mdx b/docs/ko/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/ko/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/ko/cli/harness.mdx b/docs/ko/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/ko/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/ko/cli/migrate.mdx b/docs/ko/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/ko/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/ko/cli/uninstall.mdx b/docs/ko/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/ko/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/ko/cli/update.mdx b/docs/ko/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/ko/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/ko/agenteye/api-keys.mdx b/docs/ko/cloud/access.mdx similarity index 95% rename from docs/ko/agenteye/api-keys.mdx rename to docs/ko/cloud/access.mdx index 8fe04b6a..57c08147 100644 --- a/docs/ko/agenteye/api-keys.mdx +++ b/docs/ko/cloud/access.mdx @@ -1,10 +1,10 @@ --- title: "API Keys" -description: "API keys는 Failproof AI Observability 서버에 접근할 수 있는 대상을 제어하므로, 컬렉터는 읽기 또는 관리자 권한 없이도 이벤트를 전송할 수 있습니다." +description: "API keys는 FailproofAI Cloud 서버에 접근할 수 있는 대상을 제어하므로, 컬렉터는 읽기 또는 관리자 권한 없이도 이벤트를 전송할 수 있습니다." --- -API keys는 Failproof AI Observability 서버에 접근할 수 있는 대상을 제어하므로, 컬렉터는 읽기 또는 관리자 권한 없이도 이벤트를 전송할 수 있습니다. 각 키는 하나 이상의 권한을 가지며, 각 권한은 특정 서버 라우트를 제어합니다. 작업에 필요한 최소한의 권한만 부여하세요. 대부분의 배포 환경에서는 세 가지 종류의 키만 생성합니다. +API keys는 FailproofAI Cloud 서버에 접근할 수 있는 대상을 제어하므로, 컬렉터는 읽기 또는 관리자 권한 없이도 이벤트를 전송할 수 있습니다. 각 키는 하나 이상의 권한을 가지며, 각 권한은 특정 서버 라우트를 제어합니다. 작업에 필요한 최소한의 권한만 부여하세요. 대부분의 배포 환경에서는 세 가지 종류의 키만 생성합니다. ## 대부분의 배포 환경에서 필요한 3가지 키 @@ -82,7 +82,7 @@ API keys는 Failproof AI Observability 서버에 접근할 수 있는 대상을 이 권한들은 대시보드의 **Users** 페이지를 지원하며, 각 멤버의 부여된 스코프가 칩으로 표시됩니다: -![Users 페이지: 각 대시보드 사용자의 이메일, 부여된 권한, 편집/비활성화 컨트롤이 포함된 카드](/agenteye/images/users.png) +![Users 페이지: 각 대시보드 사용자의 이메일, 부여된 권한, 편집/비활성화 컨트롤이 포함된 카드](/cloud/images/users.png) ### 운영 설정 @@ -91,7 +91,7 @@ API keys는 Failproof AI Observability 서버에 접근할 수 있는 대상을 | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | 대시보드 관리 운영 설정 및 메타데이터 보기, per-model 컨텍스트 윈도우 오버라이드 목록 조회, 모델의 유효 윈도우 확인. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | 운영 설정 편집 및 per-model 컨텍스트 윈도우 오버라이드 추가, 변경, 삭제. 변경사항은 서버 재시작 없이 새 이벤트에 적용됩니다. | -![Settings 페이지: 허용 로그인 방법, 세션/OTP 유효기간 등 대시보드 관리 운영 설정을 재시작 없이 편집 가능](/agenteye/images/settings.png) +![Settings 페이지: 허용 로그인 방법, 세션/OTP 유효기간 등 대시보드 관리 운영 설정을 재시작 없이 편집 가능](/cloud/images/settings.png) ### 알림 및 인시던트 @@ -247,7 +247,7 @@ curl -s -X POST http://your-server/keys//regenerate \ 대시보드에서 키를 생성할 때 시크릿을 직접 입력하지 않습니다. 대시보드가 강력한 시크릿을 생성하여 생성 시 **한 번** 표시합니다. 즉시 복사하여 안전하게 보관하세요. 재생성과 마찬가지로 다시는 표시되지 않습니다. 키의 권한을 직접 선택하거나 권한 집합에서 시드할 수 있습니다(아래 참조). -![API Keys 페이지: 각 키의 이름, 부여된 권한, 생성 시간이 표시된 카드, 재생성 및 비활성화 액션 포함; `admin` 같은 보호된 키는 표시됨](/agenteye/images/api-keys.png) +![API Keys 페이지: 각 키의 이름, 부여된 권한, 생성 시간이 표시된 카드, 재생성 및 비활성화 액션 포함; `admin` 같은 보호된 키는 표시됨](/cloud/images/api-keys.png) --- @@ -276,5 +276,5 @@ curl -s -X POST http://your-server/keys//regenerate \ ## 다음 단계 -- [Python SDK](/ko/agenteye/python-sdk): 에이전트 코드가 이벤트를 전송할 때 인증하는 방법. -- [Security](/ko/agenteye/security): 로그인, 액세스 제어, per-organization 데이터 격리 작동 방식. \ No newline at end of file +- [Python SDK](/ko/cloud/sdk): 에이전트 코드가 이벤트를 전송할 때 인증하는 방법. +- [Security](/ko/cloud/security): 로그인, 액세스 제어, per-organization 데이터 격리 작동 방식. \ No newline at end of file diff --git a/docs/ko/cloud/agent-skills.mdx b/docs/ko/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/ko/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
    loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/ko/agenteye/alerts.mdx b/docs/ko/cloud/alerts.mdx similarity index 75% rename from docs/ko/agenteye/alerts.mdx rename to docs/ko/cloud/alerts.mdx index fcf7f613..68145785 100644 --- a/docs/ko/agenteye/alerts.mdx +++ b/docs/ko/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "고객으로부터 먼저 듣는 대신, 팀이 이미 사용하 --- -고객으로부터 먼저 듣는 대신, 팀이 이미 사용하는 채널에서 문제가 발생하는 즉시 알림을 받으세요. 규칙을 한 번 설정하면 Failproof AI Observability가 일정에 따라 확인하고, 이메일, Slack, 웹훅, 또는 대시보드에서 직접 알림을 보내드립니다. +고객으로부터 먼저 듣는 대신, 팀이 이미 사용하는 채널에서 문제가 발생하는 즉시 알림을 받으세요. 규칙을 한 번 설정하면 FailproofAI Cloud가 일정에 따라 확인하고, 이메일, Slack, 웹훅, 또는 대시보드에서 직접 알림을 보내드립니다. -![알림 페이지: 각 트리거, 평가 기간, 채널, 정보·경고·심각 심각도 배지를 보여주는 알림 규칙 카드 그리드](/agenteye/images/alerts.png) +![알림 페이지: 각 트리거, 평가 기간, 채널, 정보·경고·심각 심각도 배지를 보여주는 알림 규칙 카드 그리드](/cloud/images/alerts.png) *모든 알림 규칙을 한눈에: 무엇을 감시하는지, 얼마나 자주, 어디로 알리는지, 얼마나 긴급한지.* ## 사용자보다 먼저 문제를 파악하세요 @@ -22,9 +22,9 @@ description: "고객으로부터 먼저 듣는 대신, 팀이 이미 사용하 ## JSON이 아닌 폼으로 규칙 작성 -무엇이 "고장"인지 폼으로 설명하면 Failproof AI Observability가 내부 규칙을 대신 작성해 줍니다. JSON 사양은 그 폼이 내부적으로 생성하는 결과물이므로, 규칙을 이해하기 위해 읽을 수는 있지만 직접 타이핑할 일은 거의 없습니다. +무엇이 "고장"인지 폼으로 설명하면 FailproofAI Cloud가 내부 규칙을 대신 작성해 줍니다. JSON 사양은 그 폼이 내부적으로 생성하는 결과물이므로, 규칙을 이해하기 위해 읽을 수는 있지만 직접 타이핑할 일은 거의 없습니다. -![새 알림 폼: 이름과 설명, 활성화 토글, 메트릭 임계값·커스텀 SQL·평가 점수·복합 평가·이벤트별 조건을 제공하는 트리거 선택기](/agenteye/images/alert-new.png) +![새 알림 폼: 이름과 설명, 활성화 토글, 메트릭 임계값·커스텀 SQL·평가 점수·복합 평가·이벤트별 조건을 제공하는 트리거 선택기](/cloud/images/alert-new.png) *트리거를 선택하면 폼에 해당 필드가 표시됩니다. 저장을 누르면 규칙이 작성됩니다.* 기본 흐름은 빠릅니다: 이름을 입력하고, **트리거**(무엇을 감시할지)를 선택하고, **임계값과 기간**(얼마나 나쁜지, 얼마나 오래)을 설정하고, **채널**을 하나 이상 연결한 다음 **저장**하고 **테스트**를 눌러 가상 알림을 발송해 모든 수신처가 올바르게 연결되었는지 확인하세요. 내부적으로는 다음과 같은 작은 사양이 생성됩니다: @@ -43,7 +43,7 @@ description: "고객으로부터 먼저 듣는 대신, 팀이 이미 사용하 | **복합 평가** | 여러 점수 조건을 any, all, 또는 최소 N개 논리로 결합하여 여러 점수에 걸쳐 나타나는 회귀를 감지할 때 | | **이벤트별** | 특정 에이전트, 특정 오류 유형, 또는 메시지 하위 문자열과 일치하는 단일 이벤트가 발생할 때 | -[오류 페이지](/ko/agenteye/error-tracking)에서 이미 장애를 보고 계신가요? 각 행에는 **+ alert** 버튼이 있어 동일한 폼이 해당 장애를 다시 감지하도록 미리 채워진 상태로 열립니다. 방금 분류한 인시던트가 다음번에 알림을 보내는 항목이 됩니다. +[오류 페이지](/ko/cloud/errors)에서 이미 장애를 보고 계신가요? 각 행에는 **+ alert** 버튼이 있어 동일한 폼이 해당 장애를 다시 감지하도록 미리 채워진 상태로 열립니다. 방금 분류한 인시던트가 다음번에 알림을 보내는 항목이 됩니다. **위치:** 알림은 `//alerts`에 있습니다. 규칙 생성, 편집, 삭제, 테스트에는 **`alerts:write`** 권한이 필요하며, 조회는 `alerts:read`로 충분합니다. 수신자 선택기에는 조직 구성원이 이름으로 표시되므로, 폼을 벗어나지 않고도 특정 사람에게 알림을 보낼 수 있습니다. @@ -53,11 +53,11 @@ description: "고객으로부터 먼저 듣는 대신, 팀이 이미 사용하 ## 알림이 발동되면 어떻게 되나요 -기준 위반이 발생하면 **인시던트**가 생성되고 채널에 한 번 알림이 전송됩니다. 이후 팀이 인지하고, 담당자를 지정하고, 논의하고, 해결하는 과정이 깔끔하고 명확한 기록으로 남습니다. 해당 분류 워크플로우는 별도의 페이지에 있습니다: [인시던트](/ko/agenteye/incidents)를 참조하세요. +기준 위반이 발생하면 **인시던트**가 생성되고 채널에 한 번 알림이 전송됩니다. 이후 팀이 인지하고, 담당자를 지정하고, 논의하고, 해결하는 과정이 깔끔하고 명확한 기록으로 남습니다. 해당 분류 워크플로우는 별도의 페이지에 있습니다: [인시던트](/ko/cloud/incidents)를 참조하세요. ## 관련 항목 -- [인시던트](/ko/agenteye/incidents): 발동된 알림을 열림에서 인지됨, 해결됨까지 추적합니다. -- [오류 추적](/ko/agenteye/error-tracking): 에이전트 장애를 그룹화하고 클릭 한 번으로 알림으로 승격합니다. -- [대시보드](/ko/agenteye/dashboards): 알림 임계값의 기반이 되는 공유 보드를 확인합니다. -- [CLI 및 에이전트](/ko/agenteye/cli-and-agents): 터미널에서 알림을 생성하고 인시던트를 확인하거나, CI에 스크립트로 통합합니다. \ No newline at end of file +- [인시던트](/ko/cloud/incidents): 발동된 알림을 열림에서 인지됨, 해결됨까지 추적합니다. +- [오류 추적](/ko/cloud/errors): 에이전트 장애를 그룹화하고 클릭 한 번으로 알림으로 승격합니다. +- [대시보드](/ko/cloud/dashboards): 알림 임계값의 기반이 되는 공유 보드를 확인합니다. +- [CLI 및 에이전트](/ko/cloud/cli): 터미널에서 알림을 생성하고 인시던트를 확인하거나, CI에 스크립트로 통합합니다. \ No newline at end of file diff --git a/docs/ko/agenteye/assistant.mdx b/docs/ko/cloud/assistant.mdx similarity index 84% rename from docs/ko/agenteye/assistant.mdx rename to docs/ko/cloud/assistant.mdx index f7c0aee4..822324eb 100644 --- a/docs/ko/agenteye/assistant.mdx +++ b/docs/ko/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "에이전트 데이터에 대해 일반 영어로 질문하고, --- -에이전트 데이터에 대해 평문으로 질문하고, 근거로 바로 연결되는 답변을 받으세요. SQL을 작성하거나 대시보드를 뒤질 필요 없이 — **Failproof AI Observability** 어시스턴트는 팀 누구든 에이전트에 대한 답변을 가장 빠르게 얻을 수 있는 방법입니다. +에이전트 데이터에 대해 평문으로 질문하고, 근거로 바로 연결되는 답변을 받으세요. SQL을 작성하거나 대시보드를 뒤질 필요 없이 — **FailproofAI Cloud** 어시스턴트는 팀 누구든 에이전트에 대한 답변을 가장 빠르게 얻을 수 있는 방법입니다. -![대시보드 내에서 평문 질문에 답하는 Failproof AI Observability 어시스턴트. 실시간 Agent Activity 테이블, 에이전트별 모델 사용 현황, 작성된 인사이트, 그리고 실행된 쿼리가 인라인으로 표시됨](/agenteye/images/assistant.png) +![대시보드 내에서 평문 질문에 답하는 FailproofAI Cloud 어시스턴트. 실시간 Agent Activity 테이블, 에이전트별 모델 사용 현황, 작성된 인사이트, 그리고 실행된 쿼리가 인라인으로 표시됨](/cloud/images/assistant.png) *평문으로 질문하면 내 데이터를 기반으로 한 답변을 받을 수 있습니다. 여기서는 어떤 에이전트가 가장 바쁘고 어떤 모델을 사용하는지 분석하며, 모든 숫자를 검증할 수 있도록 실행된 쿼리도 함께 보여줍니다.* 별도로 배울 것이 없습니다. 채팅을 열고, 알고 싶은 것을 입력하고, 돌아온 링크를 따라가세요: @@ -36,7 +36,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a **Queries** 페이지에서는 한 단계 더 나아가 SQL 작성자 역할을 합니다. 원하는 쿼리를 설명하면("지난 7일간 에이전트별 오류율 보기") SQL이 편집기에 바로 스트리밍되고, **수락** 또는 **거부**할 수 있는 diff 뷰가 열립니다. -![Observability Queries 페이지와 SQL 편집기](/agenteye/images/query-lab.png) +![FailproofAI Cloud Queries 페이지와 SQL 편집기](/cloud/images/query-lab.png) *Queries 페이지: 어시스턴트가 초안 읽기 전용 쿼리를 스트리밍하면 수락하거나 거부할 수 있는 편집기입니다.* 여기서 질문을 통해 SQL을 작성하는 기능은 편집기의 **실행** 버튼과 동일한 `queries:run` 권한을 사용합니다. 다른 곳에서의 채팅에는 `agent:use` 권한이 필요합니다. @@ -49,7 +49,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a - **모든 쓰기 작업은 승인을 기다립니다.** 저장된 쿼리와 대시보드는 명시적인 승인 클릭 이후에만 생성되며, 이 게이트를 끄는 설정은 없습니다. - **절대 삭제할 수 없습니다.** 삭제 도구가 노출되지 않으며 어시스턴트는 삭제 권한을 가지지 않습니다. 삭제는 대시보드에서 내 손으로만 가능합니다. - **내 조직 내에서만 작동합니다.** 어시스턴트는 현재 보고 있는 조직만 접근할 수 있습니다. -- **내 질문은 내 것입니다.** 프롬프트와 답변은 내 Observability 데이터베이스에 저장되며, 제품 분석은 사용 메타데이터만 기록하고 프롬프트 내용은 기록하지 않습니다. +- **내 질문은 내 것입니다.** 프롬프트와 답변은 내 FailproofAI Cloud 데이터베이스에 저장되며, 제품 분석은 사용 메타데이터만 기록하고 프롬프트 내용은 기록하지 않습니다. ## 찾는 방법 @@ -57,7 +57,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a ## 관련 항목 -- [CLI and agents](/ko/agenteye/cli-and-agents) -- [Queries](/ko/agenteye/queries) -- [Dashboards](/ko/agenteye/dashboards) -- [Evaluation suite](/ko/agenteye/evaluation-suite) \ No newline at end of file +- [CLI and agents](/ko/cloud/cli) +- [Queries](/ko/cloud/queries) +- [Dashboards](/ko/cloud/dashboards) +- [Evaluation suite](/ko/cloud/evaluators) \ No newline at end of file diff --git a/docs/ko/agenteye/audits.mdx b/docs/ko/cloud/audits.mdx similarity index 81% rename from docs/ko/agenteye/audits.mdx rename to docs/ko/cloud/audits.mdx index 46635619..22527174 100644 --- a/docs/ko/agenteye/audits.mdx +++ b/docs/ko/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "감사(Audits): 자동 신뢰성 분석가" -description: "Failproof AI Observability는 여러분이 규칙으로 정의하지 않은 장애를 찾아내고, 정확히 무엇을 수정해야 하는지 우선순위와 근거가 담긴 목록으로 제공합니다." +description: "FailproofAI Cloud는 여러분이 규칙으로 정의하지 않은 장애를 찾아내고, 정확히 무엇을 수정해야 하는지 우선순위와 근거가 담긴 목록으로 제공합니다." --- -Failproof AI Observability는 여러분이 규칙으로 정의하지 않은 장애를 찾아내고, 정확히 무엇을 수정해야 하는지 우선순위와 근거가 담긴 목록으로 제공합니다. 마치 분석가가 매일 밤 로그를 살펴보고, 아침이 되면 짧은 요약 목록을 책상 위에 남겨두는 것과 같습니다. +FailproofAI Cloud는 여러분이 규칙으로 정의하지 않은 장애를 찾아내고, 정확히 무엇을 수정해야 하는지 우선순위와 근거가 담긴 목록으로 제공합니다. 마치 분석가가 매일 밤 로그를 살펴보고, 아침이 되면 짧은 요약 목록을 책상 위에 남겨두는 것과 같습니다.
    @@ -12,7 +12,7 @@ Failproof AI Observability는 여러분이 규칙으로 정의하지 않은 장 *2분 투어: 예약 실행부터 즉시 실행할 수 있는 수정 방안까지.* -![감사(Audits) 페이지: 각 세션에서 장애 패턴을 스캔하는 반복 작업 목록으로, 일정과 민감도가 표시됩니다](/agenteye/images/audits.png) +![감사(Audits) 페이지: 각 세션에서 장애 패턴을 스캔하는 반복 작업 목록으로, 일정과 민감도가 표시됩니다](/cloud/images/audits.png) *각 감사(audit)는 세션 데이터를 분석하여 우선순위와 근거가 담긴 권고사항을 작성하는 반복 작업입니다.* ## 다음에 무엇을 수정할지 추측하지 마세요 @@ -48,7 +48,7 @@ Failproof AI Observability는 여러분이 규칙으로 정의하지 않은 장 ## 관련 항목 -- [알림(Alerts)](/ko/agenteye/alerts): 이미 알고 있는 임계값이 초과되는 순간 즉시 알림을 받습니다. -- [평가(Evaluations)](/ko/agenteye/evaluations): 모든 실행에 점수를 매겨 품질 저하가 자동으로 드러나도록 합니다. -- [오류 추적(Error tracking)](/ko/agenteye/error-tracking): 에이전트가 발생시키는 오류를 그룹화하고 추적합니다. -- [인시던트(Incidents)](/ko/agenteye/incidents): 감사에서 발견된 문제를 수정 완료까지 추적합니다. \ No newline at end of file +- [알림(Alerts)](/ko/cloud/alerts): 이미 알고 있는 임계값이 초과되는 순간 즉시 알림을 받습니다. +- [평가(Evaluations)](/ko/cloud/evaluations): 모든 실행에 점수를 매겨 품질 저하가 자동으로 드러나도록 합니다. +- [오류 추적(Error tracking)](/ko/cloud/errors): 에이전트가 발생시키는 오류를 그룹화하고 추적합니다. +- [인시던트(Incidents)](/ko/cloud/incidents): 감사에서 발견된 문제를 수정 완료까지 추적합니다. \ No newline at end of file diff --git a/docs/ko/cloud/capture.mdx b/docs/ko/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/ko/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/ko/agenteye/cli-recipes.mdx b/docs/ko/cloud/cli-recipes.mdx similarity index 91% rename from docs/ko/agenteye/cli-recipes.mdx rename to docs/ko/cloud/cli-recipes.mdx index ddd25fc1..d4b84f5e 100644 --- a/docs/ko/agenteye/cli-recipes.mdx +++ b/docs/ko/cloud/cli-recipes.mdx @@ -4,9 +4,9 @@ description: "세션, 이벤트, 평가 데이터를 스크립트나 코딩 에 --- -스크립트나 코딩 에이전트에서 세션, 이벤트, 평가 데이터를 직접 가져오고(재평가 트리거 포함) `jq`로 바로 파이프할 수 있는 깔끔한 JSON을 stdout으로 출력합니다. 이 레시피들은 Failproof AI Observability의 데이터를 터미널 사용자나 AI 코딩 에이전트(Claude Code, Cursor)가 대시보드를 클릭하지 않고도 쿼리하고 자동화할 수 있도록 해줍니다. +스크립트나 코딩 에이전트에서 세션, 이벤트, 평가 데이터를 직접 가져오고(재평가 트리거 포함) `jq`로 바로 파이프할 수 있는 깔끔한 JSON을 stdout으로 출력합니다. 이 레시피들은 FailproofAI Cloud의 데이터를 터미널 사용자나 AI 코딩 에이전트(Claude Code, Cursor)가 대시보드를 클릭하지 않고도 쿼리하고 자동화할 수 있도록 해줍니다. -아래 패턴들은 Failproof AI Observability CLI(`agenteye`)에서 바로 복사-붙여넣기하여 사용할 수 있습니다. 설치, 인증, 전체 옵션 목록은 [CLI](/ko/agenteye/cli)를 참고하세요. 내장 도움말은 `agenteye -h` 또는 `agenteye -h`로 확인할 수 있습니다. +아래 패턴들은 FailproofAI Cloud CLI(`agenteye`)에서 바로 복사-붙여넣기하여 사용할 수 있습니다. 설치, 인증, 전체 옵션 목록은 [CLI](/ko/cloud/cli)를 참고하세요. 내장 도움말은 `agenteye -h` 또는 `agenteye -h`로 확인할 수 있습니다. ## 기본 원칙 @@ -173,7 +173,7 @@ esac ## 다음 단계 -- [CLI](/ko/agenteye/cli): 모든 명령어의 설치, 인증, 전체 옵션 레퍼런스. -- [CLI 에이전트 스킬](/ko/agenteye/cli-skill): 이 레시피들을 코딩 에이전트가 로드할 수 있는 스킬로 패키징하기. -- [API 키](/ko/agenteye/api-keys): CLI, SDK, 컬렉터가 인증에 사용하는 키 생성 및 범위 설정. -- [Python SDK](/ko/agenteye/python-sdk): Failproof AI Observability로 이벤트를 전송하여 이 레시피가 쿼리할 데이터를 만들기. \ No newline at end of file +- [CLI](/ko/cloud/cli): 모든 명령어의 설치, 인증, 전체 옵션 레퍼런스. +- [CLI 에이전트 스킬](/ko/cloud/agent-skills): 이 레시피들을 코딩 에이전트가 로드할 수 있는 스킬로 패키징하기. +- [API 키](/ko/cloud/access): CLI, SDK, 컬렉터가 인증에 사용하는 키 생성 및 범위 설정. +- [Python SDK](/ko/cloud/sdk): FailproofAI Cloud로 이벤트를 전송하여 이 레시피가 쿼리할 데이터를 만들기. \ No newline at end of file diff --git a/docs/ko/agenteye/cli.mdx b/docs/ko/cloud/cli.mdx similarity index 91% rename from docs/ko/agenteye/cli.mdx rename to docs/ko/cloud/cli.mdx index 89d06d2c..de70e9fa 100644 --- a/docs/ko/agenteye/cli.mdx +++ b/docs/ko/cloud/cli.mdx @@ -1,10 +1,10 @@ --- title: "CLI" -description: "터미널이나 스크립트에서 Failproof AI Observability를 완전히 제어하세요: 대시보드를 오갈 필요가 없습니다." +description: "터미널이나 스크립트에서 FailproofAI Cloud를 완전히 제어하세요: 대시보드를 오갈 필요가 없습니다." --- -터미널이나 스크립트에서 Failproof AI Observability를 완전히 제어하세요: 대시보드를 오갈 필요가 없습니다. `agenteye` CLI는 데이터(세션, 이벤트 로그, 평가)를 조회하고 조직(API 키, 사용자, 설정, 알림, 인시던트, 저장된 쿼리)을 관리합니다. 자동화된 검사를 실행하거나, Observability를 CI에 연동하거나, 코딩 에이전트가 프로덕션을 점검하도록 할 때 활용하세요. 모든 커맨드는 `--json` 플래그를 지원하므로, 터미널에서 직접 사용하거나 코딩 에이전트(Claude Code, Cursor)가 셸을 호출해 결과를 파싱할 때 모두 동일하게 작동합니다. +터미널이나 스크립트에서 FailproofAI Cloud를 완전히 제어하세요: 대시보드를 오갈 필요가 없습니다. `agenteye` CLI는 데이터(세션, 이벤트 로그, 평가)를 조회하고 조직(API 키, 사용자, 설정, 알림, 인시던트, 저장된 쿼리)을 관리합니다. 자동화된 검사를 실행하거나, Observability를 CI에 연동하거나, 코딩 에이전트가 프로덕션을 점검하도록 할 때 활용하세요. 모든 커맨드는 `--json` 플래그를 지원하므로, 터미널에서 직접 사용하거나 코딩 에이전트(Claude Code, Cursor)가 셸을 호출해 결과를 파싱할 때 모두 동일하게 작동합니다. 하나의 바이너리로 다음을 수행할 수 있습니다: @@ -69,7 +69,7 @@ agenteye --version agenteye --help ``` -> **참고:** Failproof AI Observability Python SDK도 `agenteye` 배포 이름을 사용합니다. `pipx` 또는 `uv tool`로 CLI를 설치하면(공유 가상 환경에 `pip install`하는 것과 달리) 두 패키지가 충돌하지 않습니다. 동일한 환경에 SDK가 설치되어 있지 않다면 `pip install agenteye`도 괜찮습니다. +> **참고:** FailproofAI Cloud Python SDK도 `agenteye` 배포 이름을 사용합니다. `pipx` 또는 `uv tool`로 CLI를 설치하면(공유 가상 환경에 `pip install`하는 것과 달리) 두 패키지가 충돌하지 않습니다. 동일한 환경에 SDK가 설치되어 있지 않다면 `pip install agenteye`도 괜찮습니다. --- @@ -91,7 +91,7 @@ agenteye logout # 세션을 취소하고 저장된 토큰 삭제 `whoami`는 세션이 없거나 만료된 경우에도 오류를 발생시키지 않으며, 대신 `logged_in: false`를 반환합니다. 스크립트나 에이전트가 인증 상태를 안전하게 확인할 수 있습니다(다만 base URL이 설정되지 않았거나 대시보드에 접근할 수 없으면 여전히 비정상 종료될 수 있습니다). -**요구 사항:** 이메일이 대시보드 로그인 허용 목록에 있어야 하며(Failproof AI Observability 관리자에게 문의), 대시보드가 base URL에서 접근 가능해야 합니다([설정](#configuration) 참조). 코드를 요청했는데 도착하지 않는다면 이메일이 아직 대시보드 접근 권한이 없는 것일 수 있습니다. +**요구 사항:** 이메일이 대시보드 로그인 허용 목록에 있어야 하며(FailproofAI Cloud 관리자에게 문의), 대시보드가 base URL에서 접근 가능해야 합니다([설정](#configuration) 참조). 코드를 요청했는데 도착하지 않는다면 이메일이 아직 대시보드 접근 권한이 없는 것일 수 있습니다. --- @@ -338,13 +338,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | 인증은 됐지만 계정에 필요한 권한이 없음 (메시지에 권한 이름 표시) | | 6 | 요청한 리소스를 찾을 수 없음 (예: 알 수 없는 세션 또는 인시던트 id) | -종료 코드 덕분에 CLI를 안전하게 스크립트화할 수 있습니다: 코딩 에이전트는 `4`가 반환되면 재인증을 요청하거나, `5`가 반환되면 누락된 권한을 표시하도록 분기할 수 있습니다. 종료 코드 처리 패턴과 JSON 출력 형태는 [에이전트를 위한 CLI 레시피](/ko/agenteye/cli-recipes)를 참조하세요. +종료 코드 덕분에 CLI를 안전하게 스크립트화할 수 있습니다: 코딩 에이전트는 `4`가 반환되면 재인증을 요청하거나, `5`가 반환되면 누락된 권한을 표시하도록 분기할 수 있습니다. 종료 코드 처리 패턴과 JSON 출력 형태는 [에이전트를 위한 CLI 레시피](/ko/cloud/cli-recipes)를 참조하세요. --- ## 다음 단계 -- **[에이전트를 위한 CLI 레시피](/ko/agenteye/cli-recipes)**: 복사해서 바로 쓸 수 있는 쿼리 패턴, `jq` 원라이너, `--fields` 프로젝션, 종료 코드 처리, JSON 출력 형태 — 코딩 에이전트가 CLI를 구동하는 것을 염두에 두고 작성되었습니다. -- **[CLI 에이전트 스킬](/ko/agenteye/cli-skill)**: 이 CLI를 설치 가능한 Claude Code / Codex *스킬*로 패키징하여 코딩 에이전트가 자연어로 Failproof AI Observability를 제어할 수 있게 합니다. -- **[API 키](/ko/agenteye/api-keys)**: `keys create --add …` 뒤에 있는 권한 모델. -- **[AI 어시스턴트](/ko/agenteye/assistant)**: `agent ask`가 사용하는 어시스턴트 활성화 방법. \ No newline at end of file +- **[에이전트를 위한 CLI 레시피](/ko/cloud/cli-recipes)**: 복사해서 바로 쓸 수 있는 쿼리 패턴, `jq` 원라이너, `--fields` 프로젝션, 종료 코드 처리, JSON 출력 형태 — 코딩 에이전트가 CLI를 구동하는 것을 염두에 두고 작성되었습니다. +- **[CLI 에이전트 스킬](/ko/cloud/agent-skills)**: 이 CLI를 설치 가능한 Claude Code / Codex *스킬*로 패키징하여 코딩 에이전트가 자연어로 FailproofAI Cloud를 제어할 수 있게 합니다. +- **[API 키](/ko/cloud/access)**: `keys create --add …` 뒤에 있는 권한 모델. +- **[AI 어시스턴트](/ko/cloud/assistant)**: `agent ask`가 사용하는 어시스턴트 활성화 방법. \ No newline at end of file diff --git a/docs/ko/cloud/connect.mdx b/docs/ko/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/ko/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/ko/agenteye/dashboards.mdx b/docs/ko/cloud/dashboards.mdx similarity index 69% rename from docs/ko/agenteye/dashboards.mdx rename to docs/ko/cloud/dashboards.mdx index fc446bc7..3f76c9e5 100644 --- a/docs/ko/agenteye/dashboards.mdx +++ b/docs/ko/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "실시간 에이전트 데이터를 팀 전체가 함께 보는 실시간 에이전트 데이터를 팀 전체가 함께 보는 하나의 화면으로 만드세요. 중요한 쿼리를 차트로 고정해 두면, 누구나 단 한 번의 쿼리 재실행 없이 동일한 수치를 한눈에 확인할 수 있습니다. -![저장된 쿼리로 구성된 대시보드: 시간당 이벤트 라인 차트, 유형별 오류 막대 차트, 지연 시간 영역 차트, 모델별 토큰 수](/agenteye/images/dashboard-fleet.png) +![저장된 쿼리로 구성된 대시보드: 시간당 이벤트 라인 차트, 유형별 오류 막대 차트, 지연 시간 영역 차트, 모델별 토큰 수](/cloud/images/dashboard-fleet.png) *하나의 보드, 네 개의 저장된 쿼리: 시간당 이벤트, 유형별 오류, 지연 시간, 모델별 토큰 수.* @@ -25,15 +25,15 @@ description: "실시간 에이전트 데이터를 팀 전체가 함께 보는 ## 이미 저장한 쿼리를 고정하세요 -모든 타일은 저장된 쿼리에서 시작합니다. [쿼리](/ko/agenteye/queries) 라이브러리(기본 제공 프리셋과 이벤트 및 평가 데이터를 기반으로 한 직접 작성 쿼리 포함)에서 원하는 쿼리를 빌드하고 저장한 다음, 데이터에 맞는 차트 형식으로 대시보드에 고정하세요. 시간에 따른 추세에는 **라인**, 카테고리 비교에는 **막대**, 볼륨에는 **영역**, 구성 비율에는 **파이** 차트가 적합합니다. +모든 타일은 저장된 쿼리에서 시작합니다. [쿼리](/ko/cloud/queries) 라이브러리(기본 제공 프리셋과 이벤트 및 평가 데이터를 기반으로 한 직접 작성 쿼리 포함)에서 원하는 쿼리를 빌드하고 저장한 다음, 데이터에 맞는 차트 형식으로 대시보드에 고정하세요. 시간에 따른 추세에는 **라인**, 카테고리 비교에는 **막대**, 볼륨에는 **영역**, 구성 비율에는 **파이** 차트가 적합합니다. 타일은 저장된 쿼리를 차트로 렌더링한 것에 불과하기 때문에 수동으로 동기화할 필요가 없습니다. 쿼리를 한 번 업데이트하면 해당 쿼리를 사용하는 모든 대시보드가 자동으로 업데이트됩니다. ## 단순한 양이 아닌 품질을 모니터링하세요 -양은 에이전트가 바쁘다는 것을 알려줍니다. 품질은 에이전트가 실제로 제대로 역할을 하고 있는지를 알려줍니다. [평가 점수](/ko/agenteye/evaluations)를 대시보드에 연결하면 실행 품질이 시간에 따라 어떻게 변하는지 추적할 수 있어, 품질 저하가 고객의 불만으로 이어지기 전에 차트의 하락으로 먼저 나타납니다. +양은 에이전트가 바쁘다는 것을 알려줍니다. 품질은 에이전트가 실제로 제대로 역할을 하고 있는지를 알려줍니다. [평가 점수](/ko/cloud/evaluations)를 대시보드에 연결하면 실행 품질이 시간에 따라 어떻게 변하는지 추적할 수 있어, 품질 저하가 고객의 불만으로 이어지기 전에 차트의 하락으로 먼저 나타납니다. -![저장된 평가 쿼리로 구성된 품질 중심 대시보드](/agenteye/images/dashboard-quality.png) +![저장된 평가 쿼리로 구성된 품질 중심 대시보드](/cloud/images/dashboard-quality.png) *품질 보드는 운영 지표 바로 옆에 평가 점수를 전면에 배치합니다.* @@ -41,6 +41,6 @@ description: "실시간 에이전트 데이터를 팀 전체가 함께 보는 ## 관련 항목 -- [쿼리](/ko/agenteye/queries): 타일의 기반이 되는 쿼리를 빌드하고 저장하세요. -- [평가](/ko/agenteye/evaluations): 실행을 채점하여 시간에 따른 품질을 차트로 확인하세요. -- [알림](/ko/agenteye/alerts): 이러한 지표의 임계값을 알림으로 전환하세요. \ No newline at end of file +- [쿼리](/ko/cloud/queries): 타일의 기반이 되는 쿼리를 빌드하고 저장하세요. +- [평가](/ko/cloud/evaluations): 실행을 채점하여 시간에 따른 품질을 차트로 확인하세요. +- [알림](/ko/cloud/alerts): 이러한 지표의 임계값을 알림으로 전환하세요. \ No newline at end of file diff --git a/docs/ko/agenteye/error-tracking.mdx b/docs/ko/cloud/errors.mdx similarity index 75% rename from docs/ko/agenteye/error-tracking.mdx rename to docs/ko/cloud/errors.mdx index f15b5121..ed7a5834 100644 --- a/docs/ko/agenteye/error-tracking.mdx +++ b/docs/ko/cloud/errors.mdx @@ -5,14 +5,14 @@ description: "에이전트에서 발생하는 모든 실패를 한 곳에서 확 에이전트에서 발생하는 모든 실패를 한 곳에서 확인하세요. 동일한 오류가 연속으로 발생해도 하나의 문제로 묶어서 보여줍니다. 라이브 피드를 스크롤하지 않아도, "뭔가 빨간색이다"에서 문제가 발생한 정확한 실행까지 클릭 한 번으로 이동할 수 있습니다. -![오류 페이지: 시간별 실패 히스토그램 아래에 빨간색 오류 행이 그룹으로 표시되며, 각 행에는 원클릭 "+ alert" 버튼이 있습니다](/agenteye/images/errors.png) +![오류 페이지: 시간별 실패 히스토그램 아래에 빨간색 오류 행이 그룹으로 표시되며, 각 행에는 원클릭 "+ alert" 버튼이 있습니다](/cloud/images/errors.png) *오류 페이지: 시간별 실패 히스토그램과 반복 실패를 인시던트 단위로 하나의 행으로 묶어서 표시합니다.* ## 모든 실패를 자동으로 수집 에이전트가 중단되었을 때, 빨간색 행이 사라지기 전에 잡으려고 라이브 이벤트 스트림을 스크롤할 필요가 없어야 합니다. **오류** 페이지가 대신 수집해 드립니다. 대시보드에서 빨간색으로 표시될 모든 항목을 하나의 트리아지 화면으로 모아주기 때문에, 처음 보는 화면에서 바로 무엇이 실패하고 있는지 확인할 수 있습니다. -또한 명확한 오류뿐만 아니라 조용한 실패도 포착합니다. 명시적인 `error` 이벤트 외에도, Failproof AI Observability는 `tool_result`, `hook_completed`, `agent_end`의 페이로드에 실패가 포함된 경우도 모두 여기에 표시합니다. 오류를 반환한 도구나 비정상 종료된 훅도, 큰 예외를 던지지 않았다는 이유만으로 그냥 지나치지 않습니다. +또한 명확한 오류뿐만 아니라 조용한 실패도 포착합니다. 명시적인 `error` 이벤트 외에도, FailproofAI Cloud는 `tool_result`, `hook_completed`, `agent_end`의 페이로드에 실패가 포함된 경우도 모두 여기에 표시합니다. 오류를 반환한 도구나 비정상 종료된 훅도, 큰 예외를 던지지 않았다는 이유만으로 그냥 지나치지 않습니다. 페이지 상단에는 히스토그램이 시간에 따른 오류를 시각화합니다. 한 눈에 보면 현재 상황이 지속적인 백그라운드 오류인지, 아니면 몇 분 전에 시작된 급증인지 즉시 파악할 수 있어 지금 당장 대응해야 할지 판단할 수 있습니다. @@ -22,7 +22,7 @@ description: "에이전트에서 발생하는 모든 실패를 한 곳에서 확 단일 의존성 오류 하나가 분당 수백 번 동일한 오류를 발생시킬 수 있습니다. 그대로 두면 거의 동일한 줄이 벽처럼 쌓여 실제로 확인해야 할 중요한 정보가 묻혀버립니다. -Failproof AI Observability는 동일한 세션과 오류 유형을 공유하는 반복 실패를 하나의 행으로 묶습니다. 연속 발생은 하나의 인시던트로 읽힙니다. 로그 라인이 아닌 문제 수를 세게 되고, 중요한 신호가 자체 볼륨에 묻히는 대신 상단에 유지됩니다. +FailproofAI Cloud는 동일한 세션과 오류 유형을 공유하는 반복 실패를 하나의 행으로 묶습니다. 연속 발생은 하나의 인시던트로 읽힙니다. 로그 라인이 아닌 문제 수를 세게 되고, 중요한 신호가 자체 볼륨에 묻히는 대신 상단에 유지됩니다. ## "뭔가 빨간색이다"에서 정확한 이벤트로 @@ -34,7 +34,7 @@ Failproof AI Observability는 동일한 세션과 오류 유형을 공유하는 ## 관련 항목 -- [알림](/ko/agenteye/alerts): 모든 실패를 페이징 규칙으로 전환합니다. -- [인시던트](/ko/agenteye/incidents): 발생한 알림을 열림에서 해결까지 추적합니다. -- [세션](/ko/agenteye/sessions): 오류 뒤에 있는 전체 실행을 엽니다. -- [감사](/ko/agenteye/audits): Observability가 실행 전반에 걸친 실패 패턴을 자동으로 찾아줍니다. \ No newline at end of file +- [알림](/ko/cloud/alerts): 모든 실패를 페이징 규칙으로 전환합니다. +- [인시던트](/ko/cloud/incidents): 발생한 알림을 열림에서 해결까지 추적합니다. +- [세션](/ko/cloud/sessions): 오류 뒤에 있는 전체 실행을 엽니다. +- [감사](/ko/cloud/audits): Observability가 실행 전반에 걸친 실패 패턴을 자동으로 찾아줍니다. \ No newline at end of file diff --git a/docs/ko/agenteye/evaluations.mdx b/docs/ko/cloud/evaluations.mdx similarity index 66% rename from docs/ko/agenteye/evaluations.mdx rename to docs/ko/cloud/evaluations.mdx index a06a2bf8..ede771c1 100644 --- a/docs/ko/agenteye/evaluations.mdx +++ b/docs/ko/cloud/evaluations.mdx @@ -3,15 +3,15 @@ title: "평가(Evaluations)" description: "품질 문제가 사용자 불만으로 접수되기 전에 먼저 알 수 있습니다." --- -품질 문제가 사용자 불만으로 접수되기 전에 먼저 알 수 있습니다. 자체 채점 서비스를 한 번만 연결하면 Failproof AI Observability가 완료된 모든 실행을 자동으로 평가합니다. 따라서 유용성 저하나 환각 급증이 고객이 느끼기 전에 자동으로 표시됩니다. +품질 문제가 사용자 불만으로 접수되기 전에 먼저 알 수 있습니다. 자체 채점 서비스를 한 번만 연결하면 FailproofAI Cloud가 완료된 모든 실행을 자동으로 평가합니다. 따라서 유용성 저하나 환각 급증이 고객이 느끼기 전에 자동으로 표시됩니다. -![점수 열이 있는 세션 그리드: 각 실행에 평가 상태 배지와 유용성, 사실성, 도구 효율성에 대한 색상 코딩 배지가 표시됩니다](/agenteye/images/sessions-list.png) +![점수 열이 있는 세션 그리드: 각 실행에 평가 상태 배지와 유용성, 사실성, 도구 효율성에 대한 색상 코딩 배지가 표시됩니다](/cloud/images/sessions-list.png) *세션 그리드의 모든 실행에 점수가 표시되며, 빨간색·황색·녹색 배지 덕분에 트랜스크립트를 하나도 열지 않아도 문제 있는 실행이 바로 눈에 띕니다.* ## 수동 샘플링 중단 -이전에는 일부 실행만 무작위로 점검하며 나머지도 괜찮을 거라 기대했을 것입니다. 이제는 완료된 모든 세션이 종료되는 즉시 원하는 기준, 즉 유용성·도구 효율성·사실성·안전성 등 여러분의 품질 기준에 따라 자동으로 채점됩니다. 점수 키는 여러분이 직접 정의하고, Failproof AI Observability는 평가기가 반환하는 모든 값을 저장·추적·표시합니다. 채점되지 않고 넘어가는 실행은 없으며, 지원 티켓을 통해 회귀를 뒤늦게 파악하는 일도 없어집니다. +이전에는 일부 실행만 무작위로 점검하며 나머지도 괜찮을 거라 기대했을 것입니다. 이제는 완료된 모든 세션이 종료되는 즉시 원하는 기준, 즉 유용성·도구 효율성·사실성·안전성 등 여러분의 품질 기준에 따라 자동으로 채점됩니다. 점수 키는 여러분이 직접 정의하고, FailproofAI Cloud는 평가기가 반환하는 모든 값을 저장·추적·표시합니다. 채점되지 않고 넘어가는 실행은 없으며, 지원 티켓을 통해 회귀를 뒤늦게 파악하는 일도 없어집니다. 점수는 **`//sessions`**(사이드바 → *observe* → *sessions*)의 세션 그리드에 행마다 배지 묶음으로 표시됩니다. 기준에 미달한 실행만 보고 싶다면 점수 범위로 그리드를 필터링하세요. 예를 들어 유용성 0.5 미만으로 필터링하면 검토할 가치가 있는 실행만 정확히 불러올 수 있습니다. 점수 조회에는 `evaluations:read` 권한이 필요합니다. @@ -19,7 +19,7 @@ description: "품질 문제가 사용자 불만으로 접수되기 전에 먼저 숫자는 실행이 부진했음을 알려주고, 세션 페이지는 그 이유를 알려줍니다. 실행을 열면 오른쪽 패널 상단에 핵심 요약이 나타나고, 각 항목별로 평가기가 제공한 근거와 함께 막대 그래프가 표시됩니다. 덕분에 "사실성 점수가 0.4"에서 "어떤 주장이 틀렸는지"까지 몇 초 만에 확인할 수 있습니다. -![세션 오른쪽 패널: 상단에 평가 요약, 그 아래에 항목별 점수 막대와 근거 설명이 전체 이벤트 타임라인 옆에 표시됩니다](/agenteye/images/session-detail.png) +![세션 오른쪽 패널: 상단에 평가 요약, 그 아래에 항목별 점수 막대와 근거 설명이 전체 이벤트 타임라인 옆에 표시됩니다](/cloud/images/session-detail.png) *세션 상세 보기: 요약, 항목별 점수 막대, 각 점수의 근거가 실행 이벤트 타임라인 바로 옆에 표시됩니다.* @@ -29,7 +29,7 @@ description: "품질 문제가 사용자 불만으로 접수되기 전에 먼저 실행 하나의 낮은 점수는 노이즈일 수 있지만, 전체 코호트가 하락하면 명확한 신호입니다. 저장된 대시보드는 점수를 한눈에 파악할 수 있는 추세로 변환해줍니다. 에이전트별·환경별로 이번 주와 지난주의 평균 유용성을 비교할 수 있습니다. -![품질 대시보드: 평가 항목별 평균 점수 막대와 시간에 따른 추세 그래프](/agenteye/images/dashboard-quality.png) +![품질 대시보드: 평가 항목별 평균 점수 막대와 시간에 따른 추세 그래프](/cloud/images/dashboard-quality.png) *저장된 품질 대시보드는 주요 점수 키의 추세를 보여주므로, 서서히 하락하는 추세가 장애로 번지기 훨씬 전에 명확하게 인지할 수 있습니다.* @@ -37,14 +37,14 @@ description: "품질 문제가 사용자 불만으로 접수되기 전에 먼저 ## 평가기 한 번만 연결하기 -채점은 옵트인 방식이며, Failproof AI Observability에 채점기를 연결하기 전까지는 완전히 비활성화 상태입니다. 소형 HTTP 서비스를 하나 실행하고(Observability에서 복사할 수 있는 참조 구현을 제공합니다), 서버에 두 가지 값을 설정하면 이후 모든 실행이 자동으로 채점됩니다. 전체 안내, 채점 계약, SDK는 상세 가이드에서 확인할 수 있습니다. +채점은 옵트인 방식이며, FailproofAI Cloud에 채점기를 연결하기 전까지는 완전히 비활성화 상태입니다. 소형 HTTP 서비스를 하나 실행하고(Observability에서 복사할 수 있는 참조 구현을 제공합니다), 서버에 두 가지 값을 설정하면 이후 모든 실행이 자동으로 채점됩니다. 전체 안내, 채점 계약, SDK는 상세 가이드에서 확인할 수 있습니다. -어떤 항목을 채점해야 할지 모르겠다면? [평가기 에이전트 스킬](/ko/agenteye/evaluator-skill)을 사용하면 코딩 에이전트가 여러분의 세션을 분석해 점수 항목을 결정하고 서비스를 빌드·배포합니다. +어떤 항목을 채점해야 할지 모르겠다면? [평가기 에이전트 스킬](/ko/cloud/agent-skills)을 사용하면 코딩 에이전트가 여러분의 세션을 분석해 점수 항목을 결정하고 서비스를 빌드·배포합니다. ## 관련 문서 -- [평가 suite](/ko/agenteye/evaluation-suite): 평가기 연결, 채점 계약, SDK. -- [평가기 에이전트 스킬](/ko/agenteye/evaluator-skill): 코딩 에이전트가 점수 항목을 선택하고 평가기를 빌드합니다. -- [Sessions](/ko/agenteye/sessions): 점수가 표시되는 실행별 그리드. -- [Dashboards](/ko/agenteye/dashboards): 조직 전체의 품질 추세를 저장하고 공유합니다. -- [Audits](/ko/agenteye/audits): 세션 간 조사를 위한 Observability의 또 다른 자동 품질 기능. \ No newline at end of file +- [평가 suite](/ko/cloud/evaluators): 평가기 연결, 채점 계약, SDK. +- [평가기 에이전트 스킬](/ko/cloud/agent-skills): 코딩 에이전트가 점수 항목을 선택하고 평가기를 빌드합니다. +- [Sessions](/ko/cloud/sessions): 점수가 표시되는 실행별 그리드. +- [Dashboards](/ko/cloud/dashboards): 조직 전체의 품질 추세를 저장하고 공유합니다. +- [Audits](/ko/cloud/audits): 세션 간 조사를 위한 Observability의 또 다른 자동 품질 기능. \ No newline at end of file diff --git a/docs/ko/agenteye/evaluation-suite.mdx b/docs/ko/cloud/evaluators.mdx similarity index 88% rename from docs/ko/agenteye/evaluation-suite.mdx rename to docs/ko/cloud/evaluators.mdx index 851169cf..f7251c4b 100644 --- a/docs/ko/agenteye/evaluation-suite.mdx +++ b/docs/ko/cloud/evaluators.mdx @@ -1,10 +1,10 @@ --- title: "평가 Suite" -description: "Failproof AI Observability는 완료된 모든 에이전트 실행을 자동으로 품질 점수화할 수 있습니다: 소규모 점수화 서비스를 제공하면 Observability가 나머지를 처리합니다." +description: "FailproofAI Cloud는 완료된 모든 에이전트 실행을 자동으로 품질 점수화할 수 있습니다: 소규모 점수화 서비스를 제공하면 Observability가 나머지를 처리합니다." --- -Failproof AI Observability는 완료된 모든 에이전트 실행을 자동으로 품질 점수화할 수 있습니다: 소규모 점수화 서비스를 제공하면 Observability가 나머지를 처리합니다. 이를 통해 관심 있는 차원(유용성, 도구 효율성, 사실성, 안전성 등 원하는 항목을 선택)을 추적하고, 회귀를 조기에 감지하며, 에이전트나 환경을 한눈에 비교할 수 있습니다. 점수화는 선택 사항입니다: 서버에 `EVALUATOR_ENDPOINT`를 설정하기 전까지는 파이프라인이 아무것도 수행하지 않습니다. +FailproofAI Cloud는 완료된 모든 에이전트 실행을 자동으로 품질 점수화할 수 있습니다: 소규모 점수화 서비스를 제공하면 Observability가 나머지를 처리합니다. 이를 통해 관심 있는 차원(유용성, 도구 효율성, 사실성, 안전성 등 원하는 항목을 선택)을 추적하고, 회귀를 조기에 감지하며, 에이전트나 환경을 한눈에 비교할 수 있습니다. 점수화는 선택 사항입니다: 서버에 `EVALUATOR_ENDPOINT`를 설정하기 전까지는 파이프라인이 아무것도 수행하지 않습니다. > **참고:** 점수 차원은 직접 정의합니다. 평가자는 원하는 숫자형 키를 반환할 수 있으며, Observability는 전송된 값을 저장, 추세 분석, 표시합니다. @@ -14,7 +14,7 @@ Failproof AI Observability는 완료된 모든 에이전트 실행을 자동으 2. **Observability가 해당 서비스를 가리키도록 설정합니다.** 서버 프로세스에 `EVALUATOR_ENDPOINT`(및 공유 `EVALUATOR_TOKEN`)를 설정합니다. 3. **점수가 기록되는 것을 확인합니다.** 완료된 모든 세션은 자동으로 점수화되며, 결과는 세션 상세 페이지, 세션 그리드, 저장된 대시보드에 표시됩니다. -![평가 요약, 차원별 점수 바, 오른쪽 패널의 추론 텍스트가 포함된 세션 상세 보기](/agenteye/images/session-detail.png) +![평가 요약, 차원별 점수 바, 오른쪽 패널의 추론 텍스트가 포함된 세션 상세 보기](/cloud/images/session-detail.png) *평가자를 구성하면 완료된 각 실행이 점수화되고 결과가 세션의 오른쪽 패널에 표시됩니다: 상단의 요약, 그 아래 추론이 포함된 차원별 점수 바.* @@ -24,7 +24,7 @@ Failproof AI Observability는 완료된 모든 에이전트 실행을 자동으 ```mermaid flowchart LR - ING["ingest /events
    agent_end"] --> SRV["Observability server"] + ING["ingest /events
    agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -32,7 +32,7 @@ flowchart LR SRV --> RES["evaluations
    terminal results"] ``` -Observability SDK가 세션에 대한 `agent_end` 이벤트를 전송하면, 서버는 +FailproofAI Cloud SDK가 세션에 대한 `agent_end` 이벤트를 전송하면, 서버는 평가를 예약합니다. 그런 다음 전체 이벤트 트랜스크립트를 평가자 서비스에 POST하며, 평가자 서비스는 다음 중 하나를 수행할 수 있습니다: @@ -57,7 +57,7 @@ Observability SDK가 세션에 대한 `agent_end` 이벤트를 전송하면, 서 모든 인증된 라우트는 **베어러 토큰 인증**을 사용합니다. 양쪽에 동일한 값이 구성되어야 합니다: -- Observability 서버: 환경 변수 `EVALUATOR_TOKEN` +- FailproofAI Cloud 서버: 환경 변수 `EVALUATOR_TOKEN` - 평가자 서비스: 동일한 방식으로 구성 (`agenteye-evaluator` SDK는 관례에 따라 `EVALUATOR_TOKEN`을 읽음) `EVALUATOR_TOKEN`이 설정되지 않은 경우 서버는 `Authorization` 헤더를 전송하지 않습니다; 평가자는 익명 요청을 수락할 수 있으며, 내부 전용 네트워크에서는 괜찮지만 공개 인터넷에서는 권장하지 않습니다. @@ -129,7 +129,7 @@ Observability SDK가 세션에 대한 `agent_end` 이벤트를 전송하면, 서 HTTP 계약을 직접 구현할 필요가 없습니다. `agenteye-evaluator` Python 패키지는 인증, 라우팅, 요청/응답 형태를 자동으로 처리하는 타입이 지정된 FastAPI 래퍼를 제공합니다. -Failproof AI Observability는 트랜스크립트 형태에서 `helpfulness`, `tool_efficiency`, `factuality`를 점수화하는 **작동하는 참조 평가자**도 함께 제공합니다. 이를 시작점으로 복사하고 LLM 판단자, 규칙 엔진 등 품질 기준에 맞는 자체 로직으로 교체하세요. +FailproofAI Cloud는 트랜스크립트 형태에서 `helpfulness`, `tool_efficiency`, `factuality`를 점수화하는 **작동하는 참조 평가자**도 함께 제공합니다. 이를 시작점으로 복사하고 LLM 판단자, 규칙 엔진 등 품질 기준에 맞는 자체 로직으로 교체하세요. 최소 실행 가능한 평가자: @@ -152,7 +152,7 @@ def run(req: EvalRequest) -> EvalResponse: `app` 인스턴스는 모든 ASGI 서버에서 실행되므로 `uvicorn module:app`으로 시작할 수 있습니다. -비용이 많이 드는 작업을 지연해야 하는 평가자의 경우 대신 `JobPending`을 반환하고 `@app.job_lookup` 핸들러를 등록하세요; Observability 서버는 평가자가 최종 상태를 반환하거나 `EVALUATOR_MAX_POLL_DURATION_SECS` 제한(기본값 1시간)이 경과할 때까지 `GET /evaluate/{job_id}`를 폴링합니다. +비용이 많이 드는 작업을 지연해야 하는 평가자의 경우 대신 `JobPending`을 반환하고 `@app.job_lookup` 핸들러를 등록하세요; FailproofAI Cloud 서버는 평가자가 최종 상태를 반환하거나 `EVALUATOR_MAX_POLL_DURATION_SECS` 제한(기본값 1시간)이 경과할 때까지 `GET /evaluate/{job_id}`를 폴링합니다. 전체 API 참조, 비동기 패턴, 이벤트 스키마는 `agenteye-evaluator` SDK의 README에 문서화되어 있습니다. @@ -160,7 +160,7 @@ def run(req: EvalRequest) -> EvalResponse: ## 평가자 실행 -평가자는 **사용자의 서비스**입니다 — Failproof AI Observability는 기본 평가자를 제공하지 않으므로, 자체 서비스를 실행하는 곳에서 구축하고 실행해야 합니다. 모든 ASGI 서버에서 실행됩니다(예: `uvicorn my_evaluator:app`); [HTTP 계약](#http-contract)의 `/health`, `/config`, `/evaluate` 라우트를 제공한 다음 서버가 해당 서비스를 가리키도록 설정합니다([서버 구성](#configuring-the-server) 참조). +평가자는 **사용자의 서비스**입니다 — FailproofAI Cloud는 기본 평가자를 제공하지 않으므로, 자체 서비스를 실행하는 곳에서 구축하고 실행해야 합니다. 모든 ASGI 서버에서 실행됩니다(예: `uvicorn my_evaluator:app`); [HTTP 계약](#http-contract)의 `/health`, `/config`, `/evaluate` 라우트를 제공한 다음 서버가 해당 서비스를 가리키도록 설정합니다([서버 구성](#configuring-the-server) 참조). 평가자에 접근할 수 있으면 `GET /health`는 `{"status":"ok"}`를 반환합니다. 에이전트가 엔드-투-엔드 실행을 완료한 후, 서버의 `GET /evaluations`는 `status: "done"` 및 평가자가 생성한 점수가 포함된 행을 반환합니다. @@ -258,7 +258,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. - **`/sessions`**: 필터링 가능한 세션 그리드; 점수 열에는 각 세션의 평가 상태와 점수가 한눈에 표시됩니다. - **`/dashboards`**: 저장된 평가 상태 뷰(아래 [대시보드](#dashboards) 참조). -![세션별 평가 상태 필과 색상으로 구분된 점수 배지(helpfulness, factuality, tool_efficiency, safety, coherence)가 있는 세션 그리드](/agenteye/images/sessions-list.png) +![세션별 평가 상태 필과 색상으로 구분된 점수 배지(helpfulness, factuality, tool_efficiency, safety, coherence)가 있는 세션 그리드](/cloud/images/sessions-list.png) *세션 그리드는 각 실행의 평가 상태와 점수를 한눈에 보여줍니다; 빨간색/주황색/녹색 배지로 낮은 점수가 눈에 띄게 표시됩니다.* @@ -275,7 +275,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. 각 카드에는 일치하는 세션 수, 완료/오류/타임아웃 분류, 각 특성화된 점수의 평균, 소형 추세 스파크라인이 표시됩니다. 대시보드를 열면 전체 크기 패널이 표시되며; **"세션에서 열기"**를 누르면 정확히 해당 슬라이스로 미리 필터링된 세션 페이지로 이동합니다. 메트릭은 전체 일치 집합에 대해 서버 측에서 계산됩니다(`GET /evaluations/aggregate` 사용), 따라서 숫자는 샘플링이 아닌 정확한 값입니다. -![평가자 차원별 평균 점수 바, 도구 성공/오류 분류, 상위 도구, 시간당 이벤트 추세가 있는 평가 상태 대시보드](/agenteye/images/dashboard-quality.png) +![평가자 차원별 평균 점수 바, 도구 성공/오류 분류, 상위 도구, 시간당 이벤트 추세가 있는 평가 상태 대시보드](/cloud/images/dashboard-quality.png) **권한:** 보기에는 `dashboards:read`와 `evaluations:read` 모두 필요합니다; 생성 및 편집에는 `dashboards:write`가 필요합니다; 삭제에는 `dashboards:delete`가 필요합니다. 부트스트랩 관리자는 이 모든 권한을 자동으로 받습니다. @@ -297,7 +297,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. ## 다음 단계 -- [평가자 에이전트 스킬](/ko/agenteye/evaluator-skill): 코딩 에이전트가 실제 세션을 바탕으로 차원을 설계하고 이 서비스를 구축하도록 합니다. -- [Python SDK](/ko/agenteye/python-sdk): 점수화를 트리거하는 `agent_end` 이벤트를 전송합니다. -- [API 키](/ko/agenteye/api-keys): `evaluations:read` 및 `evaluations:trigger` 권한. -- [감사](/ko/agenteye/audits): 정책 기반 검토를 위한 Observability의 또 다른 자동화된 품질 기능. \ No newline at end of file +- [평가자 에이전트 스킬](/ko/cloud/agent-skills): 코딩 에이전트가 실제 세션을 바탕으로 차원을 설계하고 이 서비스를 구축하도록 합니다. +- [Python SDK](/ko/cloud/sdk): 점수화를 트리거하는 `agent_end` 이벤트를 전송합니다. +- [API 키](/ko/cloud/access): `evaluations:read` 및 `evaluations:trigger` 권한. +- [감사](/ko/cloud/audits): 정책 기반 검토를 위한 Observability의 또 다른 자동화된 품질 기능. \ No newline at end of file diff --git a/docs/ko/agenteye/event-stream.mdx b/docs/ko/cloud/event-stream.mdx similarity index 80% rename from docs/ko/agenteye/event-stream.mdx rename to docs/ko/cloud/event-stream.mdx index 52067e53..b5883921 100644 --- a/docs/ko/agenteye/event-stream.mdx +++ b/docs/ko/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "에이전트가 무언가를 하는 순간, 바로 확인할 수 에이전트가 무언가를 하는 순간, 바로 확인할 수 있습니다. 이벤트 스트림은 프로덕션의 모든 에이전트를 실시간으로 파악할 수 있는 창구입니다. 기다릴 필요도, 로그를 grep할 필요도, 방금 무슨 일이 일어났는지 추측할 필요도 없습니다. -![실시간 이벤트 스트림: 색상으로 구분된 이벤트 행이 실시간으로 업데이트되며, 환경·에이전트·세션·이벤트 유형·자유 텍스트로 필터링 가능](/agenteye/images/events-stream.png) +![실시간 이벤트 스트림: 색상으로 구분된 이벤트 행이 실시간으로 업데이트되며, 환경·에이전트·세션·이벤트 유형·자유 텍스트로 필터링 가능](/cloud/images/events-stream.png) *조직 내 모든 에이전트의 모든 이벤트가 최신순으로 표시되며, 발생하는 즉시 업데이트됩니다.* @@ -37,14 +37,14 @@ description: "에이전트가 무언가를 하는 순간, 바로 확인할 수 이벤트 스트림은 조직의 홈 화면입니다. 로그인하면 `//`에서 가장 먼저 보이는 화면이 바로 이벤트 스트림이므로, 도착하는 순간부터 트리아지를 시작할 수 있습니다. -이면에서는 에이전트가 SDK를 통해 이벤트를 내보내고, 수집기가 이를 Failproof AI Observability 서버로 전송하며, 스트림이 여러분이 관리하는 인프라에 도착하는 대로 이벤트를 추적합니다. 원시 로그 대신 집계된 뷰를 원한다면, 각 실행의 이벤트가 Sessions에서 단일 행으로 접혀 표시되며 클릭 한 번으로 확인할 수 있습니다. +이면에서는 에이전트가 SDK를 통해 이벤트를 내보내고, 수집기가 이를 FailproofAI Cloud 서버로 전송하며, 스트림이 여러분이 관리하는 인프라에 도착하는 대로 이벤트를 추적합니다. 원시 로그 대신 집계된 뷰를 원한다면, 각 실행의 이벤트가 Sessions에서 단일 행으로 접혀 표시되며 클릭 한 번으로 확인할 수 있습니다. 이벤트 스트림은 다른 모든 관측 화면이 기반으로 삼는 원시 진실의 원천입니다. 다른 곳에서 숫자가 이상해 보인다면, 실제로 무슨 일이 있었는지 확인하는 곳은 바로 이 스트림입니다. ## 관련 항목 -- [Sessions](/ko/agenteye/sessions): 동일한 이벤트를 실행 단위의 한 행으로 집계하며, git 스타일의 실행 그래프를 제공합니다. -- [Telemetry](/ko/agenteye/telemetry): 에이전트가 전송하는 내용과 이벤트가 스트림에 도달하는 방식. -- [Error tracking](/ko/agenteye/error-tracking): 모든 오류를 한 곳에서 트리아지할 수 있는 화면. -- [Alerts](/ko/agenteye/alerts): 임계값을 알림 규칙으로 전환. -- [CLI and agents](/ko/agenteye/cli-and-agents): 터미널에서 동일한 실시간 추적. \ No newline at end of file +- [Sessions](/ko/cloud/sessions): 동일한 이벤트를 실행 단위의 한 행으로 집계하며, git 스타일의 실행 그래프를 제공합니다. +- [Telemetry](/ko/cloud/performance): 에이전트가 전송하는 내용과 이벤트가 스트림에 도달하는 방식. +- [Error tracking](/ko/cloud/errors): 모든 오류를 한 곳에서 트리아지할 수 있는 화면. +- [Alerts](/ko/cloud/alerts): 임계값을 알림 규칙으로 전환. +- [CLI and agents](/ko/cloud/cli): 터미널에서 동일한 실시간 추적. \ No newline at end of file diff --git a/docs/ko/cloud/fleet.mdx b/docs/ko/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/ko/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/ko/agenteye/incidents.mdx b/docs/ko/cloud/incidents.mdx similarity index 87% rename from docs/ko/agenteye/incidents.mdx rename to docs/ko/cloud/incidents.mdx index 27f1bf45..719af8ae 100644 --- a/docs/ko/agenteye/incidents.mdx +++ b/docs/ko/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "알림이 발생하면, 누구나 인시던트가 열려 있는지 알림이 발생했을 때 가장 먼저 드는 질문은 언제나 "누가 담당하고 있나요?"입니다. 인시던트가 그 답을 제공합니다. 무언가 임계값을 초과하는 순간, 누구나 인시던트가 열려 있다는 사실, 담당자가 누구인지, 지금까지 정확히 어떤 일이 있었는지를 확인할 수 있습니다. 사후 검토(post-mortem)에 바로 활용할 수 있는 깔끔하고 귀속된 기록과 함께요. -![인시던트 인박스: 알림과 연결되거나 수동으로 생성된 인시던트 카드들이 상태별로 그룹화되어 있으며, 각각에는 심각도 배지와 담당자가 표시됩니다](/agenteye/images/incidents.png) +![인시던트 인박스: 알림과 연결되거나 수동으로 생성된 인시던트 카드들이 상태별로 그룹화되어 있으며, 각각에는 심각도 배지와 담당자가 표시됩니다](/cloud/images/incidents.png) *인박스는 열린 인시던트를 상태별로 그룹화하고 심각도 및 담당자로 필터링하므로, 지금 즉시 사람이 처리해야 할 것이 무엇인지 바로 확인할 수 있습니다.* ## 담당자를 한눈에 파악하세요 @@ -17,10 +17,10 @@ description: "알림이 발생하면, 누구나 인시던트가 열려 있는지 인시던트가 종료되면 이미 보고서가 완성되어 있습니다. 인시던트를 열면 임계값 초과 증거, 담당자 및 구독자, 현장에서 협업을 위한 댓글 스레드, 그리고 추가 전용(append-only) 활동 타임라인을 확인할 수 있습니다. -![인시던트 상세 보기: 상위 알림 및 임계값 초과 요약, 담당자 및 구독자, 귀속된 활동 타임라인, 댓글 스레드](/agenteye/images/incident-detail.png) +![인시던트 상세 보기: 상위 알림 및 임계값 초과 요약, 담당자 및 구독자, 귀속된 활동 타임라인, 댓글 스레드](/cloud/images/incident-detail.png) *발생한 모든 일이 순서대로 기록되며, 각 항목마다 실행한 담당자의 서명이 붙습니다.* -모든 액션(열림, 확인, 해결 등)은 해당 타임라인에 기록되며 절대 수정되거나 삭제되지 않습니다. 각 항목은 귀속됩니다. 액션을 취한 운영자의 이메일로, 또는 임계값 초과 시 인시던트를 여는 것처럼 Failproof AI Observability가 자체적으로 수행한 작업에는 **automated**로 표시됩니다. 익명 처리되거나 손실되는 것은 없으므로, 사후 검토가 거의 자동으로 완성됩니다. +모든 액션(열림, 확인, 해결 등)은 해당 타임라인에 기록되며 절대 수정되거나 삭제되지 않습니다. 각 항목은 귀속됩니다. 액션을 취한 운영자의 이메일로, 또는 임계값 초과 시 인시던트를 여는 것처럼 FailproofAI Cloud가 자체적으로 수행한 작업에는 **automated**로 표시됩니다. 익명 처리되거나 손실되는 것은 없으므로, 사후 검토가 거의 자동으로 완성됩니다. ## 인시던트의 상태 전환 @@ -45,6 +45,6 @@ stateDiagram-v2 ## 관련 항목 -- [알림](/ko/agenteye/alerts): 임계값이 초과될 때 인시던트를 여는 규칙입니다. -- [오류 추적](/ko/agenteye/error-tracking): 모든 장애를 한 곳에서 확인하고 알림으로 승격시킵니다. -- [감사](/ko/agenteye/audits): 어떤 규칙도 감지하지 못한 장애를 찾아내는 예약된 분석기입니다. \ No newline at end of file +- [알림](/ko/cloud/alerts): 임계값이 초과될 때 인시던트를 여는 규칙입니다. +- [오류 추적](/ko/cloud/errors): 모든 장애를 한 곳에서 확인하고 알림으로 승격시킵니다. +- [감사](/ko/cloud/audits): 어떤 규칙도 감지하지 못한 장애를 찾아내는 예약된 분석기입니다. \ No newline at end of file diff --git a/docs/ko/cloud/managed-policies.mdx b/docs/ko/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/ko/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/ko/agenteye/overview.mdx b/docs/ko/cloud/overview.mdx similarity index 51% rename from docs/ko/agenteye/overview.mdx rename to docs/ko/cloud/overview.mdx index b3c99b28..4366597c 100644 --- a/docs/ko/agenteye/overview.mdx +++ b/docs/ko/cloud/overview.mdx @@ -1,16 +1,16 @@ --- title: "Failproof AI: 에이전트 장애 관측" -description: "Failproof AI Observability는 프로덕션 환경에서 AI 에이전트를 관측, 평가, 개선하기 위한 자체 호스팅 플랫폼입니다." +description: "FailproofAI Cloud는 프로덕션 환경에서 AI 에이전트를 관측, 평가, 개선하기 위한 자체 호스팅 플랫폼입니다." --- -Failproof AI Observability는 프로덕션 환경에서 AI 에이전트를 관측, 평가, 개선하기 위한 자체 호스팅 플랫폼입니다. 에이전트의 모든 동작(모든 도구 호출, 모델 요청, 훅, 오류)을 기록하고, 각 실행의 품질을 점수화하며, 미처 발견하지 못했던 장애를 찾아내 여러분의 인프라 내에서 직접 운영하는 대시보드에 표시합니다. +FailproofAI Cloud는 프로덕션 환경에서 AI 에이전트를 관측, 평가, 개선하기 위한 자체 호스팅 플랫폼입니다. 에이전트의 모든 동작(모든 도구 호출, 모델 요청, 훅, 오류)을 기록하고, 각 실행의 품질을 점수화하며, 미처 발견하지 못했던 장애를 찾아내 여러분의 인프라 내에서 직접 운영하는 대시보드에 표시합니다. -AI 에이전트를 운영 중이고 실행이 왜 잘못됐는지 계속 추측하는 데 지쳤다면, 여기서 시작하세요. 이 문서는 설치 전에 Failproof AI Observability가 제공하는 것과 각 구성 요소가 어떻게 연결되는지 설명합니다. +AI 에이전트를 운영 중이고 실행이 왜 잘못됐는지 계속 추측하는 데 지쳤다면, 여기서 시작하세요. 이 문서는 설치 전에 FailproofAI Cloud가 제공하는 것과 각 구성 요소가 어떻게 연결되는지 설명합니다. -> **Failproof AI Observability는 Failproof AI의 엔터프라이즈 제품입니다.** 실제 동작을 보고 싶으신가요? 데모를 요청하세요: [nikita@befailproof.ai](mailto:nikita@befailproof.ai)로 이메일 보내주세요. +> **FailproofAI Cloud는 Failproof AI의 엔터프라이즈 제품입니다.** 실제 동작을 보고 싶으신가요? 데모를 요청하세요: [nikita@befailproof.ai](mailto:nikita@befailproof.ai)로 이메일 보내주세요. -![git 스타일의 실행 그래프와 이벤트 타임라인이 나란히 표시된 Failproof AI Observability 세션, 오른쪽 패널에는 도구·모델·훅의 실행별 분석 정보 표시](/agenteye/images/session-detail.png) +![git 스타일의 실행 그래프와 이벤트 타임라인이 나란히 표시된 FailproofAI Cloud 세션, 오른쪽 패널에는 도구·모델·훅의 실행별 분석 정보 표시](/cloud/images/session-detail.png) *모든 에이전트 실행은 git 스타일의 실행 그래프(왼쪽)와 이벤트 타임라인이 나란히 표시됩니다. 병렬 서브 에이전트는 각각 별도의 레인을 가지며, 오른쪽 패널에서 해당 실행의 도구, 모델, 훅, 토큰 사용량을 상세히 확인할 수 있습니다.* @@ -30,52 +30,52 @@ AI 에이전트를 운영 중이고 실행이 왜 잘못됐는지 계속 추측
    -*Failproof 감사: Failproof AI Observability가 세션 전반의 로그를 분석해 수정이 필요한 사항을 알려줍니다.* +*Failproof 감사: FailproofAI Cloud가 세션 전반의 로그를 분석해 수정이 필요한 사항을 알려줍니다.* --- ## 팀이 이 도구를 사용하는 이유 - **에이전트가 실제로 무엇을 했는지 확인하세요.** 모든 실행이 읽기 쉬운 git 스타일의 실행 그래프로 변환됩니다: 어떤 도구가 병렬로 실행됐는지, 어떤 서브 에이전트가 분기했는지, 어디서 멈췄는지, 무엇을 사용했는지 한눈에 볼 수 있습니다. -- **품질 저하를 자동으로 감지하세요.** 소규모 점수화 서비스를 연결하면 Failproof AI Observability가 완료된 모든 실행을 점수화하여, 유용성 하락이나 환각 급증을 자동으로 감지합니다. +- **품질 저하를 자동으로 감지하세요.** 소규모 점수화 서비스를 연결하면 FailproofAI Cloud가 완료된 모든 실행을 점수화하여, 유용성 하락이나 환각 급증을 자동으로 감지합니다. - **미리 규칙을 작성하지 않아도 장애를 찾아냅니다.** 반복 감사가 세션 전반의 로그에서 오류 클러스터, 지연 이상값, 낮은 점수, 중단된 실행을 발굴하고, 증거가 뒷받침된 우선순위 결과를 제시합니다. - **중요한 순간에 알림을 받으세요.** 오류율, 지연 시간, 비용, 또는 평가 점수에 대한 임계값 규칙이 발동되면 인시던트가 생성되어 확인, 담당자 지정, 해결까지 처리할 수 있습니다. - **일반 언어로 질문하세요.** 대시보드 내 AI 어시스턴트가 여러분의 데이터를 기반으로 "이번 주 프로덕션에서 품질 트렌드는 어떤가요?" 같은 질문에 답합니다. 어시스턴트가 변경하는 모든 사항은 승인이 필요합니다. -- **데이터를 직접 관리하세요.** Failproof AI Observability는 자체 호스팅 방식으로, 이벤트, 프롬프트, 분석 데이터가 여러분이 제어하는 인프라 안에 머뭅니다. +- **데이터를 직접 관리하세요.** FailproofAI Cloud는 자체 호스팅 방식으로, 이벤트, 프롬프트, 분석 데이터가 여러분이 제어하는 인프라 안에 머뭅니다. --- ## 제공 기능 -Failproof AI Observability는 세 가지 개념(**관측**, **분석**, **관리**)을 중심으로 구성되며, 이는 대시보드 왼쪽 사이드바에 그대로 반영됩니다. +FailproofAI Cloud는 세 가지 개념(**관측**, **분석**, **관리**)을 중심으로 구성되며, 이는 대시보드 왼쪽 사이드바에 그대로 반영됩니다. **관측** (실제로 무슨 일이 있었는지의 원본 데이터): -- **[이벤트 스트림](/ko/agenteye/event-stream)**: 모든 실행의 단계별 실시간 기록 (도구 호출, 모델 호출, 훅, 오류). -- **[세션](/ko/agenteye/sessions)**: 실행별로 집계된 이벤트로, 각 실행은 점수화 준비가 된 한 행으로 표시되며 git 스타일의 실행 그래프를 포함합니다. -- **[성능 메트릭](/ko/agenteye/telemetry)**: 표면별 지연 시간 히트맵과 모델, 도구, 훅에 대한 p50/p95/p99 지표로, 꼬리 구간의 급증을 중앙값과 비교해 식별합니다. -- **[오류 추적](/ko/agenteye/error-tracking)**: 발생한 모든 문제를 한 곳에서 트리아지하고, 발동된 알림에서 한 번의 클릭으로 접근할 수 있습니다. +- **[이벤트 스트림](/ko/cloud/event-stream)**: 모든 실행의 단계별 실시간 기록 (도구 호출, 모델 호출, 훅, 오류). +- **[세션](/ko/cloud/sessions)**: 실행별로 집계된 이벤트로, 각 실행은 점수화 준비가 된 한 행으로 표시되며 git 스타일의 실행 그래프를 포함합니다. +- **[성능 메트릭](/ko/cloud/performance)**: 표면별 지연 시간 히트맵과 모델, 도구, 훅에 대한 p50/p95/p99 지표로, 꼬리 구간의 급증을 중앙값과 비교해 식별합니다. +- **[오류 추적](/ko/cloud/errors)**: 발생한 모든 문제를 한 곳에서 트리아지하고, 발동된 알림에서 한 번의 클릭으로 접근할 수 있습니다. -![도구 관측 페이지: 24개의 시간 구간에 걸친 지연 시간 히트맵, 백분위수 밴드, 도구 분포 바](/agenteye/images/tools.png) +![도구 관측 페이지: 24개의 시간 구간에 걸친 지연 시간 히트맵, 백분위수 밴드, 도구 분포 바](/cloud/images/tools.png) *각 관측 화면은 스파크라인과 p50/p95/p99 지표를 지연 시간 히트맵 및 백분위수 밴드와 함께 표시합니다. 여기서는 도구(Tools) 화면을 보여줍니다.* **분석** (활동을 인사이트로 전환): -- **[쿼리](/ko/agenteye/queries)** 및 **[대시보드](/ko/agenteye/dashboards)**: 이벤트와 평가 데이터에 대해 저장된 SQL을 실행하고, 조직 범위의 공유 대시보드로 시각화합니다. -- **[평가](/ko/agenteye/evaluations)**: 자체 평가 서비스가 생성하는 품질 점수와 점수별 근거. -- **[감사](/ko/agenteye/audits)**: 세션 전반에서 장애 패턴을 발굴하는 반복 조사. -- **[알림](/ko/agenteye/alerts)** 및 **[인시던트](/ko/agenteye/incidents)**: 알림을 발송하는 임계값 규칙과, 이를 트리아지할 수 있는 인시던트 워크플로우. +- **[쿼리](/ko/cloud/queries)** 및 **[대시보드](/ko/cloud/dashboards)**: 이벤트와 평가 데이터에 대해 저장된 SQL을 실행하고, 조직 범위의 공유 대시보드로 시각화합니다. +- **[평가](/ko/cloud/evaluations)**: 자체 평가 서비스가 생성하는 품질 점수와 점수별 근거. +- **[감사](/ko/cloud/audits)**: 세션 전반에서 장애 패턴을 발굴하는 반복 조사. +- **[알림](/ko/cloud/alerts)** 및 **[인시던트](/ko/cloud/incidents)**: 알림을 발송하는 임계값 규칙과, 이를 트리아지할 수 있는 인시던트 워크플로우. **인터페이스** (원하는 방식으로 데이터에 접근): -- **[CLI](/ko/agenteye/cli-and-agents)**: 터미널이나 스크립트에서 전체 배포를 제어하고, 코딩 에이전트가 일반 언어로 대신 처리하도록 할 수 있습니다. -- **[AI 어시스턴트](/ko/agenteye/assistant)**: 대시보드 내에서 일반 언어로 에이전트에 대해 질문하세요. -- **REST API**: 대시보드와 CLI의 모든 기능은 범위가 지정된 [API 키](/ko/agenteye/api-keys)로 직접 호출할 수 있는 REST API로 지원됩니다 — 이벤트 수집, 세션 및 평가 쿼리, 대시보드·알림·감사·사용자·키 관리까지 가능하여, Failproof AI Observability를 자체 도구와 연동할 수 있습니다. +- **[CLI](/ko/cloud/cli)**: 터미널이나 스크립트에서 전체 배포를 제어하고, 코딩 에이전트가 일반 언어로 대신 처리하도록 할 수 있습니다. +- **[AI 어시스턴트](/ko/cloud/assistant)**: 대시보드 내에서 일반 언어로 에이전트에 대해 질문하세요. +- **REST API**: 대시보드와 CLI의 모든 기능은 범위가 지정된 [API 키](/ko/cloud/access)로 직접 호출할 수 있는 REST API로 지원됩니다 — 이벤트 수집, 세션 및 평가 쿼리, 대시보드·알림·감사·사용자·키 관리까지 가능하여, FailproofAI Cloud를 자체 도구와 연동할 수 있습니다. **관리** (팀을 위한 운영): -- **[API 키](/ko/agenteye/api-keys)**: 수집기, 대시보드, 어시스턴트용 범위 지정 토큰. +- **[API 키](/ko/cloud/access)**: 수집기, 대시보드, 어시스턴트용 범위 지정 토큰. - **사용자**: 허용 목록 기반의 이메일 패스워드리스 로그인. - **설정**: 모델 컨텍스트 윈도우 재정의를 포함한 조직별 구성. @@ -91,18 +91,18 @@ Failproof AI Observability는 세 가지 개념(**관측**, **분석**, **관리 - **대시보드**: 모든 것을 탐색하는 공간. - **선택적 서비스**: 점수화 서비스(평가)와 AI 어시스턴트 서비스(대시보드 내 채팅). -문서 전반에서 사용하는 용어(*이벤트, 세션, 평가, 감사, 결과, 인시던트*)에 대해서는 [개념](/ko/agenteye/concepts)을 참고하세요. +문서 전반에서 사용하는 용어(*이벤트, 세션, 평가, 감사, 결과, 인시던트*)에 대해서는 [개념](/ko/concepts)을 참고하세요. --- -## Failproof AI Observability 도입하기 +## FailproofAI Cloud 도입하기 -Failproof AI Observability는 Failproof AI의 엔터프라이즈 제품으로, Failproof AI 브랜드 아래 정책 및 가드레일 제품인 Failproof AI Enforcement와 함께 동작합니다. 완전히 여러분의 환경에서 실행됩니다. 아직 패키지에 대한 접근 권한이 없다면, 데모를 요청해 주세요: [nikita@befailproof.ai](mailto:nikita@befailproof.ai)로 이메일을 보내주시면 시작을 도와드리겠습니다. +FailproofAI Cloud는 Failproof AI의 엔터프라이즈 제품으로, Failproof AI 브랜드 아래 정책 및 가드레일 제품인 FailproofAI guardrails와 함께 동작합니다. 완전히 여러분의 환경에서 실행됩니다. 아직 패키지에 대한 접근 권한이 없다면, 데모를 요청해 주세요: [nikita@befailproof.ai](mailto:nikita@befailproof.ai)로 이메일을 보내주시면 시작을 도와드리겠습니다. --- ## 다음 단계 -- [개념](/ko/agenteye/concepts): Failproof AI Observability 용어를 한 곳에서 정리한 문서. -- [Observability](/ko/agenteye/observability): 에이전트의 동작을 실행별로 추적하기. -- [보안](/ko/agenteye/security): Failproof AI Observability가 데이터를 격리하고 여러분의 통제 하에 유지하는 방법. \ No newline at end of file +- [개념](/ko/concepts): FailproofAI Cloud 용어를 한 곳에서 정리한 문서. +- [FailproofAI Cloud](/ko/cloud/overview): 에이전트의 동작을 실행별로 추적하기. +- [보안](/ko/cloud/security): FailproofAI Cloud가 데이터를 격리하고 여러분의 통제 하에 유지하는 방법. \ No newline at end of file diff --git a/docs/ko/agenteye/telemetry.mdx b/docs/ko/cloud/performance.mdx similarity index 82% rename from docs/ko/agenteye/telemetry.mdx rename to docs/ko/cloud/performance.mdx index f8a36493..4b02daf4 100644 --- a/docs/ko/agenteye/telemetry.mdx +++ b/docs/ko/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "모델, 도구, 훅이 느려지거나 비용이 급증하는 순 모델, 도구, 훅이 느려지거나 비용이 급증하는 순간을 즉시 파악하고, 사용자가 체감하기 전에 테일 레이턴시 스파이크를 잡아내세요. 세 개의 전용 페이지가 원시 타이밍 데이터를 한눈에 읽을 수 있는 p50, p95, p99 수치로 변환해 줍니다. -![Models 페이지에 레이턴시 히트맵, 백분위 밴드, 모델별 토큰·비용·컨텍스트 윈도우 수치가 표시된 화면](/agenteye/images/models.png) +![Models 페이지에 레이턴시 히트맵, 백분위 밴드, 모델별 토큰·비용·컨텍스트 윈도우 수치가 표시된 화면](/cloud/images/models.png) *Models 페이지: 레이턴시 히트맵, 백분위 밴드, 모델별 토큰 수·예상 비용·컨텍스트 윈도우 사용률.* ## 평균값이 최악의 실행을 숨기지 못하게 하세요 @@ -24,13 +24,13 @@ description: "모델, 도구, 훅이 느려지거나 비용이 급증하는 순 Models 페이지(위 이미지 참고)는 청구서를 받을 때 항상 드는 두 가지 질문에 답합니다. 어떤 모델인가, 그리고 얼마인가. 공유 레이턴시 뷰 위에 **모델별 토큰 소비량**, **예상 비용**, **컨텍스트 윈도우 사용률**이 추가되므로, 프롬프트가 통제 불능으로 늘어나는 상황이나 임박한 압축(compaction)을 미리 감지할 수 있습니다. -Failproof AI Observability는 일반적인 모델 ID를 자동으로 인식합니다. 윈도우 크기가 잘못 표시되거나 자체 프라이빗 모델을 사용하는 경우, **Settings**의 **model context windows**에서 수정하거나 추가하면 사용률 수치에 즉시 반영됩니다. +FailproofAI Cloud는 일반적인 모델 ID를 자동으로 인식합니다. 윈도우 크기가 잘못 표시되거나 자체 프라이빗 모델을 사용하는 경우, **Settings**의 **model context windows**에서 수정하거나 추가하면 사용률 수치에 즉시 반영됩니다. ## Tools: 느린 것과 고장난 것을 구분하세요 도구 호출은 느릴 수도 있고, 조용히 실패하고 있을 수도 있습니다. 로그를 뒤지는 것이 아니라 몇 초 안에 어느 쪽인지 알아야 합니다. -![Tools 페이지에 공유 레이턴시 히트맵과 백분위 밴드, 성공·실패 분류, 도구 분포 막대가 표시된 화면](/agenteye/images/tools.png) +![Tools 페이지에 공유 레이턴시 히트맵과 백분위 밴드, 성공·실패 분류, 도구 분포 막대가 표시된 화면](/cloud/images/tools.png) *Tools 페이지: 동일한 히트맵과 백분위 밴드에 성공·실패 분류 및 도구 분포 막대 추가.* 공유 레이턴시 뷰와 함께 Tools 페이지는 **성공·실패 분류**와 **도구 분포 막대**를 제공합니다. 어떤 도구를 가장 많이 사용하는지, 어떤 도구가 에러 버짓을 갉아먹고 있는지 한눈에 확인할 수 있습니다. @@ -39,14 +39,14 @@ Failproof AI Observability는 일반적인 모델 ID를 자동으로 인식합 라이프사이클 훅이 실행을 지연시킬 때, "훅이 느리다"는 말만으로는 조치를 취할 수 없습니다. Hooks 페이지는 문제가 되는 바로 그 훅으로 곧장 안내합니다. -![Hooks 페이지에 훅 이름과 트리거 이벤트별로 분류된 레이턴시가 공유 히트맵과 백분위 밴드 위에 표시된 화면](/agenteye/images/hooks.png) +![Hooks 페이지에 훅 이름과 트리거 이벤트별로 분류된 레이턴시가 공유 히트맵과 백분위 밴드 위에 표시된 화면](/cloud/images/hooks.png) *Hooks 페이지: 훅 이름과 트리거 이벤트별로 분류된 레이턴시.* 동일한 레이턴시 히트맵과 백분위 밴드 위에서, Hooks 페이지는 활동을 **훅 이름**과 **트리거 이벤트**별로 세분화합니다. 주의가 필요한 단 하나의 훅과 단 하나의 이벤트를 바로 찾아낼 수 있습니다. ## 관련 문서 -- [이벤트 스트림](/ko/agenteye/event-stream): 모든 이벤트의 실시간 컬러 코딩 추적. -- [세션](/ko/agenteye/sessions): 이벤트를 실행 단위의 단일 행으로 집계하고 실행 그래프를 열람. -- [에러 트래킹](/ko/agenteye/error-tracking): 대시보드에서 빨간색으로 표시된 모든 항목을 위한 단일 트리아지 화면. -- [대시보드](/ko/agenteye/dashboards): 전체 플릿에 걸친 롤업 뷰. \ No newline at end of file +- [이벤트 스트림](/ko/cloud/event-stream): 모든 이벤트의 실시간 컬러 코딩 추적. +- [세션](/ko/cloud/sessions): 이벤트를 실행 단위의 단일 행으로 집계하고 실행 그래프를 열람. +- [에러 트래킹](/ko/cloud/errors): 대시보드에서 빨간색으로 표시된 모든 항목을 위한 단일 트리아지 화면. +- [대시보드](/ko/cloud/dashboards): 전체 플릿에 걸친 롤업 뷰. \ No newline at end of file diff --git a/docs/ko/agenteye/queries.mdx b/docs/ko/cloud/queries.mdx similarity index 77% rename from docs/ko/agenteye/queries.mdx rename to docs/ko/cloud/queries.mdx index a5541fd3..ab824623 100644 --- a/docs/ko/agenteye/queries.mdx +++ b/docs/ko/cloud/queries.mdx @@ -4,9 +4,9 @@ description: "에이전트 데이터에 어떤 질문이든 던지고 몇 초 --- -에이전트 데이터에 어떤 질문이든 던지고 몇 초 안에 답을 얻으세요. Failproof AI Observability는 이벤트와 평가 데이터에 대한 저장된 실행 가능 쿼리 라이브러리를 제공합니다. 빈 SQL 편집기 대신 이미 동작하는 예제에서 바로 시작할 수 있습니다. +에이전트 데이터에 어떤 질문이든 던지고 몇 초 안에 답을 얻으세요. FailproofAI Cloud는 이벤트와 평가 데이터에 대한 저장된 실행 가능 쿼리 라이브러리를 제공합니다. 빈 SQL 편집기 대신 이미 동작하는 예제에서 바로 시작할 수 있습니다. -![저장된 쿼리 라이브러리: 기본 제공 프리셋과 사용자 지정 쿼리가 함께 표시된 그리드](/agenteye/images/queries.png) +![저장된 쿼리 라이브러리: 기본 제공 프리셋과 사용자 지정 쿼리가 함께 표시된 그리드](/cloud/images/queries.png) *`//queries`에 있는 저장된 쿼리 라이브러리: 기본 제공 프리셋과 팀이 저장한 쿼리가 나란히 배치됩니다.* @@ -22,7 +22,7 @@ description: "에이전트 데이터에 어떤 질문이든 던지고 몇 초 쿼리를 열면 SQL 작성기로 이동하며, 여기서 바로 수정하고 즉시 결과를 확인할 수 있습니다. 내보내기도, 왕복 요청도, 다른 사람을 기다릴 필요도 없습니다. -![저장된 쿼리를 실행 중인 SQL 작성기 — 스키마 사이드바와 실시간 결과 그리드 포함](/agenteye/images/query-lab.png) +![저장된 쿼리를 실행 중인 SQL 작성기 — 스키마 사이드바와 실시간 결과 그리드 포함](/cloud/images/query-lab.png) *SQL 작성기: 왼쪽에 쿼리, 컬럼 이름을 추측하지 않아도 되는 스키마 사이드바, 아래에 실시간 결과 그리드.* @@ -43,14 +43,14 @@ agenteye query list # 터미널에서 동일한 저장된 쿼리 agenteye query run errs --arg prod # 실행하고 행 출력 (파이프 연결 시 --json 추가) ``` - 전체 명령어 목록은 [CLI and agents](/ko/agenteye/cli-and-agents)를 참조하세요. + 전체 명령어 목록은 [CLI and agents](/ko/cloud/cli)를 참조하세요. -- **AI 어시스턴트에서.** SQL 표현이 어렵다면? 대시보드 내 [AI 어시스턴트](/ko/agenteye/assistant)에게 평범한 언어로 질문하면 쿼리를 작성하고 라이브러리에 저장해 드립니다. +- **AI 어시스턴트에서.** SQL 표현이 어렵다면? 대시보드 내 [AI 어시스턴트](/ko/cloud/assistant)에게 평범한 언어로 질문하면 쿼리를 작성하고 라이브러리에 저장해 드립니다. 저장된 쿼리 실행은 `queries:run` 권한으로 제어되며, 쿼리 생성 및 삭제 권한과 별도로 분리되어 있습니다. 따라서 라이브러리 수정 권한 없이 읽기 전용 접근만 부여할 수 있습니다. ## 관련 문서 -- [Dashboards](/ko/agenteye/dashboards): 쿼리 결과를 조직 전체가 공유하는 차트에 고정합니다. -- [AI assistant](/ko/agenteye/assistant): 평범한 언어로 질문하고 쿼리를 받아보세요. -- [CLI and agents](/ko/agenteye/cli-and-agents): 터미널에서 동일한 쿼리를 실행하고 저장합니다. \ No newline at end of file +- [Dashboards](/ko/cloud/dashboards): 쿼리 결과를 조직 전체가 공유하는 차트에 고정합니다. +- [AI assistant](/ko/cloud/assistant): 평범한 언어로 질문하고 쿼리를 받아보세요. +- [CLI and agents](/ko/cloud/cli): 터미널에서 동일한 쿼리를 실행하고 저장합니다. \ No newline at end of file diff --git a/docs/ko/agenteye/python-sdk.mdx b/docs/ko/cloud/sdk.mdx similarity index 92% rename from docs/ko/agenteye/python-sdk.mdx rename to docs/ko/cloud/sdk.mdx index 7bf29eb5..ef7b9e58 100644 --- a/docs/ko/agenteye/python-sdk.mdx +++ b/docs/ko/cloud/sdk.mdx @@ -4,11 +4,11 @@ description: "AI 에이전트가 프로덕션에서 수행한 모든 작업을 --- -AI 에이전트가 프로덕션에서 수행한 모든 작업을 확인하세요: 모든 에이전트 실행, 툴 호출, 모델 요청, hook, 그리고 사람의 개입까지. Failproof AI Observability Python SDK는 에이전트 코드 내부에서 그 기록을 남겨 디버깅, 감사, 평가에 활용할 수 있게 해줍니다. 에이전트를 Failproof AI Observability로 관찰하고 싶을 때마다 사용하세요. +AI 에이전트가 프로덕션에서 수행한 모든 작업을 확인하세요: 모든 에이전트 실행, 툴 호출, 모델 요청, hook, 그리고 사람의 개입까지. FailproofAI Cloud Python SDK는 에이전트 코드 내부에서 그 기록을 남겨 디버깅, 감사, 평가에 활용할 수 있게 해줍니다. 에이전트를 FailproofAI Cloud로 관찰하고 싶을 때마다 사용하세요. 내부적으로 SDK는 구조화된 이벤트를 로컬 JSONL 파일에 기록하며, 콜렉터 데몬이 이를 감지해 플랫폼으로 자동 전송합니다. 파일을 직접 관리할 필요가 없습니다. -> **팁:** Failproof AI Observability가 처음이신가요? 이 페이지는 SDK 이벤트의 완전한 레퍼런스입니다. +> **팁:** FailproofAI Cloud가 처음이신가요? 이 페이지는 SDK 이벤트의 완전한 레퍼런스입니다.
    @@ -26,7 +26,7 @@ SDK는 공개 패키지 인덱스가 아닌 프라이빗 휠로 고객에게 배 python -c "import agenteye; print(agenteye.__version__)" ``` -코딩 에이전트가 전체 통합을 처리하게 하고 싶으신가요? [Python SDK Agent Skill](/ko/agenteye/python-sdk-skill)은 설치 경로를 파악하고, 계측 지점을 계획·작성하며, 이벤트가 정상적으로 수신되는지 검증합니다. +코딩 에이전트가 전체 통합을 처리하게 하고 싶으신가요? [Python SDK Agent Skill](/ko/cloud/agent-skills)은 설치 경로를 파악하고, 계측 지점을 계획·작성하며, 이벤트가 정상적으로 수신되는지 검증합니다. --- @@ -60,7 +60,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### 실제 호출 계측하기 -실제로는 기존 에이전트 코드를 감싸는 방식으로 사용합니다. 모델 호출 앞에 `model_request`를, 뒤에 `model_response`를 배치하면 두 이벤트가 실제 요청을 감싸게 되어 Failproof AI Observability가 두 이벤트를 서로 연결할 수 있습니다: +실제로는 기존 에이전트 코드를 감싸는 방식으로 사용합니다. 모델 호출 앞에 `model_request`를, 뒤에 `model_response`를 배치하면 두 이벤트가 실제 요청을 감싸게 되어 FailproofAI Cloud가 두 이벤트를 서로 연결할 수 있습니다: ```python import anthropic @@ -99,7 +99,7 @@ agenteye.event.model_response( 아래는 이벤트가 대시보드에 도달했을 때의 모습입니다. 이벤트 유형별로 색상이 구분되며 환경, 에이전트, 세션별로 필터링할 수 있습니다: -![이벤트 유형별로 색상이 구분되고 환경, 에이전트, 세션별로 필터링 가능한 라이브 이벤트 스트림](/agenteye/images/events-stream.png) +![이벤트 유형별로 색상이 구분되고 환경, 에이전트, 세션별로 필터링 가능한 라이브 이벤트 스트림](/cloud/images/events-stream.png) --- @@ -153,9 +153,9 @@ SDK는 명시적으로 전달한 필드만 기록합니다. 프롬프트, 메시 ## 이벤트 레퍼런스 -대부분의 이벤트는 상관 ID를 공유하는 시작/종료 쌍으로 구성됩니다: `tool_use`와 `tool_result`는 `tool_call_id`를 공유하고, `hook_triggered`와 `hook_completed`는 `hook_id`를 공유하며, `human_wait`와 `human_input`은 `input_id`를 공유합니다. 시작 이벤트를 발행하고 작업을 수행한 뒤, 동일한 ID로 종료 이벤트를 발행하세요. Failproof AI Observability가 쌍을 매칭하고 `duration_ms`를 자동으로 계산하므로 직접 전달할 필요가 없습니다. +대부분의 이벤트는 상관 ID를 공유하는 시작/종료 쌍으로 구성됩니다: `tool_use`와 `tool_result`는 `tool_call_id`를 공유하고, `hook_triggered`와 `hook_completed`는 `hook_id`를 공유하며, `human_wait`와 `human_input`은 `input_id`를 공유합니다. 시작 이벤트를 발행하고 작업을 수행한 뒤, 동일한 ID로 종료 이벤트를 발행하세요. FailproofAI Cloud가 쌍을 매칭하고 `duration_ms`를 자동으로 계산하므로 직접 전달할 필요가 없습니다. -![페어드 이벤트로 재구성된 실행 그래프 및 타임라인과 툴/모델/hook 분류 패널이 나란히 표시된 세션 상세 화면](/agenteye/images/session-detail.png) +![페어드 이벤트로 재구성된 실행 그래프 및 타임라인과 툴/모델/hook 분류 패널이 나란히 표시된 세션 상세 화면](/cloud/images/session-detail.png) 모든 이벤트 메서드에는 다음 두 필드가 필요합니다: @@ -429,5 +429,5 @@ agenteye.event.tool_use( ## 다음 단계 -- [이벤트 스트림](/ko/agenteye/event-stream): 이벤트 유형별로 색상이 구분되고 환경, 에이전트, 세션별로 필터링 가능한 라이브 이벤트 스트림을 확인하세요. -- [세션](/ko/agenteye/sessions): 페어드 이벤트가 각 에이전트 실행을 실행 그래프 및 타임라인으로 어떻게 재구성하는지 확인하세요. \ No newline at end of file +- [이벤트 스트림](/ko/cloud/event-stream): 이벤트 유형별로 색상이 구분되고 환경, 에이전트, 세션별로 필터링 가능한 라이브 이벤트 스트림을 확인하세요. +- [세션](/ko/cloud/sessions): 페어드 이벤트가 각 에이전트 실행을 실행 그래프 및 타임라인으로 어떻게 재구성하는지 확인하세요. \ No newline at end of file diff --git a/docs/ko/cloud/security.mdx b/docs/ko/cloud/security.mdx new file mode 100644 index 00000000..af9948f9 --- /dev/null +++ b/docs/ko/cloud/security.mdx @@ -0,0 +1,68 @@ +--- +title: "보안" +description: "FailproofAI Cloud는 프로덕션 에이전트 가까이에서 동작하도록 설계되어 있으며, 프롬프트, 도구 입력값, 출력값을 모두 처리합니다." +--- + + +FailproofAI Cloud는 프로덕션 에이전트 가까이에서 동작하도록 설계되어 있으며, 프롬프트, 도구 입력값, 출력값을 모두 처리합니다. 이 페이지에서는 해당 데이터를 격리하고, 통제하며, 여러분의 손에 유지하는 방법을 설명합니다. 보안 검토를 위해 FailproofAI Cloud를 평가 중이라면 여기서 시작하세요. + +--- + +## 데이터는 여러분의 환경에 보관됩니다 + +FailproofAI Cloud는 자체 호스팅 방식입니다. 이벤트, 프롬프트, 모델 응답, 분석 데이터는 모두 여러분 자신의 환경에 있는 데이터베이스에 저장됩니다. 서드파티 SaaS에 데이터가 전송되거나 저장되지 않으며, 모든 데이터는 여러분의 클라우드 계정 내에 유지됩니다. + +--- + +## 테넌트 격리 + +하나의 FailproofAI Cloud 인스턴스에서 여러 조직을 호스팅할 수 있으며, 각 조직은 스토리지 계층에서 격리됩니다. 이 격리는 UI가 아닌 데이터베이스 수준에서 강제됩니다. + +- 조직의 운영 데이터(사용자, 키, 대시보드, 저장된 쿼리)는 해당 조직 범위로 한정되며, 조직 간 데이터 읽기는 데이터베이스 자체에서 차단됩니다. +- 수집된 모든 이벤트에는 소유 조직 정보가 기록되므로, 한 조직의 이벤트를 다른 조직에서 절대 읽을 수 없습니다. + +모든 대시보드 라우트는 org 슬러그(`//…`) 하위에 범위가 지정됩니다. + +--- + +## 로그인 + +FailproofAI Cloud는 비밀번호 없는 이메일 기반 로그인을 사용합니다. 피싱하거나 유출될 비밀번호 자체가 없습니다. 사용자가 일회용 코드(또는 원클릭 매직 링크)를 요청하면 이메일로 전송되며, 짧은 시간 내에 만료됩니다. 로그인은 **허용 목록**으로 제한됩니다. 여러분이 허용한 이메일 주소(또는 도메인)만 인증할 수 있습니다. + +![이메일로 일회용 코드를 전송하는 FailproofAI Cloud 로그인 화면](/cloud/images/login.png) + +--- + +## API 키를 이용한 범위 기반 접근 제어 + +모든 클라이언트는 세분화된 최소 권한을 가진 API 키로 인증합니다. 수집기는 `events:add` 권한만 필요하고, 대시보드 또는 어시스턴트 키는 읽기 전용으로 설정할 수 있습니다. 삭제, 재생성과 같은 파괴적인 작업은 별도의 권한으로 관리하며, 여러분이 직접 부여 여부를 결정합니다. + +![각 키의 권한 부여 현황을 읽기, 쓰기, 파괴적 범위별로 색상 구분하여 표시하는 API 키 페이지](/cloud/images/api-keys.png) + +관리자 부트스트랩 키는 설정용으로만 보관하고, 그 외 모든 용도에는 제한된 키를 발급하세요. [API 키](/ko/cloud/access) 문서를 참고하세요. + +--- + +## 읽기 전용, 승인 기반 어시스턴트 + +대시보드 내 [AI 어시스턴트](/ko/cloud/assistant)는 여러분의 데이터를 기반으로 질문에 답변하지만, 설계상 다음과 같은 제약이 있습니다. + +- **기본적으로 읽기 전용**입니다. 어시스턴트의 SQL은 `SELECT`/`WITH` 쿼리만 허용하고, 단일 구문으로 제한되며, 행 수 상한이 적용되는 가드를 통해 실행됩니다. +- 어시스턴트가 생성하는 모든 것(저장된 쿼리, 대시보드)은 **승인 기반**으로 처리됩니다. 모든 쓰기 작업은 실행 전에 여러분이 검토하고 승인해야 합니다. +- 어시스턴트는 **절대 삭제할 수 없습니다**. + +따라서 팀원이 "이번 주에 가장 많이 오류가 발생한 에이전트는 무엇인가요?"라고 묻고 결과를 활용하더라도, 어시스턴트가 스스로 데이터를 변경하거나 삭제하는 것은 불가능합니다. + +--- + +## 전송 중 보안 + +모든 트래픽은 HTTPS를 통해 전송됩니다. 여러분이 직접 인증서로 TLS를 종료하므로, 수집기-서버 간 및 브라우저-서버 간 트래픽은 전송 중 암호화됩니다. + +--- + +## 다음 단계 + +- [개요](/ko/cloud/overview): FailproofAI Cloud의 전체 구조를 확인하세요. +- [API 키](/ko/cloud/access): 수집기, 대시보드, 어시스턴트에 대한 접근 범위를 설정하세요. +- [관찰 가능성](/ko/cloud/overview): FailproofAI Cloud가 에이전트에서 수집하는 정보를 확인하세요. \ No newline at end of file diff --git a/docs/ko/agenteye/sessions.mdx b/docs/ko/cloud/sessions.mdx similarity index 73% rename from docs/ko/agenteye/sessions.mdx rename to docs/ko/cloud/sessions.mdx index 89aedb5e..d86c85db 100644 --- a/docs/ko/agenteye/sessions.mdx +++ b/docs/ko/cloud/sessions.mdx @@ -4,9 +4,9 @@ description: "한 번의 실행에서 발생한 모든 이벤트를 하나의 --- -실행이 실패한 이유를 더 이상 추측할 필요가 없습니다. Failproof AI Observability는 한 번의 실행에서 발생한 모든 이벤트를 하나의 읽기 쉬운 행으로 정리하고, 전체 실행 흐름을 몇 초 만에 파악할 수 있는 git 스타일의 그림으로 그려냅니다. 에이전트가 단계별로 정확히 무엇을 했는지 한눈에 확인할 수 있습니다. +실행이 실패한 이유를 더 이상 추측할 필요가 없습니다. FailproofAI Cloud는 한 번의 실행에서 발생한 모든 이벤트를 하나의 읽기 쉬운 행으로 정리하고, 전체 실행 흐름을 몇 초 만에 파악할 수 있는 git 스타일의 그림으로 그려냅니다. 에이전트가 단계별로 정확히 무엇을 했는지 한눈에 확인할 수 있습니다. -![세션 목록: 환경과 에이전트 전반에 걸쳐 실행별로 한 행씩 표시되며, 상태 뱃지와 평가 점수 뱃지가 함께 표시됩니다](/agenteye/images/sessions-list.png) +![세션 목록: 환경과 에이전트 전반에 걸쳐 실행별로 한 행씩 표시되며, 상태 뱃지와 평가 점수 뱃지가 함께 표시됩니다](/cloud/images/sessions-list.png) *실행당 한 행: 상태 뱃지를 통해 실행 결과를 한눈에 파악할 수 있으며, 평가자를 연결하면 점수 뱃지도 함께 표시됩니다.* @@ -30,7 +30,7 @@ description: "한 번의 실행에서 발생한 모든 이벤트를 하나의 ## 전체 실행을 그림으로 읽기 -![이벤트 타임라인 옆에 표시된 세션의 git 스타일 실행 그래프와 도구, 모델, 훅 분석 패널](/agenteye/images/session-detail.png) +![이벤트 타임라인 옆에 표시된 세션의 git 스타일 실행 그래프와 도구, 모델, 훅 분석 패널](/cloud/images/session-detail.png) *실행 그래프(왼쪽)가 이벤트 타임라인 옆에 표시되며, 오른쪽 패널에서는 실행에 사용된 도구, 모델, 훅, 토큰 소비량을 상세히 확인할 수 있습니다.* @@ -38,7 +38,7 @@ description: "한 번의 실행에서 발생한 모든 이벤트를 하나의 오른쪽 패널에서는 실행별 세부 내역을 확인할 수 있습니다. 어떤 도구와 모델이 실행되었는지, 어떤 훅이 실행되었는지, 해당 실행에서 토큰을 얼마나 소비했는지가 그래프 바로 옆에 표시됩니다. "이 실행은 왜 이렇게 비쌌지?" 또는 "느린 도구가 뭐지?"에 대한 답이 바로 거기 있습니다. -개별 이벤트에는 고유 링크가 있으므로, "세션에서 3분의 2 지점쯤"이라고 설명하는 대신 특정 순간의 링크를 바로 공유할 수 있습니다. 이벤트에서 링크를 복사하거나, [감사](/ko/agenteye/audits) 결과나 오류에서 링크를 따라가면 해당 이벤트가 선택되고 스크롤된 상태로 세션이 열립니다. 매우 긴 실행에서도 마찬가지입니다. 타임라인은 브라우저 성능을 위해 제한된 범위를 로드하지만, 해당 범위를 벗어난 이벤트를 가리키는 링크도 시작 지점으로 떨어지지 않고 해당 이벤트를 정확히 찾아줍니다. 이벤트가 보존 기간을 초과한 경우, 페이지는 아무것도 선택하지 않고 넘어가는 대신 그 사실을 명시적으로 알려줍니다. +개별 이벤트에는 고유 링크가 있으므로, "세션에서 3분의 2 지점쯤"이라고 설명하는 대신 특정 순간의 링크를 바로 공유할 수 있습니다. 이벤트에서 링크를 복사하거나, [감사](/ko/cloud/audits) 결과나 오류에서 링크를 따라가면 해당 이벤트가 선택되고 스크롤된 상태로 세션이 열립니다. 매우 긴 실행에서도 마찬가지입니다. 타임라인은 브라우저 성능을 위해 제한된 범위를 로드하지만, 해당 범위를 벗어난 이벤트를 가리키는 링크도 시작 지점으로 떨어지지 않고 해당 이벤트를 정확히 찾아줍니다. 이벤트가 보존 기간을 초과한 경우, 페이지는 아무것도 선택하지 않고 넘어가는 대신 그 사실을 명시적으로 알려줍니다. --- @@ -46,12 +46,12 @@ description: "한 번의 실행에서 발생한 모든 이벤트를 하나의 모든 대시보드 페이지는 조직 단위(`//…`)로 범위가 지정됩니다. 세션은 왼쪽 사이드바의 **Observe** 메뉴 아래, Events 옆에 위치하며, 목록 상단에 날짜 범위, 환경, 에이전트, 세션 필터가 제공됩니다. 모든 행에서 클릭 한 번으로 전체 실행 그래프를 확인할 수 있습니다. -점수 뱃지와 점수 범위 필터링을 활성화하려면 평가자를 연결하세요: [평가](/ko/agenteye/evaluations)를 참고하세요. +점수 뱃지와 점수 범위 필터링을 활성화하려면 평가자를 연결하세요: [평가](/ko/cloud/evaluations)를 참고하세요. --- ## 관련 문서 -- [이벤트 스트림](/ko/agenteye/event-stream): 각 세션이 집약되는 원시 단계별 트레일. -- [평가](/ko/agenteye/evaluations): 각 실행에 필터링 가능한 점수 뱃지를 부여하기 위한 평가자 연결 방법. -- [텔레메트리](/ko/agenteye/telemetry): 에이전트의 실행 결과가 세션으로 전달되는 방식. \ No newline at end of file +- [이벤트 스트림](/ko/cloud/event-stream): 각 세션이 집약되는 원시 단계별 트레일. +- [평가](/ko/cloud/evaluations): 각 실행에 필터링 가능한 점수 뱃지를 부여하기 위한 평가자 연결 방법. +- [텔레메트리](/ko/cloud/performance): 에이전트의 실행 결과가 세션으로 전달되는 방식. \ No newline at end of file diff --git a/docs/ko/concepts.mdx b/docs/ko/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/ko/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/ko/daemon.mdx b/docs/ko/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/ko/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/ko/dashboard.mdx b/docs/ko/dashboard.mdx index 221155f9..80af4419 100644 --- a/docs/ko/dashboard.mdx +++ b/docs/ko/dashboard.mdx @@ -69,7 +69,7 @@ Hermes와 OpenClaw는 사용자 범위이며 그룹화할 작업 디렉터리가 4. **개선 방법** — 권장 정책별 목록: 흰색으로 정책 이름, 한 줄 설명, 오른쪽에 설치 명령어 + 복사 버튼. 섹션 헤더는 `enable all N → projected · `(모든 수정 사항 적용 시 도달할 점수)로 표시되며, `[install all]` 버튼은 모든 권장 정책에 대한 `failproofai policy add a b c …` 명령어 전체를 복사합니다. 5. **더 나은 복귀** — 나란히 배치된 두 개의 카드. 왼쪽: 알림 설정(`3d` / `7d` / `14d` / `30d` 주기 선택기; 인증 후 `/api/auth/reminder`를 통해 저장). 오른쪽: failproof 혜택 잠금 해제 — `invite a friend`는 친구 이메일을 쉼표/공백/줄바꿈으로 구분하여 입력하는 모달을 열고(한 번에 최대 10명), `/api/audit/invite`로 POST하여 api-server의 `POST /v0/invite`로 전달합니다. api-server는 `invite@failproof.ai`에서 수신자당 한 통의 이메일을 발송하며 발신자를 참조(Cc)로 추가하고 `Reply-To`를 설정합니다. 따라서 수신자는 누가 초대했는지 확인할 수 있고 발신자는 받은 편지함에 사본을 받게 됩니다. 익명 사용자는 초대 발송 전에 발신자 이메일을 확인하기 위해 먼저 `AuthDialog`로 안내됩니다. 권한 부여 / 혜택 이행은 추후 진행될 예정입니다. -`failproofai audit` 런타임으로 구동됩니다 — 기본 스캔 엔진, 지원 플래그, 트랜스크립트별 캐시 불변성에 대해서는 [감사 CLI](/ko/cli/audit)를 참고하세요. 대시보드는 최신 결과를 `~/.failproofai/audit-dashboard.json`(모드 `0600`, 단일 슬롯, 새 실행 시 덮어씀)에 캐시하므로 재방문 시 즉시 로드됩니다. **트랜스크립트별 캐시와 전체 결과 캐시 모두 7일이 지나면 읽기 시 거부되어** 대시보드가 1주일 된 결과를 조용히 제공하는 일이 없습니다. TTL이 지나면 `/audit`는 빈 상태로 돌아가 새 실행을 요청합니다. 보고서 하단의 `[ re-audit now ]`를 클릭하면 `noCache: true`와 함께 `/api/audit/run`에 POST합니다. 재감사는 트랜스크립트별 캐시를 우회하고 캐시된 결과를 조용히 반환하는 대신 모든 트랜스크립트를 처음부터 다시 스캔합니다. 대시보드는 실행이 완료될 때까지 1Hz로 `/api/audit/status`를 폴링하며, 실행 중에는 경과 타이머와 함께 핑크색 진행 표시줄이 뷰포트 상단에 고정됩니다. 성공하면 새 결과가 전체 페이지 새로고침 없이 즉시 교체됩니다. 재감사 실패 시 표시줄은 `RerunError.kind`(`timeout` / `network` / `post_failed`)에 따른 메시지와 함께 빨간색으로 변하며 이전 보고서는 그대로 유지됩니다. 빈 상태(캐시 없음 또는 만료)와 세션 없는 상태(캐시는 있지만 스캔에서 트랜스크립트를 찾지 못함)는 별도로 표시됩니다. +`failproofai audit` 런타임으로 구동됩니다 — 기본 스캔 엔진, 지원 플래그, 트랜스크립트별 캐시 불변성에 대해서는 [감사 CLI](/ko/audit)를 참고하세요. 대시보드는 최신 결과를 `~/.failproofai/audit-dashboard.json`(모드 `0600`, 단일 슬롯, 새 실행 시 덮어씀)에 캐시하므로 재방문 시 즉시 로드됩니다. **트랜스크립트별 캐시와 전체 결과 캐시 모두 7일이 지나면 읽기 시 거부되어** 대시보드가 1주일 된 결과를 조용히 제공하는 일이 없습니다. TTL이 지나면 `/audit`는 빈 상태로 돌아가 새 실행을 요청합니다. 보고서 하단의 `[ re-audit now ]`를 클릭하면 `noCache: true`와 함께 `/api/audit/run`에 POST합니다. 재감사는 트랜스크립트별 캐시를 우회하고 캐시된 결과를 조용히 반환하는 대신 모든 트랜스크립트를 처음부터 다시 스캔합니다. 대시보드는 실행이 완료될 때까지 1Hz로 `/api/audit/status`를 폴링하며, 실행 중에는 경과 타이머와 함께 핑크색 진행 표시줄이 뷰포트 상단에 고정됩니다. 성공하면 새 결과가 전체 페이지 새로고침 없이 즉시 교체됩니다. 재감사 실패 시 표시줄은 `RerunError.kind`(`timeout` / `network` / `post_failed`)에 따른 메시지와 함께 빨간색으로 변하며 이전 보고서는 그대로 유지됩니다. 빈 상태(캐시 없음 또는 만료)와 세션 없는 상태(캐시는 있지만 스캔에서 트랜스크립트를 찾지 못함)는 별도로 표시됩니다. ### 정책 diff --git a/docs/ko/architecture.mdx b/docs/ko/how-it-works.mdx similarity index 100% rename from docs/ko/architecture.mdx rename to docs/ko/how-it-works.mdx diff --git a/docs/ko/introduction.mdx b/docs/ko/introduction.mdx index 15a73178..e01f65dc 100644 --- a/docs/ko/introduction.mdx +++ b/docs/ko/introduction.mdx @@ -54,4 +54,4 @@ failproofai policies --install # enable policies (or skip — `failproofai` wi failproofai # launch the dashboard ``` -전체 안내는 [시작 가이드](/ko/getting-started)를 참고하세요. \ No newline at end of file +전체 안내는 [시작 가이드](/ko/quickstart)를 참고하세요. \ No newline at end of file diff --git a/docs/ko/policies.mdx b/docs/ko/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/ko/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/ko/getting-started.mdx b/docs/ko/quickstart.mdx similarity index 100% rename from docs/ko/getting-started.mdx rename to docs/ko/quickstart.mdx diff --git a/docs/ko/reference/files.mdx b/docs/ko/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/ko/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/package-aliases.mdx b/docs/package-aliases.mdx index a9be5626..50e45d9a 100644 --- a/docs/package-aliases.mdx +++ b/docs/package-aliases.mdx @@ -1,5 +1,5 @@ --- -title: Package Aliases +title: "Package aliases" description: "Registered typosquat-prevention aliases and how they work" icon: copy --- diff --git a/docs/policies.mdx b/docs/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/pt-br/agent-support.mdx b/docs/pt-br/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/pt-br/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/pt-br/agenteye/cli-and-agents.mdx b/docs/pt-br/agenteye/cli-and-agents.mdx deleted file mode 100644 index 38873d7f..00000000 --- a/docs/pt-br/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "Todo o seu deployment de Observabilidade do Failproof AI a um comando de distância." ---- - - -Todo o seu deployment de Observabilidade do Failproof AI a um comando de distância. Verifique produção, gere uma chave de API ou reconheça um incidente sem sair do terminal — e ainda automatize tudo isso em CI, ou deixe um agente de código fazer por você em linguagem natural. - -```bash -pipx install agenteye -agenteye login --email você@exemplo.com # um código de 6 dígitos chega na sua caixa de entrada -agenteye --json sessions --since 24h # todas as execuções de agentes das últimas 24h, mais recentes primeiro -``` - -*O CLI `agenteye` se comunica com o seu dashboard. É uma ferramenta diferente do coletor, que envia eventos para o servidor.* - -## Todo o seu deployment, a um comando de distância - -Pare de alternar entre abas para responder uma pergunta rápida. O CLI `agenteye` lê seus dados e administra sua organização a partir de um único binário, então uma verificação que antes exigia clicar pelo dashboard vira uma linha que você pode reexecutar, criar um alias ou colar em um runbook. Você tem quatro superfícies: - -- **Leia seus dados:** `sessions`, `events`, `evals` e `errors`, filtrados por tempo, agente e ambiente. -- **Gerencie sua organização:** `keys`, `users`, `settings`, `alerts` e `incidents`. -- **Execute análises:** SQL salvo mais um executor `query` ad-hoc sobre seus dados de eventos. -- **Consulte o assistente:** `agent ask` acessa o mesmo analista somente leitura com quem você conversa no dashboard. - -Instale uma vez com `pipx`, faça login com um código de 6 dígitos enviado por e-mail e está pronto. A sessão dura cerca de um dia; reexecute `agenteye login` quando expirar. Use-o para verificar produção, provisionar uma chave ou triagear um incidente ativo — tudo sem abrir um navegador: - -```bash -agenteye errors --since 24h --aggregate # o que está quebrando, agrupado por tipo de erro -agenteye incidents list --state firing # o que está pegando fogo agora -agenteye keys create ci --add events:add # uma chave que só pode enviar eventos, secret exibido uma vez -``` - -Um hábito importante: opções globais como `--json` vão antes do comando. `agenteye --json sessions` está correto; `agenteye sessions --json` não está. - -## Automatize, integre ao CI - -Todo comando aceita `--json`, e isso muda tudo. JSON limpo vai para stdout enquanto status e avisos para humanos vão para stderr, então uma captura com `--json` vai direto para o `jq` sem nenhuma linha extra para remover. É isso que torna o CLI igualmente útil para você no terminal e para um agente de código analisando a saída: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -Ele foi construído para rodar de forma não supervisionada. Prompts de confirmação são ignorados automaticamente quando não há terminal conectado, então nada trava em um pipeline, e todo comando retorna um código de saída significativo: `0` sucesso, `4` não autenticado, `5` sem permissão (a mensagem nomeia qual, por exemplo `alerts:write`), `3` dashboard inacessível. Um script pode ramificar em um `4` para reautenticar ou em um `5` para dizer exatamente o que pedir a um administrador, em vez de falhar silenciosamente. - -## Deixe um agente de código conduzir em linguagem natural - -Ainda melhor: você não deveria precisar lembrar nenhuma dessas flags. A **CLI skill** é uma pequena pasta Agent Skill chamada `agenteye-cli` que ensina um agente de código como Claude Code ou Codex a conduzir o CLI a partir de pedidos em linguagem natural. Pergunte "tem algo quebrado hoje?" e o agente escolhe o comando, executa como você e responde em prosa. - -Para Claude Code, coloque a pasta `agenteye-cli` em `~/.claude/skills/` e ela é descoberta automaticamente. O Failproof AI Observability fornece a pasta; não há nada extra para instalar, pois ela apenas conduz o CLI que você já instalou. Faça login você mesmo primeiro: a skill não consegue completar o login por código enviado por e-mail por você. - -Como o agente executa o CLI como você, ele pode fazer tudo que seu login permite, tanto leituras quanto escritas: criar chaves, alterar configurações, resolver incidentes. O prompt "tem certeza?" do CLI não é acionado para um agente, então a skill foi escrita para declarar o comando exato e aguardar sua aprovação antes de qualquer alteração. Você é a etapa de confirmação. - -```text -você Por que a sessão run-001 falhou? - -agente Executando: agenteye --json events --session-id run-001 --all - O checkout-agent encontrou um TimeoutError em sua terceira chamada de ferramenta. -``` - -Leituras são instantâneas, e toda escrita pausa para você: - -```text -você Dê ao CI uma chave que só possa enviar eventos. - -agente Isso cria uma chave de API. Vou executar: - agenteye keys create ci --add events:add - Pode prosseguir? - -você sim - -agente Feito. Chave "ci" criada com events:add apenas. O secret é exibido uma vez, então guarde agora. -``` - -## Relacionado - -- [Referência do CLI](/pt-br/agenteye/cli): todos os comandos, flags e formatos JSON. -- [Receitas de CLI para agentes](/pt-br/agenteye/cli-recipes): padrões `jq` para copiar e colar e tratamento de códigos de saída. -- [CLI agent skill](/pt-br/agenteye/cli-skill): instale e execute a skill `agenteye-cli`. -- [Assistente de IA](/pt-br/agenteye/assistant): o analista no dashboard com quem `agent ask` se comunica. \ No newline at end of file diff --git a/docs/pt-br/agenteye/cli-skill.mdx b/docs/pt-br/agenteye/cli-skill.mdx deleted file mode 100644 index 3c9b15fb..00000000 --- a/docs/pt-br/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Skill de Agente CLI de Observabilidade do Failproof AI" -description: "Pergunte ao seu agente de codificação 'tem algo quebrado hoje?' e deixe-o responder com dados ao vivo do Failproof AI Observability, sem precisar memorizar nenhum comando." ---- - - -Pergunte ao seu agente de codificação *"tem algo quebrado hoje?"* e deixe-o responder com seus dados ao vivo do Failproof AI Observability, sem precisar memorizar nenhum comando. A **skill de CLI do Failproof AI Observability** (`agenteye-cli`) é uma *Agent Skill*: uma pequena pasta de instruções que um agente de codificação como Claude Code ou Codex carrega sob demanda. Ela ensina o agente a operar seu deployment de Observability por meio do [`agenteye` CLI](/pt-br/agenteye/cli) a partir de pedidos em linguagem natural como *"dê ao CI uma chave que só pode enviar eventos"* ou *"confirme o incidente ativo e atribua a mim."* - -Ela **não** é um serviço nem um binário separado; não há nada para implantar. Ela funciona sobre o CLI que você já tem instalado: o agente executa `agenteye --json …`, analisa o JSON limpo e responde em texto. Tudo o que ela pode fazer, você mesmo poderia fazer digitando os mesmos comandos. - ---- - -## Como ela se relaciona com as outras interfaces do Failproof AI Observability - -O Failproof AI Observability oferece quatro formas de acessar os mesmos dados e controles. Elas se complementam: - -| Interface | O que é | Onde roda | Use quando | -|---|---|---|---| -| **[CLI](/pt-br/agenteye/cli)** | Referência de comandos e flags para `agenteye` | Seu terminal | Você quer executar ou automatizar um comando específico | -| **[Receitas de CLI](/pt-br/agenteye/cli-recipes)** | Padrões de `jq`/pipeline para copiar e colar | Seu terminal / scripts | Você está integrando o CLI em automações | -| **CLI skill** (este doc) | Uma interface em linguagem natural sobre o CLI | Seu agente de codificação, na sua estação de trabalho | Você quer *simplesmente perguntar* e deixar o agente escolher o comando | -| **[Evaluator skill](/pt-br/agenteye/evaluator-skill)** | Uma skill irmã que projeta e constrói seu serviço de pontuação | Seu agente de codificação, na sua estação de trabalho | Você quer *produzir* pontuações de avaliação em vez de lê-las | -| **[Python SDK skill](/pt-br/agenteye/python-sdk-skill)** | Uma skill irmã que instrumenta seu agente para emitir telemetria | Seu agente de codificação, na sua estação de trabalho | Você quer que seu agente *produza* os eventos que esta skill lê | -| **[Assistente de IA no dashboard](/pt-br/agenteye/assistant)** | Um chat embutido no dashboard | No servidor (dentro do dashboard) | Você quer Q&A dentro do dashboard sobre seus dados | - -A skill em si não tem privilégios próprios; ela apenas transforma suas palavras em chamadas de CLI que rodam como você: - -```mermaid -flowchart TD - YOU["você: 'confirme o incidente ativo'"] --> AGENT["agente de codificação (Claude Code / Codex)
    carrega a skill agenteye-cli"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|sua sessão de CLI autenticada| API["API do dashboard de Observabilidade"] -``` - -### vs. o assistente de IA no dashboard: uma distinção importante - -Essas são duas ferramentas diferentes com alcances de ação muito distintos: - -- O **assistente de IA no dashboard** ([AI assistant](/pt-br/agenteye/assistant)) é um chat embutido no dashboard, com suporte do serviço de agente. Ele é **somente leitura mais criação com aprovação**: pode rascunhar queries salvas e dashboards, mas toda escrita pausa para sua aprovação explícita com um clique, e ele nunca exclui nada. É restrito pela permissão `agent:use` e só enxerga dados da organização que você está visualizando. -- A **CLI skill** roda na *sua* estação de trabalho dentro do *seu* agente de codificação e aciona o `agenteye` CLI **como você**. Ela pode executar toda a **superfície do CLI, incluindo mutações** (criar/rotacionar/desativar chaves de API, alterar configurações da org, resolver incidentes, excluir queries salvas), limitada apenas pelas permissões do seu login no CLI. Trate-a com exatamente o mesmo cuidado com que trataria executar esses comandos manualmente. - ---- - -## Pré-requisitos - -1. O **`agenteye` CLI instalado** e no `PATH` (veja a referência do [CLI](/pt-br/agenteye/cli): `pipx install agenteye`). -2. Sua **URL do dashboard** configurada (`AGENTEYE_DASHBOARD_URL`, ou o agente passa `--base-url`). -3. Uma **sessão autenticada**: execute `agenteye login` você mesmo primeiro. A skill **não consegue** completar o login por código enviado por e-mail para você; ela dirá para executar `agenteye login` se a sessão estiver ausente ou expirada (código de saída do CLI `4`). - ---- - -## Onde obter - -A skill está publicada na coleção pública de skills do Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -Não há nenhuma restrição de acesso — o repositório é público e a skill não precisa de nenhuma credencial própria, pois ela apenas aciona o `agenteye` CLI **público** contra o *seu* dashboard, usando a sessão com a qual *você* fez login. Você não precisa pedir permissão a ninguém. - -Observe que ela é publicada como sua própria pasta e **não** está dentro do pacote `pipx install agenteye`, portanto não procure por ela lá. - -## Instalando a skill - -O caminho mais rápido é o CLI [`skills`](https://skills.sh), que busca a pasta e a coloca onde seu agente procura: - -```bash -# Claude Code, somente este projeto -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# todos os projetos (instala em ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex em vez disso -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -Depois gerencie como qualquer outra skill: - -```bash -npx skills list -a claude-code # o que está instalado -npx skills update agenteye-cli # baixar a versão mais recente -npx skills remove agenteye-cli # remover -``` - -Prefere instalar manualmente? Uma Agent Skill é apenas uma pasta contendo um `SKILL.md` (mais referências opcionais), então copiá-la também funciona: - -- **Claude Code**: coloque a pasta `agenteye-cli/` em `~/.claude/skills/` (todos os projetos) ou `/.claude/skills/` (somente aquele repositório). Claude Code a descobre automaticamente — verifique com a lista `/skills`, ou simplesmente faça uma pergunta que corresponda à sua descrição. -- **Codex (OpenAI)**: o Codex lê o mesmo `SKILL.md`. O `agents/openai.yaml` incluído define `allow_implicit_invocation: true`, então o Codex seleciona a skill automaticamente quando uma tarefa combina; caso contrário, invoque-a explicitamente como `$agenteye-cli`. - ---- - -## Segurança: mutações NÃO solicitam confirmação quando um agente executa o CLI - -> **Aviso:** Leia isso antes de permitir que um agente faça alterações. - -O `agenteye` CLI normalmente pergunta *"tem certeza?"* antes de uma ação destrutiva. Ele **pula automaticamente essa confirmação sempre que não está conectado a um terminal (que é exatamente como um agente de codificação o executa), e `--json` também a pula.** Portanto, o aviso de segurança **não** será acionado para o agente. - -A skill foi escrita para compensar isso: ela é instruída a declarar o comando exato que irá executar e obter seu **OK explícito antes de qualquer alteração de estado**. Mantenha essa disciplina. Quando você aciona o Failproof AI Observability por meio de um agente, *você* é a etapa de confirmação. Os comandos que alteram estado para ficar atento: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- os subcomandos de escrita de `incidents`: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -Tudo em **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) é somente leitura e não altera nada. - -Como o agente age **como você**, ele só pode fazer o que seu login tem permissão para fazer; as permissões são resolvidas **por org** (veja [API keys](/pt-br/agenteye/api-keys)). Um comando para o qual você não tem permissão retorna o código de saída `5` com o nome exato da permissão faltante, para que o agente possa informar precisamente o que pedir a um administrador em vez de falhar de forma opaca. - ---- - -## O que você pode perguntar - -Veja como é uma interação real — uma leitura, depois uma alteração que aguarda sua confirmação: - -```text -você ▸ Tem algo quebrado no último dia? - -agente▸ Executando: agenteye --json errors --since 24h --aggregate - 3 tipos de erro nas últimas 24h. O maior é TimeoutError - (18 ocorrências, todas no checkout-agent), depois RateLimited (5). - Quer ver as sessões com falha de TimeoutError? - -você ▸ Dê ao CI uma chave que só pode enviar eventos. - -agente▸ Isso cria uma chave de API. Vou executar: - agenteye keys create ci --add events:add - Pode prosseguir? - -você ▸ sim - -agente▸ Concluído. Chave "ci" criada com events:add apenas. - O segredo é exibido apenas uma vez, então guarde agora. Não consigo reexibi-lo. -``` - -A skill mapeia cada intenção em linguagem natural para o comando `agenteye` correto, descobrindo valores válidos primeiro (`list `, `whoami`) para não adivinhar, e declarando o comando exato antes de qualquer alteração. Mais exemplos: - -- *"Tem algo quebrado / falhando nas últimas 24 horas?"* → `errors --since 24h --aggregate`, depois um resumo. -- *"Por que a sessão `run-001` falhou?"* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *"Como está a qualidade esta semana?"* → `evals --aggregate --since 7d`, depois detalhes das execuções com baixa pontuação. -- *"Dê ao CI uma chave que só pode enviar eventos."* → `keys create ci --add events:add` (declara o comando, depois cria e captura o segredo único). -- *"Quem tem acesso? Torne a Dana somente leitura."* → `users list` → `users update dana@… --permission-set read-only` (após confirmar com você). -- *"Confirme o incidente ativo e atribua a mim."* → `incidents list --state firing` → `incidents ack ` / `incidents assign você@…`. - -Para os comandos exatos, flags e estruturas JSON por trás dessas interações, veja a referência do [CLI](/pt-br/agenteye/cli) e as [receitas de CLI para agentes](/pt-br/agenteye/cli-recipes). - ---- - -## Próximos passos - -- **[CLI](/pt-br/agenteye/cli)**: referência completa de comandos e flags para `agenteye`. -- **[Receitas de CLI para agentes](/pt-br/agenteye/cli-recipes)**: padrões de `jq` para copiar e colar e tratamento de códigos de saída. -- **[Evaluator agent skill](/pt-br/agenteye/evaluator-skill)**: a skill irmã, para construir o avaliador cujas pontuações `agenteye evals` lê. -- **[Python SDK agent skill](/pt-br/agenteye/python-sdk-skill)**: a skill irmã, para instrumentar um agente para que ele emita a telemetria que `agenteye` lê. -- **[AI assistant](/pt-br/agenteye/assistant)**: o assistente no dashboard (não confundir com esta skill de terminal). -- **[API keys](/pt-br/agenteye/api-keys)**: o modelo de permissões por org que delimita o que a skill pode fazer. \ No newline at end of file diff --git a/docs/pt-br/agenteye/codex-capture.mdx b/docs/pt-br/agenteye/codex-capture.mdx deleted file mode 100644 index 4db292e5..00000000 --- a/docs/pt-br/agenteye/codex-capture.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "Captura de sessão do Codex" -description: "Transmita as sessões locais do OpenAI Codex da sua equipe para o AgentEye como sessões e eventos comuns — sem nenhuma alteração na forma como eles executam o Codex." ---- - -Seus engenheiros já executam o OpenAI Codex todos os dias. A captura de sessão do Codex traz essas sessões de codificação para o AgentEye como sessões e eventos comuns, para que você possa pesquisar, reproduzir e avaliar junto a tudo o mais que você observa. Ela complementa o [Python SDK](/pt-br/agenteye/python-sdk): o SDK instrumenta os agentes que você escreve, enquanto este recurso captura o trabalho que sua equipe já realiza no Codex — sem nenhuma alteração na forma como eles o executam. - -Um pequeno coletor em segundo plano lê os transcritos de sessão locais do Codex à medida que são gravados e os envia para o AgentEye. Um coletor por máquina captura todas as superfícies locais do Codex de uma vez — sem necessidade de configuração por superfície. - -O mesmo coletor também captura outros agentes — veja [OpenClaw](/pt-br/agenteye/openclaw-capture) e [Hermes](/pt-br/agenteye/hermes-capture). Ative cada um que você utiliza; um único coletor pode capturar vários ao mesmo tempo. - ---- - -## O que é capturado - -Toda superfície do Codex que executa **localmente** produz os mesmos transcritos de sessão em disco, e o coletor processa todos eles: - -- o **CLI** do Codex e o `codex exec` -- a **extensão para VS Code / IDE** -- o **aplicativo desktop**, quando executa uma sessão localmente - -Cada sessão do Codex se torna uma [sessão](/pt-br/agenteye/sessions) no AgentEye; suas mensagens de usuário e assistente, raciocínio, chamadas de ferramentas, resultados de ferramentas e uso de tokens se tornam os [eventos](/pt-br/agenteye/event-stream) correspondentes. A superfície de origem de cada sessão (CLI, IDE ou desktop) é registrada, permitindo diferenciá-las. - -> **Sessões na nuvem não são capturadas.** O aplicativo desktop executa cada vez mais sessões na nuvem do Codex e mantém apenas os metadados na máquina — não há transcrito local para leitura. Somente sessões executadas localmente são capturadas. - ---- - -## Como ativar - -A captura está desativada até que você a habilite. Instale o coletor com uma chave de API que tenha a permissão `events:add` (veja [Chaves de API](/pt-br/agenteye/api-keys)) e ative a captura do Codex: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -Isso instala o coletor, registra-o como um serviço em segundo plano e inicia a captura. Confirme que está em execução: - -```bash -agenteye-collector health -``` - -Na primeira execução, suas sessões do Codex existentes são preenchidas retroativamente uma vez e, em seguida, novas atividades são transmitidas em segundos. Os arquivos do próprio Codex são apenas lidos — nunca modificados, movidos ou excluídos — e cada sessão é enviada exatamente uma vez, mesmo entre reinicializações. - ---- - -## Onde aparece - -As sessões capturadas aparecem em **Sessions**, e seus eventos no fluxo **Events**, da mesma forma que qualquer outro agente que você observa — portanto, [replay de sessão](/pt-br/agenteye/sessions), [pesquisa](/pt-br/agenteye/queries), [avaliações](/pt-br/agenteye/evaluations) e [alertas](/pt-br/agenteye/alerts) funcionam normalmente nelas. Filtre pelo agente do Codex para visualizá-las separadamente. - ---- - -## Privacidade - -Os transcritos do Codex contêm a sessão completa — incluindo saída de comandos, conteúdo de arquivos e tudo o que o Codex leu ou escreveu — e podem conter segredos. As sessões capturadas são enviadas como estão, portanto, ative a captura apenas em máquinas e para equipes nas quais centralizar esse conteúdo no AgentEye seja adequado, e forneça ao coletor uma chave com escopo apenas para `events:add`. Veja [Segurança](/pt-br/agenteye/security) para entender como seus dados são mantidos isolados. \ No newline at end of file diff --git a/docs/pt-br/agenteye/concepts.mdx b/docs/pt-br/agenteye/concepts.mdx deleted file mode 100644 index 8aef517f..00000000 --- a/docs/pt-br/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "Conceitos" -description: "O vocabulário por trás da Observabilidade do Failproof AI — eventos, sessões, avaliações, auditorias, descobertas e incidentes — definido em um só lugar." ---- - - -Esta página define o vocabulário utilizado pela Observabilidade do Failproof AI. Se algum termo em outro guia parecer desconhecido, ele está definido aqui. Você não precisa ler do início ao fim: percorra rapidamente ou volte quando quiser esclarecer uma palavra específica. - ---- - -## O modelo de dados - -**Evento** -A menor unidade de dados. Um evento registra um único passo executado pelo seu agente: um `tool_use`, um `model_request`, um `hook_completed`, um `error`, entre outros. Seu agente emite eventos por meio do [Python SDK](/pt-br/agenteye/python-sdk); eles aparecem em tempo real na página **Events**. - -**Sessão** -Uma execução do agente, identificada por um `session_id`. Uma sessão é composta por todos os eventos que compartilham esse id, consolidados em uma única linha na página **Sessions** e representados como um grafo de execução na página de detalhes. Uma sessão normalmente começa com `agent_start` e termina com `agent_end`. - -**Agente** -Um ator nomeado dentro de uma execução, identificado por um `agent_id`. Uma execução pode envolver vários agentes: por exemplo, um planejador que cria um sub-agente de resumo. Sub-agentes carregam um `parent_id`, que é o que permite ao Failproof AI Observability exibi-los em suas próprias faixas no grafo de execução. - -**Ambiente** -Um rótulo que indica onde a execução ocorreu: `production`, `staging`, `dev`. Você o define uma única vez ao configurar o SDK. Quase todas as páginas do dashboard permitem filtrar por ambiente. - -**Preenchimento da janela de contexto** -O percentual da janela de contexto de um modelo consumido por uma resposta. O Failproof AI Observability registra esse valor em eventos `model_response` para os modelos que reconhece, tornando o crescimento do prompt e a compactação iminente visíveis diretamente no fluxo de eventos. - ---- - -## Qualidade - -**Avaliação** -Uma pontuação de qualidade para uma sessão concluída, produzida por um serviço de pontuação que você executa. As avaliações são opcionais: até que você conecte um avaliador, as sessões são registradas, mas não pontuadas. Cada avaliação pode conter várias pontuações nomeadas (por exemplo, `helpfulness`, `factuality`, `tool_efficiency`), cada uma com uma breve nota de raciocínio. Veja [Evaluation suite](/pt-br/agenteye/evaluation-suite). - -**Chave de pontuação** -O nome de uma dimensão que um avaliador reporta, como `helpfulness`. Alertas e auditorias podem monitorar uma chave de pontuação específica ao longo do tempo. - -**Avaliador** -Seu serviço de pontuação. O Failproof AI Observability faz um POST com a transcrição de uma execução concluída para ele e armazena as pontuações retornadas. Não há um avaliador padrão incluído; a lógica de pontuação é sua. - ---- - -## Identificando e corrigindo falhas - -**Hook** -Uma salvaguarda ou efeito colateral que seu framework de agentes executa em torno de um passo: uma verificação de segurança de conteúdo, anonimização de dados pessoais, um controle de orçamento. Hooks emitem eventos `hook_triggered` / `hook_completed` com um `outcome` (allow, deny, modify) e têm sua própria página de observação. - -**Regra de alerta** -Uma regra que é acionada quando uma métrica ultrapassa um limite definido por você: taxa de erros, latência p95, custo em tokens ou uma pontuação de avaliação. Quando uma regra é acionada, ela abre um incidente e notifica os canais escolhidos (e-mail, Slack, webhook, no dashboard). Veja [Alerts](/pt-br/agenteye/alerts). - -**Incidente** -Uma questão em aberto criada quando uma regra de alerta é acionada. Incidentes têm um ciclo de vida (reconhecer, atribuir, resolver) e uma linha do tempo de atividades que registra cada ação. Você também pode abrir um manualmente. - -**Auditoria** -Uma investigação recorrente (de hora em hora a semanalmente) que analisa seus logs *entre* sessões em busca de padrões de falha para os quais você ainda não escreveu uma regra: clusters de erros, pontuações baixas, outliers de latência, loops de chamadas de ferramentas e execuções que nunca foram concluídas. Enquanto um alerta monitora uma métrica que você já conhece, uma auditoria indica o que você deve examinar a seguir. Veja [Audits](/pt-br/agenteye/audits). - -**Descoberta** -Um resultado classificado e embasado em evidências proveniente de uma execução de auditoria. Uma descoberta nomeia um padrão, vincula às sessões exatas que o sustentam e possui um ciclo de vida de triagem (reconhecer, resolver, silenciar, descartar). O Failproof AI Observability deduplica descobertas entre execuções, de forma que um padrão conhecido seja atualizado em vez de se acumular. - -**O assistente de IA** -O chat integrado ao dashboard que responde perguntas sobre seus agentes em linguagem natural, utilizando seus próprios dados. Por padrão, é somente leitura; qualquer coisa que ele crie (uma consulta salva, um dashboard) requer aprovação, e ele nunca pode excluir dados. Veja [AI assistant](/pt-br/agenteye/assistant). - ---- - -## Operação - -**Organização (tenant)** -Um espaço de trabalho isolado. Uma instância do Failproof AI Observability pode hospedar várias organizações, cada uma com seus próprios usuários, chaves e dados. Toda URL do dashboard é delimitada pelo slug da sua organização (`//…`). - -**Coletor** -`agenteye-collector`, o daemon leve que é executado em cada máquina de agente, agrupa os eventos que o SDK grava em disco e os envia para o servidor. - -**Chave de API** -Um token com escopo definido que autentica um cliente junto ao servidor. As chaves carregam permissões granulares (por exemplo, `events:add` para o coletor, escopos somente leitura para uma chave de dashboard). Veja [API keys](/pt-br/agenteye/api-keys). - -**Servidor** -O serviço de ingestão e API. Ele ingere eventos, armazena o estado operacional nos seus bancos de dados e serve o dashboard e a CLI. - -**Dashboard** -A interface web. Cada página é delimitada a uma organização e lê os dados por meio da API do servidor. - ---- - -## Próximos passos - -- [Overview](/pt-br/agenteye/overview): como essas peças se encaixam. -- [Observability](/pt-br/agenteye/observability): as superfícies de observação (Events, Sessions, Models, Tools, Hooks, Errors). \ No newline at end of file diff --git a/docs/pt-br/agenteye/evaluator-skill.mdx b/docs/pt-br/agenteye/evaluator-skill.mdx deleted file mode 100644 index c38fdd66..00000000 --- a/docs/pt-br/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: "Habilidade de Agente Avaliador de Observabilidade Failproof AI" -description: "Vá de 'acho que nosso agente às vezes falha' a um serviço de pontuação implantado, com seu agente de codificação tanto decidindo quanto construindo." ---- - - -Vá de *"acho que nosso agente às vezes falha"* a um serviço de pontuação implantado, com seu agente de codificação tanto decidindo quanto construindo. A **habilidade de avaliador de Observabilidade Failproof AI** (`agenteye-evaluator`) é uma *Agent Skill*: uma pequena pasta de instruções que um agente de codificação como Claude Code ou Codex carrega sob demanda. Ela ensina o agente a determinar quais dimensões de qualidade valem a pena rastrear para o *seu* agente, e então escrever, testar e implantar o [serviço avaliador](/pt-br/agenteye/evaluation-suite) que os pontua. - -Ela **não** é um pontuador hospedado, um registro para o qual você faz upload, ou um sistema de plugins. Seu avaliador permanece sendo seu próprio serviço HTTP na sua própria infraestrutura, exatamente como descrito no guia [Evaluation suite](/pt-br/agenteye/evaluation-suite). A habilidade apenas ensina seu agente a construí-lo bem, de modo que tudo o que ela faz, você poderia fazer escrevendo o mesmo código. - ---- - -## A parte difícil é decidir o que pontuar - -A superfície do SDK é pequena — um decorator e dois modelos — e um agente pode escrever isso apenas com o [contrato](/pt-br/agenteye/evaluation-suite#http-contract). Não é aí que os avaliadores falham. Eles falham porque pontuam a coisa errada, e um avaliador que pontua a coisa errada é pior do que nenhum: ele produz um dashboard que todos aprendem a ignorar. - -Por isso, a maior parte da habilidade é a etapa anterior a qualquer código. Ela faz o agente entrevistá-lo (*"descreva uma execução que correu bem; agora uma que correu mal"*), depois puxa suas sessões reais pelo [`agenteye` CLI](/pt-br/agenteye/cli) e as lê do início ao fim. Essas duas metades geralmente discordam, e a lacuna é exatamente o ponto: o que você pretende medir versus o que suas transcrições podem realmente suportar. Uma dimensão só sobrevive se for **computável** a partir dos eventos e **discriminatória** — se pontua 0,9 tanto na sua boa execução quanto na ruim, não ensina nada e é cortada. - -O resultado é uma proposta de 2 a 4 dimensões com o raciocínio anexado, para você aprovar antes que uma linha seja escrita. - -```mermaid -flowchart TD - YOU["você: 'quero avaliações para meu bot de suporte'"] --> AGENT["agente de codificação (Claude Code / Codex)
    carrega a habilidade agenteye-evaluator"] - AGENT -->|"entrevista: como é bom vs ruim?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["suas sessões reais
    o que realmente acontece"] - DATA --> DIMS["2-4 dimensões, você aprova"] - DIMS --> SVC["seu serviço avaliador
    SDK agenteye-evaluator"] - SVC --> SCORES["pontuações chegam no dashboard
    e em agenteye evals"] -``` - ---- - -## Como ela se relaciona com as outras partes de avaliação - -Quatro documentos cobrem pontuação, e eles se encadeiam em ordem: - -| Página | O que é | Consulte quando | -|---|---|---| -| **[Evaluations](/pt-br/agenteye/evaluations)** | O recurso: pontuações na grade de sessões, dashboards, reavaliar | Você quer saber o que a pontuação automática oferece | -| **[Evaluation suite](/pt-br/agenteye/evaluation-suite)** | O contrato HTTP, o SDK, as variáveis de ambiente do servidor | Você está implementando ou depurando o avaliador por conta própria | -| **Habilidade de avaliador** (este documento) | Uma porta de entrada em linguagem natural para projetar *e* construir o pontuador | Você quer ir de "quero avaliações" a um serviço em execução | -| **[CLI skill](/pt-br/agenteye/cli-skill)** | Uma porta de entrada em linguagem natural para o `agenteye` CLI | Você quer *ler* as pontuações que já possui | -| **[Python SDK skill](/pt-br/agenteye/python-sdk-skill)** | Uma porta de entrada em linguagem natural para instrumentar seu agente | Seu agente ainda não está emitindo sessões — não há nada para pontuar | - -### vs. a CLI skill: construir versus ler - -As duas habilidades são deliberadamente não sobrepostas, e instalar ambas é a configuração normal — o agente escolhe entre elas com base no que você pede: - -- **`agenteye-evaluator`** (este documento) constrói a coisa que *produz* pontuações. Seu trabalho termina quando as pontuações chegam pela primeira vez. -- **[`agenteye-cli`](/pt-br/agenteye/cli-skill)** lê pontuações que já existem (`agenteye evals`). *"A qualidade caiu esta semana?"* é a pergunta dela, não desta habilidade. - ---- - -## Pré-requisitos - -1. O **`agenteye` CLI instalado e com login efetuado** (`pipx install agenteye`, depois `agenteye login`). A habilidade depende dele duas vezes: para puxar as sessões reais com as quais projeta, e para confirmar que suas pontuações chegaram ao final. Seu login precisa de `events:read`, mais `evaluations:read` para essa verificação final. Como acontece com a CLI skill, ela **não pode** completar o login com código único enviado por e-mail por você. -2. **Um lugar para o avaliador residir.** Ele é construído em uma imagem e executado como um serviço de longa duração, portanto precisa de um repositório real, não de um arquivo temporário. Avaliadores geralmente vivem em seu próprio repositório, separado do agente sendo pontuado — a habilidade procura um existente e pergunta antes de criar um novo. -3. **O wheel do SDK `agenteye-evaluator`** — leia a próxima seção antes de deixar seu agente começar a digitar comandos `pip`. - ---- - -## Onde obtê-la - -A habilidade está publicada na coleção pública de habilidades da Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -O repositório é público e a habilidade não precisa de nenhuma credencial própria — ela apenas aciona o `agenteye` CLI com a sessão com a qual *você* fez login, e escreve código no *seu* repositório. Observe que ela é distribuída como sua própria pasta e **não** está dentro do pacote `pipx install agenteye`, portanto não a procure lá. - -## Instalando a habilidade - -O caminho mais rápido é o CLI [`skills`](https://skills.sh), que busca a pasta e a coloca onde seu agente procura: - -```bash -# Claude Code, somente este projeto -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# todos os projetos (instala em ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codex em vez disso -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -Depois gerencie como qualquer outra habilidade: - -```bash -npx skills list -a claude-code # o que está instalado -npx skills update agenteye-evaluator # baixar a versão mais recente -npx skills remove agenteye-evaluator # remover -``` - -Prefere instalar manualmente? Uma Agent Skill é apenas uma pasta contendo um `SKILL.md` (mais referências opcionais), então copiá-la também funciona: - -- **Claude Code**: coloque a pasta `agenteye-evaluator/` em `~/.claude/skills/` (todos os projetos) ou `/.claude/skills/` (somente aquele repositório). Claude Code a descobre automaticamente — verifique com a lista `/skills`, ou simplesmente peça avaliações. -- **Codex (OpenAI)**: Codex lê o mesmo `SKILL.md`. O arquivo `agents/openai.yaml` incluído define `allow_implicit_invocation: true`, então o Codex seleciona automaticamente a habilidade quando uma tarefa combina; caso contrário, invoque-a explicitamente como `$agenteye-evaluator`. - ---- - -## O SDK não está no PyPI público - -> **Aviso:** Leia isso antes de deixar um agente instalar o SDK. - -A habilidade é pública; o SDK que ela aciona não é. O `agenteye-evaluator` é distribuído apenas como um artefato de release privado e, ao contrário do `agenteye`, o nome está **sem registro no PyPI público** — então um simples `pip install agenteye-evaluator` poderia puxar o pacote de um desconhecido para o serviço que lê suas transcrições de produção. Isso é um problema de cadeia de suprimentos, não um erro de digitação. - -A habilidade sabe disso e percorre uma escada de instalação em vez disso, parando no primeiro degrau que se aplica: o código-fonte do monorepo se você estiver dentro do repositório AgentEye, caso contrário o wheel de release privado do GitHub Releases (requer acesso), e se nenhum estiver acessível ela **para e diz para você perguntar ao seu contato na Failproof AI pelo wheel** em vez de improvisar. - -Portanto, se seu agente propõe um simples `pip install agenteye-evaluator` do PyPI público, isso é o sinal de que a habilidade nunca foi carregada. Pare aí e verifique se ela está instalada. - ---- - -## O que você pode pedir a ela - -Uma rodada completa começa com uma solicitação vaga e termina com um design aprovado, não com código: - -```text -você ▸ Quero avaliações para nosso bot de suporte. Acho que às vezes ele falha. - -agente▸ Antes de escrever qualquer coisa: descreva uma execução que correu bem, - e uma que correu mal. - -você ▸ Boa: ele responde pela documentação. Ruim: ele inventa coisas, - ou desiste e diz "entre em contato com o suporte". - -agente▸ Executando: agenteye --json sessions --status error --since 7d - Lendo 5 sessões, incluindo duas que você consideraria ruins. - - Suas palavras dizem factualidade e desvio. As transcrições - concordam com desvio (2 de 5 terminam com encaminhamento ao suporte, - e está claro em model_response.content), mas factualidade - precisa de uma resposta de referência que seus eventos não carregam. - - Então eu pontuaria três coisas: - - deflection regras, barato, separa suas duas execuções - - tool_efficiency regras, captura o loop de retentativa em run-004 - - groundedness juiz LLM: cada afirmação está nos documentos recuperados? - - "factuality" precisaria de resposta de referência que não temos; groundedness - é o mais próximo que suas transcrições podem realmente suportar. - OK com esses três nomes? Renomear depois divide o histórico. - -você ▸ sim - -agente▸ Nenhum avaliador neste repositório. Devo criar um aqui, ou você - tem um em outro lugar? -``` - -A partir daí ele escreve primeiro as dimensões baseadas em regras (gratuitas, instantâneas, determinísticas), testa-as contra uma sessão real capturada, incluindo as vazias e as nunca concluídas que travam avaliadores ingênuos, e só recorre a um juiz LLM na dimensão subjetiva. Ele conhece os [limites do dispatcher](/pt-br/agenteye/evaluation-suite#configuring-the-server) — um timeout de requisição de 30s e 8 chamadas concorrentes em todo o deployment — portanto, se o juiz não couber de forma confiável, ele vai assíncrono com `JobPending` em vez de deixar seu juiz ser cancelado e repetido cinco vezes ao custo de cinco vezes mais. - -Depois implanta, configura as duas variáveis de ambiente do servidor e confirma com `agenteye --json evals --session-id ` que as pontuações realmente chegaram. As pontuações chegando é a única prova. - ---- - -## O que observar - -- **Nomes de dimensões são quase permanentes.** As chaves de pontuação são strings arbitrárias e a plataforma rastreia tendências de tudo o que você envia, o que significa que nada downstream corrige uma escolha ruim. Renomeie depois e o histórico se divide: sessões antigas mantêm a chave antiga e a tendência se quebra. É por isso que a habilidade obtém aprovação explícita antes de escrever código — leve esse prompt a sério. -- **Fixtures são transcrições reais de produção.** Projetar contra sessões reais significa baixá-las para o disco, e elas podem conter dados de clientes. A habilidade pergunta antes de commitá-las no git; em caso de dúvida, mantenha `fixtures/` fora do repositório e peça a cada desenvolvedor que baixe as suas próprias. -- **O agente escreve e implanta um serviço que lê todas as transcrições.** Ele age como você, limitado pelas permissões do seu login no CLI, mas revise o avaliador como qualquer outro código que toca dados de produção. - ---- - -## Próximos passos - -- **[Evaluation suite](/pt-br/agenteye/evaluation-suite)**: o contrato HTTP, o SDK e as variáveis de ambiente do servidor que a habilidade configura. -- **[Evaluations](/pt-br/agenteye/evaluations)**: onde as pontuações aparecem assim que chegam. -- **[CLI skill](/pt-br/agenteye/cli-skill)**: a habilidade irmã, para ler resultados em vez de construir o pontuador. -- **[CLI](/pt-br/agenteye/cli)**: a referência de comandos por trás dos dados de sessão com os quais a habilidade projeta. \ No newline at end of file diff --git a/docs/pt-br/agenteye/hermes-capture.mdx b/docs/pt-br/agenteye/hermes-capture.mdx deleted file mode 100644 index 73b07cf6..00000000 --- a/docs/pt-br/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Captura de sessões do Hermes" -description: "Traga as sessões do gateway Hermes da sua equipe — Slack, Telegram, CLI e execuções agendadas — para o AgentEye como sessões e eventos comuns." ---- - -O [Hermes](https://hermes-agent.nousresearch.com) responde à sua equipe de onde quer que ela já trabalhe — Slack, Telegram, CLI, execuções agendadas. A captura de sessões do Hermes traz tudo isso para o AgentEye como sessões e eventos comuns, tornando o assistente com o qual sua equipe conversa todos os dias tão observável quanto os agentes que você mesmo escreve. - -Um pequeno coletor em segundo plano lê o armazenamento local de sessões do Hermes conforme ele é escrito e envia as sessões para o AgentEye. Funciona da mesma forma que a captura do [Codex](/pt-br/agenteye/codex-capture) e do [OpenClaw](/pt-br/agenteye/openclaw-capture), e um único coletor pode capturar vários ao mesmo tempo. - ---- - -## O que é capturado - -Todas as sessões do Hermes na máquina são capturadas, independentemente do canal de origem. Cada uma se torna uma [sessão](/pt-br/agenteye/sessions) no AgentEye; suas mensagens de usuário e assistente, chamadas de ferramenta e resultados de ferramenta se tornam os [eventos](/pt-br/agenteye/event-stream) correspondentes. - -O canal pelo qual uma sessão foi iniciada — Slack, Telegram, CLI ou uma execução agendada — é registrado na sessão, para que você possa diferenciá-las e filtrar por uma de cada vez. Junto a isso, são registrados o modelo usado na sessão, o chat e a pessoa que a iniciou e, quando uma sessão originou outra, o vínculo com a sessão pai. - -As sessões aparecem assim que o Hermes as inicia, mesmo que nada tenha sido dito ainda, e a resposta de um turno e suas chamadas de ferramenta são mantidas na ordem em que realmente ocorreram. Quando uma sessão termina, você também obtém o motivo do encerramento, o custo e a quantidade de tokens utilizados. - ---- - -## Como ativar - -A captura fica desativada até você habilitá-la. Instale o coletor com uma chave de API que tenha a permissão `events:add` (consulte [Chaves de API](/pt-br/agenteye/api-keys)) e ative a captura do Hermes: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -Isso instala o coletor, o registra como um serviço em segundo plano e inicia a captura. Confirme que está em execução: - -```bash -agenteye-collector health -``` - -Capturando mais de um agente na mesma máquina? Adicione a flag de cada um ao mesmo comando — por exemplo, `--hermes-enabled --codex-enabled`. - -Na primeira execução, suas sessões existentes do Hermes são preenchidas retroativamente uma vez, e a nova atividade passa a ser transmitida em segundos. Os dados do próprio Hermes são apenas lidos — nunca modificados ou excluídos — e cada mensagem é enviada uma única vez, mesmo após reinicializações. - -O `health` também informa se tudo que o coletor capturou realmente chegou ao AgentEye. Se um lote não puder ser entregue, ele é mantido e reenviado em vez de descartado, e a verificação reporta estado não saudável enquanto houver itens pendentes — portanto, "saudável" significa que seus dados chegaram, não apenas que o processo está ativo. - ---- - -## Onde aparece - -As sessões capturadas aparecem em **Sessions**, e seus eventos na stream de **Events**, da mesma forma que qualquer outro agente que você observa — portanto, o [replay de sessão](/pt-br/agenteye/sessions), a [busca](/pt-br/agenteye/queries), as [avaliações](/pt-br/agenteye/evaluations) e os [alertas](/pt-br/agenteye/alerts) funcionam normalmente com elas. Filtre pelo agente Hermes para visualizá-las separadamente. - ---- - -## Privacidade - -As sessões do Hermes contêm a transcrição completa — incluindo saída de comandos, conteúdo de arquivos e tudo que o agente leu ou escreveu — e podem conter segredos. As sessões capturadas são enviadas no estado em que se encontram, portanto, ative a captura somente onde centralizar esse conteúdo no AgentEye for adequado, e forneça ao coletor uma chave com escopo limitado a `events:add`. Consulte [Segurança](/pt-br/agenteye/security) para saber como seus dados são mantidos isolados. \ No newline at end of file diff --git a/docs/pt-br/agenteye/observability.mdx b/docs/pt-br/agenteye/observability.mdx deleted file mode 100644 index 4db702b2..00000000 --- a/docs/pt-br/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Observar" -description: "As superfícies de observação são onde você acompanha o que seus agentes estão fazendo agora e analisa qualquer execução individual." ---- - - -As superfícies de observação são onde você acompanha o que seus agentes estão fazendo agora e analisa qualquer execução individual. Tudo aqui é em tempo real, com escopo para sua organização e filtrável por intervalo de datas, ambiente, agente e sessão — para que você passe de "algo parece errado" para a execução exata em segundos. - -![O Event Stream ao vivo, com código de cores por tipo e filtrável por ambiente, agente e sessão](/agenteye/images/events-stream.png) - -Quatro superfícies, cada uma com sua própria página: - -- **[Event stream](/pt-br/agenteye/event-stream)**: o rastro ao vivo, passo a passo, de cada execução em todos os agentes, da mais recente para a mais antiga. É a página inicial da sua organização e o primeiro ponto de triagem. -- **[Sessões e grafo de execução](/pt-br/agenteye/sessions)**: esses eventos consolidados em uma linha por execução, além de uma visualização no estilo git de como cada execução se desenrolou. -- **[Métricas de desempenho](/pt-br/agenteye/telemetry)**: mapas de calor de latência e indicadores p50/p95/p99 para seus modelos, ferramentas e hooks, para que um pico na cauda se destaque da mediana. -- **[Rastreamento de erros](/pt-br/agenteye/error-tracking)**: uma única superfície de triagem para tudo que deu errado, a um clique de um alerta disparado até a execução que falhou. - -## Relacionado - -- [Avaliações](/pt-br/agenteye/evaluations): pontue cada execução pela qualidade. -- [Alertas](/pt-br/agenteye/alerts): transforme qualquer limite em uma regra de notificação. -- [Auditorias](/pt-br/agenteye/audits): deixe a Observabilidade do Failproof AI encontrar padrões de falha entre sessões para você. -- [CLI e agentes](/pt-br/agenteye/cli-and-agents): a mesma observabilidade a partir do seu terminal. \ No newline at end of file diff --git a/docs/pt-br/agenteye/openclaw-capture.mdx b/docs/pt-br/agenteye/openclaw-capture.mdx deleted file mode 100644 index 23649dd5..00000000 --- a/docs/pt-br/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "Captura de sessão OpenClaw" -description: "Envie as sessões locais OpenClaw da sua equipe para o AgentEye como sessões e eventos comuns — sem alterar a forma como o OpenClaw é executado." ---- - -Se sua equipe utiliza o [OpenClaw](https://docs.openclaw.ai), a captura de sessão OpenClaw traz essas sessões para o AgentEye como sessões e eventos comuns, permitindo que você pesquise, reproduza e avalie-as junto com tudo o mais que você observa. Ela complementa o [Python SDK](/pt-br/agenteye/python-sdk): o SDK instrumenta os agentes que você escreve, enquanto esta captura o trabalho OpenClaw que sua equipe já realiza — sem nenhuma alteração na forma como eles o executam. - -Um pequeno coletor em segundo plano lê os transcritos de sessão locais do OpenClaw conforme são gravados e os envia para o AgentEye. Ele funciona da mesma forma que a [captura do Codex](/pt-br/agenteye/codex-capture), e um único coletor pode capturar ambos ao mesmo tempo. - ---- - -## O que é capturado - -Cada agente configurado na instalação OpenClaw de uma máquina é capturado pelo coletor dessa máquina — não é necessária nenhuma configuração por agente. - -Cada sessão OpenClaw se torna uma [sessão](/pt-br/agenteye/sessions) no AgentEye; suas mensagens de usuário e assistente, chamadas de ferramentas e resultados de ferramentas se tornam os [eventos](/pt-br/agenteye/event-stream) correspondentes. - ---- - -## Como ativar - -A captura fica desativada até que você a habilite. Instale o coletor com uma chave de API que tenha a permissão `events:add` (consulte [Chaves de API](/pt-br/agenteye/api-keys)) e ative a captura OpenClaw: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -Isso instala o coletor, registra-o como um serviço em segundo plano e inicia a captura. Confirme que está em execução: - -```bash -agenteye-collector health -``` - -Capturando mais de um agente na mesma máquina? Adicione a flag de cada um ao mesmo comando — por exemplo, `--openclaw-enabled --codex-enabled`. - -Na primeira execução, suas sessões OpenClaw existentes são preenchidas retroativamente uma vez, e as novas atividades passam a ser transmitidas em segundos. Os próprios arquivos do OpenClaw são apenas lidos — nunca modificados, movidos ou excluídos — e cada sessão é enviada exatamente uma vez, mesmo entre reinicializações. - ---- - -## Onde aparecem - -As sessões capturadas aparecem em **Sessões**, e seus eventos no fluxo de **Eventos**, da mesma forma que qualquer outro agente que você observa — portanto, [replay de sessão](/pt-br/agenteye/sessions), [pesquisa](/pt-br/agenteye/queries), [avaliações](/pt-br/agenteye/evaluations) e [alertas](/pt-br/agenteye/alerts) funcionam normalmente nelas. Filtre pelo agente OpenClaw para visualizá-las de forma isolada. - ---- - -## Privacidade - -Os transcritos do OpenClaw contêm a sessão completa — incluindo saída de comandos, conteúdo de arquivos e tudo o que o agente leu ou escreveu — e podem conter segredos. As sessões capturadas são enviadas como estão, portanto, ative a captura somente em máquinas e para equipes onde centralizar esse conteúdo no AgentEye seja apropriado, e forneça ao coletor uma chave com escopo restrito a `events:add`. Consulte [Segurança](/pt-br/agenteye/security) para saber como seus dados são mantidos isolados. \ No newline at end of file diff --git a/docs/pt-br/agenteye/python-sdk-skill.mdx b/docs/pt-br/agenteye/python-sdk-skill.mdx deleted file mode 100644 index fd5981db..00000000 --- a/docs/pt-br/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,136 +0,0 @@ ---- -title: "Agent Skill Python SDK de Observabilidade do Failproof AI" -description: "Saia de um agente sem instrumentação para eventos visíveis, com seu agente de código encontrando os pontos de instrumentação, escrevendo-os e provando que funcionaram." ---- - -Diga ao seu agente de código *"adicione Observabilidade do Failproof AI a este agente"* e deixe-o ler seu loop, identificar onde a instrumentação deve ficar, escrevê-la e verificar os eventos antes de declarar o trabalho concluído. - -A **skill Python SDK** (`agenteye-python-sdk`) é um *Agent Skill*: uma pasta de instruções que um agente de código como Claude Code ou Codex carrega sob demanda quando uma tarefa corresponde a ela. Ela ensina o agente a usar o [Python SDK](/pt-br/agenteye/python-sdk) — não é uma biblioteca e não muda nada sobre como o SDK funciona. - -## Instrumentação é fácil de escrever e fácil de errar silenciosamente - -O SDK é pequeno: treze métodos de eventos, todos com argumentos nomeados. Um agente de código pode ler a referência do [Python SDK](/pt-br/agenteye/python-sdk) e produzir instrumentação plausível em um minuto. - -O problema é que esse SDK não levanta exceções quando você erra, e uma instrumentação incorreta parece exatamente com uma correta — até alguém abrir um dashboard e encontrá-lo vazio. Os erros que custam tempo real são todos silêncios: - -| O erro | O que você vê | -|---|---| -| Sem `agent_start` | Todos os eventos chegam. Zero sessões. | -| Ambiente nunca definido | Tudo funciona, arquivado como `dev`. | -| `outcome="failure"` | A execução aparece como verde — apenas `failed`, `error`, `timeout`, `rejected` contam. | -| Nome de campo com erro de digitação | Aceito e armazenado como um novo campo. | -| Eventos emitidos de um thread pool | Descartados silenciosamente. | - -Nenhum desses levanta exceções. Nenhum aparece em testes. Cada um está na skill, declarado como um contrato com a verificação que o detecta. - -## O que ela faz, em ordem - -A skill executa os mesmos três passos que um engenheiro cuidadoso seguiria: - -1. **Planejar.** Ela lê seu loop de agente e faz as duas perguntas que só você pode responder: o que conta como uma execução (seu `session_id`) e quem são os atores distinguíveis (seu `agent_id`). Ela obtém essas respostas antes de escrever código, porque mudá-las depois divide seu histórico e quebra as tendências. -2. **Escrever.** Ela vincula a identidade uma vez por execução em vez de passá-la por todos os pontos de chamada, e escolhe uma forma segura para concorrência — um detalhe que importa, porque o atalho óbvio silenciosamente mistura duas execuções sobrepostas em uma única sessão. -3. **Verificar.** Ela executa seu agente e lê os arquivos de eventos resultantes, verificando se `agent_start` está presente, se o ambiente está correto e se uma execução produziu uma sessão. - -Esse terceiro passo é o que as pessoas pulam. O SDK grava eventos em arquivos locais, então uma integração completa pode ser provada em um laptop sem servidor, sem chave de API e sem rede — e é exatamente por isso que a skill insiste em fazê-lo. - -## Como ela se relaciona com as outras skills - -Três skills, uma divisão clara: - -| Skill | Use quando | O que ela toca | -|---|---|---| -| **Skill Python SDK** (esta página) | Você quer que seu agente *emita* telemetria — "adicionar observabilidade", "por que meu agente não está aparecendo?" | Escreve código no repositório do seu agente. Não lê nada. | -| **[Skill Evaluator](/pt-br/agenteye/evaluator-skill)** | Você quer *pontuar* execuções — "o que devemos medir?" | Escreve código no seu repositório; lê telemetria | -| **[Skill CLI](/pt-br/agenteye/cli-skill)** | Você quer *ler* o que aconteceu, ou operar seu deployment | Usa o CLI como você, incluindo alterações | - -Elas se encadeiam nessa ordem: esta skill faz os eventos fluírem, o evaluator os pontua, e o CLI os lê de volta. Não há nada para avaliar e nada para ler até que seu agente emita sessões — então, se você está começando do zero, comece aqui. - -## Pré-requisitos - -1. **Python 3.10+** e a base de código do agente que você deseja instrumentar. -2. **O SDK.** Ele é distribuído aos clientes como um wheel privado, não de um índice público — seu onboarding cobre como obtê-lo e instalá-lo. A skill conhece o caminho de instalação e perguntará a você em vez de adivinhar, caso não consiga encontrá-lo. -3. **Nada mais.** Sem login no dashboard, sem chave de API, sem rede. A skill verifica contra os arquivos de eventos que o SDK grava, portanto pode concluir e provar seu trabalho offline. - -## Onde encontrá-la - -A skill está na coleção pública [`FailproofAI/skills`](https://github.com/FailproofAI/skills): - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -Adicione `-g` para instalá-la em todos os projetos em vez de apenas no atual, e `--copy` se seu ambiente não suporta symlinks. Para Codex, passe `-a codex`. - -## Instalando manualmente - -Agent Skills são pastas contendo um `SKILL.md` mais referências. Se preferir não usar o instalador: - -- **Claude Code**: copie a pasta `agenteye-python-sdk/` para `~/.claude/skills/` (todos os projetos) ou `/.claude/skills/` (somente aquele repositório). Claude Code a descobre automaticamente — verifique a lista `/skills`, ou simplesmente pergunte algo que corresponda a ela. -- **Codex**: o Codex lê o mesmo `SKILL.md`. O arquivo `agents/openai.yaml` incluído define `allow_implicit_invocation: true`, então ela é selecionada automaticamente quando uma tarefa corresponde; caso contrário, invoque-a como `$agenteye-python-sdk`. - -Execute seu agente **no repositório que contém o código que você deseja instrumentar** — a skill lê seu loop de agente antes de propor qualquer coisa. - -## Como uma sessão se parece - -```text -você ▸ Adicione Observabilidade do Failproof AI a este agente. - -agente▸ Li seu loop — um dispatcher de ferramentas em tools.py, um - wrapper do Anthropic em llm.py. Duas perguntas antes de - escrever qualquer coisa. - - O que conta como uma execução? Você lida com requisições - concorrentemente, então eu usaria o id de requisição que você - já tem em ctx.request_id — assim uma sessão aqui e uma - requisição nos seus logs são a mesma string. - - E quem são os atores? Vejo um caminho de planejador e um - caminho de pesquisador. Rotulo-os separadamente, ou é um único - agente para você? - -você ▸ O id de requisição está certo. São um único agente, chame de - "assistant". - -agente▸ Pronto. Envolvi o dispatcher e o cliente LLM; agent_start e - agent_end delimitam handle_request, incluindo o caminho de erro. - Usei um context-local para identidade — seu dispatcher executa - ferramentas em um thread pool, e a abordagem mais simples teria - misturado execuções concorrentes em uma única sessão sem - reclamar. - - Verificado contra uma execução real: 11 eventos em 4 arquivos — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, uma sessão - por requisição, durações pareadas. - - Uma coisa que eu NÃO fiz: seu worker pool em batch.py submete - diretamente ao executor, então eventos de lá seriam descartados. - Quer que eu corrija esses também? -``` - -O padrão a observar: ele leu o código antes de propor, fez apenas as perguntas que só você pode responder, reutilizou um id que você já tinha, escolheu a forma segura para concorrência *porque* viu um thread pool, e **verificou lendo os eventos reais** em vez de declarar sucesso — e então sinalizou o único lugar que sabia que falharia silenciosamente. - -## O que você pode pedir - -- *"Por que meu agente não está aparecendo no dashboard?"* → percorre a escada: os eventos estão sendo gravados, `agent_start` está lá, o ambiente está correto, o collector está lendo o mesmo lugar. -- *"Tudo está chegando como dev."* → o ambiente nunca foi definido, ou foi redefinido por uma chamada posterior. -- *"Adicione rastreamento de tokens."* → encontra seu wrapper LLM e registra modelo, motivo de parada e uso. -- *"Instrumente os sub-agentes também."* → uma sessão, rótulos de agentes distintos, aninhados sob seu pai. -- *"Escreva testes para a instrumentação."* → aponta o SDK para um diretório temporário e faz asserções sobre os eventos que ele gravou. - -## O que observar - -**Deixe-o verificar.** O passo que torna esta skill útil é o último — executar seu agente e ler os eventos de volta. Um agente que escreve instrumentação e para fez a metade fácil, e a metade que falha silenciosamente é a outra. - -**Concorde com os nomes antes do código.** `session_id` e `agent_id` são os eixos pelos quais toda superfície agrupa. Renomeá-los depois divide o histórico: execuções antigas mantêm os rótulos antigos e suas tendências se quebram. A skill vai perguntar; a resposta vale um minuto de reflexão. - -**Se seu agente propuser instalar o SDK de um índice público, a skill não carregou.** O SDK é distribuído de forma privada. Essa proposta é um sinal confiável de que seu agente de código está adivinhando em vez de seguir a skill — pare-o ali e verifique se a skill está instalada. - -Além disso, seu raio de ação é pequeno: ela escreve código no seu diretório de trabalho e arquivos de eventos onde você mandar. Não lê nada do seu deployment e não muda nada nele. - -## Próximos passos - -- **[Python SDK](/pt-br/agenteye/python-sdk)**: a referência completa de eventos — cada tipo de evento e campo — por trás do que esta skill automatiza. -- **[Sessions](/pt-br/agenteye/sessions)**: o que sua instrumentação produz quando os eventos chegam. -- **[Evaluator Agent Skill](/pt-br/agenteye/evaluator-skill)**: o próximo passo quando as execuções estiverem chegando — pontuá-las. -- **[CLI Agent Skill](/pt-br/agenteye/cli-skill)**: lendo sua telemetria de volta. \ No newline at end of file diff --git a/docs/pt-br/agenteye/security.mdx b/docs/pt-br/agenteye/security.mdx deleted file mode 100644 index e66cb71c..00000000 --- a/docs/pt-br/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "Segurança" -description: "O Failproof AI Observability foi projetado para ficar próximo aos seus agentes em produção, o que significa que ele vê seus prompts, entradas de ferramentas e saídas." ---- - - -O Failproof AI Observability foi projetado para ficar próximo aos seus agentes em produção, o que significa que ele vê seus prompts, entradas de ferramentas e saídas. Esta página explica como esses dados são mantidos isolados, controlados e nas suas mãos. Se você está avaliando o Failproof AI Observability para uma revisão de segurança, comece por aqui. - ---- - -## Seus dados ficam no seu ambiente - -O Failproof AI Observability é auto-hospedado. Eventos, prompts, respostas do modelo e análises são armazenados nos seus próprios bancos de dados, no seu próprio ambiente. Nada é enviado para um SaaS de terceiros para armazenamento, e seus dados permanecem na sua própria conta de nuvem. - ---- - -## Isolamento de tenant - -Uma instância do Failproof AI Observability pode hospedar várias organizações, e cada uma é isolada na camada de armazenamento — aplicado pelo banco de dados, não apenas pela interface: - -- Os dados operacionais de uma organização (usuários, chaves, dashboards, consultas salvas) são restritos àquela org, e leituras entre organizações são bloqueadas pelo próprio banco de dados. -- Todo evento ingerido é marcado com a organização proprietária, de modo que os eventos de uma organização nunca podem ser lidos por outra. - -Cada rota de dashboard é delimitada por um slug de org (`//…`). - ---- - -## Login - -O Failproof AI Observability utiliza login sem senha, baseado em e-mail. Não há senha para ser furtada ou vazada. Um usuário solicita um código de uso único (ou um magic link de clique único), que é enviado por e-mail e expira rapidamente. O login é controlado por uma **lista de permissões**: somente endereços de e-mail (ou domínios) que você autorizar podem se autenticar. - -![A tela de login do Failproof AI Observability, que envia um código de uso único para seu e-mail](/agenteye/images/login.png) - ---- - -## Acesso restrito com chaves de API - -Cada cliente se autentica com uma chave de API que carrega permissões granulares e de menor privilégio. Um coletor precisa apenas de `events:add`; uma chave de dashboard ou assistente pode ser somente leitura; ações destrutivas (exclusão, regeneração) são concessões separadas que você escolhe incluir. - -![A página de chaves de API: as permissões de cada chave, com código de cores por escopo de leitura, escrita e destrutivo](/agenteye/images/api-keys.png) - -Mantenha a chave de bootstrap de administrador para a configuração inicial e emita chaves restritas para todo o resto. Consulte [Chaves de API](/pt-br/agenteye/api-keys). - ---- - -## Um assistente somente leitura com aprovação obrigatória - -O [assistente de IA](/pt-br/agenteye/assistant) integrado ao dashboard responde perguntas sobre seus dados, mas é restrito por design: - -- É **somente leitura por padrão**: o SQL que ele executa passa por um guard que permite apenas consultas `SELECT`/`WITH`, instrução única, com limite de linhas. -- Tudo que ele cria (uma consulta salva, um dashboard) **requer aprovação**: você revisa e aprova cada escrita antes que ela aconteça. -- Ele **nunca pode excluir**. - -Assim, um colega de equipe pode perguntar "quais agentes tiveram mais erros esta semana?" e agir com base na resposta, sem que o assistente consiga alterar ou remover seus dados por conta própria. - ---- - -## Em trânsito - -Todo o tráfego é transmitido via HTTPS. Você encerra o TLS com seus próprios certificados, de modo que o tráfego do coletor para o servidor e do navegador para o servidor é criptografado em trânsito. - ---- - -## Próximos passos - -- [Visão geral](/pt-br/agenteye/overview): como o Failproof AI Observability se encaixa. -- [Chaves de API](/pt-br/agenteye/api-keys): restrinja o acesso para o coletor, dashboard e assistente. -- [Observabilidade](/pt-br/agenteye/observability): o que o Failproof AI Observability captura dos seus agentes. \ No newline at end of file diff --git a/docs/pt-br/cli/audit.mdx b/docs/pt-br/audit.mdx similarity index 100% rename from docs/pt-br/cli/audit.mdx rename to docs/pt-br/audit.mdx diff --git a/docs/pt-br/cli/backfill.mdx b/docs/pt-br/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/pt-br/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/pt-br/cli/config.mdx b/docs/pt-br/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/pt-br/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/pt-br/cli/flush.mdx b/docs/pt-br/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/pt-br/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/pt-br/cli/harness.mdx b/docs/pt-br/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/pt-br/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/pt-br/cli/migrate.mdx b/docs/pt-br/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/pt-br/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/pt-br/cli/uninstall.mdx b/docs/pt-br/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/pt-br/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/pt-br/cli/update.mdx b/docs/pt-br/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/pt-br/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/pt-br/agenteye/api-keys.mdx b/docs/pt-br/cloud/access.mdx similarity index 98% rename from docs/pt-br/agenteye/api-keys.mdx rename to docs/pt-br/cloud/access.mdx index 084e1c50..73015e49 100644 --- a/docs/pt-br/agenteye/api-keys.mdx +++ b/docs/pt-br/cloud/access.mdx @@ -82,7 +82,7 @@ O servidor aplica um catálogo fixo de permissões; cada uma controla rotas HTTP Essas permissões sustentam a página **Users** do dashboard, onde os escopos concedidos a cada membro são exibidos como chips: -![A página Users: um card por usuário do dashboard com seu e-mail, permissões concedidas e controles de edição/desativação](/agenteye/images/users.png) +![A página Users: um card por usuário do dashboard com seu e-mail, permissões concedidas e controles de edição/desativação](/cloud/images/users.png) ### Configurações operacionais @@ -91,7 +91,7 @@ Essas permissões sustentam a página **Users** do dashboard, onde os escopos co | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | Visualizar configurações operacionais gerenciadas pelo dashboard e seus metadados; listar substituições de janela de contexto por modelo; e resolver a janela efetiva para um modelo. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | Editar configurações operacionais e adicionar, alterar ou remover substituições de janela de contexto por modelo. As alterações afetam novos eventos sem reiniciar o servidor. | -![A página Settings: configurações operacionais gerenciadas pelo dashboard, como logins permitidos e tempos de vida de sessão/OTP, editáveis sem reinicialização](/agenteye/images/settings.png) +![A página Settings: configurações operacionais gerenciadas pelo dashboard, como logins permitidos e tempos de vida de sessão/OTP, editáveis sem reinicialização](/cloud/images/settings.png) ### Alertas e incidentes @@ -247,7 +247,7 @@ A página **Keys** no dashboard fornece uma interface para todas as operações Ao criar uma chave pelo dashboard, você não fornece o segredo; o dashboard gera um segredo forte para você e o exibe **uma única vez** na criação. Copie-o imediatamente e armazene-o com segurança; ele nunca será exibido novamente, exatamente como em uma regeneração. Você ainda pode escolher as permissões da chave diretamente ou gerá-las a partir de um conjunto de permissões (veja abaixo). -![A página API Keys: um card por chave mostrando seu nome, permissões concedidas e horário de criação, com ações de regenerar e desativar; chaves protegidas como `admin` são marcadas](/agenteye/images/api-keys.png) +![A página API Keys: um card por chave mostrando seu nome, permissões concedidas e horário de criação, com ações de regenerar e desativar; chaves protegidas como `admin` são marcadas](/cloud/images/api-keys.png) --- @@ -276,5 +276,5 @@ Você só precisa disso se estiver atualizando uma instância existente; novos d ## Próximos passos -- [Python SDK](/pt-br/agenteye/python-sdk): como o código do seu agente se autentica ao enviar eventos. -- [Segurança](/pt-br/agenteye/security): como funcionam o login, o controle de acesso e o isolamento de dados por organização. \ No newline at end of file +- [Python SDK](/pt-br/cloud/sdk): como o código do seu agente se autentica ao enviar eventos. +- [Segurança](/pt-br/cloud/security): como funcionam o login, o controle de acesso e o isolamento de dados por organização. \ No newline at end of file diff --git a/docs/pt-br/cloud/agent-skills.mdx b/docs/pt-br/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/pt-br/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
    loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/pt-br/agenteye/alerts.mdx b/docs/pt-br/cloud/alerts.mdx similarity index 82% rename from docs/pt-br/agenteye/alerts.mdx rename to docs/pt-br/cloud/alerts.mdx index 0397e07a..dc8cdb66 100644 --- a/docs/pt-br/agenteye/alerts.mdx +++ b/docs/pt-br/cloud/alerts.mdx @@ -6,7 +6,7 @@ description: "Saiba no momento em que algo ultrapassa seu limite, no canal que s Saiba no momento em que algo ultrapassa seu limite, no canal que sua equipe já monitora, em vez de ficar sabendo pelo cliente. Configure uma regra uma vez e a Observabilidade do Failproof AI verifica ela periodicamente, depois te notifica por e-mail, Slack, webhook ou direto no dashboard. -![A página de Alertas: uma grade de cartões de regras de alerta, cada um mostrando seu gatilho, janela de avaliação, canais e um selo de severidade informativo, de aviso ou crítico](/agenteye/images/alerts.png) +![A página de Alertas: uma grade de cartões de regras de alerta, cada um mostrando seu gatilho, janela de avaliação, canais e um selo de severidade informativo, de aviso ou crítico](/cloud/images/alerts.png) *Todas as regras de alerta de relance: o que monitoram, com que frequência, onde notificam e qual a urgência.* ## Saiba dos problemas antes dos seus usuários @@ -24,7 +24,7 @@ Combine qualquer combinação em uma única regra, e a severidade (informativo, Você descreve o que "quebrado" significa em um formulário, e a Observabilidade do Failproof AI escreve a regra subjacente para você. A especificação JSON é apenas o que esse formulário produz nos bastidores, então você pode lê-la para entender uma regra, mas raramente precisa digitá-la. -![O formulário de novo alerta: nome e descrição, um botão de ativar/desativar, e um seletor de gatilho oferecendo limite de métrica, SQL personalizado, pontuação de avaliação, avaliação composta e condições por evento](/agenteye/images/alert-new.png) +![O formulário de novo alerta: nome e descrição, um botão de ativar/desativar, e um seletor de gatilho oferecendo limite de métrica, SQL personalizado, pontuação de avaliação, avaliação composta e condições por evento](/cloud/images/alert-new.png) *Escolha um gatilho e o formulário exibe os campos corretos; Salvar grava a regra.* O caminho feliz é rápido: dê um nome, escolha um **gatilho** (o que monitorar), defina o **limite e a janela** (quão grave, por quanto tempo), adicione pelo menos um **canal**, depois **Salve** e clique em **Testar** para disparar uma notificação sintética e confirmar que cada destino está configurado. Por baixo dos panos, isso produz uma pequena especificação como: @@ -43,7 +43,7 @@ Você não está limitado a um tipo de sinal. Escolha o gatilho que corresponde | **Avaliação composta** | várias verificações de pontuação se combinam com lógica any, all ou pelo-menos-N, para capturar uma regressão que só aparece entre pontuações | | **Por evento** | um único evento correspondente ocorre: um agente específico, um tipo de erro específico ou uma substring de mensagem | -Já está olhando para uma falha na [página de Erros](/pt-br/agenteye/error-tracking)? Cada linha lá tem um botão **+ alerta** que abre esse mesmo formulário preenchido para capturar exatamente aquela falha novamente, de modo que o incidente que você acabou de triar se torna o próximo a te notificar. +Já está olhando para uma falha na [página de Erros](/pt-br/cloud/errors)? Cada linha lá tem um botão **+ alerta** que abre esse mesmo formulário preenchido para capturar exatamente aquela falha novamente, de modo que o incidente que você acabou de triar se torna o próximo a te notificar. **Onde encontrar:** Os alertas ficam em `//alerts`. Criar, editar, excluir e testar regras requer **`alerts:write`**; `alerts:read` é suficiente para visualizar. O seletor de destinatários lista os membros da sua organização por nome, para que você possa notificar uma pessoa sem sair do formulário. @@ -53,11 +53,11 @@ Uma medição ruim não deveria te acordar. O filtro de ruído **M de N** contro ## O que acontece quando um alerta dispara -Uma violação abre um **incidente** e notifica seus canais uma vez. A partir daí, sua equipe confirma o recebimento, atribui um responsável, discute o problema e o resolve, tudo contra um registro limpo e atribuído. Esse fluxo de triagem tem seu próprio espaço: veja [Incidentes](/pt-br/agenteye/incidents). +Uma violação abre um **incidente** e notifica seus canais uma vez. A partir daí, sua equipe confirma o recebimento, atribui um responsável, discute o problema e o resolve, tudo contra um registro limpo e atribuído. Esse fluxo de triagem tem seu próprio espaço: veja [Incidentes](/pt-br/cloud/incidents). ## Relacionados -- [Incidentes](/pt-br/agenteye/incidents): acompanhe um alerta disparado do estado aberto ao confirmado e ao resolvido. -- [Rastreamento de erros](/pt-br/agenteye/error-tracking): agrupe falhas de agentes e promova uma delas a um alerta com um clique. -- [Dashboards](/pt-br/agenteye/dashboards): monitore os painéis compartilhados de onde vêm os limites que você alerta. -- [CLI e agentes](/pt-br/agenteye/cli-and-agents): crie alertas e confirme incidentes pelo terminal, ou automatize-os no CI. \ No newline at end of file +- [Incidentes](/pt-br/cloud/incidents): acompanhe um alerta disparado do estado aberto ao confirmado e ao resolvido. +- [Rastreamento de erros](/pt-br/cloud/errors): agrupe falhas de agentes e promova uma delas a um alerta com um clique. +- [Dashboards](/pt-br/cloud/dashboards): monitore os painéis compartilhados de onde vêm os limites que você alerta. +- [CLI e agentes](/pt-br/cloud/cli): crie alertas e confirme incidentes pelo terminal, ou automatize-os no CI. \ No newline at end of file diff --git a/docs/pt-br/agenteye/assistant.mdx b/docs/pt-br/cloud/assistant.mdx similarity index 84% rename from docs/pt-br/agenteye/assistant.mdx rename to docs/pt-br/cloud/assistant.mdx index 628d1547..fb9e3ef5 100644 --- a/docs/pt-br/agenteye/assistant.mdx +++ b/docs/pt-br/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "Faça uma pergunta em português simples sobre os dados do seu age --- -Faça uma pergunta em linguagem natural sobre os dados do seu agente e receba uma resposta com links diretos para as evidências. Sem SQL para escrever, sem dashboards para vasculhar — o assistente do **Failproof AI Observability** é a forma mais rápida de qualquer pessoa da sua equipe obter respostas sobre seus agentes. +Faça uma pergunta em linguagem natural sobre os dados do seu agente e receba uma resposta com links diretos para as evidências. Sem SQL para escrever, sem dashboards para vasculhar — o assistente do **FailproofAI Cloud** é a forma mais rápida de qualquer pessoa da sua equipe obter respostas sobre seus agentes. -![O assistente do Failproof AI Observability respondendo uma pergunta em linguagem natural dentro do dashboard, exibindo uma tabela de Atividade de Agentes ao vivo, um detalhamento de uso de modelos por agente e conclusões por escrito, com as consultas executadas mostradas inline](/agenteye/images/assistant.png) +![O assistente do FailproofAI Cloud respondendo uma pergunta em linguagem natural dentro do dashboard, exibindo uma tabela de Atividade de Agentes ao vivo, um detalhamento de uso de modelos por agente e conclusões por escrito, com as consultas executadas mostradas inline](/cloud/images/assistant.png) *Pergunte em linguagem natural e receba uma resposta construída a partir dos seus próprios dados. Aqui, ele detalha quais agentes estão mais ocupados e quais modelos utilizam, e mostra as consultas executadas para que você possa verificar cada número.* Não há nada para aprender. Abra o chat, digite o que você quer saber e siga os links que ele retorna: @@ -36,7 +36,7 @@ Quando uma resposta vale a pena guardar, peça ao assistente para salvá-la. Ele Na página de **Queries**, ele vai além e assume o papel de autor de SQL: descreva a consulta que você quer ("mostrar taxa de erros por agente nos últimos 7 dias") e ele transmite o SQL diretamente para o editor, abrindo uma visualização de diff para que você possa **Aceitar** ou **Rejeitar** a alteração antes que ela seja aplicada. -![A página de Queries do Observability e seu editor de SQL](/agenteye/images/query-lab.png) +![A página de Queries do FailproofAI Cloud e seu editor de SQL](/cloud/images/query-lab.png) *A página de Queries: é neste editor que o assistente transmite um rascunho de consulta, somente leitura, para você aceitar ou rejeitar.* Criar SQL por meio de perguntas aqui usa a permissão `queries:run`, a mesma por trás do botão **Run** do editor. O chat em todos os outros lugares requer `agent:use`. @@ -49,7 +49,7 @@ Você pode abrir o assistente para todos sem se preocupar com o que ele pode ace - **Toda escrita aguarda sua aprovação.** Consultas salvas e dashboards só são criados após seu clique explícito em Aprovar, e não há nenhuma configuração que desative essa barreira. - **Ele nunca pode excluir nada.** Nenhuma ferramenta de exclusão está exposta e o assistente não possui permissão de exclusão. As exclusões permanecem em suas mãos, no dashboard. - **Ele fica dentro da sua organização.** O assistente só visualiza a organização que você está acessando no momento. -- **Suas perguntas são suas.** Prompts e respostas ficam armazenados no seu próprio banco de dados do Observability; a análise de produto registra apenas metadados de uso, nunca o texto dos seus prompts. +- **Suas perguntas são suas.** Prompts e respostas ficam armazenados no seu próprio banco de dados do FailproofAI Cloud; a análise de produto registra apenas metadados de uso, nunca o texto dos seus prompts. ## Onde encontrá-lo @@ -57,7 +57,7 @@ O assistente acompanha a borda direita de cada página dentro da sua organizaç ## Relacionados -- [CLI e agentes](/pt-br/agenteye/cli-and-agents) -- [Queries](/pt-br/agenteye/queries) -- [Dashboards](/pt-br/agenteye/dashboards) -- [Suite de avaliação](/pt-br/agenteye/evaluation-suite) \ No newline at end of file +- [CLI e agentes](/pt-br/cloud/cli) +- [Queries](/pt-br/cloud/queries) +- [Dashboards](/pt-br/cloud/dashboards) +- [Suite de avaliação](/pt-br/cloud/evaluators) \ No newline at end of file diff --git a/docs/pt-br/agenteye/audits.mdx b/docs/pt-br/cloud/audits.mdx similarity index 81% rename from docs/pt-br/agenteye/audits.mdx rename to docs/pt-br/cloud/audits.mdx index abf2cecb..5228c74f 100644 --- a/docs/pt-br/agenteye/audits.mdx +++ b/docs/pt-br/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "Auditorias: seu analista de confiabilidade automático" -description: "O Failproof AI Observability vai atrás das falhas para as quais você nunca criou uma regra e entrega uma lista de tarefas priorizadas, com evidências, mostrando exatamente o que corrigir." +description: "O FailproofAI Cloud vai atrás das falhas para as quais você nunca criou uma regra e entrega uma lista de tarefas priorizadas, com evidências, mostrando exatamente o que corrigir." --- -O Failproof AI Observability vai atrás das falhas para as quais você nunca criou uma regra e entrega uma lista de tarefas priorizadas, com evidências, mostrando exatamente o que corrigir. É como ter um analista vasculhando seus logs toda noite e deixando o resumo na sua mesa de manhã. +O FailproofAI Cloud vai atrás das falhas para as quais você nunca criou uma regra e entrega uma lista de tarefas priorizadas, com evidências, mostrando exatamente o que corrigir. É como ter um analista vasculhando seus logs toda noite e deixando o resumo na sua mesa de manhã.
    @@ -12,7 +12,7 @@ O Failproof AI Observability vai atrás das falhas para as quais você nunca cri *Um tour de dois minutos: de uma execução agendada a uma correção que você pode tomar como ação.* -![A página de Auditorias: jobs recorrentes que analisam suas sessões em busca de padrões de falha, cada um com um cronograma e sensibilidade](/agenteye/images/audits.png) +![A página de Auditorias: jobs recorrentes que analisam suas sessões em busca de padrões de falha, cada um com um cronograma e sensibilidade](/cloud/images/audits.png) *Cada auditoria é um job recorrente que minera suas sessões e gera recomendações priorizadas com base em evidências.* ## Pare de adivinhar o que corrigir a seguir @@ -48,7 +48,7 @@ As auditorias ficam no dashboard em **`//audits`** (barra lateral em * ## Relacionados -- [Alertas](/pt-br/agenteye/alerts): seja notificado no momento em que um limite que você já conhece for ultrapassado. -- [Avaliações](/pt-br/agenteye/evaluations): pontue cada execução para que regressões de qualidade apareçam por conta própria. -- [Rastreamento de erros](/pt-br/agenteye/error-tracking): agrupe e acompanhe os erros que seus agentes lançam. -- [Incidentes](/pt-br/agenteye/incidents): acompanhe um problema encontrado por uma auditoria até a sua correção. \ No newline at end of file +- [Alertas](/pt-br/cloud/alerts): seja notificado no momento em que um limite que você já conhece for ultrapassado. +- [Avaliações](/pt-br/cloud/evaluations): pontue cada execução para que regressões de qualidade apareçam por conta própria. +- [Rastreamento de erros](/pt-br/cloud/errors): agrupe e acompanhe os erros que seus agentes lançam. +- [Incidentes](/pt-br/cloud/incidents): acompanhe um problema encontrado por uma auditoria até a sua correção. \ No newline at end of file diff --git a/docs/pt-br/cloud/capture.mdx b/docs/pt-br/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/pt-br/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/pt-br/agenteye/cli-recipes.mdx b/docs/pt-br/cloud/cli-recipes.mdx similarity index 91% rename from docs/pt-br/agenteye/cli-recipes.mdx rename to docs/pt-br/cloud/cli-recipes.mdx index 6c1d2280..74d2c55c 100644 --- a/docs/pt-br/agenteye/cli-recipes.mdx +++ b/docs/pt-br/cloud/cli-recipes.mdx @@ -4,9 +4,9 @@ description: "Padrões de consulta prontos para copiar e receitas jq que transfo --- -Extraia dados de sessão, evento e avaliação (e dispare reavaliações) diretamente de um script ou agente de codificação, com JSON limpo no stdout que pode ser redirecionado diretamente para `jq`. Essas receitas transformam os dados da Failproof AI Observability em algo que um usuário de terminal ou um agente de codificação com IA (Claude Code, Cursor) pode consultar e automatizar, sem precisar clicar no dashboard. +Extraia dados de sessão, evento e avaliação (e dispare reavaliações) diretamente de um script ou agente de codificação, com JSON limpo no stdout que pode ser redirecionado diretamente para `jq`. Essas receitas transformam os dados da FailproofAI Cloud em algo que um usuário de terminal ou um agente de codificação com IA (Claude Code, Cursor) pode consultar e automatizar, sem precisar clicar no dashboard. -Os padrões abaixo estão prontos para copiar e usar com a CLI da Failproof AI Observability (`agenteye`). Para instalação, autenticação e a lista completa de opções, consulte [CLI](/pt-br/agenteye/cli); execute `agenteye -h` ou `agenteye -h` para a ajuda integrada. +Os padrões abaixo estão prontos para copiar e usar com a CLI da FailproofAI Cloud (`agenteye`). Para instalação, autenticação e a lista completa de opções, consulte [CLI](/pt-br/cloud/cli); execute `agenteye -h` ou `agenteye -h` para a ajuda integrada. ## Regras de ouro @@ -173,7 +173,7 @@ O `--fields` de cada comando aceita exatamente os nomes de campos do seu própri ## Próximos passos -- [CLI](/pt-br/agenteye/cli): instalação, autenticação e a referência completa de opções para cada comando. -- [Skill de agente CLI](/pt-br/agenteye/cli-skill): empacote essas receitas como uma skill que seu agente de codificação pode carregar. -- [Chaves de API](/pt-br/agenteye/api-keys): crie e delimite as chaves com as quais a CLI, o SDK e o coletor se autenticam. -- [Python SDK](/pt-br/agenteye/python-sdk): envie eventos para a Failproof AI Observability para que haja dados que essas receitas possam consultar. \ No newline at end of file +- [CLI](/pt-br/cloud/cli): instalação, autenticação e a referência completa de opções para cada comando. +- [Skill de agente CLI](/pt-br/cloud/agent-skills): empacote essas receitas como uma skill que seu agente de codificação pode carregar. +- [Chaves de API](/pt-br/cloud/access): crie e delimite as chaves com as quais a CLI, o SDK e o coletor se autenticam. +- [Python SDK](/pt-br/cloud/sdk): envie eventos para a FailproofAI Cloud para que haja dados que essas receitas possam consultar. \ No newline at end of file diff --git a/docs/pt-br/agenteye/cli.mdx b/docs/pt-br/cloud/cli.mdx similarity index 95% rename from docs/pt-br/agenteye/cli.mdx rename to docs/pt-br/cloud/cli.mdx index 9e17758b..f9081214 100644 --- a/docs/pt-br/agenteye/cli.mdx +++ b/docs/pt-br/cloud/cli.mdx @@ -91,7 +91,7 @@ agenteye logout # revoga a sessão e limpa o token armazenado `whoami` nunca retorna erro por sessão ausente ou expirada; em vez disso, retorna `logged_in: false`, para que um script ou agente possa verificar o estado de autenticação com segurança (ainda pode sair com código diferente de zero se nenhuma URL base estiver definida ou se o dashboard estiver inacessível). -**Requisitos:** seu e-mail deve ter permissão para acessar o dashboard (solicite ao administrador do Failproof AI Observability), e o dashboard deve estar acessível na sua URL base (consulte [Configuração](#configuration)). Se você solicitar um código e ele não chegar, provavelmente seu e-mail ainda não está habilitado para acesso ao dashboard. +**Requisitos:** seu e-mail deve ter permissão para acessar o dashboard (solicite ao administrador do FailproofAI Cloud), e o dashboard deve estar acessível na sua URL base (consulte [Configuração](#configuration)). Se você solicitar um código e ele não chegar, provavelmente seu e-mail ainda não está habilitado para acesso ao dashboard. --- @@ -338,13 +338,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | Autenticado, mas sua conta não tem a permissão necessária (a mensagem a identifica) | | 6 | O recurso solicitado não foi encontrado (ex.: sessão ou id de incidente desconhecido) | -Esses códigos tornam o CLI seguro para scripts: um agente de codificação pode ramificar em um `4` para solicitar reautenticação, ou em um `5` para expor a permissão ausente. Consulte [Receitas de CLI para agentes](/pt-br/agenteye/cli-recipes) para padrões de tratamento de códigos de saída e formatos de saída JSON. +Esses códigos tornam o CLI seguro para scripts: um agente de codificação pode ramificar em um `4` para solicitar reautenticação, ou em um `5` para expor a permissão ausente. Consulte [Receitas de CLI para agentes](/pt-br/cloud/cli-recipes) para padrões de tratamento de códigos de saída e formatos de saída JSON. --- ## Próximos passos -- **[Receitas de CLI para agentes](/pt-br/agenteye/cli-recipes)**: padrões de consulta prontos para uso, one-liners com `jq`, projeções com `--fields`, tratamento de códigos de saída e formatos de saída JSON, escritos para agentes de codificação que operam o CLI. -- **[Skill de agente CLI](/pt-br/agenteye/cli-skill)**: empacote este CLI como uma *skill* instalável para Claude Code / Codex, permitindo que um agente de codificação opere o Failproof AI Observability com solicitações em linguagem natural. -- **[Chaves de API](/pt-br/agenteye/api-keys)**: o modelo de permissões por trás de `keys create --add …`. -- **[Assistente de IA](/pt-br/agenteye/assistant)**: habilitando o assistente que `agent ask` utiliza. \ No newline at end of file +- **[Receitas de CLI para agentes](/pt-br/cloud/cli-recipes)**: padrões de consulta prontos para uso, one-liners com `jq`, projeções com `--fields`, tratamento de códigos de saída e formatos de saída JSON, escritos para agentes de codificação que operam o CLI. +- **[Skill de agente CLI](/pt-br/cloud/agent-skills)**: empacote este CLI como uma *skill* instalável para Claude Code / Codex, permitindo que um agente de codificação opere o FailproofAI Cloud com solicitações em linguagem natural. +- **[Chaves de API](/pt-br/cloud/access)**: o modelo de permissões por trás de `keys create --add …`. +- **[Assistente de IA](/pt-br/cloud/assistant)**: habilitando o assistente que `agent ask` utiliza. \ No newline at end of file diff --git a/docs/pt-br/cloud/connect.mdx b/docs/pt-br/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/pt-br/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/pt-br/agenteye/dashboards.mdx b/docs/pt-br/cloud/dashboards.mdx similarity index 68% rename from docs/pt-br/agenteye/dashboards.mdx rename to docs/pt-br/cloud/dashboards.mdx index 72b6704f..21f7f2af 100644 --- a/docs/pt-br/agenteye/dashboards.mdx +++ b/docs/pt-br/cloud/dashboards.mdx @@ -5,7 +5,7 @@ description: "Transforme os dados ao vivo do seu agente em uma visão compartilh Transforme os dados ao vivo do seu agente em uma visão compartilhada que toda a equipe acompanha. Fixe as consultas mais importantes como gráficos e todos têm acesso aos mesmos números de forma imediata, sem precisar executar uma única consulta novamente. -![Um dashboard construído a partir de consultas salvas: uma linha de eventos por hora, uma barra de erros por tipo, um gráfico de área de latência e tokens por modelo](/agenteye/images/dashboard-fleet.png) +![Um dashboard construído a partir de consultas salvas: uma linha de eventos por hora, uma barra de erros por tipo, um gráfico de área de latência e tokens por modelo](/cloud/images/dashboard-fleet.png) *Um painel, quatro consultas salvas: eventos por hora, erros por tipo, latência e tokens por modelo.* @@ -24,15 +24,15 @@ Você encontrará seus painéis em `//dashboards`. ## Fixe as consultas que você já salvou -Cada tile começa como uma consulta salva. Crie e salve a consulta desejada na biblioteca de [Queries](/pt-br/agenteye/queries) (presets integrados mais os seus próprios, sobre seus eventos e avaliações) e, em seguida, fixe-a em um dashboard como o gráfico que melhor representa os dados: uma **linha** para tendências ao longo do tempo, uma **barra** para comparar categorias, uma **área** para volume ou um **pizza** para mostrar distribuição percentual. +Cada tile começa como uma consulta salva. Crie e salve a consulta desejada na biblioteca de [Queries](/pt-br/cloud/queries) (presets integrados mais os seus próprios, sobre seus eventos e avaliações) e, em seguida, fixe-a em um dashboard como o gráfico que melhor representa os dados: uma **linha** para tendências ao longo do tempo, uma **barra** para comparar categorias, uma **área** para volume ou um **pizza** para mostrar distribuição percentual. Como um tile é apenas sua consulta salva renderizada como gráfico, não há nada para sincronizar manualmente. Atualize a consulta uma vez e todos os dashboards que a utilizam são atualizados automaticamente. ## Monitore qualidade, não apenas volume -Volume indica que os agentes estão ocupados. Qualidade indica que eles estão realmente fazendo o trabalho. Aponte um dashboard para suas [pontuações de avaliação](/pt-br/agenteye/evaluations) e você terá um painel que acompanha o desempenho das execuções ao longo do tempo — assim, uma regressão de qualidade aparece como uma queda no gráfico, e não como uma surpresa vinda de um cliente. +Volume indica que os agentes estão ocupados. Qualidade indica que eles estão realmente fazendo o trabalho. Aponte um dashboard para suas [pontuações de avaliação](/pt-br/cloud/evaluations) e você terá um painel que acompanha o desempenho das execuções ao longo do tempo — assim, uma regressão de qualidade aparece como uma queda no gráfico, e não como uma surpresa vinda de um cliente. -![Um dashboard focado em qualidade construído a partir de consultas de avaliação salvas](/agenteye/images/dashboard-quality.png) +![Um dashboard focado em qualidade construído a partir de consultas de avaliação salvas](/cloud/images/dashboard-quality.png) *Um painel de qualidade mantém suas pontuações de avaliação em destaque, lado a lado com os números operacionais.* @@ -40,6 +40,6 @@ Mantenha um painel de operações e um painel de qualidade lado a lado e sua equ ## Relacionados -- [Queries](/pt-br/agenteye/queries): crie e salve as consultas que se tornarão seus tiles. -- [Evaluations](/pt-br/agenteye/evaluations): pontue suas execuções para poder visualizar a qualidade ao longo do tempo. -- [Alerts](/pt-br/agenteye/alerts): transforme um limite em qualquer uma dessas métricas em um alerta. \ No newline at end of file +- [Queries](/pt-br/cloud/queries): crie e salve as consultas que se tornarão seus tiles. +- [Evaluations](/pt-br/cloud/evaluations): pontue suas execuções para poder visualizar a qualidade ao longo do tempo. +- [Alerts](/pt-br/cloud/alerts): transforme um limite em qualquer uma dessas métricas em um alerta. \ No newline at end of file diff --git a/docs/pt-br/agenteye/error-tracking.mdx b/docs/pt-br/cloud/errors.mdx similarity index 65% rename from docs/pt-br/agenteye/error-tracking.mdx rename to docs/pt-br/cloud/errors.mdx index aa61661d..66cc672f 100644 --- a/docs/pt-br/agenteye/error-tracking.mdx +++ b/docs/pt-br/cloud/errors.mdx @@ -5,14 +5,14 @@ description: "Veja todas as falhas dos seus agentes em um único lugar, agrupada Veja todas as falhas dos seus agentes em um único lugar, agrupadas para que uma enxurrada de erros apareça como um único problema. Você tem um caminho de um clique entre "algo está vermelho" e a execução exata que quebrou, sem precisar rolar um feed ao vivo para encontrá-la. -![A página de Erros: um histograma de falhas ao longo do tempo acima de linhas de erros vermelhas agrupadas, cada uma com um botão "+ alerta" de um clique](/agenteye/images/errors.png) +![A página de Erros: um histograma de falhas ao longo do tempo acima de linhas de erros vermelhas agrupadas, cada uma com um botão "+ alerta" de um clique](/cloud/images/errors.png) *A página de Erros: um histograma de falhas ao longo do tempo, com falhas repetidas agrupadas em uma única linha por incidente.* ## Todas as falhas, já coletadas para você Quando um agente quebra, você não deveria precisar rolar um stream de eventos ao vivo esperando capturar as linhas vermelhas antes que desapareçam. A página **Errors** faz a coleta por você. Ela reúne tudo o que o dashboard pintaria de vermelho em uma única superfície de triagem, para que a primeira coisa que você veja seja o que está falhando, não onde procurar. -E ela captura mais do que as falhas óbvias. Além dos eventos explícitos de `error`, o Failproof AI Observability também exibe as falhas silenciosas: qualquer `tool_result`, `hook_completed` ou `agent_end` cujo payload indique uma falha aparece aqui. Uma ferramenta que retornou um erro ou um hook que terminou com problema não passa mais despercebido só porque nenhuma exceção barulhenta foi lançada. +E ela captura mais do que as falhas óbvias. Além dos eventos explícitos de `error`, o FailproofAI Cloud também exibe as falhas silenciosas: qualquer `tool_result`, `hook_completed` ou `agent_end` cujo payload indique uma falha aparece aqui. Uma ferramenta que retornou um erro ou um hook que terminou com problema não passa mais despercebido só porque nenhuma exceção barulhenta foi lançada. No topo, um histograma plota os erros ao longo do tempo. Uma olhada já diz se é um gotejamento constante de fundo ou um pico que começou há alguns minutos, para que você saiba imediatamente se deve largar o que está fazendo. @@ -22,19 +22,19 @@ Como toda superfície de observação, a página de Errors é limitada à sua or Uma única dependência quebrada pode disparar o mesmo erro centenas de vezes por minuto. Sem tratamento, isso é uma parede de linhas quase idênticas que enterra exatamente o que você precisa ver. -O Failproof AI Observability agrupa falhas repetidas que compartilham a mesma sessão e tipo de erro em uma única linha. Uma enxurrada aparece como um único incidente. Você acaba contando problemas, não linhas de log, e o sinal que importa permanece no topo em vez de ser afogado pelo seu próprio volume. +O FailproofAI Cloud agrupa falhas repetidas que compartilham a mesma sessão e tipo de erro em uma única linha. Uma enxurrada aparece como um único incidente. Você acaba contando problemas, não linhas de log, e o sinal que importa permanece no topo em vez de ser afogado pelo seu próprio volume. ## De "algo está vermelho" ao evento exato Clique em qualquer linha para ir direto para a sessão daquela execução, posicionado no evento exato que falhou. Sem copiar IDs de sessão, sem rolar para encontrar o momento em que deu errado: você chega direto nele, com o gráfico de execução completo a uma olhada de distância para ver o que o agente fez nos momentos antes de quebrar. -Se você tiver `alerts:write`, cada linha também traz um botão **+ alert**. Clique nele e o Observability abre uma nova regra de alerta já preenchida para capturar essa mesma falha novamente. O incidente que você acabou de triar se torna o que vai te notificar na próxima vez, em vez de te surpreender duas vezes. +Se você tiver `alerts:write`, cada linha também traz um botão **+ alert**. Clique nele e o FailproofAI Cloud abre uma nova regra de alerta já preenchida para capturar essa mesma falha novamente. O incidente que você acabou de triar se torna o que vai te notificar na próxima vez, em vez de te surpreender duas vezes. **Onde encontrar:** a página **Errors** fica na seção de observação do dashboard, em `//errors`. ## Relacionados -- [Alertas](/pt-br/agenteye/alerts): transforme qualquer falha em uma regra de notificação. -- [Incidentes](/pt-br/agenteye/incidents): acompanhe um alerta ativo do início à resolução. -- [Sessões](/pt-br/agenteye/sessions): abra a execução completa por trás de qualquer erro. -- [Auditorias](/pt-br/agenteye/audits): deixe o Observability encontrar padrões de falha nas suas execuções para você. \ No newline at end of file +- [Alertas](/pt-br/cloud/alerts): transforme qualquer falha em uma regra de notificação. +- [Incidentes](/pt-br/cloud/incidents): acompanhe um alerta ativo do início à resolução. +- [Sessões](/pt-br/cloud/sessions): abra a execução completa por trás de qualquer erro. +- [Auditorias](/pt-br/cloud/audits): deixe o FailproofAI Cloud encontrar padrões de falha nas suas execuções para você. \ No newline at end of file diff --git a/docs/pt-br/agenteye/evaluations.mdx b/docs/pt-br/cloud/evaluations.mdx similarity index 83% rename from docs/pt-br/agenteye/evaluations.mdx rename to docs/pt-br/cloud/evaluations.mdx index 0823e4a7..70542d67 100644 --- a/docs/pt-br/agenteye/evaluations.mdx +++ b/docs/pt-br/cloud/evaluations.mdx @@ -6,7 +6,7 @@ description: "Problemas de qualidade chegam até você antes de virar reclamaç Problemas de qualidade chegam até você antes de virar reclamação de usuário. Conecte seu próprio serviço de pontuação uma única vez e a Observabilidade do Failproof AI avalia automaticamente cada execução concluída — assim, uma queda na utilidade ou um pico de alucinações aparece sozinho, antes que qualquer cliente sinta. -![A grade de Sessões com uma coluna de pontuação: cada execução exibe um indicador de status de avaliação e badges com código de cores para utilidade, factualidade e eficiência de ferramentas](/agenteye/images/sessions-list.png) +![A grade de Sessões com uma coluna de pontuação: cada execução exibe um indicador de status de avaliação e badges com código de cores para utilidade, factualidade e eficiência de ferramentas](/cloud/images/sessions-list.png) *Cada execução na grade de sessões carrega suas pontuações; badges vermelhos, âmbar e verdes destacam as execuções problemáticas sem que você precise abrir uma única transcrição.* @@ -20,7 +20,7 @@ As pontuações aparecem na grade de sessões em **`//sessions`** (bar Um número te diz que uma execução foi fraca; a página da sessão te diz o porquê. Abra qualquer execução e o painel lateral começa com o resumo geral, depois exibe uma barra por dimensão com o próprio raciocínio do avaliador abaixo de cada uma — assim você vai de "essa execução tirou 0,4 em factualidade" até a afirmação exata que deu errado, em segundos. -![O painel lateral de uma sessão: o resumo da avaliação no topo, depois barras de pontuação por dimensão cada uma com uma linha de raciocínio, ao lado da linha do tempo completa de eventos](/agenteye/images/session-detail.png) +![O painel lateral de uma sessão: o resumo da avaliação no topo, depois barras de pontuação por dimensão cada uma com uma linha de raciocínio, ao lado da linha do tempo completa de eventos](/cloud/images/session-detail.png) *A visualização de detalhe da sessão: resumo, barras de pontuação por dimensão e o raciocínio por trás de cada pontuação, bem ao lado da linha do tempo de eventos da execução.* @@ -30,7 +30,7 @@ Implantou um avaliador mais preciso, ou está olhando para uma execução que tr Uma execução com pontuação baixa é ruído; uma coorte inteira caindo é um sinal. Dashboards salvos transformam suas pontuações em uma tendência que você pode acompanhar de relance: média de utilidade desta semana versus a semana passada, por agente, por ambiente. -![Um dashboard de qualidade: barras de pontuação média por dimensão do avaliador ao lado de uma tendência ao longo do tempo](/agenteye/images/dashboard-quality.png) +![Um dashboard de qualidade: barras de pontuação média por dimensão do avaliador ao lado de uma tendência ao longo do tempo](/cloud/images/dashboard-quality.png) *Um dashboard de qualidade salvo mostra a tendência das chaves de pontuação que você destaca, tornando uma deriva lenta óbvia muito antes de se tornar um incidente.* @@ -40,12 +40,12 @@ Os dashboards ficam em **`//dashboards`** (barra lateral → *analyze* A pontuação é opt-in e fica completamente desativada até que você aponte a Observabilidade do Failproof AI para um avaliador. Você sobe um pequeno serviço HTTP (a Observabilidade inclui uma referência funcional que você pode copiar), define dois valores no seu servidor, e a partir daí toda execução é pontuada automaticamente. O guia completo, o contrato de pontuação e o SDK estão no guia detalhado. -Não tem certeza de quais dimensões valem a pena pontuar? A [habilidade de agente avaliador](/pt-br/agenteye/evaluator-skill) faz com que seu agente de código descubra isso com base nas suas próprias sessões, depois cria e implanta o serviço. +Não tem certeza de quais dimensões valem a pena pontuar? A [habilidade de agente avaliador](/pt-br/cloud/agent-skills) faz com que seu agente de código descubra isso com base nas suas próprias sessões, depois cria e implanta o serviço. ## Relacionados -- [Suite de avaliação](/pt-br/agenteye/evaluation-suite): conecte seu avaliador, o contrato de pontuação e o SDK. -- [Habilidade de agente avaliador](/pt-br/agenteye/evaluator-skill): deixe um agente de código escolher suas dimensões de pontuação e construir o avaliador. -- [Sessões](/pt-br/agenteye/sessions): a grade execução por execução onde as pontuações aparecem. -- [Dashboards](/pt-br/agenteye/dashboards): salve e compartilhe tendências de qualidade em toda a sua organização. -- [Auditorias](/pt-br/agenteye/audits): outro recurso automático de qualidade da Observabilidade, para investigações entre sessões. \ No newline at end of file +- [Suite de avaliação](/pt-br/cloud/evaluators): conecte seu avaliador, o contrato de pontuação e o SDK. +- [Habilidade de agente avaliador](/pt-br/cloud/agent-skills): deixe um agente de código escolher suas dimensões de pontuação e construir o avaliador. +- [Sessões](/pt-br/cloud/sessions): a grade execução por execução onde as pontuações aparecem. +- [Dashboards](/pt-br/cloud/dashboards): salve e compartilhe tendências de qualidade em toda a sua organização. +- [Auditorias](/pt-br/cloud/audits): outro recurso automático de qualidade da Observabilidade, para investigações entre sessões. \ No newline at end of file diff --git a/docs/pt-br/agenteye/evaluation-suite.mdx b/docs/pt-br/cloud/evaluators.mdx similarity index 89% rename from docs/pt-br/agenteye/evaluation-suite.mdx rename to docs/pt-br/cloud/evaluators.mdx index 20517a6c..f4673db8 100644 --- a/docs/pt-br/agenteye/evaluation-suite.mdx +++ b/docs/pt-br/cloud/evaluators.mdx @@ -1,20 +1,20 @@ --- title: "Suite de Avaliação" -description: "O Failproof AI Observability pode pontuar automaticamente cada execução de agente concluída em termos de qualidade: você fornece um pequeno serviço de pontuação e o Observability cuida do restante." +description: "O FailproofAI Cloud pode pontuar automaticamente cada execução de agente concluída em termos de qualidade: você fornece um pequeno serviço de pontuação e o FailproofAI Cloud cuida do restante." --- -O Failproof AI Observability pode pontuar automaticamente cada execução de agente concluída em termos de qualidade: você fornece um pequeno serviço de pontuação e o Observability cuida do restante. Use-o para acompanhar as dimensões que importam para você (utilidade, eficiência de ferramentas, veracidade, segurança — você escolhe), identificar regressões cedo e comparar agentes ou ambientes de forma rápida. A pontuação é opcional: o pipeline não faz nada até que você defina `EVALUATOR_ENDPOINT` no servidor. +O FailproofAI Cloud pode pontuar automaticamente cada execução de agente concluída em termos de qualidade: você fornece um pequeno serviço de pontuação e o FailproofAI Cloud cuida do restante. Use-o para acompanhar as dimensões que importam para você (utilidade, eficiência de ferramentas, veracidade, segurança — você escolhe), identificar regressões cedo e comparar agentes ou ambientes de forma rápida. A pontuação é opcional: o pipeline não faz nada até que você defina `EVALUATOR_ENDPOINT` no servidor. -> **Nota:** Você define as dimensões de pontuação. Seu avaliador pode retornar quaisquer chaves numéricas que desejar; o Observability armazena, acompanha tendências e exibe tudo o que você enviar. +> **Nota:** Você define as dimensões de pontuação. Seu avaliador pode retornar quaisquer chaves numéricas que desejar; o FailproofAI Cloud armazena, acompanha tendências e exibe tudo o que você enviar. ## Resumo -1. **Escreva um avaliador.** Suba um pequeno serviço HTTP que leia a transcrição de uma sessão e retorne pontuações. O Observability inclui um exemplo funcional que você pode copiar. Veja [Escrevendo um avaliador com o SDK](#writing-an-evaluator-with-the-sdk). -2. **Aponte o Observability para ele.** Defina `EVALUATOR_ENDPOINT` (e um `EVALUATOR_TOKEN` compartilhado) no processo do servidor. +1. **Escreva um avaliador.** Suba um pequeno serviço HTTP que leia a transcrição de uma sessão e retorne pontuações. O FailproofAI Cloud inclui um exemplo funcional que você pode copiar. Veja [Escrevendo um avaliador com o SDK](#writing-an-evaluator-with-the-sdk). +2. **Aponte o FailproofAI Cloud para ele.** Defina `EVALUATOR_ENDPOINT` (e um `EVALUATOR_TOKEN` compartilhado) no processo do servidor. 3. **Acompanhe as pontuações.** Cada sessão concluída é pontuada automaticamente; os resultados aparecem na página de detalhes da sessão, na grade de sessões e nos dashboards salvos. -![Uma visualização de detalhes da sessão com o resumo da avaliação, barras de pontuação por dimensão e texto de raciocínio no painel direito](/agenteye/images/session-detail.png) +![Uma visualização de detalhes da sessão com o resumo da avaliação, barras de pontuação por dimensão e texto de raciocínio no painel direito](/cloud/images/session-detail.png) *Após configurar um avaliador, cada execução concluída é pontuada e os resultados aparecem no painel direito da sessão: o resumo no topo, seguido pelas barras de pontuação por dimensão com o raciocínio correspondente.* @@ -24,7 +24,7 @@ O Failproof AI Observability pode pontuar automaticamente cada execução de age ```mermaid flowchart LR - ING["ingest /events
    agent_end"] --> SRV["Observability server"] + ING["ingest /events
    agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -32,26 +32,26 @@ flowchart LR SRV --> RES["evaluations
    terminal results"] ``` -Quando o SDK do Observability emite um evento `agent_end` para uma sessão, o servidor +Quando o SDK do FailproofAI Cloud emite um evento `agent_end` para uma sessão, o servidor agenda uma avaliação. Em seguida, ele envia via POST a transcrição completa de eventos para o seu serviço avaliador, que pode: - **Retornar o resultado inline** com `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`. O resultado é anexado à linha do tempo de avaliações da sessão. `reasoning` e `summary` são opcionais. -- **Adiar** com `{"status":"pending", "job_id":"abc-123"}`. O Observability então +- **Adiar** com `{"status":"pending", "job_id":"abc-123"}`. O FailproofAI Cloud então chama `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` até que seu avaliador retorne `{"status":"done", ...}` ou `{"status":"error", "error":"..."}`. O intervalo de polling é por job: uma resposta `pending` pode incluir - `next_poll_secs` para sobrescrever o valor padrão; caso contrário, o Observability usa o + `next_poll_secs` para sobrescrever o valor padrão; caso contrário, o FailproofAI Cloud usa o valor `default_poll_interval_secs` de `GET /config`; caso contrário, o servidor recorre a `EVALUATOR_POLLING_INTERVAL_SECS` (padrão: 10s). Todos os valores são limitados ao intervalo [1s, 1h]. Sessões que nunca emitem `agent_end` (por exemplo, um processo de agente que travou) também podem ser processadas: o `GET /config` do avaliador pode retornar -`{"inactivity_timeout_secs": 1800}`, e o Observability avaliará qualquer sessão +`{"inactivity_timeout_secs": 1800}`, e o FailproofAI Cloud avaliará qualquer sessão que estiver inativa por esse tempo. Defina o campo como `null` ou omita-o para desabilitar esse fallback. @@ -74,7 +74,7 @@ além de `inactivity_timeout_secs`, uma nova avaliação é enfileirada. Falhas transitórias (5xx, 429, timeouts, erros de rede) são repetidas com backoff exponencial até `EVALUATOR_MAX_ATTEMPTS`; respostas 4xx são -terminais. O Observability pode ser executado com múltiplas instâncias de servidor +terminais. O FailproofAI Cloud pode ser executado com múltiplas instâncias de servidor com escalonamento horizontal; o trabalho é particionado para que a mesma sessão nunca seja despachada duas vezes simultaneamente. @@ -85,7 +85,7 @@ despachada duas vezes simultaneamente. Todas as rotas autenticadas usam **autenticação por bearer token**. O mesmo valor deve ser configurado nos dois lados: -- Servidor do Observability: variável de ambiente `EVALUATOR_TOKEN` +- Servidor do FailproofAI Cloud: variável de ambiente `EVALUATOR_TOKEN` - Serviço avaliador: configurado da mesma forma (o SDK `agenteye-evaluator` lê `EVALUATOR_TOKEN` por convenção) @@ -169,7 +169,7 @@ Você não precisa implementar o contrato HTTP manualmente. O pacote Python `age fornece um wrapper FastAPI tipado que cuida da autenticação, roteamento e dos formatos de requisição/resposta por você. -O Failproof AI Observability também inclui um **avaliador de referência funcional** que +O FailproofAI Cloud também inclui um **avaliador de referência funcional** que pontua `helpfulness`, `tool_efficiency` e `factuality` a partir do formato da transcrição. Copie-o como ponto de partida e substitua pela sua própria lógica: um juiz LLM, um motor de regras, o que melhor se adequar ao seu padrão de qualidade. @@ -196,7 +196,7 @@ def run(req: EvalRequest) -> EvalResponse: A instância `app` roda sob qualquer servidor ASGI, portanto `uvicorn module:app` a inicializa. Para avaliadores que precisam adiar trabalho pesado, retorne `JobPending` -em vez disso e registre um handler `@app.job_lookup`; o servidor do Observability +em vez disso e registre um handler `@app.job_lookup`; o servidor do FailproofAI Cloud faz polling em `GET /evaluate/{job_id}` até que você retorne um status terminal ou o limite `EVALUATOR_MAX_POLL_DURATION_SECS` (padrão: 1 h) seja atingido. @@ -207,7 +207,7 @@ README do SDK `agenteye-evaluator`. ## Executando seu avaliador -O avaliador é **seu serviço** — o Failproof AI Observability não inclui um +O avaliador é **seu serviço** — o FailproofAI Cloud não inclui um avaliador padrão, então você o constrói e executa onde preferir. Ele roda sob qualquer servidor ASGI (por exemplo, `uvicorn my_evaluator:app`); sirva as rotas `/health`, `/config` e `/evaluate` conforme o @@ -326,7 +326,7 @@ O admin bootstrap (`ADMIN_KEY`, `ADMIN_EMAIL`) recebe todas essas permissões au status de avaliação e as pontuações de cada sessão de forma rápida. - **`/dashboards`**: visualizações salvas de saúde de avaliação (veja [Dashboards](#dashboards) abaixo). -![A grade de Sessões com pílulas de status de avaliação por sessão e emblemas de pontuação codificados por cor (helpfulness, factuality, tool_efficiency, safety, coherence)](/agenteye/images/sessions-list.png) +![A grade de Sessões com pílulas de status de avaliação por sessão e emblemas de pontuação codificados por cor (helpfulness, factuality, tool_efficiency, safety, coherence)](/cloud/images/sessions-list.png) *A grade de sessões exibe o status de avaliação e as pontuações de cada execução de forma rápida; emblemas em vermelho/âmbar/verde destacam pontuações baixas.* @@ -354,7 +354,7 @@ página de sessões pré-filtrada exatamente para aquele subconjunto. As métric no servidor sobre todo o conjunto correspondente (via `GET /evaluations/aggregate`), portanto os números são exatos em vez de amostrados. -![Um dashboard de saúde de avaliação com barras de pontuação média por dimensão do avaliador, um breakdown de ferramenta ok vs. erro, principais ferramentas e uma tendência de eventos por hora](/agenteye/images/dashboard-quality.png) +![Um dashboard de saúde de avaliação com barras de pontuação média por dimensão do avaliador, um breakdown de ferramenta ok vs. erro, principais ferramentas e uma tendência de eventos por hora](/cloud/images/dashboard-quality.png) **Permissões:** visualizar requer tanto `dashboards:read` quanto `evaluations:read`; criar e editar requer `dashboards:write`; excluir requer `dashboards:delete`. @@ -395,7 +395,7 @@ de mais tempo do que o padrão. ## Próximos passos -- [Habilidade de agente avaliador](/pt-br/agenteye/evaluator-skill): tenha um agente de código projetando suas dimensões a partir de sessões reais e construindo este serviço para você. -- [SDK Python](/pt-br/agenteye/python-sdk): emita os eventos `agent_end` que acionam a pontuação. -- [Chaves de API](/pt-br/agenteye/api-keys): as permissões `evaluations:read` e `evaluations:trigger`. -- [Auditorias](/pt-br/agenteye/audits): o outro recurso de qualidade automatizado do Observability, para revisão baseada em políticas. \ No newline at end of file +- [Habilidade de agente avaliador](/pt-br/cloud/agent-skills): tenha um agente de código projetando suas dimensões a partir de sessões reais e construindo este serviço para você. +- [SDK Python](/pt-br/cloud/sdk): emita os eventos `agent_end` que acionam a pontuação. +- [Chaves de API](/pt-br/cloud/access): as permissões `evaluations:read` e `evaluations:trigger`. +- [Auditorias](/pt-br/cloud/audits): o outro recurso de qualidade automatizado do FailproofAI Cloud, para revisão baseada em políticas. \ No newline at end of file diff --git a/docs/pt-br/agenteye/event-stream.mdx b/docs/pt-br/cloud/event-stream.mdx similarity index 85% rename from docs/pt-br/agenteye/event-stream.mdx rename to docs/pt-br/cloud/event-stream.mdx index 530a552f..0359f385 100644 --- a/docs/pt-br/agenteye/event-stream.mdx +++ b/docs/pt-br/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "No momento em que seu agente faz algo, você vê." No momento em que seu agente faz algo, você vê. O Event Stream é o seu pulso em tempo real sobre cada agente em produção: sem espera, sem vasculhar logs, sem precisar adivinhar o que acabou de acontecer. -![O Event Stream ao vivo: linhas de eventos com código de cores atualizando em tempo real, filtráveis por ambiente, agente, sessão, tipo de evento e texto livre](/agenteye/images/events-stream.png) +![O Event Stream ao vivo: linhas de eventos com código de cores atualizando em tempo real, filtráveis por ambiente, agente, sessão, tipo de evento e texto livre](/cloud/images/events-stream.png) *Todos os eventos de todos os agentes da sua organização, os mais recentes primeiro, atualizando conforme acontecem.* @@ -43,8 +43,8 @@ Esta é a fonte primária de verdade sobre a qual todas as outras superfícies d ## Relacionado -- [Sessions](/pt-br/agenteye/sessions): os mesmos eventos consolidados em uma linha por execução, com um gráfico de execução no estilo git. -- [Telemetry](/pt-br/agenteye/telemetry): o que seus agentes enviam e como os eventos chegam ao stream. -- [Error tracking](/pt-br/agenteye/error-tracking): uma única superfície de triagem para tudo que deu errado. -- [Alerts](/pt-br/agenteye/alerts): transforme qualquer limite em uma regra de notificação. -- [CLI and agents](/pt-br/agenteye/cli-and-agents): o mesmo rastro ao vivo pelo seu terminal. \ No newline at end of file +- [Sessions](/pt-br/cloud/sessions): os mesmos eventos consolidados em uma linha por execução, com um gráfico de execução no estilo git. +- [Telemetry](/pt-br/cloud/performance): o que seus agentes enviam e como os eventos chegam ao stream. +- [Error tracking](/pt-br/cloud/errors): uma única superfície de triagem para tudo que deu errado. +- [Alerts](/pt-br/cloud/alerts): transforme qualquer limite em uma regra de notificação. +- [CLI and agents](/pt-br/cloud/cli): o mesmo rastro ao vivo pelo seu terminal. \ No newline at end of file diff --git a/docs/pt-br/cloud/fleet.mdx b/docs/pt-br/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/pt-br/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/pt-br/agenteye/incidents.mdx b/docs/pt-br/cloud/incidents.mdx similarity index 84% rename from docs/pt-br/agenteye/incidents.mdx rename to docs/pt-br/cloud/incidents.mdx index a96d6085..6bf9b7f6 100644 --- a/docs/pt-br/agenteye/incidents.mdx +++ b/docs/pt-br/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "Quando um alerta dispara, todos podem ver que o incidente está ab Quando um alerta dispara, a primeira pergunta é sempre "quem está cuidando disso?" Os incidentes respondem a essa questão: no momento em que algo ultrapassa um limiar, todos podem ver que o incidente está aberto, quem é o responsável e exatamente o que aconteceu até agora, com um registro limpo e atribuído que pode ser entregue diretamente para uma análise pós-incidente. -![A caixa de entrada de Incidentes: cartões de incidentes vinculados a alertas e abertos manualmente, agrupados por estado, cada um com um badge de severidade e um responsável](/agenteye/images/incidents.png) +![A caixa de entrada de Incidentes: cartões de incidentes vinculados a alertas e abertos manualmente, agrupados por estado, cada um com um badge de severidade e um responsável](/cloud/images/incidents.png) *A caixa de entrada agrupa os incidentes abertos por estado e filtra por severidade e responsável, para que você veja o que precisa de atenção humana agora.* ## Saiba quem está cuidando, de relance @@ -17,10 +17,10 @@ Chega de "alguém está olhando para isso?" em uma thread de chat. Uma violaçã Quando o incidente termina, você já tem o relatório. Abra qualquer incidente e você terá as evidências da violação, seus responsáveis e assinantes, uma thread de comentários para coordenação no local e uma linha do tempo de atividade somente de acréscimo. -![Uma visualização detalhada de incidente: o alerta pai e o resumo da violação, responsáveis e assinantes, uma linha do tempo de atividade atribuída e uma thread de comentários](/agenteye/images/incident-detail.png) +![Uma visualização detalhada de incidente: o alerta pai e o resumo da violação, responsáveis e assinantes, uma linha do tempo de atividade atribuída e uma thread de comentários](/cloud/images/incident-detail.png) *Tudo o que aconteceu, em ordem, cada linha assinada por quem fez a ação.* -Cada ação (aberto, reconhecido, resolvido, e assim por diante) é gravada nessa linha do tempo e nunca é editada. Cada entrada é atribuída: ao operador que a executou, por e-mail, ou como **automatizado** para qualquer coisa que o Failproof AI Observability fez por conta própria, como abrir o incidente na violação. Nada é anônimo e nada se perde, então a análise pós-incidente praticamente se escreve sozinha. +Cada ação (aberto, reconhecido, resolvido, e assim por diante) é gravada nessa linha do tempo e nunca é editada. Cada entrada é atribuída: ao operador que a executou, por e-mail, ou como **automatizado** para qualquer coisa que o FailproofAI Cloud fez por conta própria, como abrir o incidente na violação. Nada é anônimo e nada se perde, então a análise pós-incidente praticamente se escreve sozinha. ## Como um incidente evolui @@ -45,6 +45,6 @@ Os incidentes estão em `//incidents`. Para visualizar, é necessária ## Relacionados -- [Alertas](/pt-br/agenteye/alerts): as regras que abrem esses incidentes quando um limiar é ultrapassado. -- [Rastreamento de erros](/pt-br/agenteye/error-tracking): veja todas as falhas em um único lugar e promova uma delas a um alerta. -- [Auditorias](/pt-br/agenteye/audits): o analista agendado que encontra as falhas que nenhuma regra estava monitorando. \ No newline at end of file +- [Alertas](/pt-br/cloud/alerts): as regras que abrem esses incidentes quando um limiar é ultrapassado. +- [Rastreamento de erros](/pt-br/cloud/errors): veja todas as falhas em um único lugar e promova uma delas a um alerta. +- [Auditorias](/pt-br/cloud/audits): o analista agendado que encontra as falhas que nenhuma regra estava monitorando. \ No newline at end of file diff --git a/docs/pt-br/cloud/managed-policies.mdx b/docs/pt-br/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/pt-br/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/pt-br/agenteye/overview.mdx b/docs/pt-br/cloud/overview.mdx similarity index 50% rename from docs/pt-br/agenteye/overview.mdx rename to docs/pt-br/cloud/overview.mdx index c121944c..a0d1831f 100644 --- a/docs/pt-br/agenteye/overview.mdx +++ b/docs/pt-br/cloud/overview.mdx @@ -1,16 +1,16 @@ --- title: "Failproof AI: Observe Agentes em Busca de Falhas" -description: "Failproof AI Observability é uma plataforma auto-hospedada para observar, avaliar e aprimorar seus agentes de IA em produção." +description: "FailproofAI Cloud é uma plataforma auto-hospedada para observar, avaliar e aprimorar seus agentes de IA em produção." --- -Failproof AI Observability é uma plataforma auto-hospedada para observar, avaliar e aprimorar seus agentes de IA em produção. Ela registra tudo o que seus agentes fazem (cada chamada de ferramenta, requisição ao modelo, hook e erro), pontua a qualidade de cada execução e expõe as falhas que você não sabia que precisava procurar — tudo em um dashboard que roda dentro da sua própria infraestrutura. +FailproofAI Cloud é uma plataforma auto-hospedada para observar, avaliar e aprimorar seus agentes de IA em produção. Ela registra tudo o que seus agentes fazem (cada chamada de ferramenta, requisição ao modelo, hook e erro), pontua a qualidade de cada execução e expõe as falhas que você não sabia que precisava procurar — tudo em um dashboard que roda dentro da sua própria infraestrutura. -Se você coloca agentes de IA em produção e está cansado de tentar adivinhar por que uma execução deu errado, este é o ponto de partida certo. Aqui você entende o que Failproof AI Observability oferece e como as peças se encaixam, antes mesmo de instalar qualquer coisa. +Se você coloca agentes de IA em produção e está cansado de tentar adivinhar por que uma execução deu errado, este é o ponto de partida certo. Aqui você entende o que FailproofAI Cloud oferece e como as peças se encaixam, antes mesmo de instalar qualquer coisa. -> **Failproof AI Observability é um produto empresarial da Failproof AI.** Quer ver em ação? Solicite uma demonstração: envie um e-mail para [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +> **FailproofAI Cloud é um produto empresarial da Failproof AI.** Quer ver em ação? Solicite uma demonstração: envie um e-mail para [nikita@befailproof.ai](mailto:nikita@befailproof.ai). -![Uma sessão do Failproof AI Observability representada como um grafo de execução no estilo git ao lado da sua linha do tempo de eventos, com um detalhamento por execução de ferramentas, modelos e hooks na coluna da direita](/agenteye/images/session-detail.png) +![Uma sessão do FailproofAI Cloud representada como um grafo de execução no estilo git ao lado da sua linha do tempo de eventos, com um detalhamento por execução de ferramentas, modelos e hooks na coluna da direita](/cloud/images/session-detail.png) *Cada execução de agente é representada como um grafo de execução no estilo git (esquerda) ao lado da sua linha do tempo de eventos. Sub-agentes paralelos recebem sua própria faixa; a coluna da direita detalha as ferramentas, modelos, hooks e consumo de tokens da execução.* @@ -30,52 +30,52 @@ Dois vídeos curtos mostram as duas coisas que as equipes buscam primeiro: rastr
    -*Failproof Audit: deixe o Failproof AI Observability minerar seus logs entre sessões e identificar o que precisa ser corrigido.* +*Failproof Audit: deixe o FailproofAI Cloud minerar seus logs entre sessões e identificar o que precisa ser corrigido.* --- ## Por que as equipes usam - **Veja o que seu agente realmente fez.** Cada execução se torna um grafo de execução legível no estilo git: quais ferramentas rodaram em paralelo, quais sub-agentes se ramificaram, onde travou e quanto consumiu. -- **Detecte regressões de qualidade automaticamente.** Conecte um pequeno serviço de pontuação e o Failproof AI Observability pontua cada execução concluída — uma queda na utilidade ou um pico de alucinações aparece por conta própria. +- **Detecte regressões de qualidade automaticamente.** Conecte um pequeno serviço de pontuação e o FailproofAI Cloud pontua cada execução concluída — uma queda na utilidade ou um pico de alucinações aparece por conta própria. - **Encontre falhas para as quais você não escreveu uma regra.** Auditorias recorrentes mineram seus logs entre sessões em busca de clusters de erros, outliers de latência, pontuações baixas e execuções travadas, entregando descobertas classificadas e fundamentadas em evidências. - **Seja alertado quando importa.** Regras de threshold disparam sobre taxa de erro, latência, custo ou pontuações de avaliadores e abrem incidentes que você pode reconhecer, atribuir e resolver. - **Faça perguntas em linguagem natural.** Um assistente de IA integrado ao dashboard responde perguntas como "como está a qualidade em prod esta semana?" sobre seus próprios dados. Qualquer alteração que ele faça requer aprovação. -- **Mantenha seus dados.** Failproof AI Observability é auto-hospedado: eventos, prompts e análises ficam na infraestrutura que você controla. +- **Mantenha seus dados.** FailproofAI Cloud é auto-hospedado: eventos, prompts e análises ficam na infraestrutura que você controla. --- ## O que você recebe -Failproof AI Observability é organizado em torno de três ideias (**observe**, **analyze** e **admin**), refletidas na barra lateral esquerda do dashboard. +FailproofAI Cloud é organizado em torno de três ideias (**observe**, **analyze** e **admin**), refletidas na barra lateral esquerda do dashboard. **Observe** (a realidade bruta do que aconteceu): -- **[Event stream](/pt-br/agenteye/event-stream)**: o rastro em tempo real, passo a passo, de cada execução (chamadas de ferramentas, chamadas ao modelo, hooks, erros). -- **[Sessions](/pt-br/agenteye/sessions)**: esses eventos consolidados em uma linha por execução, cada uma pronta para ser pontuada, com um grafo de execução no estilo git. -- **[Performance metrics](/pt-br/agenteye/telemetry)**: mapas de calor de latência por superfície e métricas p50/p95/p99 para modelos, ferramentas e hooks, de modo que um pico na cauda se destaque da mediana. -- **[Error tracking](/pt-br/agenteye/error-tracking)**: uma superfície de triagem unificada para tudo que deu errado, a um clique de um alerta disparado. +- **[Event stream](/pt-br/cloud/event-stream)**: o rastro em tempo real, passo a passo, de cada execução (chamadas de ferramentas, chamadas ao modelo, hooks, erros). +- **[Sessions](/pt-br/cloud/sessions)**: esses eventos consolidados em uma linha por execução, cada uma pronta para ser pontuada, com um grafo de execução no estilo git. +- **[Performance metrics](/pt-br/cloud/performance)**: mapas de calor de latência por superfície e métricas p50/p95/p99 para modelos, ferramentas e hooks, de modo que um pico na cauda se destaque da mediana. +- **[Error tracking](/pt-br/cloud/errors)**: uma superfície de triagem unificada para tudo que deu errado, a um clique de um alerta disparado. -![A página de observação de Tools: um mapa de calor de latência, uma faixa de percentil e uma barra de distribuição de ferramentas ao longo de 24 intervalos de tempo](/agenteye/images/tools.png) +![A página de observação de Tools: um mapa de calor de latência, uma faixa de percentil e uma barra de distribuição de ferramentas ao longo de 24 intervalos de tempo](/cloud/images/tools.png) *Cada superfície de observação combina um sparkline e métricas p50/p95/p99 com um mapa de calor de latência e uma faixa de percentil. Mostrado aqui: Tools.* **Analyze** (transforme atividade em respostas): -- **[Queries](/pt-br/agenteye/queries)** e **[dashboards](/pt-br/agenteye/dashboards)**: SQL salvo sobre seus eventos e avaliações, transformado em dashboards compartilhados com escopo de organização. -- **[Evaluations](/pt-br/agenteye/evaluations)**: pontuações de qualidade produzidas pelo seu próprio serviço de avaliação, com raciocínio por pontuação. -- **[Audits](/pt-br/agenteye/audits)**: investigações recorrentes que expõem padrões de falha entre sessões. -- **[Alerts](/pt-br/agenteye/alerts)** e **[incidents](/pt-br/agenteye/incidents)**: regras de threshold que alertam você, mais um fluxo de trabalho de incidentes para triagem. +- **[Queries](/pt-br/cloud/queries)** e **[dashboards](/pt-br/cloud/dashboards)**: SQL salvo sobre seus eventos e avaliações, transformado em dashboards compartilhados com escopo de organização. +- **[Evaluations](/pt-br/cloud/evaluations)**: pontuações de qualidade produzidas pelo seu próprio serviço de avaliação, com raciocínio por pontuação. +- **[Audits](/pt-br/cloud/audits)**: investigações recorrentes que expõem padrões de falha entre sessões. +- **[Alerts](/pt-br/cloud/alerts)** e **[incidents](/pt-br/cloud/incidents)**: regras de threshold que alertam você, mais um fluxo de trabalho de incidentes para triagem. **Interfaces** (acesse seus dados do seu jeito): -- **[CLI](/pt-br/agenteye/cli-and-agents)**: controle todo o seu deployment pelo terminal ou por um script, e deixe um agente de codificação fazer isso por você em linguagem natural. -- **[AI assistant](/pt-br/agenteye/assistant)**: faça perguntas sobre seus agentes em linguagem natural, diretamente no dashboard. -- **REST API**: tudo o que o dashboard e o CLI fazem é respaldado por uma REST API que você pode chamar diretamente com uma [chave de API](/pt-br/agenteye/api-keys) com escopo — ingira eventos, consulte sessões e avaliações, e gerencie dashboards, alertas, auditorias, usuários e chaves, para integrar o Failproof AI Observability às suas próprias ferramentas. +- **[CLI](/pt-br/cloud/cli)**: controle todo o seu deployment pelo terminal ou por um script, e deixe um agente de codificação fazer isso por você em linguagem natural. +- **[AI assistant](/pt-br/cloud/assistant)**: faça perguntas sobre seus agentes em linguagem natural, diretamente no dashboard. +- **REST API**: tudo o que o dashboard e o CLI fazem é respaldado por uma REST API que você pode chamar diretamente com uma [chave de API](/pt-br/cloud/access) com escopo — ingira eventos, consulte sessões e avaliações, e gerencie dashboards, alertas, auditorias, usuários e chaves, para integrar o FailproofAI Cloud às suas próprias ferramentas. **Admin** (gerencie para sua equipe): -- **[API keys](/pt-br/agenteye/api-keys)**: tokens com escopo para o coletor, o dashboard e o assistente. +- **[API keys](/pt-br/cloud/access)**: tokens com escopo para o coletor, o dashboard e o assistente. - **Users**: login sem senha, baseado em e-mail, com lista de permissões. - **Settings**: configuração por organização, incluindo substituições de janela de contexto do modelo. @@ -91,18 +91,18 @@ Os dados fluem em uma única direção, do código do seu agente até o dashboar - **Dashboard**: onde você explora tudo. - **Serviços opcionais**: um serviço de pontuação (avaliações) e um serviço de assistente de IA (o chat integrado ao dashboard). -Para o vocabulário usado ao longo da documentação (*event, session, evaluation, audit, finding, incident*), consulte [Concepts](/pt-br/agenteye/concepts). +Para o vocabulário usado ao longo da documentação (*event, session, evaluation, audit, finding, incident*), consulte [Concepts](/pt-br/concepts). --- -## Obtendo o Failproof AI Observability +## Obtendo o FailproofAI Cloud -Failproof AI Observability é um produto empresarial da Failproof AI e funciona em conjunto com o Failproof AI Enforcement — o produto de políticas e guardrails — sob a marca Failproof AI. Ele roda inteiramente no seu próprio ambiente. Se você ainda não tem acesso aos pacotes, solicite uma demonstração e entraremos em contato: envie um e-mail para [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +FailproofAI Cloud é um produto empresarial da Failproof AI e funciona em conjunto com o FailproofAI guardrails — o produto de políticas e guardrails — sob a marca Failproof AI. Ele roda inteiramente no seu próprio ambiente. Se você ainda não tem acesso aos pacotes, solicite uma demonstração e entraremos em contato: envie um e-mail para [nikita@befailproof.ai](mailto:nikita@befailproof.ai). --- ## Próximos passos -- [Concepts](/pt-br/agenteye/concepts): o vocabulário do Failproof AI Observability em um único lugar. -- [Observability](/pt-br/agenteye/observability): acompanhe o que seus agentes fazem, execução por execução. -- [Security](/pt-br/agenteye/security): como o Failproof AI Observability mantém seus dados isolados e sob seu controle. \ No newline at end of file +- [Concepts](/pt-br/concepts): o vocabulário do FailproofAI Cloud em um único lugar. +- [FailproofAI Cloud](/pt-br/cloud/overview): acompanhe o que seus agentes fazem, execução por execução. +- [Security](/pt-br/cloud/security): como o FailproofAI Cloud mantém seus dados isolados e sob seu controle. \ No newline at end of file diff --git a/docs/pt-br/agenteye/telemetry.mdx b/docs/pt-br/cloud/performance.mdx similarity index 81% rename from docs/pt-br/agenteye/telemetry.mdx rename to docs/pt-br/cloud/performance.mdx index 5cdfbf84..10cdadb9 100644 --- a/docs/pt-br/agenteye/telemetry.mdx +++ b/docs/pt-br/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "Veja no instante em que seus modelos, ferramentas ou hooks ficam l Veja no instante em que seus modelos, ferramentas ou hooks ficam lentos ou aumentam sua fatura, e detecte um pico de latência na cauda antes que seus usuários percebam. Três páginas dedicadas transformam tempos brutos em p50, p95 e p99 que você lê de relance. -![A página Models exibindo um mapa de calor de latência, uma faixa de percentis e valores de tokens, custo e janela de contexto por modelo](/agenteye/images/models.png) +![A página Models exibindo um mapa de calor de latência, uma faixa de percentis e valores de tokens, custo e janela de contexto por modelo](/cloud/images/models.png) *A página Models: um mapa de calor de latência, uma faixa de percentis e, por modelo, tokens, custo estimado e ocupação da janela de contexto.* ## Pare de deixar as médias esconderem suas piores execuções @@ -24,13 +24,13 @@ Um crosshair de hover compartilhado conecta o mapa de calor e a faixa, então um A página Models (exibida acima) responde às duas perguntas que uma fatura sempre levanta: qual modelo e quanto. Além da visão de latência compartilhada, ela adiciona **consumo de tokens por modelo**, **custo estimado** e **ocupação da janela de contexto**, para que o crescimento descontrolado de prompts e uma compactação iminente sejam visíveis antes de te surpreenderem. -O Failproof AI Observability reconhece IDs de modelos comuns automaticamente. Se uma janela parecer incorreta, ou se você rodar um modelo próprio privado, corrija ou adicione um em **Settings**, em **model context windows**, e as leituras de ocupação se atualizam. +O FailproofAI Cloud reconhece IDs de modelos comuns automaticamente. Se uma janela parecer incorreta, ou se você rodar um modelo próprio privado, corrija ou adicione um em **Settings**, em **model context windows**, e as leituras de ocupação se atualizam. ## Tools: distinga o lento do quebrado Uma chamada de ferramenta pode ser lenta ou pode estar falhando silenciosamente, e você quer saber qual é o caso em segundos, não após vasculhar logs. -![A página Tools exibindo o mapa de calor de latência e a faixa de percentis compartilhados ao lado de uma divisão de sucesso e falha e uma barra de distribuição de ferramentas](/agenteye/images/tools.png) +![A página Tools exibindo o mapa de calor de latência e a faixa de percentis compartilhados ao lado de uma divisão de sucesso e falha e uma barra de distribuição de ferramentas](/cloud/images/tools.png) *A página Tools: o mesmo mapa de calor e faixa de percentis, mais uma divisão de sucesso e falha e uma barra de distribuição de ferramentas.* Junto à visão de latência compartilhada, a página Tools adiciona uma **divisão de sucesso e falha** e uma **barra de distribuição de ferramentas**, para que você veja de relance em quais ferramentas você mais depende e quais estão consumindo seu orçamento de erros. @@ -39,14 +39,14 @@ Junto à visão de latência compartilhada, a página Tools adiciona uma **divis Quando um hook de ciclo de vida atrasa uma execução, "os hooks estão lentos" não é algo sobre o qual você pode agir. A página Hooks leva você até o que importa. -![A página Hooks exibindo a latência detalhada por nome de hook e evento de gatilho sobre o mapa de calor e a faixa de percentis compartilhados](/agenteye/images/hooks.png) +![A página Hooks exibindo a latência detalhada por nome de hook e evento de gatilho sobre o mapa de calor e a faixa de percentis compartilhados](/cloud/images/hooks.png) *A página Hooks: latência detalhada por nome de hook e evento de gatilho.* Sobre o mesmo mapa de calor de latência e faixa de percentis, a página Hooks detalha a atividade por **nome do hook** e **evento de gatilho**, para que você chegue ao único hook e ao único evento que precisam de atenção. ## Relacionados -- [Event stream](/pt-br/agenteye/event-stream): o rastro em tempo real, com código de cores, de cada evento. -- [Sessions](/pt-br/agenteye/sessions): agrupe eventos em uma linha por execução e abra seu grafo de execução. -- [Error tracking](/pt-br/agenteye/error-tracking): uma superfície de triagem unificada para tudo que o dashboard pinta de vermelho. -- [Dashboards](/pt-br/agenteye/dashboards): visões consolidadas de toda a sua frota. \ No newline at end of file +- [Event stream](/pt-br/cloud/event-stream): o rastro em tempo real, com código de cores, de cada evento. +- [Sessions](/pt-br/cloud/sessions): agrupe eventos em uma linha por execução e abra seu grafo de execução. +- [Error tracking](/pt-br/cloud/errors): uma superfície de triagem unificada para tudo que o dashboard pinta de vermelho. +- [Dashboards](/pt-br/cloud/dashboards): visões consolidadas de toda a sua frota. \ No newline at end of file diff --git a/docs/pt-br/agenteye/queries.mdx b/docs/pt-br/cloud/queries.mdx similarity index 84% rename from docs/pt-br/agenteye/queries.mdx rename to docs/pt-br/cloud/queries.mdx index e69dbbda..ca55e2ce 100644 --- a/docs/pt-br/agenteye/queries.mdx +++ b/docs/pt-br/cloud/queries.mdx @@ -6,7 +6,7 @@ description: "Faça qualquer pergunta sobre os dados do seu agente e obtenha uma Faça qualquer pergunta sobre os dados do seu agente e obtenha uma resposta em segundos. A Observabilidade do Failproof AI oferece uma biblioteca de consultas salvas, prontas para execução, sobre seus eventos e avaliações — assim você começa a partir de um exemplo funcional em vez de um editor SQL em branco. -![A biblioteca de consultas salvas: uma grade de consultas reutilizáveis, incluindo predefinições integradas e personalizadas](/agenteye/images/queries.png) +![A biblioteca de consultas salvas: uma grade de consultas reutilizáveis, incluindo predefinições integradas e personalizadas](/cloud/images/queries.png) *Sua biblioteca de consultas salvas em `//queries`: predefinições integradas ao lado das consultas que sua equipe salvou.* @@ -22,7 +22,7 @@ Acesse em `//queries`. Abra qualquer consulta e ela será carregada no compositor SQL, onde você pode ajustá-la e ver a resposta imediatamente: sem exportação, sem idas e vindas, sem esperar por outra pessoa. -![O compositor de consultas SQL executando uma consulta salva, com uma barra lateral de esquema e uma grade de resultados ao vivo](/agenteye/images/query-lab.png) +![O compositor de consultas SQL executando uma consulta salva, com uma barra lateral de esquema e uma grade de resultados ao vivo](/cloud/images/query-lab.png) *O compositor SQL: sua consulta à esquerda, uma barra lateral de esquema para que você nunca precise adivinhar o nome de uma coluna, e uma grade de resultados ao vivo abaixo.* @@ -43,14 +43,14 @@ agenteye query list # as mesmas consultas salvas, pelo seu termi agenteye query run errs --arg prod # execute uma e imprima as linhas (adicione --json para redirecionar) ``` - Consulte [CLI e agentes](/pt-br/agenteye/cli-and-agents) para o conjunto completo de comandos. + Consulte [CLI e agentes](/pt-br/cloud/cli) para o conjunto completo de comandos. -- **Pelo assistente de IA.** Não tem certeza de como formular o SQL? Pergunte ao [assistente de IA](/pt-br/agenteye/assistant) no dashboard em linguagem natural e ele rascunhará a consulta e a salvará na sua biblioteca. +- **Pelo assistente de IA.** Não tem certeza de como formular o SQL? Pergunte ao [assistente de IA](/pt-br/cloud/assistant) no dashboard em linguagem natural e ele rascunhará a consulta e a salvará na sua biblioteca. A execução de uma consulta salva é controlada pela permissão `queries:run`, separada das permissões para criar ou excluir consultas — assim você pode conceder acesso de leitura sem permitir que todos reescrevam a biblioteca. ## Relacionados -- [Dashboards](/pt-br/agenteye/dashboards): fixe resultados de consultas em gráficos compartilhados para toda a organização. -- [Assistente de IA](/pt-br/agenteye/assistant): faça perguntas em linguagem natural e obtenha uma consulta como resposta. -- [CLI e agentes](/pt-br/agenteye/cli-and-agents): execute e salve as mesmas consultas pelo seu terminal. \ No newline at end of file +- [Dashboards](/pt-br/cloud/dashboards): fixe resultados de consultas em gráficos compartilhados para toda a organização. +- [Assistente de IA](/pt-br/cloud/assistant): faça perguntas em linguagem natural e obtenha uma consulta como resposta. +- [CLI e agentes](/pt-br/cloud/cli): execute e salve as mesmas consultas pelo seu terminal. \ No newline at end of file diff --git a/docs/pt-br/agenteye/python-sdk.mdx b/docs/pt-br/cloud/sdk.mdx similarity index 96% rename from docs/pt-br/agenteye/python-sdk.mdx rename to docs/pt-br/cloud/sdk.mdx index 76bb31cf..1f0f0887 100644 --- a/docs/pt-br/agenteye/python-sdk.mdx +++ b/docs/pt-br/cloud/sdk.mdx @@ -26,7 +26,7 @@ Após a instalação, confirme que está disponível: python -c "import agenteye; print(agenteye.__version__)" ``` -Prefere deixar um agente de código fazer toda a integração? A [Python SDK Agent Skill](/pt-br/agenteye/python-sdk-skill) conhece o caminho de instalação, planeja os pontos de instrumentação, os implementa e verifica se os eventos chegam corretamente. +Prefere deixar um agente de código fazer toda a integração? A [Python SDK Agent Skill](/pt-br/cloud/agent-skills) conhece o caminho de instalação, planeja os pontos de instrumentação, os implementa e verifica se os eventos chegam corretamente. --- @@ -99,7 +99,7 @@ Envolva as chamadas de ferramenta da mesma forma com `tool_use` e `tool_result`, Veja como esses eventos aparecem no dashboard, com código de cores por tipo e filtráveis por ambiente, agente e sessão: -![O stream de Eventos ao vivo, com código de cores por tipo de evento e filtrável por ambiente, agente e sessão](/agenteye/images/events-stream.png) +![O stream de Eventos ao vivo, com código de cores por tipo de evento e filtrável por ambiente, agente e sessão](/cloud/images/events-stream.png) --- @@ -158,7 +158,7 @@ Isso torna a redação uma escolha e responsabilidade sua. Se um prompt ou paylo A maioria dos eventos vem em pares início/fim que compartilham um ID de correlação: `tool_use` e `tool_result` compartilham um `tool_call_id`, `hook_triggered` e `hook_completed` compartilham um `hook_id`, e `human_wait` e `human_input` compartilham um `input_id`. Emita o evento de início, execute o trabalho e, em seguida, emita o evento de fim com o mesmo ID. A Observabilidade do Failproof AI associa o par e calcula o `duration_ms` para você, portanto, você nunca passa `duration_ms` diretamente. -![O gráfico de execução no estilo git de uma sessão ao lado de sua linha do tempo de eventos, reconstruído a partir dos eventos pareados, com o painel de detalhamento de ferramenta/modelo/hook](/agenteye/images/session-detail.png) +![O gráfico de execução no estilo git de uma sessão ao lado de sua linha do tempo de eventos, reconstruído a partir dos eventos pareados, com o painel de detalhamento de ferramenta/modelo/hook](/cloud/images/session-detail.png) Todos os métodos de evento exigem estes dois campos: @@ -432,5 +432,5 @@ Cada arquivo é gravado atomicamente: o SDK grava em um arquivo temporário e en ## Próximos passos -- [Stream de eventos](/pt-br/agenteye/event-stream): acompanhe esses eventos chegando ao vivo, com código de cores e filtráveis por ambiente, agente e sessão. -- [Sessões](/pt-br/agenteye/sessions): veja como os eventos pareados reconstroem cada execução de agente como um gráfico de execução e uma linha do tempo. \ No newline at end of file +- [Stream de eventos](/pt-br/cloud/event-stream): acompanhe esses eventos chegando ao vivo, com código de cores e filtráveis por ambiente, agente e sessão. +- [Sessões](/pt-br/cloud/sessions): veja como os eventos pareados reconstroem cada execução de agente como um gráfico de execução e uma linha do tempo. \ No newline at end of file diff --git a/docs/pt-br/cloud/security.mdx b/docs/pt-br/cloud/security.mdx new file mode 100644 index 00000000..70932835 --- /dev/null +++ b/docs/pt-br/cloud/security.mdx @@ -0,0 +1,68 @@ +--- +title: "Segurança" +description: "O FailproofAI Cloud foi projetado para ficar próximo aos seus agentes em produção, o que significa que ele vê seus prompts, entradas de ferramentas e saídas." +--- + + +O FailproofAI Cloud foi projetado para ficar próximo aos seus agentes em produção, o que significa que ele vê seus prompts, entradas de ferramentas e saídas. Esta página explica como esses dados são mantidos isolados, controlados e nas suas mãos. Se você está avaliando o FailproofAI Cloud para uma revisão de segurança, comece por aqui. + +--- + +## Seus dados ficam no seu ambiente + +O FailproofAI Cloud é auto-hospedado. Eventos, prompts, respostas do modelo e análises são armazenados nos seus próprios bancos de dados, no seu próprio ambiente. Nada é enviado para um SaaS de terceiros para armazenamento, e seus dados permanecem na sua própria conta de nuvem. + +--- + +## Isolamento de tenant + +Uma instância do FailproofAI Cloud pode hospedar várias organizações, e cada uma é isolada na camada de armazenamento — aplicado pelo banco de dados, não apenas pela interface: + +- Os dados operacionais de uma organização (usuários, chaves, dashboards, consultas salvas) são restritos àquela org, e leituras entre organizações são bloqueadas pelo próprio banco de dados. +- Todo evento ingerido é marcado com a organização proprietária, de modo que os eventos de uma organização nunca podem ser lidos por outra. + +Cada rota de dashboard é delimitada por um slug de org (`//…`). + +--- + +## Login + +O FailproofAI Cloud utiliza login sem senha, baseado em e-mail. Não há senha para ser furtada ou vazada. Um usuário solicita um código de uso único (ou um magic link de clique único), que é enviado por e-mail e expira rapidamente. O login é controlado por uma **lista de permissões**: somente endereços de e-mail (ou domínios) que você autorizar podem se autenticar. + +![A tela de login do FailproofAI Cloud, que envia um código de uso único para seu e-mail](/cloud/images/login.png) + +--- + +## Acesso restrito com chaves de API + +Cada cliente se autentica com uma chave de API que carrega permissões granulares e de menor privilégio. Um coletor precisa apenas de `events:add`; uma chave de dashboard ou assistente pode ser somente leitura; ações destrutivas (exclusão, regeneração) são concessões separadas que você escolhe incluir. + +![A página de chaves de API: as permissões de cada chave, com código de cores por escopo de leitura, escrita e destrutivo](/cloud/images/api-keys.png) + +Mantenha a chave de bootstrap de administrador para a configuração inicial e emita chaves restritas para todo o resto. Consulte [Chaves de API](/pt-br/cloud/access). + +--- + +## Um assistente somente leitura com aprovação obrigatória + +O [assistente de IA](/pt-br/cloud/assistant) integrado ao dashboard responde perguntas sobre seus dados, mas é restrito por design: + +- É **somente leitura por padrão**: o SQL que ele executa passa por um guard que permite apenas consultas `SELECT`/`WITH`, instrução única, com limite de linhas. +- Tudo que ele cria (uma consulta salva, um dashboard) **requer aprovação**: você revisa e aprova cada escrita antes que ela aconteça. +- Ele **nunca pode excluir**. + +Assim, um colega de equipe pode perguntar "quais agentes tiveram mais erros esta semana?" e agir com base na resposta, sem que o assistente consiga alterar ou remover seus dados por conta própria. + +--- + +## Em trânsito + +Todo o tráfego é transmitido via HTTPS. Você encerra o TLS com seus próprios certificados, de modo que o tráfego do coletor para o servidor e do navegador para o servidor é criptografado em trânsito. + +--- + +## Próximos passos + +- [Visão geral](/pt-br/cloud/overview): como o FailproofAI Cloud se encaixa. +- [Chaves de API](/pt-br/cloud/access): restrinja o acesso para o coletor, dashboard e assistente. +- [Observabilidade](/pt-br/cloud/overview): o que o FailproofAI Cloud captura dos seus agentes. \ No newline at end of file diff --git a/docs/pt-br/agenteye/sessions.mdx b/docs/pt-br/cloud/sessions.mdx similarity index 82% rename from docs/pt-br/agenteye/sessions.mdx rename to docs/pt-br/cloud/sessions.mdx index 63037c8f..daf0e431 100644 --- a/docs/pt-br/agenteye/sessions.mdx +++ b/docs/pt-br/cloud/sessions.mdx @@ -5,7 +5,7 @@ description: "Todos os eventos de uma execução consolidados em uma linha legí Chega de adivinhar por que uma execução falhou. A Observabilidade do Failproof AI consolida todos os eventos de uma execução em uma única linha legível e, em seguida, desenha toda a execução como uma imagem no estilo git que você pode interpretar em segundos — assim você vê exatamente o que seu agente fez, passo a passo. -![A lista de Sessões: uma linha por execução, entre ambientes e agentes, com indicadores de status e emblemas de pontuação de avaliação](/agenteye/images/sessions-list.png) +![A lista de Sessões: uma linha por execução, entre ambientes e agentes, com indicadores de status e emblemas de pontuação de avaliação](/cloud/images/sessions-list.png) *Uma linha por execução: o indicador de status mostra como a execução terminou de relance, e um emblema de pontuação aparece assim que um avaliador é conectado.* @@ -29,7 +29,7 @@ Quando você conectar um avaliador, cada execução concluída recebe uma pontua ## Leia toda a execução como uma imagem -![O grafo de execução no estilo git de uma sessão ao lado da linha do tempo de eventos, com o painel de detalhamento de ferramentas, modelos e hooks](/agenteye/images/session-detail.png) +![O grafo de execução no estilo git de uma sessão ao lado da linha do tempo de eventos, com o painel de detalhamento de ferramentas, modelos e hooks](/cloud/images/session-detail.png) *O grafo de execução (à esquerda) fica ao lado da linha do tempo de eventos; o painel direito detalha as ferramentas, modelos, hooks e o consumo de tokens da execução.* @@ -37,7 +37,7 @@ Clique em qualquer sessão para abrir o grafo de execução: uma visualização O painel direito oferece o detalhamento por execução: quais ferramentas e modelos rodaram, quais hooks foram disparados e quanto a execução consumiu em tokens. É a resposta para "por que essa execução custou tanto?" ou "qual ferramenta está lenta?" — ali mesmo, ao lado do grafo que a gerou. -Eventos individuais são endereçáveis, então você pode passar para alguém um link para um momento específico em vez de dizer "a sessão, lá pelo terço final". Copie o link de qualquer evento, ou siga um link de uma descoberta de [auditoria](/pt-br/agenteye/audits) ou de um erro, e a sessão abre com aquele evento selecionado e na posição certa. Isso funciona mesmo em execuções muito longas: a linha do tempo carrega uma janela delimitada para poupar seu navegador, e um link que aponta para além dessa janela ainda encontra o evento em vez de te jogar no início. Se o evento tiver ultrapassado o período de retenção, a página informa isso em vez de silenciosamente não selecionar nada. +Eventos individuais são endereçáveis, então você pode passar para alguém um link para um momento específico em vez de dizer "a sessão, lá pelo terço final". Copie o link de qualquer evento, ou siga um link de uma descoberta de [auditoria](/pt-br/cloud/audits) ou de um erro, e a sessão abre com aquele evento selecionado e na posição certa. Isso funciona mesmo em execuções muito longas: a linha do tempo carrega uma janela delimitada para poupar seu navegador, e um link que aponta para além dessa janela ainda encontra o evento em vez de te jogar no início. Se o evento tiver ultrapassado o período de retenção, a página informa isso em vez de silenciosamente não selecionar nada. --- @@ -45,12 +45,12 @@ Eventos individuais são endereçáveis, então você pode passar para alguém u Cada página do dashboard é escopada à sua organização (`//…`). Sessões fica em **Observe** na barra lateral esquerda, ao lado de Eventos, com os filtros de intervalo de datas, ambiente, agente e sessão no topo da lista. Cada linha está a um clique do seu grafo de execução completo. -Para ativar os emblemas de pontuação e a filtragem por faixa de pontuação, conecte um avaliador: consulte [Avaliações](/pt-br/agenteye/evaluations). +Para ativar os emblemas de pontuação e a filtragem por faixa de pontuação, conecte um avaliador: consulte [Avaliações](/pt-br/cloud/evaluations). --- ## Relacionados -- [Fluxo de eventos](/pt-br/agenteye/event-stream): o rastro bruto por etapa a partir do qual cada sessão é consolidada. -- [Avaliações](/pt-br/agenteye/evaluations): conecte um avaliador para que cada execução receba um emblema de pontuação pelo qual você pode filtrar. -- [Telemetria](/pt-br/agenteye/telemetry): como as execuções chegam do seu agente até essas sessões. \ No newline at end of file +- [Fluxo de eventos](/pt-br/cloud/event-stream): o rastro bruto por etapa a partir do qual cada sessão é consolidada. +- [Avaliações](/pt-br/cloud/evaluations): conecte um avaliador para que cada execução receba um emblema de pontuação pelo qual você pode filtrar. +- [Telemetria](/pt-br/cloud/performance): como as execuções chegam do seu agente até essas sessões. \ No newline at end of file diff --git a/docs/pt-br/concepts.mdx b/docs/pt-br/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/pt-br/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/pt-br/daemon.mdx b/docs/pt-br/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/pt-br/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/pt-br/dashboard.mdx b/docs/pt-br/dashboard.mdx index 25d8ad74..bfc37610 100644 --- a/docs/pt-br/dashboard.mdx +++ b/docs/pt-br/dashboard.mdx @@ -69,7 +69,7 @@ Um relatório com personalidade sobre como seu agente realmente se comportou ao 4. **Como melhorar** — lista de linhas tranquilas, uma por política prescrita: nome da política em branco, descrição em uma linha, comando de instalação + botão de cópia no lado direito. O cabeçalho da seção exibe `enable all N → projected · ` (a pontuação que você alcançaria com todas as correções aplicadas), e seu botão `[install all]` copia o comando combinado `failproofai policy add a b c …` para cada política prescrita. 5. **Volte melhorado** — dois cards lado a lado. Esquerda: definir um lembrete (seletor de cadência `3d` / `7d` / `14d` / `30d`; persiste via `/api/auth/reminder` após autenticação). Direita: desbloquear benefícios failproof — `invite a friend` abre um modal que aceita uma lista de e-mails de amigos separados por vírgula/espaço/quebra de linha (máx. 10 por envio), faz POST para `/api/audit/invite`, que encaminha para o `POST /v0/invite` do api-server. O api-server envia um e-mail por destinatário a partir de `invite@failproof.ai` com o remetente em Cc e `Reply-To` configurado, para que o destinatário veja quem o convidou e o remetente receba uma cópia em sua caixa de entrada. Usuários anônimos são redirecionados pelo `AuthDialog` primeiro para que o e-mail do remetente seja conhecido antes de os convites serem enviados. Direitos / cumprimento de benefícios é um acompanhamento futuro. -Impulsionado pelo runtime `failproofai audit` — consulte [CLI de Auditoria](/pt-br/cli/audit) para o mecanismo de escaneamento subjacente, flags suportadas e invariantes de cache por transcrição. O dashboard armazena em cache o resultado mais recente em `~/.failproofai/audit-dashboard.json` (modo `0600`, slot único, novas execuções sobrescrevem) para que revisitas sejam instantâneas; **tanto os caches por transcrição quanto os de resultado completo são rejeitados na leitura após 7 dias** para que o dashboard nunca sirva silenciosamente um resultado com uma semana de atraso — após o TTL, `/audit` vai para seu estado vazio e solicita uma nova execução. Clicar em `[ re-audit now ]` próximo ao final do relatório faz POST em `/api/audit/run` com `noCache: true` — uma re-auditoria ignora o cache por transcrição e re-escaneia cada transcrição do zero em vez de retornar silenciosamente o resultado em cache — e o dashboard faz polling em `/api/audit/status` a 1Hz até que a execução termine; uma faixa rosa fixa de progresso é fixada no topo do viewport durante a execução com um cronômetro decorrido, e o resultado atualizado substitui o anterior ao concluir com sucesso (sem recarregamento completo da página; uma re-auditoria com falha deixa o relatório anterior intacto). Em caso de falha, a faixa fica vermelha com texto baseado no `RerunError.kind` (`timeout` / `network` / `post_failed`). Estado vazio (sem cache ou expirado) e estado de zero sessões (cache existe, mas o escaneamento não encontrou transcrições) são apresentados separadamente. +Impulsionado pelo runtime `failproofai audit` — consulte [CLI de Auditoria](/pt-br/audit) para o mecanismo de escaneamento subjacente, flags suportadas e invariantes de cache por transcrição. O dashboard armazena em cache o resultado mais recente em `~/.failproofai/audit-dashboard.json` (modo `0600`, slot único, novas execuções sobrescrevem) para que revisitas sejam instantâneas; **tanto os caches por transcrição quanto os de resultado completo são rejeitados na leitura após 7 dias** para que o dashboard nunca sirva silenciosamente um resultado com uma semana de atraso — após o TTL, `/audit` vai para seu estado vazio e solicita uma nova execução. Clicar em `[ re-audit now ]` próximo ao final do relatório faz POST em `/api/audit/run` com `noCache: true` — uma re-auditoria ignora o cache por transcrição e re-escaneia cada transcrição do zero em vez de retornar silenciosamente o resultado em cache — e o dashboard faz polling em `/api/audit/status` a 1Hz até que a execução termine; uma faixa rosa fixa de progresso é fixada no topo do viewport durante a execução com um cronômetro decorrido, e o resultado atualizado substitui o anterior ao concluir com sucesso (sem recarregamento completo da página; uma re-auditoria com falha deixa o relatório anterior intacto). Em caso de falha, a faixa fica vermelha com texto baseado no `RerunError.kind` (`timeout` / `network` / `post_failed`). Estado vazio (sem cache ou expirado) e estado de zero sessões (cache existe, mas o escaneamento não encontrou transcrições) são apresentados separadamente. ### Políticas diff --git a/docs/pt-br/architecture.mdx b/docs/pt-br/how-it-works.mdx similarity index 100% rename from docs/pt-br/architecture.mdx rename to docs/pt-br/how-it-works.mdx diff --git a/docs/pt-br/introduction.mdx b/docs/pt-br/introduction.mdx index 4b1ab604..d66ef415 100644 --- a/docs/pt-br/introduction.mdx +++ b/docs/pt-br/introduction.mdx @@ -54,4 +54,4 @@ failproofai policies --install # enable policies (or skip — `failproofai` wi failproofai # launch the dashboard ``` -Consulte o guia de [Primeiros passos](/pt-br/getting-started) para o passo a passo completo. \ No newline at end of file +Consulte o guia de [Primeiros passos](/pt-br/quickstart) para o passo a passo completo. \ No newline at end of file diff --git a/docs/pt-br/policies.mdx b/docs/pt-br/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/pt-br/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/pt-br/getting-started.mdx b/docs/pt-br/quickstart.mdx similarity index 100% rename from docs/pt-br/getting-started.mdx rename to docs/pt-br/quickstart.mdx diff --git a/docs/pt-br/reference/files.mdx b/docs/pt-br/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/pt-br/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/quickstart.mdx b/docs/quickstart.mdx new file mode 100644 index 00000000..a29ee71c --- /dev/null +++ b/docs/quickstart.mdx @@ -0,0 +1,261 @@ +--- +title: Quickstart +description: "From nothing to a guarded machine in about two minutes — then connected to your fleet." +icon: rocket +--- + +This page takes you from an empty machine to one where every agent tool call is checked, +every decision is recorded, and (optionally) the whole thing reports into your +organization's dashboard. + +## Requirements + +| | | +|---|---| +| **Node.js** | 20.9.0 or newer | +| **Platform** | macOS or Linux. Windows is not supported yet — setup refuses rather than half-configuring the machine. See [why](/daemon#platform-support). | +| **An agent CLI** | Any of the [12 supported ones](/agent-support). None installed yet? Setup still writes the hooks, and they activate the moment you install one. | + +--- + +## 1. Install + + + +```bash npm +npm install -g failproofai +``` + +```bash bun +bun add -g failproofai +``` + + + +Prefer not to install anything? Every command works through `npx -y failproofai …` — but +a guardrail you have to remember to invoke is not a guardrail, so for real use, install it. + +--- + +## 2. Run setup + +```bash +failproofai config +``` + +Setup asks **two questions**. Everything else it works out from what is already on the +machine. + + + + **Recommended** turns on 16 policies globally for every agent CLI it finds on this + machine. It is the answer for almost everyone, and it is a decision, not a shortcut: + + | What it covers | Policies | + |---|---| + | Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | + | The agent cannot switch off its own guardrails | `block-self-pause`, `block-failproofai-commands` | + | Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | + | Git history stays recoverable | `block-push-master`, `block-force-push` | + + **Customize** opens the full wizard: pick the scope (global or just this project), + combine [policy presets](/policies#presets), and choose exactly which agent CLIs to + wire up. + + + + **Paste an API key** connects this machine to your organization: policy comes down + from the dashboard, and what your agents do goes up to it. Don't have a key yet? + Create one at [befailproof.ai/get-started](https://befailproof.ai/get-started/). + + **Not now — stay local** keeps everything on this machine. Nothing is sent anywhere. + You can connect later by re-running `failproofai config`, and nothing you set up now + is lost. + + + Connecting sends both policy decisions **and** session transcripts. Transcripts + contain prompts, file contents, and command output. That is the point of connecting — + it is what makes a fleet dashboard worth having — but it is a real disclosure. Use + `--no-transcripts` if you want decisions only. See [what leaves this + machine](/cloud/connect#what-leaves-this-machine). + + + + +Setup then writes the hook entries into each agent CLI's own settings file, installs the +[`failproofaid` background service](/daemon), and confirms every file it touched before +it touches it. + + + Installing the service needs root, and setup uses `sudo -n` — it never prompts for a + password from inside its own UI. If it cannot elevate, it writes nothing and prints the + exact commands for you to run. That is deliberate: a half-configured machine is worse + than an unconfigured one. + + +--- + +## 3. Check it + +```bash +failproofai config --status +``` + +Tells you three things: whether this machine is connected to the cloud, whether the +daemon is running, and whether enforcement is currently paused on any session. + +```bash +failproofai policies +``` + +Lists every policy, whether it is on, and any parameters you have set. + +--- + +## 4. Watch it work + +Start your agent exactly as you normally do. Then ask it to do something a policy blocks: + +```text +you Run `sudo apt-get install ripgrep` for me. + +agent I tried to run that, but it was blocked: + "Blocked by failproofai: sudo command blocked" + I'll install it without elevated privileges instead. +``` + +That recovery is the whole design. A denial is not an error the agent hits and gives up +on — it is a sentence the model can read and route around. + +--- + +## 5. See what happened + +```bash +failproofai +``` + +Opens the [local dashboard](/dashboard) at `http://localhost:8020`: every project, every +session, every tool call with its input and output, and every policy decision that fired +on it. + +While you are there, run the [audit](/audit): + +```bash +failproofai audit +``` + +It replays the agent transcripts already on this machine through the policy engine and +tells you what your agents have *actually* been doing — a score, your agent's archetype, +a ranked list of what slipped through, and a copy-pasteable command for each fix. + +--- + +## 6. Make it your team's standard + +The fastest way to turn one person's guardrails into a team's is the +`.failproofai/policies/` convention. Drop a file in, commit it, done — no flags, no +config, no per-developer setup. + + + + ```bash + mkdir -p .failproofai/policies + ``` + + + ```js + // .failproofai/policies/team-policies.mjs + import { customPolicies, allow, instruct } from "failproofai"; + + customPolicies.add({ + name: "test-before-commit", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + if (/git\s+commit/.test(ctx.toolInput?.command ?? "")) { + return instruct("Run the test suite before committing."); + } + return allow(); + }, + }); + ``` + + + ```bash + git add .failproofai/policies/ + git commit -m "Add team quality policies" + ``` + + Every teammate with failproofai installed picks it up on their next pull. + + + + + Treat `.failproofai/policies/` as a living quality standard. Every time your team finds + a new way an agent wastes an afternoon, add a policy and push. The standard compounds; + nobody has to remember it. + + +Running a fleet? Skip the git round-trip entirely — [deploy policy from the +cloud](/cloud/managed-policies) and every connected machine picks it up on its next poll. + +--- + +## Where your data lives + +Everything FailproofAI writes on a machine lives under `~/.failproofai/`, plus a +`.failproofai/` directory in any project you configure. Nothing is written outside those +two places except the hook entries in each agent CLI's own settings file. + +| Path | What it holds | +|---|---| +| `~/.failproofai/policies-config.json` | Which policies are on, and their parameters | +| `~/.failproofai/policies/` | Your own policy files — drop `*policies.mjs` in, no config needed | +| `~/.failproofai/policies/cloud-policies/` | Policies your organization deployed to this machine | +| `~/.failproofai/config.json` | Machine settings: daemon, collector, audit schedule | +| `~/.failproofai/credentials.toml` | Cloud tokens. Owner-only (`0600`) | +| `~/.failproofai/hook-activity/` | The decision log the dashboard reads | +| `.failproofai/policies-config.json` | Per-project config, committed | +| `.failproofai/policies-config.local.json` | Personal overrides, gitignored | + +[Full file-layout reference →](/reference/files) + +--- + +## Uninstalling + +```bash +failproofai uninstall # remove hooks from every agent CLI + the daemon service +failproofai uninstall --purge # …and delete ~/.failproofai entirely +``` + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves the hook entries and the background + service behind. + + +--- + +## Next steps + + + + + What actually happens between a tool call and a decision. + + + + All 39 policies and what each one catches. + + + + Scopes, merge rules, and tuning a policy without code. + + + + One command, two capabilities, and exactly what gets sent. + + + diff --git a/docs/reference/files.mdx b/docs/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/ru/agent-support.mdx b/docs/ru/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/ru/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/ru/agenteye/cli-and-agents.mdx b/docs/ru/agenteye/cli-and-agents.mdx deleted file mode 100644 index 51571561..00000000 --- a/docs/ru/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "Весь ваш Failproof AI Observability, развёрнутый одной командой." ---- - - -Весь ваш Failproof AI Observability, развёрнутый одной командой. Проверьте production, создайте API-ключ или подтвердите инцидент, не покидая терминала, а затем автоматизируйте всё это в CI или позвольте coding-агенту сделать это на простом английском языке. - -```bash -pipx install agenteye -agenteye login --email you@example.com # a 6-digit code lands in your inbox -agenteye --json sessions --since 24h # every agent run from the last day, newest first -``` - -*CLI `agenteye` взаимодействует с вашим dashboard. Это отдельный инструмент от collector, который отправляет события на сервер.* - -## Весь deployment в одной команде - -Перестаньте прыгать по табам, чтобы ответить на быстрый вопрос. CLI `agenteye` читает ваши данные и управляет организацией из единого бинарного файла, поэтому проверка, которая раньше требовала кликов в dashboard, становится одной строкой, которую можно переиспользовать, создать alias или вставить в runbook. Вы получаете четыре интерфейса: - -- **Читайте ваши данные:** `sessions`, `events`, `evals` и `errors`, отфильтрованные по времени, агенту и окружению. -- **Управляйте организацией:** `keys`, `users`, `settings`, `alerts` и `incidents`. -- **Запускайте аналитику:** сохранённые SQL-запросы плюс ad-hoc `query` для анализа данных о событиях. -- **Спросите ассистента:** `agent ask` подключает того же read-only аналитика, с которым вы общаетесь в dashboard. - -Установите один раз с помощью `pipx`, войдите, используя 6-значный код из письма, и готово. Сессия длится около дня; переустановите `agenteye login` когда она истечёт. Используйте его для проверки production, подготовки ключа или триажа срабатывающего инцидента, всё без открытия браузера: - -```bash -agenteye errors --since 24h --aggregate # what is breaking, grouped by error type -agenteye incidents list --state firing # what is on fire right now -agenteye keys create ci --add events:add # a key that can only push events, secret shown once -``` - -Один привычный нюанс: глобальные опции вроде `--json` идут перед командой. `agenteye --json sessions` правильно; `agenteye sessions --json` неправильно. - -## Автоматизируйте, подключите к CI - -Каждая команда поддерживает `--json`, и это меняет всё. Чистый JSON идёт в stdout, а статус и предупреждения для человека — в stderr, поэтому захват с `--json` направляется прямо в `jq` без лишних строк для очистки. Именно это делает CLI одинаково полезным как для вас в командной строке, так и для coding-агента, парсящего выходные данные: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -Построен для работы без присмотра. Запросы подтверждения пропускаются автоматически, когда терминал не подключён, поэтому ничего не зависает в pipeline, и каждая команда возвращает значимый exit code: `0` успех, `4` не залогинены, `5` недостаточны разрешения (сообщение указывает их, например `alerts:write`), `3` dashboard недоступен. Скрипт может ветвиться на `4` для переаутентификации или на `5` чтобы рассказать вам, что именно нужно попросить у администратора, вместо того чтобы упасть без информации. - -## Позвольте coding-агенту управлять им на простом английском - -Лучше ещё — вам не должно требоваться помнить все эти флаги вообще. **CLI skill** — это небольшая папка Agent Skill под названием `agenteye-cli`, которая учит coding-агента такого как Claude Code или Codex управлять CLI из простых запросов на английском языке. Спросите "что-нибудь сломалось сегодня?" и агент выберет команду, запустит её от вашего имени и ответит прозой. - -Для Claude Code поместите папку `agenteye-cli` в `~/.claude/skills/` и она будет автоматически обнаружена. Failproof AI Observability предоставляет папку; нет ничего дополнительного для установки, потому что она только управляет CLI, который вы уже установили. Залогиньтесь сами первыми: skill не может завершить для вас login с кодом из письма. - -Поскольку агент запускает CLI от вашего имени, он может делать всё, что позволяет ваш логин, чтение и запись: создавать ключи, менять настройки, разрешать инциденты. Prompt "вы уверены?" CLI не срабатывает для агента, поэтому skill написан так, чтобы указать точную команду и ждать вашего ОК перед любым изменением. Вы — шаг подтверждения. - -```text -вы Почему упала сессия run-001? - -агент Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -Чтения остаются мгновенными, и каждая запись приостанавливается для вас: - -```text -вы Дайте CI ключ, который может только отправлять события. - -агент This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -вы yes - -агент Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## Связанное - -- [CLI reference](/ru/agenteye/cli): все команды, флаги и JSON-структуры. -- [CLI recipes for agents](/ru/agenteye/cli-recipes): паттерны `jq` и обработка exit-кодов для копирования-вставки. -- [CLI agent skill](/ru/agenteye/cli-skill): установка и запуск skill `agenteye-cli`. -- [AI assistant](/ru/agenteye/assistant): аналитик в dashboard, с которым общается `agent ask`. \ No newline at end of file diff --git a/docs/ru/agenteye/cli-skill.mdx b/docs/ru/agenteye/cli-skill.mdx deleted file mode 100644 index 272fb6c7..00000000 --- a/docs/ru/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Failproof AI Observability CLI Agent Skill" -description: "Спросите у вашего coding agent \"сегодня что-нибудь сломалось?\" и позвольте ему ответить на основе ваших live данных Failproof AI Observability, без необходимости запоминать команды." ---- - - -Спросите у вашего coding agent *«сегодня что-нибудь сломалось?»* и позвольте ему ответить на основе ваших live данных Failproof AI Observability, без необходимости запоминать команды. **Failproof AI Observability CLI skill** (`agenteye-cli`) — это *Agent Skill*: небольшая папка с инструкциями, которую coding agent, такой как Claude Code или Codex, загружает по требованию. Она учит agent управлять вашей Observability deploymentом через [`agenteye` CLI](/ru/agenteye/cli) на основе запросов на обычном английском языке, таких как *«выдай CI ключ, который может только отправлять события»* или *«подтверди активный инцидент и назначь его на меня»*. - -Это **не** сервис и не отдельный бинарный файл; нет ничего, что нужно разворачивать. Он работает поверх уже установленного вами CLI: agent выполняет `agenteye --json …`, парсит чистый JSON и отвечает вам прозой. Всё, что он может сделать, вы можете сделать сами, набрав те же команды. - ---- - -## Как это соотносится с другими интерфейсами Failproof AI Observability - -Failproof AI Observability предоставляет четыре способа доступа к одним и тем же данным и элементам управления. Они дополняют друг друга: - -| Интерфейс | Что это такое | Где выполняется | Используйте, когда | -|---|---|---|---| -| **[CLI](/ru/agenteye/cli)** | Справочник команд и флагов для `agenteye` | Ваш терминал | Вы хотите запустить или создать сценарий для конкретной команды | -| **[CLI recipes](/ru/agenteye/cli-recipes)** | Шаблоны `jq`/pipelines для копирования и вставки | Ваш терминал / скрипты | Вы интегрируете CLI в автоматизацию | -| **CLI skill** (этот документ) | Дверь с естественным языком для CLI | Ваш coding agent на рабочей станции | Вы просто хотите спросить и позволить agent выбрать команду | -| **[Evaluator skill](/ru/agenteye/evaluator-skill)** | Родственный skill, который проектирует и создаёт ваш сервис оценивания | Ваш coding agent на рабочей станции | Вы хотите *производить* баллы оценивания, а не только их читать | -| **[Python SDK skill](/ru/agenteye/python-sdk-skill)** | Родственный skill, который инструментирует agent, чтобы он вообще испускал телеметрию | Ваш coding agent на рабочей станции | Вы хотите, чтобы agent *производил* события, которые этот skill читает | -| **[In-dashboard AI assistant](/ru/agenteye/assistant)** | Чат, встроенный в приборную панель | Серверная сторона (в приборной панели) | Вы хотите Q&A в приборной панели над вашими данными | - -Сам skill не имеет собственных привилегий; он просто преобразует ваши слова в вызовы CLI, которые выполняются от вас: - -```mermaid -flowchart TD - YOU["вы: 'подтверди активный инцидент'"] --> AGENT["coding agent (Claude Code / Codex)
    загружает agenteye-cli skill"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|ваша аутентифицированная CLI сессия| API["Observability dashboard API"] -``` - -### в сравнении с in-dashboard AI assistant: важное различие - -Это два совершенно разных инструмента с очень разными областями влияния: - -- **In-dashboard AI assistant** ([AI assistant](/ru/agenteye/assistant)) — это чат, встроенный в приборную панель, поддерживаемый сервисом agent. Он **только для чтения плюс создание с одобрением**: он может создавать черновики сохранённых запросов и панелей управления, но каждая запись требует вашего явного подтверждения, и он никогда не удаляет. Он защищён разрешением `agent:use` и видит только данные организации, которую вы просматриваете. -- **CLI skill** работает на *вашей* рабочей станции внутри *вашего* coding agent и управляет `agenteye` CLI от вас. Он может выполнять **полный набор CLI, включая изменения** (создание/ротация/отключение API ключей, изменение параметров org, разрешение инцидентов, удаление сохранённых запросов), ограниченные только разрешениями вашей CLI-сессии. Относитесь к этому ровно так же осторожно, как вы относились бы к выполнению этих команд вручную. - ---- - -## Предварительные требования - -1. **`agenteye` CLI установлен** и находится в `PATH` (см. [CLI](/ru/agenteye/cli) справочник: `pipx install agenteye`). -2. Ваш **URL приборной панели установлен** (`AGENTEYE_DASHBOARD_URL`, или agent передаёт `--base-url`). -3. **Сессия с аутентификацией**: сначала запустите `agenteye login` сами. Skill **не может** завершить отправку одноразового кода по электронной почте за вас; он подскажет вам запустить `agenteye login`, если сессия отсутствует или истекла (CLI код выхода `4`). - ---- - -## Где это взять - -Skill опубликован в публичной коллекции skills Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -Ничто в нём не защищено — репозиторий открыт, и skill не требует собственных учётных данных, потому что он только управляет **публичным** `agenteye` CLI против *вашей* приборной панели, используя сессию, под которую *вы* вошли. Вам не нужно ничего у кого-то просить. - -Обратите внимание, что он поставляется как отдельная папка и **не находится** в пакете `pipx install agenteye`, поэтому не ищите его там. - -## Установка skill - -Самый быстрый способ — это [`skills`](https://skills.sh) CLI, который загружает папку и размещает её там, где ваш agent её ищет: - -```bash -# Claude Code, только этот проект -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# каждый проект (устанавливает в ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex вместо этого -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -Затем управляйте им как любым другим skill: - -```bash -npx skills list -a claude-code # что установлено -npx skills update agenteye-cli # загрузить последнюю версию -npx skills remove agenteye-cli # удалить его -``` - -Предпочитаете установку вручную? Agent Skill — это просто папка, содержащая `SKILL.md` (плюс дополнительные справки), поэтому копирование тоже работает: - -- **Claude Code**: положите папку `agenteye-cli/` в `~/.claude/skills/` (каждый проект) или `<ваш-репо>/.claude/skills/` (только этот репо). Claude Code автоматически её обнаруживает — проверьте со списком `/skills` или просто задайте вопрос, который совпадает с её описанием. -- **Codex (OpenAI)**: Codex читает тот же `SKILL.md`. Включённый `agents/openai.yaml` устанавливает `allow_implicit_invocation: true`, поэтому Codex автоматически выбирает skill, когда задача совпадает; в противном случае вызовите его явно как `$agenteye-cli`. - ---- - -## Безопасность: изменения НЕ требуют подтверждения, когда agent запускает CLI - -> **Warning:** Прочитайте это перед тем, как позволить agent вносить изменения. - -`agenteye` CLI обычно запрашивает *«ты уверен?»* перед деструктивным действием. Он **автоматически пропускает это подтверждение, когда он не подключён к терминалу (что является в точности тем, как coding agent его запускает), и `--json` тоже пропускает это.** Поэтому подсказка о безопасности **не будет** активирована для agent. - -Skill написан, чтобы это компенсировать: ему дано указание указать точную команду, которую он будет запускать, и получить ваше явное **ОК перед любым изменением состояния**. Соблюдайте эту дисциплину. Когда вы управляете Failproof AI Observability через agent, *вы* — это шаг подтверждения. Команды, изменяющие состояние, на которые нужно обратить внимание: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- команды записи `incidents`: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -Всё под **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) — только для чтения и ничего не меняет. - -Поскольку agent действует от вас, он может делать только то, что разрешено вашей сессией; разрешения разрешаются **на организацию** (см. [API keys](/ru/agenteye/api-keys)). Команда, на которую у вас нет разрешения, возвращает код выхода `5` с точно названным разрешением, поэтому agent может точно сказать вам, что просить у администратора, вместо того чтобы неясно отказать. - ---- - -## Что вы можете его попросить - -Вот что выглядит реальный обмен, сначала чтение, затем изменение, которое ждёт вашего ОК: - -```text -вы ▸ Что-нибудь сломалось за последний день? - -agent ▸ Запуск: agenteye --json errors --since 24h --aggregate - 3 типа ошибок за последние 24 часа. Самый большой — TimeoutError - (18 попаданий, все на checkout-agent), затем RateLimited (5). - Хотите получить сбойные сессии для TimeoutError? - -вы ▸ Выдай CI ключ, который может только отправлять события. - -agent ▸ Это создаёт API ключ. Я запущу: - agenteye keys create ci --add events:add - Продолжить? - -вы ▸ да - -agent ▸ Готово. Ключ "ci" создан с только events:add. - Секрет показан только один раз, поэтому сохраните его сейчас. Я не могу его переиспечатать. -``` - -Skill отображает каждое намерение на простом языке на правильную `agenteye` команду, сначала открывая допустимые значения (`list `, `whoami`), чтобы не угадывать, и указывая точную команду перед любым изменением. Больше примеров: - -- *«Что-нибудь сломалось / сбойное за последние 24 часа?»* → `errors --since 24h --aggregate`, затем разбивка. -- *«Почему сессия `run-001` сбойная?»* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *«Как качество тренднулось на этой неделе?»* → `evals --aggregate --since 7d`, затем углубиться в низко оценённые прогоны. -- *«Выдай CI ключ, который может только отправлять события»* → `keys create ci --add events:add` (он указывает команду, затем создаёт её и захватывает одноразовый секрет). -- *«Кто имеет доступ? Сделай Dana только для чтения»* → `users list` → `users update dana@… --permission-set read-only` (после подтверждения с вами). -- *«Подтверди активный инцидент и назначь его на меня»* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`. - -Для точных команд, флагов и JSON форм за этим смотрите справочник [CLI](/ru/agenteye/cli) и [CLI recipes для agents](/ru/agenteye/cli-recipes). - ---- - -## Следующие шаги - -- **[CLI](/ru/agenteye/cli)**: полный справочник команд и флагов для `agenteye`. -- **[CLI recipes для agents](/ru/agenteye/cli-recipes)**: шаблоны `jq` и обработка кодов выхода для копирования и вставки. -- **[Evaluator agent skill](/ru/agenteye/evaluator-skill)**: родственный skill для создания evaluator, чьи баллы читает `agenteye evals`. -- **[Python SDK agent skill](/ru/agenteye/python-sdk-skill)**: родственный skill для инструментирования agent, чтобы он испускал телеметрию, которую читает `agenteye`. -- **[AI assistant](/ru/agenteye/assistant)**: in-dashboard ассистент (не путать с этим terminal skill). -- **[API keys](/ru/agenteye/api-keys)**: модель разрешений на организацию, которая ограничивает то, что может делать skill. \ No newline at end of file diff --git a/docs/ru/agenteye/codex-capture.mdx b/docs/ru/agenteye/codex-capture.mdx deleted file mode 100644 index 77567f0d..00000000 --- a/docs/ru/agenteye/codex-capture.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "Запись сессий Codex" -description: "Собирайте локальные сессии OpenAI Codex вашей команды в AgentEye как обычные сессии и события — без каких-либо изменений в том, как они запускают Codex." ---- - -Ваши инженеры уже ежедневно используют OpenAI Codex. Запись сессий Codex переносит эти сессии кодирования в AgentEye как обычные сессии и события, так что вы можете искать, проигрывать и оценивать их вместе со всем остальным, что вы наблюдаете. Это дополняет [Python SDK](/ru/agenteye/python-sdk): SDK инструментирует написанные вами агенты, а это захватывает работу в Codex, которую ваша команда уже выполняет — без каких-либо изменений в том, как они его запускают. - -Небольшой фоновый сборщик считывает локальные расшифровки сессий Codex по мере их записи и отправляет их в AgentEye. Один сборщик на машину захватывает все локальные поверхности Codex одновременно — настройка каждой поверхности не требуется. - -Тот же сборщик захватывает и других агентов — см. [OpenClaw](/ru/agenteye/openclaw-capture) и [Hermes](/ru/agenteye/hermes-capture). Включите каждого, кого вы запускаете; один сборщик может захватывать несколько одновременно. - ---- - -## Что он захватывает - -Каждая поверхность Codex, работающая **локально**, создает одинаковые расшифровки сессий на диске, и сборщик захватывает все из них: - -- **CLI** Codex и `codex exec` -- **расширение VS Code / IDE** -- **настольное приложение**, когда оно запускает сессию локально - -Каждая сессия Codex становится AgentEye [сессией](/ru/agenteye/sessions); её сообщения пользователя и ассистента, рассуждения, вызовы инструментов, результаты инструментов и использование токенов становятся соответствующими [событиями](/ru/agenteye/event-stream). Записывается поверхность, из которой пришла каждая сессия (CLI, IDE или настольное приложение), так что вы можете их различить. - -> **Облачные сессии не захватываются.** Настольное приложение всё чаще запускает сессии в облаке Codex и хранит только их метаданные на машине — нет локальной расшифровки для чтения. Захватываются только локально выполняемые сессии. - ---- - -## Включение - -Захват отключен до включения. Установите сборщик с ключом API, который имеет разрешение `events:add` (см. [API keys](/ru/agenteye/api-keys)), и включите захват Codex: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -Это устанавливает сборщик, регистрирует его как фоновый сервис и начинает захват. Убедитесь, что он работает: - -```bash -agenteye-collector health -``` - -При первом запуске существующие сессии Codex заполняются один раз, а новая активность затем поступает в течение нескольких секунд. Файлы самого Codex только считываются — никогда не изменяются, не перемещаются и не удаляются — и каждая сессия отправляется ровно один раз, даже при перезагрузках. - ---- - -## Где это отображается - -Захваченные сессии появляются в **Sessions**, а их события в потоке **Events**, как и любой другой наблюдаемый вами агент — поэтому [проигрывание сессий](/ru/agenteye/sessions), [поиск](/ru/agenteye/queries), [оценки](/ru/agenteye/evaluations) и [оповещения](/ru/agenteye/alerts) работают на них. Отфильтруйте по агенту Codex, чтобы увидеть их отдельно. - ---- - -## Приватность - -Расшифровки Codex содержат полную сессию — включая вывод команд, содержимое файлов и всё, что Codex читал или писал — и могут содержать секреты. Захваченные сессии отправляются как есть, поэтому включайте захват только на машинах и для команд, где централизация этого содержимого в AgentEye целесообразна, и предоставьте сборщику ключ с областью `events:add` только. См. [Security](/ru/agenteye/security), чтобы узнать, как ваши данные остаются изолированными. \ No newline at end of file diff --git a/docs/ru/agenteye/concepts.mdx b/docs/ru/agenteye/concepts.mdx deleted file mode 100644 index 4bdc66e7..00000000 --- a/docs/ru/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "Концепции" -description: "Словарь Failproof AI Observability — события, сеансы, оценки, аудиты, результаты и инциденты — определены в одном месте." ---- - - -На этой странице определены термины, которые использует Failproof AI Observability. Если в другом руководстве вам встретится незнакомый термин, его определение здесь. Вам не нужно читать всё подряд: просто просмотрите или вернитесь сюда, когда встретите слово, которое нужно уточнить. - ---- - -## Модель данных - -**Event (событие)** -Наименьшая единица данных. Одно событие записывает один шаг, который выполнил ваш агент: `tool_use`, `model_request`, `hook_completed`, `error` и т. д. Ваш агент генерирует события через [Python SDK](/ru/agenteye/python-sdk); они отображаются в реальном времени на странице **Events** (события). - -**Session (сеанс)** -Один запуск агента, идентифицируемый `session_id`. Сеанс — это все события, имеющие этот идентификатор, сведённые в одну строку на странице **Sessions** (сеансы) и отображённые в виде графика выполнения на странице подробностей. Сеанс обычно начинается с `agent_start` и заканчивается `agent_end`. - -**Agent (агент)** -Именованный участник в запуске, идентифицируемый `agent_id`. Запуск может включать несколько агентов: например, планировщик, который порождает подагента-summarizer. Подагенты имеют `parent_id`, который позволяет Failproof AI Observability отображать их на отдельных линиях в графике выполнения. - -**Environment (окружение)** -Метка, указывающая, где происходил запуск: `production`, `staging`, `dev`. Вы устанавливаете его один раз при настройке SDK. Почти все страницы панели управления могут фильтроваться по окружению. - -**Context-window fill (заполнение контекстного окна)** -Процент контекстного окна модели, который потребил ответ. Failproof AI Observability проставляет этот показатель для событий `model_response` для распознаваемых моделей, чтобы рост промтов и предстоящая компрессия были видны прямо в потоке событий. - ---- - -## Качество - -**Evaluation (оценка)** -Оценка качества завершённого сеанса, созданная вашим сервисом оценки. Оценки опциональны: до подключения оценщика сеансы записываются, но не оцениваются. Каждая оценка может содержать несколько именованных баллов (например `helpfulness`, `factuality`, `tool_efficiency`), каждый с кратким примечанием рассуждений. См. [Evaluation suite](/ru/agenteye/evaluation-suite). - -**Score key (ключ оценки)** -Название одного измерения, о котором сообщает оценщик, например `helpfulness`. Оповещения и аудиты могут отслеживать определённый ключ оценки со временем. - -**Evaluator (оценщик)** -Ваш сервис оценки. Failproof AI Observability отправляет POST-запрос стенограмму завершённого запуска и сохраняет возвращаемые оценки. Служба не поставляется с оценщиком по умолчанию; логика оценки — ваша. - ---- - -## Поиск и исправление ошибок - -**Hook (хук)** -Guardrail или побочный эффект, который выполняет ваш фреймворк агента вокруг шага: проверка безопасности контента, редакция PII, guard для бюджета. Хуки генерируют события `hook_triggered` / `hook_completed` с `outcome` (allow, deny, modify) и имеют собственную страницу наблюдения. - -**Alert rule (правило оповещения)** -Правило, которое срабатывает, когда метрика пересекает установленный вами порог: error rate, p95 latency, token cost или оценка оценщика. Когда срабатывает правило, оно открывает инцидент и отправляет уведомления в выбранные каналы (email, Slack, webhook, в панель управления). См. [Alerts](/ru/agenteye/alerts). - -**Incident (инцидент)** -Открытая проблема, созданная при срабатывании правила оповещения. Инциденты имеют жизненный цикл (acknowledge, assign, resolve) и временную шкалу активности, которая записывает каждое действие. Вы также можете открыть инцидент вручную. - -**Audit (аудит)** -Периодическое расследование (ежечасное до еженедельного), которое анализирует журналы *между* сеансами в поисках паттернов сбоев, для которых вы ещё не написали правило: кластеры ошибок, низкие оценки, выбросы latency, циклы вызовов инструментов и запуски, которые никогда не завершились. Если оповещение отслеживает метрику, о которой вы уже знаете, аудит указывает, на что смотреть дальше. См. [Audits](/ru/agenteye/audits). - -**Finding (результат)** -Один ранжированный, подкреплённый доказательствами результат запуска аудита. Результат называет паттерн, ссылается на точные сеансы, лежащие в его основе, и имеет жизненный цикл сортировки (acknowledge, resolve, mute, dismiss). Failproof AI Observability дедублирует результаты от запуска к запуску, поэтому известный паттерн обновляется вместо накопления. - -**The AI assistant (AI-ассистент)** -Встроенный в панель управления чат, который отвечает на вопросы об ваших агентах на простом английском языке, используя ваши собственные данные. По умолчанию он работает в режиме чтения; всё, что он создаёт (сохранённый запрос, панель управления), требует одобрения, и он никогда не может удалять. См. [AI assistant](/ru/agenteye/assistant). - ---- - -## Запуск - -**Organization (tenant) (организация)** -Изолированное рабочее пространство. Один экземпляр Failproof AI Observability может размещать множество организаций, каждая со своими пользователями, ключами и данными. Каждый URL панели управления ограничена вашим слагом организации (`//…`). - -**Collector (коллектор)** -`agenteye-collector`, лёгкий демон, который работает на каждой машине агента, объединяет события, которые SDK записывает на диск, и отправляет их на сервер. - -**API key (API-ключ)** -Токен с ограниченной областью, который аутентифицирует клиент на сервере. Ключи имеют детальные разрешения (например `events:add` для коллектора, read-only области для ключа панели управления). См. [API keys](/ru/agenteye/api-keys). - -**Server (сервер)** -Сервис ingestion и API. Он принимает события, хранит операционное состояние в ваших базах данных и обслуживает панель управления и CLI. - -**Dashboard (панель управления)** -Веб-интерфейс. Каждая страница ограничена организацией и читает данные через API сервера. - ---- - -## Следующие шаги - -- [Overview](/ru/agenteye/overview): как эти компоненты работают вместе. -- [Observability](/ru/agenteye/observability): поверхности наблюдения (Events, Sessions, Models, Tools, Hooks, Errors). \ No newline at end of file diff --git a/docs/ru/agenteye/evaluator-skill.mdx b/docs/ru/agenteye/evaluator-skill.mdx deleted file mode 100644 index 1afb464b..00000000 --- a/docs/ru/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,171 +0,0 @@ ---- ---- -title: "Навык Failproof AI Observability Evaluator Agent" -description: "От «я думаю, что наш агент иногда работает плохо» к развёрнутому сервису оценки, где кодирующий агент сам принимает решения и строит решение." ---- - - -От *«я думаю, что наш агент иногда работает плохо»* к развёрнутому сервису оценки, где кодирующий агент сам принимает решения и строит решение. **Навык Failproof AI Observability evaluator** (`agenteye-evaluator`) — это *Agent Skill*: небольшая папка с инструкциями, которые кодирующий агент, такой как Claude Code или Codex, загружает по требованию. Она учит агента определять, какие показатели качества стоит отслеживать для *вашего* агента, а затем писать, тестировать и развёртывать [сервис оценки](/ru/agenteye/evaluation-suite), который их оценивает. - -Это **не** размещённый скорер, реестр для загрузки или система плагинов. Ваша оценка остаётся вашей собственной HTTP-службой на вашей инфраструктуре, точно так, как описано в руководстве [Evaluation suite](/ru/agenteye/evaluation-suite). Навык только учит вашего агента строить её правильно, поэтому всё, что она делает, вы могли бы сделать сами, написав тот же код. - ---- - -## Сложная часть — решить, что оценивать - -Поверхность SDK небольшая — декоратор и две модели — и агент может написать это просто по [контракту](/ru/agenteye/evaluation-suite#http-contract). В этом не проблема оценок. Они не работают, потому что оценивают неправильное, и оценка, которая оценивает неправильное, хуже, чем никакая: она создаёт панель управления, которую все учатся игнорировать. - -Поэтому большая часть навыка — это часть до того, как существует код. Агент берёт у вас интервью (*«опишите сеанс, который прошёл хорошо; теперь один, который прошёл плохо»*), затем загружает ваши реальные сеансы через [`agenteye` CLI](/ru/agenteye/cli) и читает их от начала до конца. Эти две части обычно не совпадают, и разрыв — это именно то, что нужно: что вы намерены измерять против того, что ваши расшифровки могут реально поддерживать. Измерение выживает только если оно **вычислимо** из событий и **дискриминирующее** — если оно выставляет 0,9 как для вашего хорошего, так и для вашего плохого сеанса, оно ничему не учит и исключается. - -То, что возвращается — предложение 2-4 измерений с приложенным обоснованием, которое вы подписываете перед тем, как будет написана строка кода. - -```mermaid -flowchart TD - YOU["вы: 'Мне нужна оценка для моего бота поддержки'"] --> AGENT["кодирующий агент (Claude Code / Codex)
    загружает навык agenteye-evaluator"] - AGENT -->|"интервью: как выглядит хорошее vs плохое?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["ваши реальные сеансы
    что действительно происходит"] - DATA --> DIMS["2-4 измерения, вы подписываете"] - DIMS --> SVC["ваш сервис оценки
    agenteye-evaluator SDK"] - SVC --> SCORES["оценки попадают на панель
    и в agenteye evals"] -``` - ---- - -## Как это относится к другим частям оценки - -Четыре документа охватывают оценку и передают информацию друг другу по очереди: - -| Страница | Что это | Используйте, когда | -|---|---|---| -| **[Evaluations](/ru/agenteye/evaluations)** | Функция: оценки на сетке сеансов, панели, переоценка | Вы хотите узнать, что вы получаете от автоматической оценки | -| **[Evaluation suite](/ru/agenteye/evaluation-suite)** | HTTP контракт, SDK, переменные окружения сервера | Вы реализуете или отлаживаете оценку самостоятельно | -| **Evaluator skill** (этот документ) | Естественный язык для проектирования *и* построения скорера | Вы хотите перейти от «мне нужна оценка» к работающему сервису | -| **[CLI skill](/ru/agenteye/cli-skill)** | Естественный язык для `agenteye` CLI | Вы хотите *читать* оценки, которые уже у вас есть | -| **[Python SDK skill](/ru/agenteye/python-sdk-skill)** | Естественный язык для инструментирования вашего агента | Ваш агент ещё не генерирует сеансы — нечего оценивать | - -### vs. CLI skill: построение vs чтение - -Два навыка намеренно неперекрывающиеся, и установка обоих — это обычная конфигурация — агент выбирает между ними в зависимости от того, что вы просите: - -- **`agenteye-evaluator`** (этот документ) строит то, что *производит* оценки. Его работа заканчивается, когда оценки появляются в первый раз. -- **[`agenteye-cli`](/ru/agenteye/cli-skill)** читает оценки, которые уже существуют (`agenteye evals`). *«Качество упало на этой неделе?»* — это его вопрос, не этого навыка. - ---- - -## Предварительные требования - -1. **`agenteye` CLI установлен и подключён** (`pipx install agenteye`, затем `agenteye login`). Навык использует его дважды: для загрузки реальных сеансов, на которых он проектирует, и для подтверждения того, что ваши оценки появились в конце. Ваш логин нуждается в `events:read`, плюс `evaluations:read` для этой окончательной проверки. Как и в случае с CLI skill, он **не может** завершить отправленный по почте вход с одноразовым кодом за вас. -2. **Место для жизни оценки.** Она строится в образ и работает как долгоживущий сервис, поэтому ей нужно настоящее хранилище, а не временный файл. Оценки часто живут в собственном хранилище, отдельно от оцениваемого агента — навык ищет существующее и спрашивает перед построением нового. -3. **Колесо `agenteye-evaluator` SDK** — прочитайте следующий раздел перед тем, как ваш агент начнёт вводить команды `pip`. - ---- - -## Где это получить - -Навык опубликован в публичной коллекции навыков Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -Хранилище публичное и навыку не нужно никаких собственных учётных данных — он только управляет `agenteye` CLI с сеансом, на который *вы* подключились, и пишет код в *ваше* хранилище. Обратите внимание, что он поставляется как собственная папка и **не** находится внутри пакета `pipx install agenteye`, поэтому не ищите его там. - -## Установка навыка - -Самый быстрый способ — это CLI [`skills`](https://skills.sh), которая загружает папку и помещает её туда, где ваш агент ищет: - -```bash -# Claude Code, только этот проект -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# каждый проект (устанавливает в ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# вместо этого Codex -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -Затем управляйте ею как любым другим навыком: - -```bash -npx skills list -a claude-code # что установлено -npx skills update agenteye-evaluator # получить последнюю версию -npx skills remove agenteye-evaluator # удалить -``` - -Предпочитаете установить вручную? Agent Skill — это просто папка, содержащая `SKILL.md` (плюс опциональные ссылки), поэтому копирование тоже работает: - -- **Claude Code**: поместите папку `agenteye-evaluator/` в `~/.claude/skills/` (каждый проект) или `/.claude/skills/` (только это хранилище). Claude Code автоматически её обнаруживает — проверьте с помощью списка `/skills` или просто попросите оценки. -- **Codex (OpenAI)**: Codex читает тот же `SKILL.md`. Включённый `agents/openai.yaml` устанавливает `allow_implicit_invocation: true`, поэтому Codex автоматически выбирает навык при совпадении задачи; иначе вызовите его явно как `$agenteye-evaluator`. - ---- - -## SDK не на публичном PyPI - -> **Предупреждение:** прочитайте это перед тем, как позволить агенту установить SDK. - -Навык публичный; SDK, который он управляет — нет. `agenteye-evaluator` поставляется только как приватный артефакт выпуска, и в отличие от `agenteye`, имя **не заявлено на публичном PyPI** — поэтому голый `pip install agenteye-evaluator` может загрузить пакет незнакомца в сервис, который читает ваши производственные расшифровки. Это проблема цепочки поставок, а не опечатка. - -Навык знает это и работает по лестнице установки, останавливаясь на первой применимой ступени: исходный код монорепозитория, если вы внутри репозитория AgentEye, иначе приватное колесо выпуска из GitHub Releases (нужен доступ), и если ни одно не достижимо, он **останавливается и говорит вам попросить колесо у вашего контакта Failproof AI** вместо импровизации. - -Поэтому если ваш агент предлагает голый `pip install agenteye-evaluator` с публичного PyPI, это признак того, что навык никогда не загружался. Остановитесь там и проверьте, что он установлен. - ---- - -## Что вы можете просить - -Настоящий полный цикл начинается с нечёткого запроса и заканчивается подписанным дизайном, а не кодом: - -```text -вы ▸ Мне нужна оценка для нашего бота поддержки. Я думаю, что - иногда он работает плохо. - -агент ▸ Перед тем, как писать: опишите сеанс, который прошёл - хорошо, и один, который прошёл плохо. - -вы ▸ Хорошо: он отвечает из документов. Плохо: он выдумывает - или сдаётся и говорит «свяжитесь с поддержкой». - -агент ▸ Запуск: agenteye --json sessions --status error --since 7d - Чтение 5 сеансов, включая два, которые вы назовёте плохими. - - Ваши слова говорят о точности фактов и отказах. - Расшифровки согласны на отказы (2 из 5 заканчиваются - переводом на поддержку, и это видно в model_response.content), - но точность фактов нужна истина, которую ваши события не содержат. - - Поэтому я буду оценивать три вещи: - - deflection правила, дёшево, разделяет ваши два сеанса - - tool_efficiency правила, ловят цикл переповтора в run-004 - - groundedness судья LLM: каждое ли утверждение в загруженных - документах? - - «точность фактов» потребовала бы истину, которой у нас нет; - groundedness — это ближайшее, что ваши расшифровки могут - действительно поддержать. Хорошо с этими тремя названиями? - Переименование позже разбивает историю. - -вы ▸ да - -агент ▸ Нет оценки в этом хранилище. Мне построить здесь новую - или у вас она есть где-то ещё? -``` - -Отсюда он пишет правила-измерения сначала (бесплатные, мгновенные, детерминированные), тестирует их против реального захваченного сеанса, включая пустые и никогда не завершённые, которые ломают наивные оценки, и только затем обращается к судье LLM для субъективного измерения. Он знает [ограничения диспетчера](/ru/agenteye/evaluation-suite#configuring-the-server) — 30-секундный таймаут запроса и 8 одновременных вызовов в развёртывании — поэтому если судья не поместится надёжно, он идёт асинхронно с `JobPending` вместо того, чтобы позволить вашему судье быть отменённым и переправленным пять раз в пять раз дороже. - -Затем он развёртывает, устанавливает две переменные окружения сервера и подтверждает с помощью `agenteye --json evals --session-id `, что оценки действительно появились. Появление оценок — единственное доказательство. - ---- - -## На что обратить внимание - -- **Названия измерений почти постоянны.** Ключи оценок — произвольные строки, и платформа тренирует всё, что вы отправляете, что означает, что ничто ниже не исправляет плохой выбор. Переименование позже и история разбивается: старые сеансы хранят старый ключ и тренд разбивается. Вот почему навык получает явное одобрение перед написанием кода — отнеситесь к этому приглашению серьёзно. -- **Фиксации — настоящие производственные расшифровки.** Проектирование против реальных сеансов означает их загрузку на диск, и они могут содержать данные клиентов. Навык спрашивает перед фиксацией в git; если сомневаетесь, держите `fixtures/` вне хранилища и попросите каждого разработчика загрузить свои собственные. -- **Агент пишет и развёртывает сервис, который читает каждую расшифровку.** Он действует от вашего имени, ограниченный разрешениями логина вашего CLI, но просмотрите оценку как любой другой код, который касается производственных данных. - ---- - -## Следующие шаги - -- **[Evaluation suite](/ru/agenteye/evaluation-suite)**: HTTP контракт, SDK и переменные окружения сервера, которые навык конфигурирует. -- **[Evaluations](/ru/agenteye/evaluations)**: где оценки показываются, когда они появляются. -- **[CLI skill](/ru/agenteye/cli-skill)**: родственный навык для чтения результатов вместо построения скорера. -- **[CLI](/ru/agenteye/cli)**: справочник команд за данными сеансов, против которых навык проектирует. \ No newline at end of file diff --git a/docs/ru/agenteye/hermes-capture.mdx b/docs/ru/agenteye/hermes-capture.mdx deleted file mode 100644 index 6a63a423..00000000 --- a/docs/ru/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Захват сессий Hermes" -description: "Переносите сессии Hermes gateway вашей команды — Slack, Telegram, CLI и запланированные запуски — в AgentEye как обычные сессии и события." ---- - -[Hermes](https://hermes-agent.nousresearch.com) отвечает вашей команде из любого места, где она уже работает — Slack, Telegram, CLI, запланированные запуски. Захват сессий Hermes переносит всё это в AgentEye как обычные сессии и события, поэтому помощник, с которым ваша команда общается каждый день, становится таким же наблюдаемым, как агенты, которых вы пишете сами. - -Небольшой фоновый сборщик читает локальное хранилище сессий Hermes по мере его обновления и отправляет сессии в AgentEye. Он работает так же, как захват [Codex](/ru/agenteye/codex-capture) и [OpenClaw](/ru/agenteye/openclaw-capture), и один сборщик может одновременно захватывать несколько сессий. - ---- - -## Что захватывается - -Каждая сессия Hermes на машине захватывается, независимо от канала, с которого она пришла. Каждая становится [сессией](/ru/agenteye/sessions) AgentEye; её сообщения пользователя и ассистента, вызовы инструментов и результаты инструментов становятся соответствующими [событиями](/ru/agenteye/event-stream). - -Канал, с которого началась сессия — Slack, Telegram, CLI или запланированный запуск — записывается в сессию, поэтому вы можете их различить и фильтровать по одному. Вместе с этим фиксируются модель, на которой выполнялась сессия, чат и человек, от которого она была запущена, и, когда сессия порождала другую, ссылка на родительскую сессию. - -Сессии появляются сразу же, когда Hermes их запускает, независимо от того, что-то ли в них было написано или нет, и ответ хода и его вызовы инструментов остаются в порядке, в котором они фактически произошли. Когда сессия завершается, вы также получаете причину завершения, её стоимость и количество использованных токенов. - ---- - -## Включение - -Захват отключен по умолчанию. Установите сборщик с API ключом, который имеет разрешение `events:add` (см. [API ключи](/ru/agenteye/api-keys)), и включите захват Hermes: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -Это установит сборщик, зарегистрирует его как фоновый сервис и начнёт захват. Убедитесь, что он работает: - -```bash -agenteye-collector health -``` - -Захватываете более одного агента на одной машине? Добавьте флаг каждого к одной команде — например `--hermes-enabled --codex-enabled`. - -При первом запуске ваши существующие сессии Hermes заполняются один раз, а новая активность затем потоком поступает в течение нескольких секунд. Данные самого Hermes только читаются — никогда не изменяются и не удаляются — и каждое сообщение отправляется один раз, даже при перезагрузках. - -`health` также показывает, всё ли, что сборщик захватил, фактически достигло AgentEye. Если пакет не удалось доставить, он сохраняется и повторяется попытка, а не отбрасывается, и проверка сообщает о неполадках, пока что-то ещё ожидает обработки — поэтому "healthy" означает, что ваши данные прибыли, а не просто что процесс живой. - ---- - -## Где это отображается - -Захваченные сессии появляются в разделе **Sessions**, а их события в потоке **Events**, так же как любой другой наблюдаемый вами агент — поэтому [воспроизведение сессии](/ru/agenteye/sessions), [поиск](/ru/agenteye/queries), [оценки](/ru/agenteye/evaluations) и [оповещения](/ru/agenteye/alerts) работают на них. Отфильтруйте по агенту Hermes, чтобы видеть их отдельно. - ---- - -## Конфиденциальность - -Сессии Hermes содержат полный транскрипт — включая вывод команд, содержимое файлов и всё, что агент читал или писал — и могут содержать секреты. Захваченные сессии отправляются как есть, поэтому включайте захват только там, где централизация этого содержимого в AgentEye уместна, и выдайте сборщику ключ, ограниченный только разрешением `events:add`. См. [Безопасность](/ru/agenteye/security), чтобы узнать, как ваши данные хранятся отдельно. \ No newline at end of file diff --git a/docs/ru/agenteye/observability.mdx b/docs/ru/agenteye/observability.mdx deleted file mode 100644 index acc5b7fd..00000000 --- a/docs/ru/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Observe" -description: "Surfaces наблюдения — это место, где вы видите, что делают ваши агенты прямо сейчас и анализируете любой отдельный запуск." ---- - - -Surfaces наблюдения — это место, где вы видите, что делают ваши агенты прямо сейчас и анализируете любой отдельный запуск. Все данные здесь поступают в реальном времени, ограничены областью вашей организации и отфильтрованы по диапазону дат, окружению, агенту и сеансу, поэтому вы переходите от «что-то не так» к точному запуску за секунды. - -![Live Event Stream с цветовой кодировкой по типам и фильтрацией по окружению, агенту и сеансу](/agenteye/images/events-stream.png) - -Четыре surface, каждый со своей страницей: - -- **[Event stream](/ru/agenteye/event-stream)**: live хронология каждого шага каждого запуска на всех агентах, новейшие сначала. Главная страница вашей организации и первая остановка для триажа. -- **[Sessions and execution graph](/ru/agenteye/sessions)**: эти события свернуты в одну строку на запуск плюс картина в стиле git того, как каждый запуск развивался. -- **[Performance metrics](/ru/agenteye/telemetry)**: heat-maps задержки и p50/p95/p99 показатели для ваших моделей, инструментов и hooks, чтобы всплески на хвосте отличались от медианы. -- **[Error tracking](/ru/agenteye/error-tracking)**: единый surface триажа для всего, что пошло не так, одним кликом от срабатывающего оповещения к запуску, который сломался. - -## Связанное - -- [Evaluations](/ru/agenteye/evaluations): оценка каждого запуска по качеству. -- [Alerts](/ru/agenteye/alerts): превратите любой порог в правило повызова. -- [Audits](/ru/agenteye/audits): позвольте Failproof AI Observability найти для вас закономерности отказов во всех сеансах. -- [CLI and agents](/ru/agenteye/cli-and-agents): та же наблюдаемость из вашего терминала. \ No newline at end of file diff --git a/docs/ru/agenteye/openclaw-capture.mdx b/docs/ru/agenteye/openclaw-capture.mdx deleted file mode 100644 index 4604477a..00000000 --- a/docs/ru/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "Захват сеансов OpenClaw" -description: "Собирайте локальные сеансы OpenClaw вашей команды в AgentEye как обычные сеансы и события — без изменения способа работы OpenClaw." ---- - -Если ваша команда использует [OpenClaw](https://docs.openclaw.ai), захват сеансов OpenClaw переносит эти сеансы в AgentEye как обычные сеансы и события, так что вы можете искать, воспроизводить и оценивать их наряду со всем остальным, что вы наблюдаете. Это дополнение к [Python SDK](/ru/agenteye/python-sdk): SDK инструментирует агентов, которых вы пишете, а захват собирает работу OpenClaw, которую ваша команда уже выполняет — без каких-либо изменений в способе её запуска. - -Небольшой фоновый сборщик читает локальные расшифровки сеансов OpenClaw по мере их записи и отправляет их в AgentEye. Он работает так же, как [захват Codex](/ru/agenteye/codex-capture), и один сборщик может захватывать оба одновременно. - ---- - -## Что захватывается - -Каждый агент, настроенный в локальной установке OpenClaw машины, захватывается сборщиком этой машины — не требуется настройка для каждого агента отдельно. - -Каждый сеанс OpenClaw становится [сеансом](/ru/agenteye/sessions) AgentEye; его сообщения пользователя и ассистента, вызовы инструментов и результаты инструментов становятся соответствующими [событиями](/ru/agenteye/event-stream). - ---- - -## Включение захвата - -Захват отключен до тех пор, пока вы его не включите. Установите сборщик с API ключом, имеющим разрешение `events:add` (см. [API ключи](/ru/agenteye/api-keys)), и включите захват OpenClaw: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -Это установит сборщик, зарегистрирует его как фоновый сервис и начнёт захват. Убедитесь, что он запущен: - -```bash -agenteye-collector health -``` - -Захватываете более одного агента на одной машине? Добавьте флаг каждого в одну команду — например `--openclaw-enabled --codex-enabled`. - -При первом запуске ваши существующие сеансы OpenClaw будут загружены задним числом один раз, а новая активность будет поступать в течение нескольких секунд. Файлы OpenClaw только читаются — никогда не изменяются, не перемещаются и не удаляются — и каждый сеанс отправляется ровно один раз, даже при перезагрузках. - ---- - -## Где это появляется - -Захваченные сеансы появляются в **Sessions**, а их события в потоке **Events**, так же как любой другой наблюдаемый агент — поэтому [воспроизведение сеансов](/ru/agenteye/sessions), [поиск](/ru/agenteye/queries), [оценки](/ru/agenteye/evaluations) и [оповещения](/ru/agenteye/alerts) работают на них. Отфильтруйте по агенту OpenClaw, чтобы увидеть их отдельно. - ---- - -## Приватность - -Расшифровки OpenClaw содержат полный сеанс — включая выходные данные команд, содержимое файлов и всё, что агент прочитал или написал — и могут содержать секреты. Захваченные сеансы отправляются как есть, поэтому включайте захват только на машинах и для команд, где централизация этого контента в AgentEye уместна, и выдайте сборщику ключ с областью действия только `events:add`. См. [Security](/ru/agenteye/security) для информации о том, как ваши данные остаются изолированными. \ No newline at end of file diff --git a/docs/ru/agenteye/python-sdk-skill.mdx b/docs/ru/agenteye/python-sdk-skill.mdx deleted file mode 100644 index f6372430..00000000 --- a/docs/ru/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,131 +0,0 @@ ---- -title: "Failproof AI Observability Python SDK Agent Skill" -description: "Переход от неинструментированного агента к событиям, которые вы можете видеть, с кодирующим агентом, находящим точки инструментирования, написанием их и проверкой их внедрения." ---- - -Скажите своему кодирующему агенту *"добавь Failproof AI Observability к этому агенту"* и позвольте ему прочитать ваш цикл, определить, где должна быть инструментировка, написать её и проверить события перед завершением работы. - -**Python SDK skill** (`agenteye-python-sdk`) — это *Agent Skill*: папка инструкций, которую кодирующий агент, такой как Claude Code или Codex, загружает по требованию при совпадении задачи. Она обучает агента использованию [Python SDK](/ru/agenteye/python-sdk) — это не библиотека и не меняет ничего в том, как работает SDK. - -## Инструментировка легко написать и легко сделать неправильно незаметно - -SDK небольшой: тринадцать методов событий, все только с ключевыми параметрами. Кодирующий агент может прочитать справочник [Python SDK](/ru/agenteye/python-sdk) и создать приемлемую инструментировку за минуту. - -Загвоздка в том, что этот SDK не выдаёт ошибку, когда вы ошибаетесь, и неправильная инструментировка выглядит точно так же, как правильная, пока кто-то не откроет панель и не обнаружит пустоту. Ошибки, которые требуют реального времени для исправления — это все молчанцы: - -| Ошибка | Что вы видите | -|---|---| -| Нет `agent_start` | Все события приходят. Нулевых сессий. | -| Окружение никогда не установлено | Всё работает, заархивировано под `dev`. | -| `outcome="failure"` | Запуск показывает зелень — только `failed`, `error`, `timeout`, `rejected` учитываются. | -| Опечатка в имени поля | Принято и сохранено как новое поле. | -| События, испущенные из пула потоков | Молча отброшены. | - -Никакая из них не выдаёт ошибку. Ни одна не появляется в тестах. Каждая есть в skill, установленная как контракт с проверкой, которая её ловит. - -## Что она делает, по порядку - -Skill выполняет те же три шага, которые выполнил бы аккуратный инженер: - -1. **План.** Он читает цикл вашего агента и задаёт два вопроса, на которые может ответить только вы: что считается одним запуском (ваш `session_id`) и кто различимые участники (ваш `agent_id`). Он получает согласие перед написанием кода, потому что изменение их позже разделяет вашу историю и ломает тренды. -2. **Написать.** Он связывает идентичность один раз за запуск, а не проводит её через каждый вызов, и выбирает форму, безопасную для одновременности — деталь, которая имеет значение, потому что очевидный ярлык молча смешивает два перекрывающихся запуска в одну сессию. -3. **Проверить.** Он запускает ваш агент и читает полученные файлы событий, проверяя наличие `agent_start`, правильность окружения и то, что один запуск произвёл одну сессию. - -Третий шаг — это тот, который люди пропускают. SDK записывает события в локальные файлы, поэтому полную интеграцию можно доказать на ноутбуке без сервера, без API ключа и без сети — что именно почему skill настаивает на этом. - -## Как это соотносится с другими skills - -Три skills, один чистый разделение: - -| Skill | Используйте его когда | Что он трогает | -|---|---|---| -| **Python SDK skill** (эта страница) | Вы хотите, чтобы ваш агент *выдавал* телеметрию — "добавить наблюдаемость", "почему мой агент не показывается?" | Пишет код в репо вашего агента. Ничего не читает. | -| **[Evaluator skill](/ru/agenteye/evaluator-skill)** | Вы хотите *оценить* запуски — "что нам вообще измерять?" | Пишет код в вашу репо; читает телеметрию | -| **[CLI skill](/ru/agenteye/cli-skill)** | Вы хотите *прочитать* что случилось, или управлять вашим развёртыванием | Управляет CLI как вы, включая изменения | - -Они передают друг другу в этом порядке: этот skill запускает поток событий, эвалюатор их оценивает, CLI читает их обратно. Нечего оценивать и нечего читать, пока ваш агент не выдаёт сессии, поэтому если вы начинаете с нуля, начните отсюда. - -## Предусловия - -1. **Python 3.10+** и кодовая база агента, которую вы хотите инструментировать. -2. **SDK.** Он распространяется среди клиентов как приватное колесо вместо общедоступного индекса — ваш онбординг охватывает как его получить и установить. Skill знает путь установки и попросит вас, если не сможет его найти. -3. **Ничего больше.** Нет входа на панель, нет API ключа, нет сети. Skill проверяет по файлам событий, которые пишет SDK, поэтому может завершиться и доказать свою работу оффлайн. - -## Где его получить - -Skill находится в общей коллекции [`FailproofAI/skills`](https://github.com/FailproofAI/skills): - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -Добавьте `-g` для установки для каждого проекта вместо только текущего, и `--copy` если ваше окружение не следует симлинкам. Для Codex передайте `-a codex`. - -## Установка вручную - -Agent Skills — это папки, содержащие `SKILL.md` плюс справки. Если вы предпочитаете не использовать установщик: - -- **Claude Code**: скопируйте папку `agenteye-python-sdk/` в `~/.claude/skills/` (каждый проект) или `/.claude/skills/` (только этот репо). Claude Code автоматически её обнаружит — проверьте список `/skills` или просто спросите что-нибудь, что на неё совпадает. -- **Codex**: Codex читает то же самое `SKILL.md`. Bundled `agents/openai.yaml` устанавливает `allow_implicit_invocation: true`, поэтому она автоматически выбирается при совпадении задачи; иначе вызовите как `$agenteye-python-sdk`. - -Запустите ваш агент **в репозитории, содержащем код, который вы хотите инструментировать** — skill читает цикл вашего агента перед тем как что-либо предложить. - -## Как выглядит сессия - -```text -you ▸ Add Failproof AI Observability to this agent. - -agent ▸ Read your loop — one tool dispatcher in tools.py, one Anthropic - wrapper in llm.py. Two questions before I write anything. - - What counts as one run? You handle requests concurrently, so I'd - use the request id you already have in ctx.request_id — then a - session here and a request in your logs are the same string. - - And who are the actors? I see a planner path and a researcher - path. Label them separately, or is that one agent to you? - -you ▸ Request id is right. They're one agent, call it "assistant". - -agent ▸ Done. Wrapped the dispatcher and the LLM client; agent_start and - agent_end bracket handle_request, including the error path. Used - a context-local for identity — your dispatcher runs tools in a - thread pool, and the simpler approach would have mixed concurrent - runs into one session without complaining. - - Verified against a real run: 11 events across 4 files — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, one session - per request, durations paired. - - One thing I did NOT do: your worker pool in batch.py submits - straight to the executor, so events from there would be dropped. - Want me to fix those too? -``` - -Закономерность, на которую стоит обратить внимание: он прочитал код перед предложением, задал только вопросы, на которые вы можете ответить, переиспользовал уже имеющийся id, выбрал форму, безопасную для одновременности *потому что* увидел пул потоков, и **проверил путём чтения фактических событий** вместо объявления успеха — затем отметил единственное место, которое, как он знал, молча провалилось. - -## Что вы можете у неё спросить - -- *"Почему мой агент не показывается на панели?"* → прочитает лестницу: записываются ли события, есть ли там `agent_start`, правильно ли окружение, читает ли сборщик из того же места. -- *"Всё приходит под dev."* → окружение никогда не было установлено, или было сброшено позже. -- *"Добавь отслеживание токенов."* → находит ваш LLM wrapper и записывает модель, причину остановки и использование. -- *"Инструментируй субагентов тоже."* → одна сессия, различные метки агентов, вложенные под своим родителем. -- *"Напиши тесты для инструментировки."* → указывает SDK на временную директорию и проверяет события, которые она написала. - -## На что нужно обратить внимание - -**Позвольте ей проверить.** Шаг, который делает этот skill стоящим использования — последний — запуск вашего агента и чтение событий обратно. Агент, который пишет инструментировку и останавливается, выполнил лёгкую половину, а половину, которая молча падает, другую. - -**Согласуйте имена перед кодом.** `session_id` и `agent_id` — оси, по которым каждая поверхность группирует. Переименование их позже разделяет историю: старые запуски сохраняют старые метки и ваши тренды ломаются. Skill попросит; ответ стоит минуты размышления. - -**Если ваш агент предлагает установить SDK из общедоступного индекса, skill не загрузился.** SDK распространяется приватно. Это предложение — надёжный признак того, что ваш кодирующий агент угадывает вместо следования skill — остановите его там и проверьте что skill установлен. - -Кроме того его радиус взрыва небольшой: он пишет код в вашу рабочую директорию и файлы событий, где вы ему скажете. Он ничего не читает из вашего развёртывания и не меняет о нём ничего. - -## Следующие шаги - -- **[Python SDK](/ru/agenteye/python-sdk)**: полная справка по событиям — каждый тип события и поле — стоящая за этим что automation этого skill. -- **[Sessions](/ru/agenteye/sessions)**: что производит ваша инструментировка как только события приходят. -- **[Evaluator Agent Skill](/ru/agenteye/evaluator-skill)**: следующий шаг как только запуски приходят — их оценка. -- **[CLI Agent Skill](/ru/agenteye/cli-skill)**: чтение вашей телеметрии обратно. \ No newline at end of file diff --git a/docs/ru/cli/audit.mdx b/docs/ru/audit.mdx similarity index 100% rename from docs/ru/cli/audit.mdx rename to docs/ru/audit.mdx diff --git a/docs/ru/cli/backfill.mdx b/docs/ru/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/ru/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/ru/cli/config.mdx b/docs/ru/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/ru/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/ru/cli/flush.mdx b/docs/ru/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/ru/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/ru/cli/harness.mdx b/docs/ru/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/ru/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/ru/cli/migrate.mdx b/docs/ru/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/ru/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/ru/cli/uninstall.mdx b/docs/ru/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/ru/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/ru/cli/update.mdx b/docs/ru/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/ru/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/ru/agenteye/api-keys.mdx b/docs/ru/cloud/access.mdx similarity index 96% rename from docs/ru/agenteye/api-keys.mdx rename to docs/ru/cloud/access.mdx index 8051a6b6..045a685d 100644 --- a/docs/ru/agenteye/api-keys.mdx +++ b/docs/ru/cloud/access.mdx @@ -1,10 +1,10 @@ --- title: "API ключи" -description: "API ключи контролируют, кто и что может получить доступ к вашему серверу Failproof AI Observability, позволяя коллектору отправлять события без предоставления прав на чтение или администрирование." +description: "API ключи контролируют, кто и что может получить доступ к вашему серверу FailproofAI Cloud, позволяя коллектору отправлять события без предоставления прав на чтение или администрирование." --- -API ключи контролируют, кто и что может получить доступ к вашему серверу Failproof AI Observability, позволяя коллектору отправлять события без предоставления прав на чтение или администрирование. Каждый ключ имеет одно или несколько разрешений, и каждое разрешение ограничивает доступ к определённым маршрутам сервера; вы даёте только те разрешения, которые необходимы для работы. В большинстве развёртываний требуется всего три типа ключей. +API ключи контролируют, кто и что может получить доступ к вашему серверу FailproofAI Cloud, позволяя коллектору отправлять события без предоставления прав на чтение или администрирование. Каждый ключ имеет одно или несколько разрешений, и каждое разрешение ограничивает доступ к определённым маршрутам сервера; вы даёте только те разрешения, которые необходимы для работы. В большинстве развёртываний требуется всего три типа ключей. ## Три ключа, необходимые большинству развёртываний @@ -82,7 +82,7 @@ API ключи контролируют, кто и что может получ Эти разрешения поддерживают страницу панели управления **Пользователи**, где предоставленные области действия каждого участника отображаются в виде чипов: -![Страница Пользователи: карточка на каждого пользователя панели управления с его электронной почтой, предоставленными разрешениями и элементами управления редактированием/отключением](/agenteye/images/users.png) +![Страница Пользователи: карточка на каждого пользователя панели управления с его электронной почтой, предоставленными разрешениями и элементами управления редактированием/отключением](/cloud/images/users.png) ### Операционные параметры @@ -91,7 +91,7 @@ API ключи контролируют, кто и что может получ | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | Просмотр операционных параметров, управляемых панелью управления, и их метаданных; список переопределений окна контекста для каждой модели; и разрешение эффективного окна для модели. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | Редактирование операционных параметров и добавление, изменение или удаление переопределений окна контекста для каждой модели. Изменения влияют на новые события без перезагрузки сервера. | -![Страница параметров: операционные параметры, управляемые панелью управления, такие как разрешённые входы и время жизни сеанса/OTP, редактируемые без перезагрузки](/agenteye/images/settings.png) +![Страница параметров: операционные параметры, управляемые панелью управления, такие как разрешённые входы и время жизни сеанса/OTP, редактируемые без перезагрузки](/cloud/images/settings.png) ### Оповещения и инциденты @@ -247,7 +247,7 @@ curl -s -X POST http://your-server/keys//regenerate \ При создании ключа с панели управления вы не предоставляете секрет; панель управления генерирует сильный секрет для вас и отображает его **один раз** при создании. Скопируйте его немедленно и храните безопасно; он никогда не будет показан снова, точно как при восстановлении. Вы всё ещё можете выбрать разрешения ключа непосредственно или инициализировать их из набора разрешений (смотрите ниже). -![Страница API ключей: карточка на каждый ключ с его именем, предоставленными разрешениями и временем создания, с действиями восстановления и отключения; защищённые ключи, такие как `admin`, отмечены](/agenteye/images/api-keys.png) +![Страница API ключей: карточка на каждый ключ с его именем, предоставленными разрешениями и временем создания, с действиями восстановления и отключения; защищённые ключи, такие как `admin`, отмечены](/cloud/images/api-keys.png) --- @@ -276,5 +276,5 @@ curl -s -X POST http://your-server/keys//regenerate \ ## Следующие шаги -- [Python SDK](/ru/agenteye/python-sdk): как ваш код агента аутентифицируется при отправке событий. -- [Безопасность](/ru/agenteye/security): как работают вход, контроль доступа и изоляция данных для каждой организации. \ No newline at end of file +- [Python SDK](/ru/cloud/sdk): как ваш код агента аутентифицируется при отправке событий. +- [Безопасность](/ru/cloud/security): как работают вход, контроль доступа и изоляция данных для каждой организации. \ No newline at end of file diff --git a/docs/ru/cloud/agent-skills.mdx b/docs/ru/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/ru/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
    loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/ru/agenteye/alerts.mdx b/docs/ru/cloud/alerts.mdx similarity index 79% rename from docs/ru/agenteye/alerts.mdx rename to docs/ru/cloud/alerts.mdx index 4126172f..1370bef4 100644 --- a/docs/ru/agenteye/alerts.mdx +++ b/docs/ru/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "Узнайте о проблеме в тот момент, ког --- -Узнайте о проблеме в тот момент, когда она возникает, в канале, который уже смотрит ваша команда, вместо того чтобы услышать об этом от клиента. Установите правило один раз, и Failproof AI Observability будет проверять его по расписанию, затем отправит вам уведомление по электронной почте, Slack, webhook или прямо в панель управления. +Узнайте о проблеме в тот момент, когда она возникает, в канале, который уже смотрит ваша команда, вместо того чтобы услышать об этом от клиента. Установите правило один раз, и FailproofAI Cloud будет проверять его по расписанию, затем отправит вам уведомление по электронной почте, Slack, webhook или прямо в панель управления. -![Страница оповещений: сетка карточек правил оповещений, каждая из которых показывает триггер, окно оценки, каналы и значок серьезности (информация, предупреждение или критический уровень)](/agenteye/images/alerts.png) +![Страница оповещений: сетка карточек правил оповещений, каждая из которых показывает триггер, окно оценки, каналы и значок серьезности (информация, предупреждение или критический уровень)](/cloud/images/alerts.png) *Все правила оповещений с первого взгляда: что они контролируют, как часто, куда отправляются уведомления и как срочны.* ## Узнайте о проблемах прежде, чем о них узнают пользователи @@ -22,9 +22,9 @@ description: "Узнайте о проблеме в тот момент, ког ## Создавайте правило в форме, а не в JSON -Вы описываете, что означает «сбой», в форме, а Failproof AI Observability создает базовое правило за вас. JSON спецификация — это просто то, что создает эта форма под капотом, поэтому вы можете его прочитать, чтобы понять правило, но редко вводите его вручную. +Вы описываете, что означает «сбой», в форме, а FailproofAI Cloud создает базовое правило за вас. JSON спецификация — это просто то, что создает эта форма под капотом, поэтому вы можете его прочитать, чтобы понять правило, но редко вводите его вручную. -![Форма нового оповещения: имя и описание, переключатель включения и выбор триггера с предложениями порога метрики, пользовательского SQL, оценки оценивания, составного оценивания и условий для каждого события](/agenteye/images/alert-new.png) +![Форма нового оповещения: имя и описание, переключатель включения и выбор триггера с предложениями порога метрики, пользовательского SQL, оценки оценивания, составного оценивания и условий для каждого события](/cloud/images/alert-new.png) *Выберите триггер и форма заменит нужные поля; нажмите Сохранить.* Быстрый путь прост: дайте имя, выберите **триггер** (что контролировать), установите **пороговое значение и окно** (насколько плохо, в течение какого времени), прикрепите по крайней мере один **канал**, затем **Сохраните** и нажмите **Тест**, чтобы отправить синтетическое уведомление и подтвердить, что все назначения настроены правильно. Под капотом это создает небольшую спецификацию вроде: @@ -43,7 +43,7 @@ description: "Узнайте о проблеме в тот момент, ког | **Составное оценивание** | несколько проверок оценки объединяются логикой any, all или at-least-N, чтобы поймать регрессию, которая проявляется только в разных оценках | | **Для каждого события** | приходит одно соответствующее событие: конкретный агент, конкретный тип ошибки или подстрока сообщения | -Уже смотрите на сбой на [странице Ошибок](/ru/agenteye/error-tracking)? Каждая строка там имеет кнопку **+ оповещение**, которая открывает эту же форму предварительно заполненную, чтобы поймать эту точную ошибку снова, так что инцидент, который вы только что разобрали, станет тем, который вас предупредит в следующий раз. +Уже смотрите на сбой на [странице Ошибок](/ru/cloud/errors)? Каждая строка там имеет кнопку **+ оповещение**, которая открывает эту же форму предварительно заполненную, чтобы поймать эту точную ошибку снова, так что инцидент, который вы только что разобрали, станет тем, который вас предупредит в следующий раз. **Где это найти:** Оповещения находятся по адресу `//alerts`. Создание, редактирование, удаление и тестирование правил требует **`alerts:write`**; `alerts:read` достаточно для просмотра. Выбор получателя показывает членов вашей организации по имени, поэтому вы можете отправить уведомление человеку, не выходя из формы. @@ -53,11 +53,11 @@ description: "Узнайте о проблеме в тот момент, ког ## Что происходит, когда срабатывает оповещение -Нарушение открывает **инцидент** и уведомляет ваши каналы один раз. После этого ваша команда подтверждает его, назначает владельца, обсуждает и разрешает, все с чистой атрибутированной записью. Этот рабочий процесс сортировки имеет свой собственный дом: см. [Инциденты](/ru/agenteye/incidents). +Нарушение открывает **инцидент** и уведомляет ваши каналы один раз. После этого ваша команда подтверждает его, назначает владельца, обсуждает и разрешает, все с чистой атрибутированной записью. Этот рабочий процесс сортировки имеет свой собственный дом: см. [Инциденты](/ru/cloud/incidents). ## Связанное -- [Инциденты](/ru/agenteye/incidents): отслеживайте срабатывающее оповещение от открытия до подтверждения до разрешения. -- [Отслеживание ошибок](/ru/agenteye/error-tracking): группируйте ошибки агентов и повысьте одну до оповещения в один клик. -- [Панели управления](/ru/agenteye/dashboards): смотрите общие доски, из которых берутся пороги, для которых вы устанавливаете оповещения. -- [CLI и агенты](/ru/agenteye/cli-and-agents): создавайте оповещения и подтверждайте инциденты из терминала, или встраивайте их в CI. \ No newline at end of file +- [Инциденты](/ru/cloud/incidents): отслеживайте срабатывающее оповещение от открытия до подтверждения до разрешения. +- [Отслеживание ошибок](/ru/cloud/errors): группируйте ошибки агентов и повысьте одну до оповещения в один клик. +- [Панели управления](/ru/cloud/dashboards): смотрите общие доски, из которых берутся пороги, для которых вы устанавливаете оповещения. +- [CLI и агенты](/ru/cloud/cli): создавайте оповещения и подтверждайте инциденты из терминала, или встраивайте их в CI. \ No newline at end of file diff --git a/docs/ru/agenteye/assistant.mdx b/docs/ru/cloud/assistant.mdx similarity index 86% rename from docs/ru/agenteye/assistant.mdx rename to docs/ru/cloud/assistant.mdx index f03273f6..5075f572 100644 --- a/docs/ru/agenteye/assistant.mdx +++ b/docs/ru/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "Задайте вопрос о данных вашего аген --- -Задайте вопрос о данных вашего агента на простом русском языке и получите ответ со ссылками прямо на источники данных. Не нужно писать SQL, не нужно копаться в дашбордах — помощник **Failproof AI Observability** — это самый быстрый способ для кого угодно в вашей команде получить ответы об агентах. +Задайте вопрос о данных вашего агента на простом русском языке и получите ответ со ссылками прямо на источники данных. Не нужно писать SQL, не нужно копаться в дашбордах — помощник **FailproofAI Cloud** — это самый быстрый способ для кого угодно в вашей команде получить ответы об агентах. -![Помощник Failproof AI Observability отвечает на вопрос на простом английском языке внутри дашборда, показывая активность агентов в реальном времени, разбор использования модели по агентам и выводы, с отображением выполненных запросов](/agenteye/images/assistant.png) +![Помощник FailproofAI Cloud отвечает на вопрос на простом английском языке внутри дашборда, показывая активность агентов в реальном времени, разбор использования модели по агентам и выводы, с отображением выполненных запросов](/cloud/images/assistant.png) *Спросите на простом языке и получите ответ на основе ваших собственных данных. Здесь показано, какие агенты загружены больше всего и какие модели они используют, с отображением выполненных запросов, чтобы вы могли проверить каждую цифру.* Нечего учить. Откройте чат, введите вопрос и переходите по ссылкам, которые он вернёт: @@ -36,7 +36,7 @@ AI: Этот сеанс выполнил 12 шагов с использова На странице **Queries** он делает ещё больше и превращается в автора SQL: опишите нужный вам запрос («показать процент ошибок по агентам за последние 7 дней»), и он выведет SQL прямо в редактор, откроет представление различий, чтобы вы смогли **принять** или **отклонить** изменение перед внедрением. -![Страница Observability Queries и её редактор SQL](/agenteye/images/query-lab.png) +![Страница FailproofAI Cloud Queries и её редактор SQL](/cloud/images/query-lab.png) *Страница Queries: в этом редакторе помощник выводит проект запроса только для чтения, который вы можете принять или отклонить.* Написание SQL через вопросы здесь использует разрешение `queries:run`, то же самое, что за кнопкой **Run** в редакторе. Чат везде остаёт требует `agent:use`. @@ -49,7 +49,7 @@ AI: Этот сеанс выполнил 12 шагов с использова - **Каждое изменение ждёт вашего подтверждения.** Сохранённые запросы и дашборды создаются только после вашего явного клика на «Одобрить», и нет никакой настройки, которая отключит эту защиту. - **Он не может удалять ничего.** Нет инструмента удаления, и помощник не имеет разрешения на удаление. Удаления остаются в ваших руках, в дашборде. - **Он остаётся внутри вашей организации.** Помощник видит только организацию, которую вы сейчас просматриваете. -- **Ваши вопросы остаются вашими.** Запросы и ответы хранятся в вашей собственной базе данных Observability; аналитика продукта записывает только метаданные использования, никогда ваш текст запроса. +- **Ваши вопросы остаются вашими.** Запросы и ответы хранятся в вашей собственной базе данных FailproofAI Cloud; аналитика продукта записывает только метаданные использования, никогда ваш текст запроса. ## Где его найти @@ -57,7 +57,7 @@ AI: Этот сеанс выполнил 12 шагов с использова ## Связанное -- [CLI and agents](/ru/agenteye/cli-and-agents) -- [Queries](/ru/agenteye/queries) -- [Dashboards](/ru/agenteye/dashboards) -- [Evaluation suite](/ru/agenteye/evaluation-suite) \ No newline at end of file +- [CLI and agents](/ru/cloud/cli) +- [Queries](/ru/cloud/queries) +- [Dashboards](/ru/cloud/dashboards) +- [Evaluation suite](/ru/cloud/evaluators) \ No newline at end of file diff --git a/docs/ru/agenteye/audits.mdx b/docs/ru/cloud/audits.mdx similarity index 82% rename from docs/ru/agenteye/audits.mdx rename to docs/ru/cloud/audits.mdx index 5406cc08..a3c87be5 100644 --- a/docs/ru/agenteye/audits.mdx +++ b/docs/ru/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "Audits: ваш автоматический аналитик надёжности" -description: "Failproof AI Observability ищет те сбои, для которых вы никогда не писали правил, и выдаёт вам ранжированный, подкреплённый доказательствами список того, что именно нужно исправить." +description: "FailproofAI Cloud ищет те сбои, для которых вы никогда не писали правил, и выдаёт вам ранжированный, подкреплённый доказательствами список того, что именно нужно исправить." --- -Failproof AI Observability ищет те сбои, для которых вы никогда не писали правил, и выдаёт вам ранжированный, подкреплённый доказательствами список того, что именно нужно исправить. Это как если бы аналитик каждую ночь прочёсывал ваши логи, а утром оставлял краткий список на вашем столе. +FailproofAI Cloud ищет те сбои, для которых вы никогда не писали правил, и выдаёт вам ранжированный, подкреплённый доказательствами список того, что именно нужно исправить. Это как если бы аналитик каждую ночь прочёсывал ваши логи, а утром оставлял краткий список на вашем столе.
    @@ -12,7 +12,7 @@ Failproof AI Observability ищет те сбои, для которых вы н *Двухминутное введение: от запланированного запуска к исправлению, на которое вы можете действовать.* -![Страница Audits: повторяющиеся задачи, которые сканируют ваши сессии в поисках паттернов сбоев, каждая с расписанием и чувствительностью](/agenteye/images/audits.png) +![Страница Audits: повторяющиеся задачи, которые сканируют ваши сессии в поисках паттернов сбоев, каждая с расписанием и чувствительностью](/cloud/images/audits.png) *Каждый аудит — это повторяющаяся задача, которая анализирует ваши сессии и составляет ранжированные, подкреплённые доказательствами рекомендации.* ## Перестаньте гадать, что исправить в следующий раз @@ -48,7 +48,7 @@ Audits находятся в панели управления по адресу ## Связанное -- [Alerts](/ru/agenteye/alerts): получайте уведомление в момент пересечения известного вам порога. -- [Evaluations](/ru/agenteye/evaluations): оценивайте каждый запуск, чтобы регрессии качества всплывали сами. -- [Error tracking](/ru/agenteye/error-tracking): группируйте и отслеживайте ошибки, которые выбрасывают ваши агенты. -- [Incidents](/ru/agenteye/incidents): отслеживайте проблему, которую аудит выявил, вплоть до её исправления. \ No newline at end of file +- [Alerts](/ru/cloud/alerts): получайте уведомление в момент пересечения известного вам порога. +- [Evaluations](/ru/cloud/evaluations): оценивайте каждый запуск, чтобы регрессии качества всплывали сами. +- [Error tracking](/ru/cloud/errors): группируйте и отслеживайте ошибки, которые выбрасывают ваши агенты. +- [Incidents](/ru/cloud/incidents): отслеживайте проблему, которую аудит выявил, вплоть до её исправления. \ No newline at end of file diff --git a/docs/ru/cloud/capture.mdx b/docs/ru/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/ru/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/agenteye/cli-recipes.mdx b/docs/ru/cloud/cli-recipes.mdx similarity index 91% rename from docs/agenteye/cli-recipes.mdx rename to docs/ru/cloud/cli-recipes.mdx index fa854979..00fc130c 100644 --- a/docs/agenteye/cli-recipes.mdx +++ b/docs/ru/cloud/cli-recipes.mdx @@ -4,9 +4,9 @@ description: "Copy-paste query patterns and jq recipes that turn session, event, --- -Pull session, event, and evaluation data (and trigger re-evaluations) straight from a script or coding agent, with clean JSON on stdout that pipes directly into `jq`. These recipes turn Failproof AI Observability's data into something a terminal user or an AI coding agent (Claude Code, Cursor) can query and automate, without clicking through the dashboard. +Pull session, event, and evaluation data (and trigger re-evaluations) straight from a script or coding agent, with clean JSON on stdout that pipes directly into `jq`. These recipes turn FailproofAI Cloud's data into something a terminal user or an AI coding agent (Claude Code, Cursor) can query and automate, without clicking through the dashboard. -The patterns below are copy-paste ready for the Failproof AI Observability CLI (`agenteye`). For installation, authentication, and the full option list see [CLI](/agenteye/cli); run `agenteye -h` or `agenteye -h` for the built-in help. +The patterns below are copy-paste ready for the FailproofAI Cloud CLI (`agenteye`). For installation, authentication, and the full option list see [CLI](/ru/cloud/cli); run `agenteye -h` or `agenteye -h` for the built-in help. ## Golden rules @@ -173,7 +173,7 @@ Each command's `--fields` accepts exactly its own item's field names. The set di ## Next steps -- [CLI](/agenteye/cli): installation, authentication, and the full option reference for every command. -- [CLI agent skill](/agenteye/cli-skill): package these recipes as a skill your coding agent can load. -- [API keys](/agenteye/api-keys): create and scope the keys the CLI, SDK, and collector authenticate with. -- [Python SDK](/agenteye/python-sdk): send events into Failproof AI Observability so there is data for these recipes to query. +- [CLI](/ru/cloud/cli): installation, authentication, and the full option reference for every command. +- [CLI agent skill](/ru/cloud/agent-skills): package these recipes as a skill your coding agent can load. +- [API keys](/ru/cloud/access): create and scope the keys the CLI, SDK, and collector authenticate with. +- [Python SDK](/ru/cloud/sdk): send events into FailproofAI Cloud so there is data for these recipes to query. \ No newline at end of file diff --git a/docs/ru/agenteye/cli.mdx b/docs/ru/cloud/cli.mdx similarity index 91% rename from docs/ru/agenteye/cli.mdx rename to docs/ru/cloud/cli.mdx index 58c0b5df..5e856e78 100644 --- a/docs/ru/agenteye/cli.mdx +++ b/docs/ru/cloud/cli.mdx @@ -1,10 +1,10 @@ --- title: "CLI" -description: "Управляйте всеми функциями Failproof AI Observability из терминала или скрипта: без навигации по веб-интерфейсу." +description: "Управляйте всеми функциями FailproofAI Cloud из терминала или скрипта: без навигации по веб-интерфейсу." --- -Управляйте всеми функциями Failproof AI Observability из терминала или скрипта: без навигации по веб-интерфейсу. CLI `agenteye` позволяет запрашивать ваши данные (сеансы, журналы событий, оценки) и администрировать организацию (API ключи, пользователи, параметры, оповещения, инциденты, сохранённые запросы), поэтому используйте его для автоматизации проверок, интеграции Observability в CI или инспекции продакшена посредством coding agent. Каждая команда поддерживает флаг `--json`, поэтому работает одинаково хорошо как для вас в терминале, так и для coding agent'а (Claude Code, Cursor), выполняющего команду и разбирающего результат. +Управляйте всеми функциями FailproofAI Cloud из терминала или скрипта: без навигации по веб-интерфейсу. CLI `agenteye` позволяет запрашивать ваши данные (сеансы, журналы событий, оценки) и администрировать организацию (API ключи, пользователи, параметры, оповещения, инциденты, сохранённые запросы), поэтому используйте его для автоматизации проверок, интеграции FailproofAI Cloud в CI или инспекции продакшена посредством coding agent. Каждая команда поддерживает флаг `--json`, поэтому работает одинаково хорошо как для вас в терминале, так и для coding agent'а (Claude Code, Cursor), выполняющего команду и разбирающего результат. С одним бинарным файлом вы можете: @@ -69,7 +69,7 @@ agenteye --version agenteye --help ``` -> **Примечание:** Python SDK Failproof AI Observability также использует имя дистрибутива `agenteye`. Установка CLI с помощью `pipx` или `uv tool` (вместо `pip install` в общую virtualenv) предотвращает их конфликт. Простой `pip install agenteye` допустим только если SDK не установлен в той же среде. +> **Примечание:** Python SDK FailproofAI Cloud также использует имя дистрибутива `agenteye`. Установка CLI с помощью `pipx` или `uv tool` (вместо `pip install` в общую virtualenv) предотвращает их конфликт. Простой `pip install agenteye` допустим только если SDK не установлен в той же среде. --- @@ -91,7 +91,7 @@ agenteye logout # revoke the session and clear the stored token `whoami` никогда не выводит ошибку при отсутствующем или истекшем сеансе; вместо этого сообщает `logged_in: false`, поэтому скрипт или агент могут безопасно проверить состояние аутентификации (он всё ещё может выйти с кодом non-zero если не установлен базовый URL или веб-интерфейс недоступен). -**Требования:** ваша электронная почта должна быть разрешена для входа в веб-интерфейс (обратитесь к администратору Failproof AI Observability), и веб-интерфейс должен быть доступен по его базовому URL (см. [Конфигурация](#configuration)). Если вы запросили код и он не приходит, ваша электронная почта вероятно ещё не активирована для доступа к веб-интерфейсу. +**Требования:** ваша электронная почта должна быть разрешена для входа в веб-интерфейс (обратитесь к администратору FailproofAI Cloud), и веб-интерфейс должен быть доступен по его базовому URL (см. [Конфигурация](#configuration)). Если вы запросили код и он не приходит, ваша электронная почта вероятно ещё не активирована для доступа к веб-интерфейсу. --- @@ -338,13 +338,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | Аутентифицирован, но ваш аккаунт не имеет требуемое разрешение (сообщение его указывает) | | 6 | Запрашиваемый ресурс не найден (например неизвестный session или incident id) | -Это делает CLI безопасным для скриптов: coding agent может ветвиться на `4` чтобы попросить вас переаутентифицироваться, или на `5` чтобы вывести отсутствующее разрешение. См. [CLI рецепты для агентов](/ru/agenteye/cli-recipes) для exit-code-handling паттернов и JSON output форм. +Это делает CLI безопасным для скриптов: coding agent может ветвиться на `4` чтобы попросить вас переаутентифицироваться, или на `5` чтобы вывести отсутствующее разрешение. См. [CLI рецепты для агентов](/ru/cloud/cli-recipes) для exit-code-handling паттернов и JSON output форм. --- ## Следующие шаги -- **[CLI рецепты для агентов](/ru/agenteye/cli-recipes)**: copy-paste паттерны запросов, `jq` one-liners, `--fields` проекции, обработка exit-code, и JSON output формы, написанные для coding agents управляющих CLI. -- **[CLI агент скилл](/ru/agenteye/cli-skill)**: упакуйте этот CLI как устанавливаемый Claude Code / Codex *скилл* чтобы coding agent управлял Failproof AI Observability из plain-English запросов. -- **[API ключи](/ru/agenteye/api-keys)**: модель разрешений за `keys create --add …`. -- **[AI помощник](/ru/agenteye/assistant)**: включение помощника на который `agent ask` разговаривает. \ No newline at end of file +- **[CLI рецепты для агентов](/ru/cloud/cli-recipes)**: copy-paste паттерны запросов, `jq` one-liners, `--fields` проекции, обработка exit-code, и JSON output формы, написанные для coding agents управляющих CLI. +- **[CLI агент скилл](/ru/cloud/agent-skills)**: упакуйте этот CLI как устанавливаемый Claude Code / Codex *скилл* чтобы coding agent управлял FailproofAI Cloud из plain-English запросов. +- **[API ключи](/ru/cloud/access)**: модель разрешений за `keys create --add …`. +- **[AI помощник](/ru/cloud/assistant)**: включение помощника на который `agent ask` разговаривает. \ No newline at end of file diff --git a/docs/ru/cloud/connect.mdx b/docs/ru/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/ru/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/ru/agenteye/dashboards.mdx b/docs/ru/cloud/dashboards.mdx similarity index 73% rename from docs/ru/agenteye/dashboards.mdx rename to docs/ru/cloud/dashboards.mdx index 4979fe13..6e8adf18 100644 --- a/docs/ru/agenteye/dashboards.mdx +++ b/docs/ru/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "Превратите ваши живые данные агенто Превратите ваши живые данные агентов в одну общую картину, за которой следит вся команда. Закрепите важные запросы в виде графиков, и каждый сможет увидеть одни и те же цифры с первого взгляда, без повторного выполнения запросов. -![Приборная панель, построенная из сохраненных запросов: линия событий в час, столбчатая диаграмма ошибок по типам, диаграмма площади для задержки и распределение токенов по модели](/agenteye/images/dashboard-fleet.png) +![Приборная панель, построенная из сохраненных запросов: линия событий в час, столбчатая диаграмма ошибок по типам, диаграмма площади для задержки и распределение токенов по модели](/cloud/images/dashboard-fleet.png) *Одна панель, четыре сохраненных запроса: события в час, ошибки по типам, задержка и токены по модели.* @@ -25,15 +25,15 @@ description: "Превратите ваши живые данные агенто ## Закрепляйте уже сохраненные запросы -Каждая плитка начинается как сохраненный запрос. Создайте и сохраните нужный вам запрос в библиотеке [Запросов](/ru/agenteye/queries) (встроенные предустановки плюс ваши собственные, по вашим событиям и оценкам), затем закрепите его на приборной панели как график, который подходит данным: **линия** для тенденций во времени, **столбцы** для сравнения категорий, **площадь** для объема или **круговая диаграмма** для распределения долей. +Каждая плитка начинается как сохраненный запрос. Создайте и сохраните нужный вам запрос в библиотеке [Запросов](/ru/cloud/queries) (встроенные предустановки плюс ваши собственные, по вашим событиям и оценкам), затем закрепите его на приборной панели как график, который подходит данным: **линия** для тенденций во времени, **столбцы** для сравнения категорий, **площадь** для объема или **круговая диаграмма** для распределения долей. Поскольку плитка — это просто ваш сохраненный запрос, отображаемый как график, нечего синхронизировать вручную. Обновите запрос один раз, и каждая приборная панель, которая его использует, обновится тоже. ## Отслеживайте качество, а не просто объем -Объем говорит вам, что агенты заняты. Качество говорит вам, что они действительно выполняют работу. Направьте приборную панель на ваши [оценки качества](/ru/agenteye/evaluations) и получите панель, которая отслеживает, насколько хорошо идут запуски с течением времени, так что регрессия качества появится как провал на графике вместо сюрприза от клиента. +Объем говорит вам, что агенты заняты. Качество говорит вам, что они действительно выполняют работу. Направьте приборную панель на ваши [оценки качества](/ru/cloud/evaluations) и получите панель, которая отслеживает, насколько хорошо идут запуски с течением времени, так что регрессия качества появится как провал на графике вместо сюрприза от клиента. -![Приборная панель, ориентированная на качество, созданная на основе сохраненных запросов оценок](/agenteye/images/dashboard-quality.png) +![Приборная панель, ориентированная на качество, созданная на основе сохраненных запросов оценок](/cloud/images/dashboard-quality.png) *Панель качества держит ваши оценки в центре внимания, прямо рядом с операционными показателями.* @@ -41,6 +41,6 @@ description: "Превратите ваши живые данные агенто ## Связанное -- [Запросы](/ru/agenteye/queries): создавайте и сохраняйте запросы, которые становятся вашими плитками. -- [Оценки](/ru/agenteye/evaluations): оценивайте ваши запуски, чтобы отслеживать качество с течением времени. -- [Оповещения](/ru/agenteye/alerts): превратите пороговое значение любой из этих метрик в уведомление. \ No newline at end of file +- [Запросы](/ru/cloud/queries): создавайте и сохраняйте запросы, которые становятся вашими плитками. +- [Оценки](/ru/cloud/evaluations): оценивайте ваши запуски, чтобы отслеживать качество с течением времени. +- [Оповещения](/ru/cloud/alerts): превратите пороговое значение любой из этих метрик в уведомление. \ No newline at end of file diff --git a/docs/ru/agenteye/error-tracking.mdx b/docs/ru/cloud/errors.mdx similarity index 67% rename from docs/ru/agenteye/error-tracking.mdx rename to docs/ru/cloud/errors.mdx index 938578f2..f0727601 100644 --- a/docs/ru/agenteye/error-tracking.mdx +++ b/docs/ru/cloud/errors.mdx @@ -6,14 +6,14 @@ description: "Просматривайте все сбои ваших агент Просматривайте все сбои ваших агентов в одном месте, сгруппированные так, чтобы множество похожих ошибок отображалось как одна проблема. Вы получаете прямой путь от "что-то красное" к точному запуску, который вызвал сбой, без прокрутки живого потока событий. -![Страница ошибок: гистограмма сбоев во времени над сгруппированными красными строками ошибок, каждая с кнопкой "+ alert" в один клик](/agenteye/images/errors.png) +![Страница ошибок: гистограмма сбоев во времени над сгруппированными красными строками ошибок, каждая с кнопкой "+ alert" в один клик](/cloud/images/errors.png) *Страница ошибок: гистограмма сбоев во времени с повторяющимися сбоями, свёрнутыми в одну строку на инцидент.* ## Каждый сбой уже собран для вас Когда агент ломается, вам не нужно прокручивать живой поток событий в надежде поймать красные строки перед тем, как они исчезнут. Страница **Errors** (Ошибки) собирает это за вас. Она объединяет всё, что приборная панель отметила бы как красное, в одну поверхность для сортировки, так что первое, что вы видите — это что именно ломается, а не где это искать. -И она ловит больше, чем только очевидные сбои. Наряду с явными событиями `error`, Failproof AI Observability выявляет и тихие сбои: любой `tool_result`, `hook_completed` или `agent_end`, в полезной нагрузке которого есть сбой, появляется здесь. Инструмент, вернувший ошибку, или хук, завершившийся неудачно, больше не пройдёт мимо вас просто потому, что ничего не выбросило громкого исключения. +И она ловит больше, чем только очевидные сбои. Наряду с явными событиями `error`, FailproofAI Cloud выявляет и тихие сбои: любой `tool_result`, `hook_completed` или `agent_end`, в полезной нагрузке которого есть сбой, появляется здесь. Инструмент, вернувший ошибку, или хук, завершившийся неудачно, больше не пройдёт мимо вас просто потому, что ничего не выбросило громкого исключения. В верхней части гистограмма отображает ошибки во времени. Один взгляд подскажет вам, это постоянный фоновый поток или всплеск, начавшийся несколько минут назад, так что вы сразу узнаете, стоит ли отвлекаться. @@ -23,19 +23,19 @@ description: "Просматривайте все сбои ваших агент Одна сломанная зависимость может вызвать одну и ту же ошибку сотни раз в минуту. В необработанном виде это стена из практически идентичных строк, которая скрывает единственное, что вам действительно нужно увидеть. -Failproof AI Observability сворачивает повторяющиеся сбои, которые имеют одинаковую сессию и тип ошибки, в одну строку. Всплеск читается как один инцидент. Вы в итоге считаете проблемы, а не строки логов, и сигнал, который имеет значение, остаётся на виду вместо того, чтобы быть захороненным своим собственным объёмом. +FailproofAI Cloud сворачивает повторяющиеся сбои, которые имеют одинаковую сессию и тип ошибки, в одну строку. Всплеск читается как один инцидент. Вы в итоге считаете проблемы, а не строки логов, и сигнал, который имеет значение, остаётся на виду вместо того, чтобы быть захороненным своим собственным объёмом. ## От "что-то красное" к точному событию Нажмите на любую строку, чтобы перейти прямо в сессию этого запуска, позиционированную на точном событии, которое привело к сбою. Никакого копирования ID сессий, никакой прокрутки в поисках момента, когда всё пошло не так: вы окажетесь прямо на нём, с полным графиком выполнения в одном взгляде, чтобы вы могли увидеть, что делал агент в моменты перед тем, как он сломался. -Если у вас есть `alerts:write`, каждая строка также содержит кнопку **+ alert**. Нажмите на неё, и Observability откроет новое правило оповещения, уже заполненное для отлова того же сбоя снова. Инцидент, который вы только что рассортировали, станет тем, который вас оповестит в следующий раз, вместо того чтобы застать вас врасплох дважды. +Если у вас есть `alerts:write`, каждая строка также содержит кнопку **+ alert**. Нажмите на неё, и FailproofAI Cloud откроет новое правило оповещения, уже заполненное для отлова того же сбоя снова. Инцидент, который вы только что рассортировали, станет тем, который вас оповестит в следующий раз, вместо того чтобы застать вас врасплох дважды. **Где это найти:** страница **Errors** находится в разделе observe приборной панели по адресу `//errors`. ## Связанное -- [Alerts](/ru/agenteye/alerts): превратите любой сбой в правило оповещения. -- [Incidents](/ru/agenteye/incidents): отслеживайте срабатывающее оповещение от открытия до разрешения. -- [Sessions](/ru/agenteye/sessions): откройте полный запуск за любой ошибкой. -- [Audits](/ru/agenteye/audits): позвольте Observability найти закономерности в сбоях ваших запусков. \ No newline at end of file +- [Alerts](/ru/cloud/alerts): превратите любой сбой в правило оповещения. +- [Incidents](/ru/cloud/incidents): отслеживайте срабатывающее оповещение от открытия до разрешения. +- [Sessions](/ru/cloud/sessions): откройте полный запуск за любой ошибкой. +- [Audits](/ru/cloud/audits): позвольте FailproofAI Cloud найти закономерности в сбоях ваших запусков. \ No newline at end of file diff --git a/docs/ru/agenteye/evaluations.mdx b/docs/ru/cloud/evaluations.mdx similarity index 70% rename from docs/ru/agenteye/evaluations.mdx rename to docs/ru/cloud/evaluations.mdx index e2b9ccca..3f9c4916 100644 --- a/docs/ru/agenteye/evaluations.mdx +++ b/docs/ru/cloud/evaluations.mdx @@ -4,15 +4,15 @@ description: "Проблемы качества находятся вами се --- -Проблемы качества находятся вами сейчас, а не узнаются из жалоб пользователей. Подключите свой сервис оценки один раз, и Failproof AI Observability автоматически оценит каждый завершённый запуск, поэтому снижение полезности или всплеск галлюцинаций проявится сами по себе, до того как это почувствует клиент. +Проблемы качества находятся вами сейчас, а не узнаются из жалоб пользователей. Подключите свой сервис оценки один раз, и FailproofAI Cloud автоматически оценит каждый завершённый запуск, поэтому снижение полезности или всплеск галлюцинаций проявится сами по себе, до того как это почувствует клиент. -![Сетка сессий с колонкой оценки: каждый запуск содержит статус оценки и цветовые значки полезности, факт-проверяемости и эффективности использования инструментов](/agenteye/images/sessions-list.png) +![Сетка сессий с колонкой оценки: каждый запуск содержит статус оценки и цветовые значки полезности, факт-проверяемости и эффективности использования инструментов](/cloud/images/sessions-list.png) *Каждый запуск в сетке сессий содержит свои оценки; красные, жёлтые и зелёные значки выделяют слабые запуски без необходимости открывать транскрипты.* ## Прекратите выборочную проверку запусков вручную -Раньше вы проверяли вручную несколько запусков и надеялись, что остальные в порядке. Теперь каждая завершённая сессия оценивается в момент завершения по интересующим вас параметрам: полезность, эффективность использования инструментов, факт-проверяемость, безопасность, любые ваши критерии качества. Вы определяете ключи оценки; Failproof AI Observability сохраняет, отслеживает и отображает любую информацию, которую возвращает ваша система оценки. Ни один запуск не остаётся без оценки, и вы перестаёте узнавать о регрессии из тикета поддержки. +Раньше вы проверяли вручную несколько запусков и надеялись, что остальные в порядке. Теперь каждая завершённая сессия оценивается в момент завершения по интересующим вас параметрам: полезность, эффективность использования инструментов, факт-проверяемость, безопасность, любые ваши критерии качества. Вы определяете ключи оценки; FailproofAI Cloud сохраняет, отслеживает и отображает любую информацию, которую возвращает ваша система оценки. Ни один запуск не остаётся без оценки, и вы перестаёте узнавать о регрессии из тикета поддержки. Оценки отображаются в сетке сессий по адресу **`//sessions`** (боковая панель → *observe* → *sessions*), с кластером значков в каждой строке. Хотите только запуски, которые не прошли? Отфильтруйте сетку по диапазону оценок, например полезность ниже 0,5, и вы получите ровно те запуски, которые стоит прочитать. Для просмотра оценок требуется разрешение `evaluations:read`. @@ -20,7 +20,7 @@ description: "Проблемы качества находятся вами се Число говорит вам, что запуск был слабым; страница сессии объясняет почему. Откройте любой запуск, и правая панель показывает краткое резюме, затем полосу для каждого параметра с собственными рассуждениями оценщика под каждой, чтобы вы перешли от «это получило 0,4 за факт-проверяемость» к точному утверждению, в котором ошибка, за секунды. -![Правая панель сессии: сводка оценки вверху, затем полосы оценок для каждого параметра с кратким обоснованием рядом с полной временной шкалой событий](/agenteye/images/session-detail.png) +![Правая панель сессии: сводка оценки вверху, затем полосы оценок для каждого параметра с кратким обоснованием рядом с полной временной шкалой событий](/cloud/images/session-detail.png) *Вид деталей сессии: резюме, полосы оценок для каждого параметра и обоснование каждой оценки прямо рядом с временной шкалой событий запуска.* @@ -30,7 +30,7 @@ description: "Проблемы качества находятся вами се Один запуск с низкой оценкой — это шум; целая группа с понижением — это сигнал. Сохранённые панели превращают ваши оценки в тенденцию, которую вы можете отслеживать с первого взгляда: средняя полезность на этой неделе против прошлой, по агентам, по окружениям. -![Панель качества: столбцы средних оценок для каждого параметра оценки рядом с графиком тренда во времени](/agenteye/images/dashboard-quality.png) +![Панель качества: столбцы средних оценок для каждого параметра оценки рядом с графиком тренда во времени](/cloud/images/dashboard-quality.png) *Сохранённая панель качества отслеживает трендовые ключи оценок, которые вы выбрали, поэтому медленный дрейф становится очевиден задолго до того, как он перейдёт в инцидент.* @@ -38,14 +38,14 @@ description: "Проблемы качества находятся вами се ## Подключите оценщика один раз -Оценка — это опциональный компонент и остаётся полностью отключённой до тех пор, пока вы не укажете Failproof AI Observability адрес оценщика. Вы поднимаете один небольшой HTTP-сервис (в Observability есть работающий эталон, который вы можете скопировать), устанавливаете два значения на вашем сервере, и каждый запуск с этого момента оценивается для вас. Полное пошаговое руководство, контракт оценки и SDK находятся в подробном руководстве. +Оценка — это опциональный компонент и остаётся полностью отключённой до тех пор, пока вы не укажете FailproofAI Cloud адрес оценщика. Вы поднимаете один небольшой HTTP-сервис (в FailproofAI Cloud есть работающий эталон, который вы можете скопировать), устанавливаете два значения на вашем сервере, и каждый запуск с этого момента оценивается для вас. Полное пошаговое руководство, контракт оценки и SDK находятся в подробном руководстве. -Не уверены, какие параметры в принципе стоит оценивать? [Навык агента-оценщика](/ru/agenteye/evaluator-skill) поможет вашему кодирующему агенту разобраться с этим на основе ваших собственных сессий, а затем построить и развернуть сервис. +Не уверены, какие параметры в принципе стоит оценивать? [Навык агента-оценщика](/ru/cloud/agent-skills) поможет вашему кодирующему агенту разобраться с этим на основе ваших собственных сессий, а затем построить и развернуть сервис. ## Связанные разделы -- [Набор оценок](/ru/agenteye/evaluation-suite): подключение оценщика, контракт оценки и SDK. -- [Навык агента-оценщика](/ru/agenteye/evaluator-skill): позвольте кодирующему агенту выбрать параметры оценки и построить оценщик. -- [Сессии](/ru/agenteye/sessions): сетка запусков, где отображаются оценки. -- [Панели](/ru/agenteye/dashboards): сохраняйте и делитесь тенденциями качества в вашей организации. -- [Аудиты](/ru/agenteye/audits): другая автоматическая функция качества Observability для кроссе-сессионных расследований. \ No newline at end of file +- [Набор оценок](/ru/cloud/evaluators): подключение оценщика, контракт оценки и SDK. +- [Навык агента-оценщика](/ru/cloud/agent-skills): позвольте кодирующему агенту выбрать параметры оценки и построить оценщик. +- [Сессии](/ru/cloud/sessions): сетка запусков, где отображаются оценки. +- [Панели](/ru/cloud/dashboards): сохраняйте и делитесь тенденциями качества в вашей организации. +- [Аудиты](/ru/cloud/audits): другая автоматическая функция качества FailproofAI Cloud для кроссе-сессионных расследований. \ No newline at end of file diff --git a/docs/ru/agenteye/evaluation-suite.mdx b/docs/ru/cloud/evaluators.mdx similarity index 89% rename from docs/ru/agenteye/evaluation-suite.mdx rename to docs/ru/cloud/evaluators.mdx index 6e73fe6c..44d8ad6d 100644 --- a/docs/ru/agenteye/evaluation-suite.mdx +++ b/docs/ru/cloud/evaluators.mdx @@ -1,20 +1,20 @@ --- title: "Evaluation Suite" -description: "Failproof AI Observability может автоматически оценивать качество каждого завершённого запуска агента: вы предоставляете небольшой сервис оценки, а Observability берёт на себя остальное." +description: "FailproofAI Cloud может автоматически оценивать качество каждого завершённого запуска агента: вы предоставляете небольшой сервис оценки, а FailproofAI Cloud берёт на себя остальное." --- -Failproof AI Observability может автоматически оценивать качество каждого завершённого запуска агента: вы предоставляете небольшой сервис оценки, а Observability берёт на себя остальное. Используйте её для отслеживания интересующих вас параметров (полезность, эффективность инструментов, фактичность, безопасность — выбираете вы), раннего выявления регрессий и быстрого сравнения агентов или окружений. Оценка является дополнительной функцией: конвейер ничего не делает, пока вы не установите `EVALUATOR_ENDPOINT` на сервере. +FailproofAI Cloud может автоматически оценивать качество каждого завершённого запуска агента: вы предоставляете небольшой сервис оценки, а FailproofAI Cloud берёт на себя остальное. Используйте её для отслеживания интересующих вас параметров (полезность, эффективность инструментов, фактичность, безопасность — выбираете вы), раннего выявления регрессий и быстрого сравнения агентов или окружений. Оценка является дополнительной функцией: конвейер ничего не делает, пока вы не установите `EVALUATOR_ENDPOINT` на сервере. -> **Примечание:** Вы определяете параметры оценки. Ваш оценивающий сервис может возвращать любые числовые ключи; Observability сохраняет, отслеживает и отображает всё, что вы отправляете. +> **Примечание:** Вы определяете параметры оценки. Ваш оценивающий сервис может возвращать любые числовые ключи; FailproofAI Cloud сохраняет, отслеживает и отображает всё, что вы отправляете. ## Кратко -1. **Напишите оценивающий сервис.** Создайте небольшой HTTP-сервис, который читает транскрипт сессии и возвращает оценки. Observability поставляется с рабочим примером, который вы можете скопировать. См. [Написание оценивающего сервиса с SDK](#writing-an-evaluator-with-the-sdk). -2. **Укажите Observability на него.** Установите `EVALUATOR_ENDPOINT` (и общий `EVALUATOR_TOKEN`) на процесс сервера. +1. **Напишите оценивающий сервис.** Создайте небольшой HTTP-сервис, который читает транскрипт сессии и возвращает оценки. FailproofAI Cloud поставляется с рабочим примером, который вы можете скопировать. См. [Написание оценивающего сервиса с SDK](#writing-an-evaluator-with-the-sdk). +2. **Укажите FailproofAI Cloud на него.** Установите `EVALUATOR_ENDPOINT` (и общий `EVALUATOR_TOKEN`) на процесс сервера. 3. **Смотрите, как появляются оценки.** Каждая завершённая сессия автоматически оценивается; результаты отображаются на странице деталей сессии, в сетке сессий и на сохранённых панелях. -![Представление деталей сессии с резюме оценки, полосами оценок по параметрам и текстом обоснования на правой панели](/agenteye/images/session-detail.png) +![Представление деталей сессии с резюме оценки, полосами оценок по параметрам и текстом обоснования на правой панели](/cloud/images/session-detail.png) *После настройки оценивающего сервиса каждый завершённый запуск оценивается, и результаты появляются на правой панели сессии: резюме вверху, затем полосы оценок по параметрам с обоснованием.* @@ -24,7 +24,7 @@ Failproof AI Observability может автоматически оценива ```mermaid flowchart LR - ING["ingest /events
    agent_end"] --> SRV["Observability server"] + ING["ingest /events
    agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -32,26 +32,26 @@ flowchart LR SRV --> RES["evaluations
    terminal results"] ``` -Когда Failproof AI Observability SDK генерирует событие `agent_end` для сессии, сервер +Когда FailproofAI Cloud SDK генерирует событие `agent_end` для сессии, сервер планирует оценку. Затем он отправляет полный транскрипт событий в ваш оценивающий сервис, который может: - **Вернуть результат сразу** с `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`. Результат добавляется в временную линию оценок сессии. `reasoning` и `summary` опциональны. -- **Отложить** с `{"status":"pending", "job_id":"abc-123"}`. Observability затем +- **Отложить** с `{"status":"pending", "job_id":"abc-123"}`. FailproofAI Cloud затем вызывает `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` до тех пор, пока ваш оценивающий сервис не вернёт `{"status":"done", ...}` или `{"status":"error", "error":"..."}`. Интервал опроса зависит от задачи: ответ `pending` может включать - `next_poll_secs` для переопределения; в противном случае Observability использует + `next_poll_secs` для переопределения; в противном случае FailproofAI Cloud использует значение `default_poll_interval_secs` из `GET /config`; если его нет, сервер использует `EVALUATOR_POLLING_INTERVAL_SECS` (по умолчанию 10 сек). Все значения ограничиваются диапазоном [1 сек, 1 ч]. Сессии, которые никогда не генерируют `agent_end` (например, упавший процесс агента), также могут быть обработаны: конфигурация оценивающего сервиса `GET /config` может возвращать -`{"inactivity_timeout_secs": 1800}`, и Observability будет оценивать любую сессию, +`{"inactivity_timeout_secs": 1800}`, и FailproofAI Cloud будет оценивать любую сессию, которая неактивна в течение этого времени. Установите поле в `null` или опустите его, чтобы отключить этот резервный механизм. @@ -74,7 +74,7 @@ flowchart LR Преходящие сбои (5xx, 429, таймауты, сетевые ошибки) повторяются с экспоненциальной задержкой до `EVALUATOR_MAX_ATTEMPTS`; ответы 4xx являются -финальными. Observability безопасно запускается с несколькими горизонтально масштабируемыми экземплярами сервера; +финальными. FailproofAI Cloud безопасно запускается с несколькими горизонтально масштабируемыми экземплярами сервера; работа разбита так, чтобы одна сессия никогда не была отправлена дважды одновременно. @@ -85,7 +85,7 @@ flowchart LR Каждый защищённый маршрут использует **аутентификацию по токену носителя**. Одно и то же значение должно быть настроено с обеих сторон: -- Сервер Observability: переменная окружения `EVALUATOR_TOKEN` +- Сервер FailproofAI Cloud: переменная окружения `EVALUATOR_TOKEN` - Сервис оценки: настроен аналогично (SDK `agenteye-evaluator` по соглашению читает `EVALUATOR_TOKEN`) @@ -169,7 +169,7 @@ flowchart LR предоставляет типизированную обёртку FastAPI, которая обрабатывает аутентификацию, маршрутизацию и формы запроса/ответа для вас. -Failproof AI Observability также поставляется с **рабочим примером оценивающего сервиса**, который +FailproofAI Cloud также поставляется с **рабочим примером оценивающего сервиса**, который оценивает `helpfulness`, `tool_efficiency` и `factuality` на основе формы транскрипта. Скопируйте его как отправную точку и замените вашей собственной логикой: судья LLM, механизм правил, что угодно, соответствующее вашему уровню качества. @@ -196,7 +196,7 @@ def run(req: EvalRequest) -> EvalResponse: Экземпляр `app` работает под любым ASGI сервером, поэтому `uvicorn module:app` его запускает. Для оценивающих сервисов, которым нужно отложить дорогостоящую работу, верните `JobPending` -вместо этого и зарегистрируйте обработчик `@app.job_lookup`; сервер Observability +вместо этого и зарегистрируйте обработчик `@app.job_lookup`; сервер FailproofAI Cloud опрашивает `GET /evaluate/{job_id}` до тех пор, пока вы не вернёте финальный статус или не истечёт лимит `EVALUATOR_MAX_POLL_DURATION_SECS` (по умолчанию 1 ч). @@ -207,7 +207,7 @@ README SDK `agenteye-evaluator`. ## Запуск вашего оценивающего сервиса -Оценивающий сервис — **ваш сервис** — Failproof AI Observability не поставляет +Оценивающий сервис — **ваш сервис** — FailproofAI Cloud не поставляет оценивающий сервис по умолчанию, поэтому вы строите и запускаете его там же, где запускаете ваши сервисы. Он работает под любым ASGI сервером (например `uvicorn my_evaluator:app`); обслуживайте маршруты `/health`, `/config` и `/evaluate` из @@ -326,7 +326,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. оценки каждой сессии и оценки с первого взгляда. - **`/dashboards`**: сохранённые представления здоровья оценок (см. [Dashboards](#dashboards) ниже). -![Сетка Sessions с табличками статуса оценки для каждой сессии и значками оценок с цветовой кодировкой (helpfulness, factuality, tool_efficiency, safety, coherence)](/agenteye/images/sessions-list.png) +![Сетка Sessions с табличками статуса оценки для каждой сессии и значками оценок с цветовой кодировкой (helpfulness, factuality, tool_efficiency, safety, coherence)](/cloud/images/sessions-list.png) *Сетка сессий показывает статус оценки каждого запуска и оценки с первого взгляда; красные/янтарные/зелёные значки выделяют низкие оценки.* @@ -354,7 +354,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. на сервере по всему совпадающему набору (через `GET /evaluations/aggregate`), поэтому числа точные вместо выборки. -![Панель здоровья оценок со средними полосами оценок для каждого измерения оценивающего сервиса, разбором инструментов ok-vs-error, топ-инструментами и трендом событий в час](/agenteye/images/dashboard-quality.png) +![Панель здоровья оценок со средними полосами оценок для каждого измерения оценивающего сервиса, разбором инструментов ok-vs-error, топ-инструментами и трендом событий в час](/cloud/images/dashboard-quality.png) **Разрешения:** просмотр нуждается в `dashboards:read` и `evaluations:read`; создание и редактирование нужны `dashboards:write`; удаление нужно `dashboards:delete`. @@ -395,7 +395,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. ## Следующие шаги -- [Evaluator agent skill](/ru/agenteye/evaluator-skill): попросите кодирующего агента спроектировать ваши параметры на основе реальных сессий и построить для вас этот сервис. -- [Python SDK](/ru/agenteye/python-sdk): генерируйте события `agent_end`, которые запускают оценку. -- [API keys](/ru/agenteye/api-keys): разрешения `evaluations:read` и `evaluations:trigger`. -- [Audits](/ru/agenteye/audits): другая автоматизированная функция качества Observability для проверки на основе политик. \ No newline at end of file +- [Evaluator agent skill](/ru/cloud/agent-skills): попросите кодирующего агента спроектировать ваши параметры на основе реальных сессий и построить для вас этот сервис. +- [Python SDK](/ru/cloud/sdk): генерируйте события `agent_end`, которые запускают оценку. +- [API keys](/ru/cloud/access): разрешения `evaluations:read` и `evaluations:trigger`. +- [Audits](/ru/cloud/audits): другая автоматизированная функция качества FailproofAI Cloud для проверки на основе политик. \ No newline at end of file diff --git a/docs/ru/agenteye/event-stream.mdx b/docs/ru/cloud/event-stream.mdx similarity index 81% rename from docs/ru/agenteye/event-stream.mdx rename to docs/ru/cloud/event-stream.mdx index 9a4d55bf..a9f90670 100644 --- a/docs/ru/agenteye/event-stream.mdx +++ b/docs/ru/cloud/event-stream.mdx @@ -7,7 +7,7 @@ description: "В момент, когда ваш агент что-то дела В момент, когда ваш агент что-то делает, вы это видите. Event Stream — это живой пульс каждого агента в продакшене: без ожидания, без поиска в логах, без угадывания того, что произошло. -![The live Event Stream: colour-coded event rows tailing in real time, filterable by environment, agent, session, event type, and free text](/agenteye/images/events-stream.png) +![The live Event Stream: colour-coded event rows tailing in real time, filterable by environment, agent, session, event type, and free text](/cloud/images/events-stream.png) *Каждое событие от каждого агента в вашей организации, новые сверху, обновляется в реальном времени.* @@ -38,14 +38,14 @@ description: "В момент, когда ваш агент что-то дела Event Stream — это ваша домашняя страница организации. Войдите, и это первая поверхность, на которую вы попадаете, по адресу `//`, поэтому классификация начинается в момент вашего прибытия. -За кулисами ваши агенты генерируют события через SDK, сборщик отправляет их на ваш сервер Failproof AI Observability, а поток отслеживает их по мере поступления в управляемую вами инфраструктуру. Когда вы хотите сводное представление вместо необработанного следа, события каждого запуска сворачиваются в одну строку на Sessions, в один клик. +За кулисами ваши агенты генерируют события через SDK, сборщик отправляет их на ваш сервер FailproofAI Cloud, а поток отслеживает их по мере поступления в управляемую вами инфраструктуру. Когда вы хотите сводное представление вместо необработанного следа, события каждого запуска сворачиваются в одну строку на Sessions, в один клик. Это основной источник истины, на котором строятся все остальные поверхности наблюдения, поэтому когда где-то числа выглядят неправильно, поток — это место, где вы подтверждаете, что на самом деле произошло. ## Связанные материалы -- [Sessions](/ru/agenteye/sessions): те же события, объединённые в одну строку за запуск, с графиком выполнения в стиле git. -- [Telemetry](/ru/agenteye/telemetry): что отправляют ваши агенты и как события попадают в поток. -- [Error tracking](/ru/agenteye/error-tracking): единая поверхность классификации для всего, что пошло не так. -- [Alerts](/ru/agenteye/alerts): превратите любой порог в правило уведомления. -- [CLI and agents](/ru/agenteye/cli-and-agents): тот же живой след прямо из вашего терминала. \ No newline at end of file +- [Sessions](/ru/cloud/sessions): те же события, объединённые в одну строку за запуск, с графиком выполнения в стиле git. +- [Telemetry](/ru/cloud/performance): что отправляют ваши агенты и как события попадают в поток. +- [Error tracking](/ru/cloud/errors): единая поверхность классификации для всего, что пошло не так. +- [Alerts](/ru/cloud/alerts): превратите любой порог в правило уведомления. +- [CLI and agents](/ru/cloud/cli): тот же живой след прямо из вашего терминала. \ No newline at end of file diff --git a/docs/ru/cloud/fleet.mdx b/docs/ru/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/ru/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/ru/agenteye/incidents.mdx b/docs/ru/cloud/incidents.mdx similarity index 86% rename from docs/ru/agenteye/incidents.mdx rename to docs/ru/cloud/incidents.mdx index c7d2b710..acd7d105 100644 --- a/docs/ru/agenteye/incidents.mdx +++ b/docs/ru/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "Когда срабатывает оповещение, все в Когда срабатывает оповещение, первый вопрос всегда один: «кто этим займётся?» Инциденты отвечают на него: в момент обнаружения нарушения все видят, что инцидент открыт, кто за него отвечает и ровно что произошло, с чистой и упорядоченной записью, которую можно сразу передать на анализ после инцидента. -![Входящие инциденты: карточки инцидентов, связанные с оповещениями и открытые вручную, сгруппированные по статусу, каждая с значком серьёзности и назначенным ответственным](/agenteye/images/incidents.png) +![Входящие инциденты: карточки инцидентов, связанные с оповещениями и открытые вручную, сгруппированные по статусу, каждая с значком серьёзности и назначенным ответственным](/cloud/images/incidents.png) *Входящие группируют открытые инциденты по статусу и фильтруют по серьёзности и ответственному, чтобы вы видели, что требует внимания человека прямо сейчас.* ## Сразу видно, кто этим занимается @@ -17,10 +17,10 @@ description: "Когда срабатывает оповещение, все в Когда инцидент завершён, у вас уже есть описание. Откройте любой инцидент — и вы увидите свидетельства нарушения, его ответственных и подписчиков, цепочку комментариев для координации и неизменяемую временную шкалу активности. -![Детальный вид инцидента: родительское оповещение и краткое описание нарушения, ответственные и подписчики, упорядоченная по времени временная шкала активности и цепочка комментариев](/agenteye/images/incident-detail.png) +![Детальный вид инцидента: родительское оповещение и краткое описание нарушения, ответственные и подписчики, упорядоченная по времени временная шкала активности и цепочка комментариев](/cloud/images/incident-detail.png) *Все события, по порядку, каждая строка подписана тем, кто её создал.* -Каждое действие (открыто, подтверждено, разрешено и так далее) записывается в эту временную шкалу и никогда не изменяется. Каждая запись имеет автора: оператора, который её выполнил, с указанием почты, или **automated** для всего, что Failproof AI Observability сделал самостоятельно, например открыл инцидент при обнаружении нарушения. Ничего не анонимно и ничего не теряется, так что анализ после инцидента практически пишется сам по себе. +Каждое действие (открыто, подтверждено, разрешено и так далее) записывается в эту временную шкалу и никогда не изменяется. Каждая запись имеет автора: оператора, который её выполнил, с указанием почты, или **automated** для всего, что FailproofAI Cloud сделал самостоятельно, например открыл инцидент при обнаружении нарушения. Ничего не анонимно и ничего не теряется, так что анализ после инцидента практически пишется сам по себе. ## Как инцидент развивается @@ -45,6 +45,6 @@ stateDiagram-v2 ## Связанное -- [Оповещения](/ru/agenteye/alerts): правила, которые открывают эти инциденты при нарушении порога. -- [Отслеживание ошибок](/ru/agenteye/error-tracking): смотрите все сбои в одном месте и повысьте один до оповещения. -- [Аудиты](/ru/agenteye/audits): запланированный аналитик, который находит сбои, за которыми не наблюдало ни одно правило. \ No newline at end of file +- [Оповещения](/ru/cloud/alerts): правила, которые открывают эти инциденты при нарушении порога. +- [Отслеживание ошибок](/ru/cloud/errors): смотрите все сбои в одном месте и повысьте один до оповещения. +- [Аудиты](/ru/cloud/audits): запланированный аналитик, который находит сбои, за которыми не наблюдало ни одно правило. \ No newline at end of file diff --git a/docs/ru/cloud/managed-policies.mdx b/docs/ru/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/ru/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/ru/agenteye/overview.mdx b/docs/ru/cloud/overview.mdx similarity index 52% rename from docs/ru/agenteye/overview.mdx rename to docs/ru/cloud/overview.mdx index 8f42c045..e2d4383a 100644 --- a/docs/ru/agenteye/overview.mdx +++ b/docs/ru/cloud/overview.mdx @@ -1,15 +1,15 @@ --- title: "Failproof AI: Наблюдение за отказами агентов" -description: "Failproof AI Observability — это самостоятельно размещаемая платформа для наблюдения, оценки и улучшения ваших AI-агентов в продакшене." +description: "FailproofAI Cloud — это самостоятельно размещаемая платформа для наблюдения, оценки и улучшения ваших AI-агентов в продакшене." --- -Failproof AI Observability — это самостоятельно размещаемая платформа для наблюдения, оценки и улучшения ваших AI-агентов в продакшене. Она фиксирует всё, что делают ваши агенты (каждый вызов инструмента, запрос к модели, hook и ошибку), оценивает качество каждого запуска и выявляет сбои, на которые вы не смотрели, всё это в панели управления, работающей в вашей инфраструктуре. +FailproofAI Cloud — это самостоятельно размещаемая платформа для наблюдения, оценки и улучшения ваших AI-агентов в продакшене. Она фиксирует всё, что делают ваши агенты (каждый вызов инструмента, запрос к модели, hook и ошибку), оценивает качество каждого запуска и выявляет сбои, на которые вы не смотрели, всё это в панели управления, работающей в вашей инфраструктуре. -Если вы развёртываете AI-агентов и устали гадать, почему запуск пошёл не так, эта страница — ваша отправная точка. Здесь объясняется, что вам даёт Failproof AI Observability и как всё взаимодействует, прежде чем вы что-то устанавливать. +Если вы развёртываете AI-агентов и устали гадать, почему запуск пошёл не так, эта страница — ваша отправная точка. Здесь объясняется, что вам даёт FailproofAI Cloud и как всё взаимодействует, прежде чем вы что-то устанавливать. -> **Failproof AI Observability — это корпоративный продукт компании Failproof AI.** Хотите увидеть его в действии? Запросите демонстрацию: напишите на [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +> **FailproofAI Cloud — это корпоративный продукт компании Failproof AI.** Хотите увидеть его в действии? Запросите демонстрацию: напишите на [nikita@befailproof.ai](mailto:nikita@befailproof.ai). -![Сеанс Failproof AI Observability, изображённый в виде графа выполнения в стиле git рядом с временной шкалой событий, с разбивкой каждого запуска на инструменты, модели и hooks в правой панели](/agenteye/images/session-detail.png) +![Сеанс FailproofAI Cloud, изображённый в виде графа выполнения в стиле git рядом с временной шкалой событий, с разбивкой каждого запуска на инструменты, модели и hooks в правой панели](/cloud/images/session-detail.png) *Каждый запуск агента отображается в виде графа выполнения в стиле git (слева) рядом с его временной шкалой событий. Параллельные под-агенты получают свою полосу; в правой панели показаны инструменты, модели, hooks и расход токенов для запуска.* @@ -29,52 +29,52 @@ Failproof AI Observability — это самостоятельно размещ
    -*Failproof Audit: позвольте Failproof AI Observability проанализировать ваши логи во всех сеансах и показать, что нужно исправить.* +*Failproof Audit: позвольте FailproofAI Cloud проанализировать ваши логи во всех сеансах и показать, что нужно исправить.* --- ## Почему команды её используют - **Узнайте, что действительно делал ваш агент.** Каждый запуск становится читаемым графом выполнения в стиле git: какие инструменты работали параллельно, какие под-агенты ветвились, где он зависал и что стоило. -- **Автоматически ловите падение качества.** Подключите небольшой сервис оценки, и Failproof AI Observability оценит каждый завершённый запуск, чтобы падение полезности или всплеск галлюцинаций стали очевидны. +- **Автоматически ловите падение качества.** Подключите небольшой сервис оценки, и FailproofAI Cloud оценит каждый завершённый запуск, чтобы падение полезности или всплеск галлюцинаций стали очевидны. - **Найдите сбои, для которых вы не написали правила.** Повторяющиеся аудиты анализируют ваши логи во всех сеансах в поиске кластеров ошибок, выбросов латентности, низких оценок и зависаний, а затем выдают вам ранжированные, подтвёрённые результаты. - **Получайте уведомления, когда это важно.** Правила по порогам срабатывают на основе частоты ошибок, латентности, стоимости или оценок оценивателя и открывают инциденты, которые вы можете подтвердить, назначить и разрешить. - **Задавайте вопросы на обычном английском.** AI-ассистент в панели управления ответит на вопросы вроде «как качество развивается в продакшене на этой неделе?» по вашим данным. Любое изменение требует одобрения. -- **Держите ваши данные под контролем.** Failproof AI Observability является самостоятельно размещаемым: события, промпты и аналитика остаются в инфраструктуре, которую вы контролируете. +- **Держите ваши данные под контролем.** FailproofAI Cloud является самостоятельно размещаемым: события, промпты и аналитика остаются в инфраструктуре, которую вы контролируете. --- ## Что вы получаете -Failproof AI Observability организована вокруг трёх концепций (**observe**, **analyze** и **admin**), отражённых в левой боковой панели панели управления. +FailproofAI Cloud организована вокруг трёх концепций (**observe**, **analyze** и **admin**), отражённых в левой боковой панели панели управления. **Observe** (сырая правда о том, что произошло): -- **[Поток событий](/ru/agenteye/event-stream)**: живая, пошаговая цепь всех запусков (вызовы инструментов, вызовы моделей, hooks, ошибки). -- **[Сеансы](/ru/agenteye/sessions)**: эти события сведены в одну строку на запуск, каждый готов к оценке, с графом выполнения в стиле git. -- **[Метрики производительности](/ru/agenteye/telemetry)**: тепловые карты латентности для каждой поверхности и жизненно важные показатели p50/p95/p99 для моделей, инструментов и hooks, чтобы всплеск на хвосте выделялся из медианы. -- **[Отслеживание ошибок](/ru/agenteye/error-tracking)**: единая поверхность для триажа всего, что пошло не так, в один клик от срабатывающего предупреждения. +- **[Поток событий](/ru/cloud/event-stream)**: живая, пошаговая цепь всех запусков (вызовы инструментов, вызовы моделей, hooks, ошибки). +- **[Сеансы](/ru/cloud/sessions)**: эти события сведены в одну строку на запуск, каждый готов к оценке, с графом выполнения в стиле git. +- **[Метрики производительности](/ru/cloud/performance)**: тепловые карты латентности для каждой поверхности и жизненно важные показатели p50/p95/p99 для моделей, инструментов и hooks, чтобы всплеск на хвосте выделялся из медианы. +- **[Отслеживание ошибок](/ru/cloud/errors)**: единая поверхность для триажа всего, что пошло не так, в один клик от срабатывающего предупреждения. -![Страница инструментов в Observe: тепловая карта латентности, полоса перцентилей и диаграмма распределения инструментов более 24 временных бинов](/agenteye/images/tools.png) +![Страница инструментов в Observe: тепловая карта латентности, полоса перцентилей и диаграмма распределения инструментов более 24 временных бинов](/cloud/images/tools.png) *Каждая поверхность наблюдения объединяет искромётную линию и жизненно важные показатели p50/p95/p99 с тепловой картой латентности и полосой перцентилей. Показано здесь: инструменты.* **Analyze** (преобразуйте активность в ответы): -- **[Запросы](/ru/agenteye/queries)** и **[панели управления](/ru/agenteye/dashboards)**: сохранённый SQL по вашим событиям и оценкам, представленный в виде общих, ориентированных на организацию панелей управления. -- **[Оценки](/ru/agenteye/evaluations)**: оценки качества, полученные от вашего собственного сервиса оценивателя, с рассуждением для каждой оценки. -- **[Аудиты](/ru/agenteye/audits)**: повторяющиеся исследования, выявляющие закономерности сбоев во всех сеансах. -- **[Предупреждения](/ru/agenteye/alerts)** и **[инциденты](/ru/agenteye/incidents)**: правила по порогам, которые вызывают уведомления, плюс рабочий процесс инцидентов для их триажа. +- **[Запросы](/ru/cloud/queries)** и **[панели управления](/ru/cloud/dashboards)**: сохранённый SQL по вашим событиям и оценкам, представленный в виде общих, ориентированных на организацию панелей управления. +- **[Оценки](/ru/cloud/evaluations)**: оценки качества, полученные от вашего собственного сервиса оценивателя, с рассуждением для каждой оценки. +- **[Аудиты](/ru/cloud/audits)**: повторяющиеся исследования, выявляющие закономерности сбоев во всех сеансах. +- **[Предупреждения](/ru/cloud/alerts)** и **[инциденты](/ru/cloud/incidents)**: правила по порогам, которые вызывают уведомления, плюс рабочий процесс инцидентов для их триажа. **Интерфейсы** (получайте доступ к вашим данным своим способом): -- **[CLI](/ru/agenteye/cli-and-agents)**: управляйте всем развёртыванием из терминала или скрипта, и позвольте кодирующему агенту делать это за вас на обычном английском. -- **[AI-ассистент](/ru/agenteye/assistant)**: задавайте вопросы о ваших агентах на обычном английском прямо в панели управления. -- **REST API**: всё, что делают панель управления и CLI, поддерживается REST API, который вы можете вызывать напрямую с помощью ограниченного [API ключа](/ru/agenteye/api-keys) — принимайте события, запрашивайте сеансы и оценки, управляйте панелями управления, предупреждениями, аудитами, пользователями и ключами, чтобы интегрировать Failproof AI Observability в свой инструментарий. +- **[CLI](/ru/cloud/cli)**: управляйте всем развёртыванием из терминала или скрипта, и позвольте кодирующему агенту делать это за вас на обычном английском. +- **[AI-ассистент](/ru/cloud/assistant)**: задавайте вопросы о ваших агентах на обычном английском прямо в панели управления. +- **REST API**: всё, что делают панель управления и CLI, поддерживается REST API, который вы можете вызывать напрямую с помощью ограниченного [API ключа](/ru/cloud/access) — принимайте события, запрашивайте сеансы и оценки, управляйте панелями управления, предупреждениями, аудитами, пользователями и ключами, чтобы интегрировать FailproofAI Cloud в свой инструментарий. **Admin** (управляйте это для своей команды): -- **[API ключи](/ru/agenteye/api-keys)**: ограниченные токены для коллектора, панели управления и ассистента. +- **[API ключи](/ru/cloud/access)**: ограниченные токены для коллектора, панели управления и ассистента. - **Пользователи**: вход без пароля на основе электронной почты с использованием списка разрешений. - **Параметры**: конфигурация для каждой организации, включая переопределения размера контекстного окна модели. @@ -90,18 +90,18 @@ Failproof AI Observability организована вокруг трёх кон - **Панель управления**: где вы изучаете всё. - **Дополнительные сервисы**: сервис оценки (оценки) и сервис AI-ассистента (чат в панели управления). -Для словаря, используемого во всей документации (*event, session, evaluation, audit, finding, incident*), см. [Concepts](/ru/agenteye/concepts). +Для словаря, используемого во всей документации (*event, session, evaluation, audit, finding, incident*), см. [Concepts](/ru/concepts). --- -## Получение Failproof AI Observability +## Получение FailproofAI Cloud -Failproof AI Observability — это корпоративный продукт компании Failproof AI, и он работает вместе с Failproof AI Enforcement — продуктом политик и ограждений — под брендом Failproof AI. Он полностью работает в вашей среде. Если у вас ещё нет доступа к пакетам, запросите демонстрацию, и мы вас настроим: напишите на [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +FailproofAI Cloud — это корпоративный продукт компании Failproof AI, и он работает вместе с FailproofAI guardrails — продуктом политик и ограждений — под брендом Failproof AI. Он полностью работает в вашей среде. Если у вас ещё нет доступа к пакетам, запросите демонстрацию, и мы вас настроим: напишите на [nikita@befailproof.ai](mailto:nikita@befailproof.ai). --- ## Следующие шаги -- [Concepts](/ru/agenteye/concepts): словарь Failproof AI Observability в одном месте. -- [Observability](/ru/agenteye/observability): следите за тем, что делают ваши агенты, запуск за запуском. -- [Security](/ru/agenteye/security): как Failproof AI Observability держит ваши данные изолированными и под вашим контролем. \ No newline at end of file +- [Concepts](/ru/concepts): словарь FailproofAI Cloud в одном месте. +- [FailproofAI Cloud](/ru/cloud/overview): следите за тем, что делают ваши агенты, запуск за запуском. +- [Security](/ru/cloud/security): как FailproofAI Cloud держит ваши данные изолированными и под вашим контролем. \ No newline at end of file diff --git a/docs/ru/agenteye/telemetry.mdx b/docs/ru/cloud/performance.mdx similarity index 84% rename from docs/ru/agenteye/telemetry.mdx rename to docs/ru/cloud/performance.mdx index 754f6bf1..374b3125 100644 --- a/docs/ru/agenteye/telemetry.mdx +++ b/docs/ru/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "Заметьте в тот же миг, когда модели, Заметьте в тот же миг, когда модели, инструменты или хуки замедляются или начинают потреблять ресурсы, и перехватите скачок хвостовой задержки до того, как это почувствуют пользователи. Три отдельные страницы превращают сырые временные данные в p50, p95 и p99, которые можно оценить с первого взгляда. -![Страница Models с тепловой картой задержки, полосой процентилей и показателями токенов, стоимости и размера контекстного окна для каждой модели](/agenteye/images/models.png) +![Страница Models с тепловой картой задержки, полосой процентилей и показателями токенов, стоимости и размера контекстного окна для каждой модели](/cloud/images/models.png) *Страница Models: тепловая карта задержки, полоса процентилей и показатели токенов, расчётная стоимость и коэффициент заполнения контекстного окна.* ## Не позволяйте средним значениям скрывать ваши худшие прогоны @@ -24,13 +24,13 @@ description: "Заметьте в тот же миг, когда модели, Страница Models (показана выше) отвечает на два вопроса, которые всегда возникают при получении счёта: какая модель и сколько. Поверх общего представления задержки она добавляет **потребление токенов для каждой модели**, **расчётную стоимость** и **заполнение контекстного окна**, чтобы неконтролируемый рост приглашения и предстоящее сжатие были видны до того, как они вас застанут врасплох. -Failproof AI Observability автоматически распознаёт обычные ID моделей. Если окно выглядит неправильно или вы используете собственную приватную модель, исправьте это или добавьте её в разделе **Settings**, в **model context windows**, и показатели заполнения будут следовать за изменениями. +FailproofAI Cloud автоматически распознаёт обычные ID моделей. Если окно выглядит неправильно или вы используете собственную приватную модель, исправьте это или добавьте её в разделе **Settings**, в **model context windows**, и показатели заполнения будут следовать за изменениями. ## Tools: отличите медленное от сломанного Вызов инструмента может быть медленным или тихо не работать, и вы хотите узнать, что именно происходит, за секунды, а не после раскопок в логах. -![Страница Tools с общей тепловой картой задержки и полосой процентилей рядом с разбивкой по успехам и ошибкам и полосой распределения инструментов](/agenteye/images/tools.png) +![Страница Tools с общей тепловой картой задержки и полосой процентилей рядом с разбивкой по успехам и ошибкам и полосой распределения инструментов](/cloud/images/tools.png) *Страница Tools: одна и та же тепловая карта и полоса процентилей, плюс разбивка по успехам и ошибкам и полоса распределения инструментов.* Рядом с общим представлением задержки страница Tools добавляет **разбивку по успехам и ошибкам** и **полосу распределения инструментов**, чтобы вы видели с первого взгляда, какими инструментами вы больше всего пользуетесь и какие съедают ваш бюджет ошибок. @@ -39,14 +39,14 @@ Failproof AI Observability автоматически распознаёт об Когда жизненный цикл хука замедляет прогон, фраза "хуки медленные" — это не то, на что вы можете действовать. Страница Hooks доставляет вас к тому, что имеет значение. -![Страница Hooks с задержкой, разбитой по имени хука и событию-триггеру поверх общей тепловой карты и полосы процентилей](/agenteye/images/hooks.png) +![Страница Hooks с задержкой, разбитой по имени хука и событию-триггеру поверх общей тепловой карты и полосы процентилей](/cloud/images/hooks.png) *Страница Hooks: задержка разбита по имени хука и событию-триггеру.* На той же тепловой карте задержки и полосе процентилей страница Hooks разбивает активность по **имени хука** и **событию-триггеру**, чтобы вы сосредоточились на одном хуке и одном событии, требующих внимания. ## Связанное -- [Event stream](/ru/agenteye/event-stream): живая, цветовая кодировка всех событий. -- [Sessions](/ru/agenteye/sessions): свёртывает события в одну строку за прогон и открывает его граф выполнения. -- [Error tracking](/ru/agenteye/error-tracking): единая поверхность сортировки для всего, что панель управления отмечает красным. -- [Dashboards](/ru/agenteye/dashboards): сводные представления для всего вашего флота. \ No newline at end of file +- [Event stream](/ru/cloud/event-stream): живая, цветовая кодировка всех событий. +- [Sessions](/ru/cloud/sessions): свёртывает события в одну строку за прогон и открывает его граф выполнения. +- [Error tracking](/ru/cloud/errors): единая поверхность сортировки для всего, что панель управления отмечает красным. +- [Dashboards](/ru/cloud/dashboards): сводные представления для всего вашего флота. \ No newline at end of file diff --git a/docs/ru/agenteye/queries.mdx b/docs/ru/cloud/queries.mdx similarity index 80% rename from docs/ru/agenteye/queries.mdx rename to docs/ru/cloud/queries.mdx index c53959c2..48a0a945 100644 --- a/docs/ru/agenteye/queries.mdx +++ b/docs/ru/cloud/queries.mdx @@ -4,9 +4,9 @@ description: "Задавайте любые вопросы о данных ва --- -Задавайте любые вопросы о данных вашего агента и получайте ответы за секунды. Observability от Failproof AI предоставляет вам библиотеку сохранённых готовых к запуску запросов над вашими событиями и оценками, чтобы вы начали с рабочего примера вместо пустого редактора SQL. +Задавайте любые вопросы о данных вашего агента и получайте ответы за секунды. FailproofAI Cloud от Failproof AI предоставляет вам библиотеку сохранённых готовых к запуску запросов над вашими событиями и оценками, чтобы вы начали с рабочего примера вместо пустого редактора SQL. -![Библиотека сохранённых запросов: сетка переиспользуемых запросов, как встроенных предустановок, так и пользовательских](/agenteye/images/queries.png) +![Библиотека сохранённых запросов: сетка переиспользуемых запросов, как встроенных предустановок, так и пользовательских](/cloud/images/queries.png) *Ваша библиотека сохранённых запросов на `//queries`: встроенные предустановки рядом с запросами, которые сохранила ваша команда.* @@ -22,7 +22,7 @@ description: "Задавайте любые вопросы о данных ва Откройте любой запрос, и он откроется в редакторе SQL, где вы сможете его изменить и сразу увидеть ответ: без экспорта, без круговорота, без ожидания помощи от кого-то другого. -![Редактор SQL-запросов с запущенным сохранённым запросом, боковой панелью схемы и таблицей результатов](/agenteye/images/query-lab.png) +![Редактор SQL-запросов с запущенным сохранённым запросом, боковой панелью схемы и таблицей результатов](/cloud/images/query-lab.png) *Редактор SQL: ваш запрос слева, боковая панель схемы, чтобы вы никогда не угадывали название колонки, и таблица результатов снизу.* @@ -43,14 +43,14 @@ agenteye query list # те же сохранённые запр agenteye query run errs --arg prod # запустить один и вывести строки (добавьте --json для передачи) ``` - См. [CLI и агенты](/ru/agenteye/cli-and-agents) для полного набора команд. + См. [CLI и агенты](/ru/cloud/cli) для полного набора команд. -- **От AI-помощника.** Не уверены, как сформулировать SQL? Спросите встроенного в панель [AI-помощника](/ru/agenteye/assistant) на простом английском языке, и он напишет запрос и сохранит его в вашу библиотеку за вас. +- **От AI-помощника.** Не уверены, как сформулировать SQL? Спросите встроенного в панель [AI-помощника](/ru/cloud/assistant) на простом английском языке, и он напишет запрос и сохранит его в вашу библиотеку за вас. Запуск сохранённого запроса контролируется разрешением `queries:run`, отделённым от разрешений на создание или удаление запросов, поэтому вы можете предоставить доступ на чтение без разрешения переписывать библиотеку. ## Связанное -- [Панели управления](/ru/agenteye/dashboards): закрепляйте результаты запросов на общих диаграммах уровня организации. -- [AI-помощник](/ru/agenteye/assistant): задавайте вопросы на простом английском языке и получайте запрос в ответ. -- [CLI и агенты](/ru/agenteye/cli-and-agents): запускайте и сохраняйте те же запросы из вашего терминала. \ No newline at end of file +- [Панели управления](/ru/cloud/dashboards): закрепляйте результаты запросов на общих диаграммах уровня организации. +- [AI-помощник](/ru/cloud/assistant): задавайте вопросы на простом английском языке и получайте запрос в ответ. +- [CLI и агенты](/ru/cloud/cli): запускайте и сохраняйте те же запросы из вашего терминала. \ No newline at end of file diff --git a/docs/ru/agenteye/python-sdk.mdx b/docs/ru/cloud/sdk.mdx similarity index 92% rename from docs/ru/agenteye/python-sdk.mdx rename to docs/ru/cloud/sdk.mdx index c83ae6bc..d5288912 100644 --- a/docs/ru/agenteye/python-sdk.mdx +++ b/docs/ru/cloud/sdk.mdx @@ -5,11 +5,11 @@ description: "Посмотрите, что именно сделали ваши --- -Посмотрите, что именно сделали ваши AI-агенты в продакшене: каждый запуск агента, вызов инструмента, запрос к модели, хук и вмешательство человека. Python SDK Failproof AI Observability записывает эту цепочку событий изнутри кода вашего агента, чтобы вы могли отлаживать, аудировать и оценивать происходящее. Используйте его, когда захотите, чтобы Failproof AI Observability наблюдал за вашими агентами. +Посмотрите, что именно сделали ваши AI-агенты в продакшене: каждый запуск агента, вызов инструмента, запрос к модели, хук и вмешательство человека. Python SDK FailproofAI Cloud записывает эту цепочку событий изнутри кода вашего агента, чтобы вы могли отлаживать, аудировать и оценивать происходящее. Используйте его, когда захотите, чтобы FailproofAI Cloud наблюдал за вашими агентами. Под капотом SDK записывает структурированные события в локальные JSONL-файлы, а демон сборщика подхватывает их и автоматически отправляет на платформу. Вам не нужно самостоятельно управлять этими файлами. -> **Совет:** Новичок в Failproof AI Observability? Эта страница является полным справочником событий SDK. +> **Совет:** Новичок в FailproofAI Cloud? Эта страница является полным справочником событий SDK.
    @@ -27,7 +27,7 @@ SDK распространяется клиентам как приватный python -c "import agenteye; print(agenteye.__version__)" ``` -Предпочитаете позволить кодирующему агенту выполнить всю интеграцию? [Python SDK Agent Skill](/ru/agenteye/python-sdk-skill) знает путь установки, планирует точки инструментирования, пишет их и проверяет, что события доходят. +Предпочитаете позволить кодирующему агенту выполнить всю интеграцию? [Python SDK Agent Skill](/ru/cloud/agent-skills) знает путь установки, планирует точки инструментирования, пишет их и проверяет, что события доходят. --- @@ -61,7 +61,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### Инструментирование реального вызова -На практике вы оборачиваете существующий код агента. Заключите вызов модели с `model_request` перед и `model_response` после, чтобы два события охватывали реальный запрос и Failproof AI Observability смогла их связать: +На практике вы оборачиваете существующий код агента. Заключите вызов модели с `model_request` перед и `model_response` после, чтобы два события охватывали реальный запрос и FailproofAI Cloud смогла их связать: ```python import anthropic @@ -100,7 +100,7 @@ agenteye.event.model_response( Вот как выглядят эти события на дашборде — они раскрашены по типам и фильтруются по среде, агенту и сессии: -![Живой поток событий, раскрашенный по типам событий и фильтруемый по среде, агенту и сессии](/agenteye/images/events-stream.png) +![Живой поток событий, раскрашенный по типам событий и фильтруемый по среде, агенту и сессии](/cloud/images/events-stream.png) --- @@ -157,9 +157,9 @@ SDK записывает только поля, которые вы явно п ## Справочник событий -Большинство событий поступают в парах начало/конец, которые разделяют идентификатор корреляции: `tool_use` и `tool_result` разделяют `tool_call_id`, `hook_triggered` и `hook_completed` разделяют `hook_id`, а `human_wait` и `human_input` разделяют `input_id`. Выпустите событие начала, выполните работу, затем выпустите событие завершения с тем же ID. Failproof AI Observability соответствует паре и вычисляет `duration_ms` за вас, поэтому вы никогда не передаёте `duration_ms` сами. +Большинство событий поступают в парах начало/конец, которые разделяют идентификатор корреляции: `tool_use` и `tool_result` разделяют `tool_call_id`, `hook_triggered` и `hook_completed` разделяют `hook_id`, а `human_wait` и `human_input` разделяют `input_id`. Выпустите событие начала, выполните работу, затем выпустите событие завершения с тем же ID. FailproofAI Cloud соответствует паре и вычисляет `duration_ms` за вас, поэтому вы никогда не передаёте `duration_ms` сами. -![Граф выполнения сессии в стиле git рядом с временной шкалой событий, реконструированный из парных событий, с панелью разбивки инструмента/модели/хука](/agenteye/images/session-detail.png) +![Граф выполнения сессии в стиле git рядом с временной шкалой событий, реконструированный из парных событий, с панелью разбивки инструмента/модели/хука](/cloud/images/session-detail.png) Все методы событий требуют эти два поля: @@ -433,5 +433,5 @@ agenteye.event.tool_use( ## Дальнейшие шаги -- [Поток событий](/ru/agenteye/event-stream): смотрите эти события в прямом эфире, раскрашенные и фильтруемые по среде, агенту и сессии. -- [Сессии](/ru/agenteye/sessions): смотрите, как парные события реконструируют каждый запуск агента как граф выполнения и временную шкалу. \ No newline at end of file +- [Поток событий](/ru/cloud/event-stream): смотрите эти события в прямом эфире, раскрашенные и фильтруемые по среде, агенту и сессии. +- [Сессии](/ru/cloud/sessions): смотрите, как парные события реконструируют каждый запуск агента как граф выполнения и временную шкалу. \ No newline at end of file diff --git a/docs/ru/agenteye/security.mdx b/docs/ru/cloud/security.mdx similarity index 51% rename from docs/ru/agenteye/security.mdx rename to docs/ru/cloud/security.mdx index 9028e2a1..4549e103 100644 --- a/docs/ru/agenteye/security.mdx +++ b/docs/ru/cloud/security.mdx @@ -1,22 +1,22 @@ --- title: "Безопасность" -description: "Failproof AI Observability разработана для работы рядом с вашими production агентами, что означает, что она видит ваши промпты, входные данные инструментов и результаты их работы." +description: "FailproofAI Cloud разработана для работы рядом с вашими production агентами, что означает, что она видит ваши промпты, входные данные инструментов и результаты их работы." --- -Failproof AI Observability разработана для работы рядом с вашими production агентами, что означает, что она видит ваши промпты, входные данные инструментов и результаты их работы. На этой странице объясняется, как она хранит данные в изолированном виде, под контролем и в ваших руках. Если вы оцениваете Failproof AI Observability для проверки безопасности, начните отсюда. +FailproofAI Cloud разработана для работы рядом с вашими production агентами, что означает, что она видит ваши промпты, входные данные инструментов и результаты их работы. На этой странице объясняется, как она хранит данные в изолированном виде, под контролем и в ваших руках. Если вы оцениваете FailproofAI Cloud для проверки безопасности, начните отсюда. --- ## Ваши данные остаются в вашей среде -Failproof AI Observability развернута локально. События, промпты, ответы модели и аналитика хранятся в ваших собственных базах данных, в вашей собственной среде. Ничто не отправляется на сторонний SaaS для хранения, и ваши данные остаются в вашем облачном аккаунте. +FailproofAI Cloud развернута локально. События, промпты, ответы модели и аналитика хранятся в ваших собственных базах данных, в вашей собственной среде. Ничто не отправляется на сторонний SaaS для хранения, и ваши данные остаются в вашем облачном аккаунте. --- ## Изоляция между тенантами -Один экземпляр Failproof AI Observability может размещать множество организаций, каждая из которых изолирована на уровне хранилища — это обеспечивается самой базой данных, а не просто интерфейсом: +Один экземпляр FailproofAI Cloud может размещать множество организаций, каждая из которых изолирована на уровне хранилища — это обеспечивается самой базой данных, а не просто интерфейсом: - Операционные данные организации (пользователи, ключи, панели управления, сохранённые запросы) относятся только к этой организации, и межорганизационное чтение блокируется самой базой данных. - Каждое поступившее событие отмечается организацией-владельцем, поэтому события одной организации никогда не могут быть прочитаны другой. @@ -27,9 +27,9 @@ Failproof AI Observability развернута локально. События ## Вход в систему -Failproof AI Observability использует вход без пароля, на основе электронной почты. Пароля нет, поэтому нечего фишировать или раскрывать. Пользователь запрашивает одноразовый код (или однокликовую волшебную ссылку), который отправляется ему по электронной почте и быстро истекает. Вход контролируется **списком разрешённых адресов**: только те адреса электронной почты (или домены), которые вы разрешите, смогут пройти проверку подлинности. +FailproofAI Cloud использует вход без пароля, на основе электронной почты. Пароля нет, поэтому нечего фишировать или раскрывать. Пользователь запрашивает одноразовый код (или однокликовую волшебную ссылку), который отправляется ему по электронной почте и быстро истекает. Вход контролируется **списком разрешённых адресов**: только те адреса электронной почты (или домены), которые вы разрешите, смогут пройти проверку подлинности. -![Экран входа Failproof AI Observability, который отправляет одноразовый код на вашу электронную почту](/agenteye/images/login.png) +![Экран входа FailproofAI Cloud, который отправляет одноразовый код на вашу электронную почту](/cloud/images/login.png) --- @@ -37,15 +37,15 @@ Failproof AI Observability использует вход без пароля, н Каждый клиент проходит проверку подлинности с помощью API ключа, который имеет детализированные разрешения минимальных привилегий. Сборщику нужно только `events:add`; ключ панели управления или помощника может быть только для чтения; деструктивные действия (удаление, переполучение) — это отдельные разрешения, которые вы решаете включить. -![Страница API ключей: разрешения каждого ключа, цветокодированные по областям чтения, записи и деструктивных операций](/agenteye/images/api-keys.png) +![Страница API ключей: разрешения каждого ключа, цветокодированные по областям чтения, записи и деструктивных операций](/cloud/images/api-keys.png) -Сохраните начальный административный ключ для настройки и выдавайте узкие ключи для всего остального. См. [API ключи](/ru/agenteye/api-keys). +Сохраните начальный административный ключ для настройки и выдавайте узкие ключи для всего остального. См. [API ключи](/ru/cloud/access). --- ## Ассистент только для чтения с одобрением -Встроенный в панель управления [AI ассистент](/ru/agenteye/assistant) отвечает на вопросы по вашим данным, но он ограничен по замыслу: +Встроенный в панель управления [AI ассистент](/ru/cloud/assistant) отвечает на вопросы по вашим данным, но он ограничен по замыслу: - Он **предназначен только для чтения по умолчанию**: его SQL проходит через защиту, которая допускает только запросы `SELECT`/`WITH`, однооператорные, с ограничением по строкам. - Все, что он создаёт (сохранённый запрос, панель управления), **требует одобрения**: вы проверяете и одобряете каждую запись перед её выполнением. @@ -63,6 +63,6 @@ Failproof AI Observability использует вход без пароля, н ## Следующие шаги -- [Обзор](/ru/agenteye/overview): как Failproof AI Observability работает вместе. -- [API ключи](/ru/agenteye/api-keys): ограничьте доступ для сборщика, панели управления и ассистента. -- [Наблюдаемость](/ru/agenteye/observability): что Failproof AI Observability захватывает из ваших агентов. \ No newline at end of file +- [Обзор](/ru/cloud/overview): как FailproofAI Cloud работает вместе. +- [API ключи](/ru/cloud/access): ограничьте доступ для сборщика, панели управления и ассистента. +- [Наблюдаемость](/ru/cloud/overview): что FailproofAI Cloud захватывает из ваших агентов. \ No newline at end of file diff --git a/docs/ru/agenteye/sessions.mdx b/docs/ru/cloud/sessions.mdx similarity index 76% rename from docs/ru/agenteye/sessions.mdx rename to docs/ru/cloud/sessions.mdx index 00e4b67b..05cc79f6 100644 --- a/docs/ru/agenteye/sessions.mdx +++ b/docs/ru/cloud/sessions.mdx @@ -4,9 +4,9 @@ description: "Каждое событие из запуска, объединё --- -Хватит гадать, почему запуск не сработал. Failproof AI Observability объединяет все события запуска в одну читаемую строку, а затем рисует весь запуск как git-подобное изображение, которое можно прочитать за секунды. Так вы видите в точности, что сделал ваш агент, шаг за шагом. +Хватит гадать, почему запуск не сработал. FailproofAI Cloud объединяет все события запуска в одну читаемую строку, а затем рисует весь запуск как git-подобное изображение, которое можно прочитать за секунды. Так вы видите в точности, что сделал ваш агент, шаг за шагом. -![Список сессий: одна строка на запуск, в разных окружениях и агентах, с индикаторами статуса и значками оценки](/agenteye/images/sessions-list.png) +![Список сессий: одна строка на запуск, в разных окружениях и агентах, с индикаторами статуса и значками оценки](/cloud/images/sessions-list.png) *Одна строка на запуск: индикатор статуса показывает, как закончился запуск с первого взгляда, а значок оценки появляется, когда подключен оценивающий модуль.* @@ -30,7 +30,7 @@ description: "Каждое событие из запуска, объединё ## Прочитайте весь запуск как картинку -![Git-подобный граф выполнения сессии рядом с временной шкалой событий, с панелью разбора инструментов, моделей и hooks](/agenteye/images/session-detail.png) +![Git-подобный граф выполнения сессии рядом с временной шкалой событий, с панелью разбора инструментов, моделей и hooks](/cloud/images/session-detail.png) *Граф выполнения (слева) находится рядом с временной шкалой событий; правая панель показывает инструменты, модели, hooks и расход токенов для запуска.* @@ -38,7 +38,7 @@ description: "Каждое событие из запуска, объединё Правая панель даёт вам разбор по запуску: какие инструменты и модели запустились, какие hooks сработали и сколько токенов потратил запуск. Это ответ на вопросы «почему этот запуск стоил так дорого?» или «какой инструмент работает медленно?» прямо рядом с графом, который это вызвал. -Отдельные события имеют адресацию, поэтому вы можете дать кому-то ссылку на один момент вместо «сессия, примерно на две трети вниз». Скопируйте ссылку любого события или следите за ней из [аудита](/ru/agenteye/audits) или ошибки, и сессия откроется с выбранным событием и прокруткой к нему. Это работает даже для очень длинных запусков: временная шкала загружает ограниченное окно для вашего браузера, а ссылка, указывающая за пределы этого окна, всё равно найдёт его событие вместо того, чтобы вернуть вас в начало. Если событие устарело из вашего окна хранения, страница скажет вам об этом вместо того, чтобы молча ничего не выбирать. +Отдельные события имеют адресацию, поэтому вы можете дать кому-то ссылку на один момент вместо «сессия, примерно на две трети вниз». Скопируйте ссылку любого события или следите за ней из [аудита](/ru/cloud/audits) или ошибки, и сессия откроется с выбранным событием и прокруткой к нему. Это работает даже для очень длинных запусков: временная шкала загружает ограниченное окно для вашего браузера, а ссылка, указывающая за пределы этого окна, всё равно найдёт его событие вместо того, чтобы вернуть вас в начало. Если событие устарело из вашего окна хранения, страница скажет вам об этом вместо того, чтобы молча ничего не выбирать. --- @@ -46,12 +46,12 @@ description: "Каждое событие из запуска, объединё Каждая страница приборной панели относится к вашей организации (`//…`). Sessions находится в разделе **Observe** на левой боковой панели рядом с Events, с фильтрами диапазона дат, окружения, агента и сессии в верхней части списка. Каждая строка находится в одном клике от её полного графа выполнения. -Чтобы включить значки оценок и фильтрацию по диапазону оценок, подключите оценивающий модуль: см. [Evaluations](/ru/agenteye/evaluations). +Чтобы включить значки оценок и фильтрацию по диапазону оценок, подключите оценивающий модуль: см. [Evaluations](/ru/cloud/evaluations). --- ## Связанное -- [Event stream](/ru/agenteye/event-stream): сырой, пошаговый журнал, из которого объединяются все сессии. -- [Evaluations](/ru/agenteye/evaluations): подключите оценивающий модуль, чтобы каждый запуск получил значок оценки, по которому можно фильтровать. -- [Telemetry](/ru/agenteye/telemetry): как запуски попадают из вашего агента в эти сессии. \ No newline at end of file +- [Event stream](/ru/cloud/event-stream): сырой, пошаговый журнал, из которого объединяются все сессии. +- [Evaluations](/ru/cloud/evaluations): подключите оценивающий модуль, чтобы каждый запуск получил значок оценки, по которому можно фильтровать. +- [Telemetry](/ru/cloud/performance): как запуски попадают из вашего агента в эти сессии. \ No newline at end of file diff --git a/docs/ru/concepts.mdx b/docs/ru/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/ru/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/ru/daemon.mdx b/docs/ru/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/ru/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/ru/dashboard.mdx b/docs/ru/dashboard.mdx index c0c7ee56..7f77e32c 100644 --- a/docs/ru/dashboard.mdx +++ b/docs/ru/dashboard.mdx @@ -73,7 +73,7 @@ Hermes и OpenClaw имеют область пользователя и не и 5. **Вернись лучше** — две карточки рядом. Слева: установить напоминание (выбор частоты `3d` / `7d` / `14d` / `30d`; сохраняется через `/api/auth/reminder` после аутентификации). Справа: разблокировать привилегии failproof — `invite a friend` открывает модальное окно, которое принимает список адресов электронной почты друзей, разделённых запятыми/пробелами/переводами строк (максимум 10 за отправку), POST их на `/api/audit/invite`, что перенаправляет на `POST /v0/invite` сервера api. Сервер api отправляет одно письмо на каждого получателя от `invite@failproof.ai` с копией отправителя и установленным `Reply-To`, так что получатель видит, кто его пригласил, а отправитель получает копию в своем почтовом ящике. Анонимные пользователи маршрутизируются через `AuthDialog` сначала, чтобы адрес электронной почты отправителя был известен до отправки приглашений. Выполнение прав / привилегий — последующее действие. -Управляется выполнением `failproofai audit` — см. [Audit CLI](/ru/cli/audit) для базового механизма сканирования, поддерживаемых флагов и инвариантов кэша для каждой записи. Панель управления кэширует последний результат в `~/.failproofai/audit-dashboard.json` (режим `0600`, одиночный слот, новые запуски перезаписывают), поэтому повторные посещения мгновенны; **оба кэши для каждой записи и полного результата отклоняются при чтении, когда они старше 7 дней**, поэтому панель управления никогда не служит результатом возрастом в неделю — после TTL `/audit` переходит в состояние пустого значения и предлагает свежий запуск. Нажатие `[ re-audit now ]` близко внизу отчёта POST `/api/audit/run` с `noCache: true` — повторный аудит пропускает кэш для каждой записи и повторно сканирует каждую запись с нуля вместо того, чтобы молча возвращать кэшированный результат — и панель управления опрашивает `/api/audit/status` на частоте 1Hz до завершения запуска; липкая розовая полоса прогресса прикрепляется к верхней части видимого экрана во время запуска с таймером истекшего времени, и свежий результат переходит на место при успехе (без перезагрузки полной страницы; неудачный повторный аудит оставляет предыдущий отчёт нетронутым). При ошибке полоса становится красной с копией, ключируемой по `RerunError.kind` (`timeout` / `network` / `post_failed`). Состояние пустого значения (нет кэша или истекло) и состояние нулевых сеансов (кэш существует, но сканирование не нашло записей) выводятся отдельно. +Управляется выполнением `failproofai audit` — см. [Audit CLI](/ru/audit) для базового механизма сканирования, поддерживаемых флагов и инвариантов кэша для каждой записи. Панель управления кэширует последний результат в `~/.failproofai/audit-dashboard.json` (режим `0600`, одиночный слот, новые запуски перезаписывают), поэтому повторные посещения мгновенны; **оба кэши для каждой записи и полного результата отклоняются при чтении, когда они старше 7 дней**, поэтому панель управления никогда не служит результатом возрастом в неделю — после TTL `/audit` переходит в состояние пустого значения и предлагает свежий запуск. Нажатие `[ re-audit now ]` близко внизу отчёта POST `/api/audit/run` с `noCache: true` — повторный аудит пропускает кэш для каждой записи и повторно сканирует каждую запись с нуля вместо того, чтобы молча возвращать кэшированный результат — и панель управления опрашивает `/api/audit/status` на частоте 1Hz до завершения запуска; липкая розовая полоса прогресса прикрепляется к верхней части видимого экрана во время запуска с таймером истекшего времени, и свежий результат переходит на место при успехе (без перезагрузки полной страницы; неудачный повторный аудит оставляет предыдущий отчёт нетронутым). При ошибке полоса становится красной с копией, ключируемой по `RerunError.kind` (`timeout` / `network` / `post_failed`). Состояние пустого значения (нет кэша или истекло) и состояние нулевых сеансов (кэш существует, но сканирование не нашло записей) выводятся отдельно. ### Политики diff --git a/docs/ru/architecture.mdx b/docs/ru/how-it-works.mdx similarity index 100% rename from docs/ru/architecture.mdx rename to docs/ru/how-it-works.mdx diff --git a/docs/ru/introduction.mdx b/docs/ru/introduction.mdx index 90038880..224a7674 100644 --- a/docs/ru/introduction.mdx +++ b/docs/ru/introduction.mdx @@ -55,4 +55,4 @@ failproofai policies --install # включить политики (или п failproofai # запустить панель мониторинга ``` -Полный обзор см. в руководстве [Начало работы](/ru/getting-started). \ No newline at end of file +Полный обзор см. в руководстве [Начало работы](/ru/quickstart). \ No newline at end of file diff --git a/docs/ru/policies.mdx b/docs/ru/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/ru/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/ru/getting-started.mdx b/docs/ru/quickstart.mdx similarity index 100% rename from docs/ru/getting-started.mdx rename to docs/ru/quickstart.mdx diff --git a/docs/ru/reference/files.mdx b/docs/ru/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/ru/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/testing.mdx b/docs/testing.mdx index 0d0fb095..2d8808f8 100644 --- a/docs/testing.mdx +++ b/docs/testing.mdx @@ -1,5 +1,5 @@ --- -title: Testing +title: "Testing policies" description: "Unit tests, E2E tests, and test helpers" icon: flask-vial --- diff --git a/docs/tr/agent-support.mdx b/docs/tr/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/tr/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/tr/agenteye/cli-and-agents.mdx b/docs/tr/agenteye/cli-and-agents.mdx deleted file mode 100644 index 7d5ad5fe..00000000 --- a/docs/tr/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "Tüm Failproof AI Observability dağıtımınız, tek bir komut uzağında." ---- - - -Tüm Failproof AI Observability dağıtımınız, tek bir komut uzağında. Production'ı kontrol edin, bir API anahtarı oluşturun veya bir olayı onaylayın—hiç terminalinizi terk etmeden. Ardından herhangi birini CI'ye entegre edin veya bir kodlama aracısının bunu düz İngilizce ile yapmasına izin verin. - -```bash -pipx install agenteye -agenteye login --email you@example.com # a 6-digit code lands in your inbox -agenteye --json sessions --since 24h # every agent run from the last day, newest first -``` - -*`agenteye` CLI'si panoyla iletişim kurar. Sunucuya olayları gönderen collector'dan farklı bir araçtır.* - -## Tüm dağıtımınız, tek bir komut uzağında - -Hızlı bir soruyu cevaplamak için sekme atlama işini bırakın. `agenteye` CLI'si verilerinizi okur ve kuruluşunuzu tek bir ikili dosyadan yönetir; böylece dashboard'da tıklayarak cevap bulmanız gereken bir kontrol, yeniden çalıştırabileceğiniz, takma ad oluşturabileceğiniz veya bir runbook'a yapıştırabileceğiniz tek bir satıra dönüşür. Dört yüzeye erişebilirsiniz: - -- **Verilerinizi okuyun:** `sessions`, `events`, `evals` ve `errors`—zaman, agent ve ortama göre filtrelenmiş. -- **Kuruluşunuzu yönetin:** `keys`, `users`, `settings`, `alerts` ve `incidents`. -- **Analitik çalıştırın:** kaydedilmiş SQL artı olay verileriniz üzerinde ad hoc `query` çalıştırıcı. -- **Asistana sorun:** `agent ask` dashboard'da sohbet ettiğiniz salt okunur analistle bağlantı kurar. - -`pipx` ile bir kez kurun, bir e-postaya gelen 6 haneli koduyla oturum açın ve hazırsınız. Oturum yaklaşık bir gün sürer; süresi dolduğunda `agenteye login` komutunu yeniden çalıştırın. Production'ı hızlıca kontrol etmek, bir anahtar sağlamak veya çalışan bir olayı triage etmek için kullanın—tarayıcı açmadan: - -```bash -agenteye errors --since 24h --aggregate # what is breaking, grouped by error type -agenteye incidents list --state firing # what is on fire right now -agenteye keys create ci --add events:add # a key that can only push events, secret shown once -``` - -Önemli bir alışkanlık: `--json` gibi global seçenekler komuttan önce gelir. `agenteye --json sessions` doğru; `agenteye sessions --json` değildir. - -## Betik haline getirin, CI'ye entegre edin - -Her komut `--json` alır ve bu her şeyi değiştirir. Temiz JSON stdout'a yazılır, insan durumu ve uyarılar stderr'e gider; böylece `--json` çıktısı `jq`'ya doğrudan gider, kırpılacak hiçbir satır olmaz. Bu, CLI'yi hem sizin bir komut isteminde hem de kodlama aracısının çıktısını ayrıştırırken eşit derecede iyi kılar: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -Gözetimsiz çalışmak için tasarlanmıştır. Terminal bağlı olmadığında onay istemleri otomatik olarak atlanır; böylece hiçbir şey bir pipeline'da kalmaz ve her komut anlamlı bir çıkış kodu döndürür: `0` başarılı, `4` oturum açılmamış, `5` izin eksik (ileti adını verir, örneğin `alerts:write`), `3` dashboard ulaşılamaz. Bir betik kodu `4` için yeniden kimlik doğrulamak veya `5` için tam olarak bir yöneticiye ne sorması gerektiğini söylemek için dallanabilir, kör şekilde başarısız olmak yerine. - -## Bir kodlama aracısının bunu düz İngilizce ile yapmasına izin verin - -Daha da iyisi, tüm bu bayrakları hatırlamanız gerekmemeli. **CLI skill'i**, bir kodlama aracısı (Claude Code veya Codex gibi) CLI'yi düz İngilizce isteklerden yönlendirir öğreten `agenteye-cli` adlı küçük bir Agent Skill klasörüdür. "Bugün bir şey bozuk mu?" sorun ve aracı komutu seçer, sizin olarak çalıştırır ve cevabı yazılı olarak verir. - -Claude Code için, `agenteye-cli` klasörünü `~/.claude/skills/` dizinine bırakın ve otomatik olarak keşfedilir. Failproof AI Observability klasörü sağlar; zaten kurduğunuz CLI'yi yönlendirdiği için ekstra kurmaya gerek yoktur. Önce kendiniz oturum açın: skill, e-postaya gelen kod oturumunu sizin için tamamlayamaz. - -Aracı CLI'yi sizin olarak çalıştırdığından, oturmunuzun izin verdiği her şeyi yapabilir—okuma ve yazma işlemleri: anahtarlar oluşturun, ayarları değiştirin, olayları çözün. CLI'nin "emin misiniz?" istemi bir aracı için ateşlenmez; böylece skill yazılmıştır—tam komutu belirtir ve herhangi bir değişiklikten önce onay bekler. Siz onay adımısınız. - -```text -you Why did session run-001 fail? - -agent Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -Okuma işlemleri anlıktır ve her yazma işlemi duraklar: - -```text -you Give CI a key that can only push events. - -agent This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you yes - -agent Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## İlgili - -- [CLI reference](/tr/agenteye/cli): her komut, bayrak ve JSON şekli. -- [Aracılar için CLI tarifleri](/tr/agenteye/cli-recipes): kopyala-yapıştır `jq` desenleri ve çıkış kodu işleme. -- [CLI aracı skill'i](/tr/agenteye/cli-skill): `agenteye-cli` skill'ini kurun ve çalıştırın. -- [AI asistanı](/tr/agenteye/assistant): `agent ask` ile konuşan pano içi analist. \ No newline at end of file diff --git a/docs/tr/agenteye/cli-skill.mdx b/docs/tr/agenteye/cli-skill.mdx deleted file mode 100644 index 7bbfbf83..00000000 --- a/docs/tr/agenteye/cli-skill.mdx +++ /dev/null @@ -1,161 +0,0 @@ ---- ---- -title: "Failproof AI Observability CLI Agent Skill" -description: "Kodlama aracınıza \"bugün bir şey bozuk mu?\" sorusu sorun ve Failproof AI Observability verilerinizden canlı yanıt alın — hiç komut ezberlemek zorunda değilsiniz." ---- - - -Kodlama aracınıza *"bugün bir şey bozuk mu?"* sorusu sorun ve Failproof AI Observability canlı verilerinizden yanıt alın — hiç komut ezberlemek zorunda değilsiniz. **Failproof AI Observability CLI becerisi** (`agenteye-cli`), bir *Agent Becerisi*dir: kodlama aracı olarak Claude Code veya Codex'in isteğe bağlı olarak yükleyebileceği küçük bir talimat klasörü. Aracınızı, *"sadece etkinlik gönderebilecek CI için bir anahtar ver"* veya *"açık olayı onayla ve bana ata"* gibi basit İngilizce isteklerle [`agenteye` CLI](/tr/agenteye/cli) aracılığıyla Observability dağıtımınızı kullanmayı öğretir. - -Bu **değildir** bir hizmet veya ayrı bir ikili dosya; dağıtılacak hiçbir şey yoktur. Zaten yüklemiş olduğunuz CLI'nin üzerinde çalışır: ajan `agenteye --json …` komutunu çalıştırır, temiz JSON'u ayrıştırır ve size cevapı düz metin şeklinde verir. Yapabileceği her şey, aynı komutları kendiniz yazarak da yapabilirsiniz. - ---- - -## Diğer Failproof AI Observability arayüzleriyle ilişkisi - -Failproof AI Observability aynı verilere ve kontrollere ulaşmanız için dört yol sunar. Birbirlerini tamamlarlar: - -| Arayüz | Ne olduğu | Nerede çalışır | Şu durumlarda kullanın | -|---|---|---|---| -| **[CLI](/tr/agenteye/cli)** | `agenteye` için komut/bayrak başvurusu | Terminaliniz | Belirli bir komutu çalıştırmak veya betiklemek istediğinizde | -| **[CLI tarifleri](/tr/agenteye/cli-recipes)** | Kopyala-yapıştır `jq`/pipeline desenleri | Terminaliniz / betikleriniz | CLI'yi otomasyon içine bağlıyorsanız | -| **CLI becerisi** (bu belge) | CLI üzerinde doğal dil giriş kapısı | Kodlama aracınız, iş istasyonunuzda | Sadece sormak ve aracın komutu seçmesini bırakmak istediğinizde | -| **[Evaluator becerisi](/tr/agenteye/evaluator-skill)** | Puanlama hizmetinizi tasarlayan ve kuran kardeş beceri | Kodlama aracınız, iş istasyonunuzda | Puanlamayı *üretmek* istediğinizde, okumak değil | -| **[Python SDK becerisi](/tr/agenteye/python-sdk-skill)** | Aracınıza telemetri yayması için enstrüman takılan kardeş beceri | Kodlama aracınız, iş istasyonunuzda | Aracınızın bu becerinin okuduğu olayları *üretmesini* istediğinizde | -| **[Panodaki AI asistanı](/tr/agenteye/assistant)** | Panoya gömülü sohbet | Sunucu tarafı (panoda) | Verileriniz üzerinde pano içi soru-cevap istediğinizde | - -Becerinin kendi imtiyazı yoktur; sadece sözlerinizi sizin olarak çalışan CLI çağrılarına dönüştürür: - -```mermaid -flowchart TD - YOU["siz: 'açık olayı onayla'"] --> AGENT["kodlama aracı (Claude Code / Codex)
    agenteye-cli becerisini yükler"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|kimliğiniz doğrulanan CLI oturumu| API["Observability panosu API"] -``` - -### Panodaki AI asistanına karşı: önemli bir fark - -Bunlar çok farklı etki alanlarına sahip iki farklı araçtır: - -- **Panodaki AI asistanı** ([AI asistanı](/tr/agenteye/assistant)) panoya gömülü bir sohbet, ajan hizmeti tarafından desteklenir. **Yalnızca okunur artı onay gerektiren yazma**: kaydedilen sorguları ve panoları hazırlayabilir, ancak her yazma işlemi açık tıklamanızı bekler ve hiçbir zaman silmez. `agent:use` izni tarafından korunan ve yalnızca görüntülediğiniz kuruluşun verilerini görür. -- **CLI becerisi** *sizin* iş istasyonunuzda *sizin* kodlama aracınız içinde çalışır ve `agenteye` CLI'yi **sizin olarak** kullanır. CLI'nin **tam yüzeyini, değişiklikleri** (API anahtarları oluşturma/döndürme/devre dışı bırakma, kuruluş ayarlarını değiştirme, olayları çözme, kaydedilen sorguları silme) gerçekleştirebilir; bunlar yalnızca CLI oturumunuzun izinleriyle sınırlanır. Bunu tam olarak bu komutları elle çalıştırıyor gibi dikkatle kullanın. - ---- - -## Ön koşullar - -1. **`agenteye` CLI yüklü** ve `PATH` içinde (bkz. [CLI](/tr/agenteye/cli) başvurusu: `pipx install agenteye`). -2. **Pano URL'niz ayarlanmış** (`AGENTEYE_DASHBOARD_URL` veya ajan `--base-url` iletir). -3. **Oturum açmış bir oturum**: önce kendiniz `agenteye login` çalıştırın. Beceri **yapamaz** e-postayla gelen tek seferlik kod girişini sizin için tamamlamak; oturum eksikse veya süresi dolmuşsa (`CLI çıkış kodu 4`) size `agenteye login` çalıştırmasını söyler. - ---- - -## Nerede bulabileceğiniz - -Beceri, Failproof AI'nın genel beceri koleksiyonunda yayınlanır: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -Hiçbir şey kilitli değildir — depo halka açıktır ve becerinin kendi kimlik bilgisine ihtiyacı yoktur, çünkü sadece **genel** `agenteye` CLI'yi *sizin* panonuza karşı, *sizin* oturum açtığınız oturumu kullanarak kullanır. Bunu almak için kimseye sormak zorunda değilsiniz. - -`pipx install agenteye` paketi içinde kendi klasörü olarak gönderildiğine, **değil** içinde olduğunu, bu nedenle orada arama yapmayın. - -## Beceriyi yükleme - -En hızlı yol [`skills`](https://skills.sh) CLI'dir; bu klasörü getirir ve aracınızın aradığı yere koyar: - -```bash -# Claude Code, yalnızca bu proje -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# her proje (~/.claude/skills/ dizinine yükler) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Bunun yerine Codex -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -Sonra bunu başka herhangi bir beceri gibi yönetin: - -```bash -npx skills list -a claude-code # yüklü olanlar -npx skills update agenteye-cli # en son sürümü al -npx skills remove agenteye-cli # kaldır -``` - -El ile yüklemeyi tercih ediyor musunuz? Bir Agent Becerisi sadece bir `SKILL.md` (artı isteğe bağlı referanslar) içeren bir klasördür; kopyalama da işe yarar: - -- **Claude Code**: `agenteye-cli/` klasörünü `~/.claude/skills/` (her proje) veya `/.claude/skills/` (yalnızca o repo) içine koyun. Claude Code otomatik olarak keşfeder — `/skills` listinde doğrulayın veya açıklamasıyla eşleşen bir soru sorun. -- **Codex (OpenAI)**: Codex aynı `SKILL.md` dosyasını okur. Paketlenen `agents/openai.yaml`, `allow_implicit_invocation: true` olarak ayarlanmıştır; bu nedenle görev eşleştiğinde Codex beceriyi otomatik olarak seçer; aksi takdirde bunu açıkça `$agenteye-cli` olarak çağırın. - ---- - -## Güvenlik: değişiklikler bir ajan CLI çalıştırdığında uyarı vermez - -> **Uyarı:** Bir ajanın değişiklik yapmasına izin vermeden önce bunu okuyun. - -`agenteye` CLI normalde yıkıcı bir eylemden önce *"emin misiniz?"* sorular. **Hiçbir zaman terminale bağlı olmadığında (tam olarak bir kodlama aracının onu çalıştırma şekli) ve `--json` de atlar onayı, bu onaylamayı otomatik olarak atlayın.** Bu nedenle güvenlik uyarısı ajan için **ateşlenmez**. - -Beceri bunu telafi etmek için yazılmıştır: çalıştıracağı tam komutu belirtmesi ve herhangi bir durum değişikliğinden önce açık **OK** alması talimatı verilir. Bu disiplini koruyun. Failproof AI Observability'yi bir ajan aracılığıyla kullanırken, *siz* onaylama adımısınız. İzlenecek durum değiştiren komutlar: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- yazma `incidents` alt komutları: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -**Gözlem** altındaki her şey (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) yalnızca okunur ve hiçbir şeyi değiştirmez. - -Ajan **sizin olarak** davrandığı için, yalnızca oturum açmanızın izin verdiği işlemleri yapabilir; izinler **kuruluş başına** çözülür (bkz. [API anahtarları](/tr/agenteye/api-keys)). İzniniz olmayan bir komut, çıkış kodu `5` ile tam izin adı döndürür; bu sayede ajan hata alması yerine tam olarak ne isteyeceğini bir yöneticiden söyleyebilir. - ---- - -## Ne sorabilirsiniz - -İşte gerçek bir gidiş-dönüş; bir okuma, ardından sizin tamam vermesini bekleyen bir değişiklik: - -```text -siz ▸ Son gün içinde bir şey bozuk mu? - -ajan ▸ Çalıştırılıyor: agenteye --json errors --since 24h --aggregate - Son 24 saat içinde 3 hata türü. En büyüğü TimeoutError - (18 isabet, tamamı checkout-agent üzerinde), ardından RateLimited (5). - TimeoutError için başarısız oturumları görmek ister misiniz? - -siz ▸ CI'ye sadece etkinlik gönderebilecek bir anahtar ver. - -ajan ▸ Bu bir API anahtarı oluşturur. Çalıştıracağım: - agenteye keys create ci --add events:add - Devam etmeye izin var mı? - -siz ▸ evet - -ajan ▸ Bitti. "ci" anahtarı events:add ile oluşturuldu. - Sır sadece bir kez gösterilir, bu nedenle şimdi kaydedin. - Yeniden yazdıramam. -``` - -Beceri her basit İngilizce niyeti doğru `agenteye` komutuna eşler; geçerli değerleri önceden keşfeder (`list `, `whoami`) böylece tahmin etmez ve herhangi bir değişiklikten önce tam komutu belirtir. Daha fazla örnek: - -- *"Son gün içinde bir şey bozuk / başarısız mı?"* → `errors --since 24h --aggregate`, sonra bir döküm. -- *"Oturum `run-001` neden başarısız oldu?"* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *"Bu hafta kalite nasıl gidiyor?"* → `evals --aggregate --since 7d`, sonra düşük puanlamaya dalın. -- *"CI'ye sadece etkinlik gönderebilecek bir anahtar ver."* → `keys create ci --add events:add` (komutu belirtir, sonra oluşturur ve tek seferlik sırrı yakalar). -- *"Kimin erişimi var? Dana'yı salt okunur yap."* → `users list` → `users update dana@… --permission-set read-only` (sizinle onayladıktan sonra). -- *"Açık olayı onayla ve bana ata."* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`. - -Bunların arkasındaki tam komutlar, bayraklar ve JSON şekilleri için bkz. [CLI](/tr/agenteye/cli) başvurusu ve [Ajanlar için CLI tarifleri](/tr/agenteye/cli-recipes). - ---- - -## Sonraki adımlar - -- **[CLI](/tr/agenteye/cli)**: `agenteye` için tam komut ve bayrak başvurusu. -- **[Ajanlar için CLI tarifleri](/tr/agenteye/cli-recipes)**: kopyala-yapıştır `jq` desenleri ve çıkış kodu işleme. -- **[Evaluator ajan becerisi](/tr/agenteye/evaluator-skill)**: kardeş beceri, `agenteye evals` içindeki puanları okuyan puanlayıcıyı kurmak için. -- **[Python SDK ajan becerisi](/tr/agenteye/python-sdk-skill)**: kardeş beceri, `agenteye` nin okuduğu telemetriyi yayması için aracı enstrüman takma için. -- **[AI asistanı](/tr/agenteye/assistant)**: pano içi asistan (bu terminal beceriyle karıştırılmamalıdır). -- **[API anahtarları](/tr/agenteye/api-keys)**: becerinin yapabileceklerini sınırlayan kuruluş başına izin modeli. \ No newline at end of file diff --git a/docs/tr/agenteye/codex-capture.mdx b/docs/tr/agenteye/codex-capture.mdx deleted file mode 100644 index 708eb7c5..00000000 --- a/docs/tr/agenteye/codex-capture.mdx +++ /dev/null @@ -1,56 +0,0 @@ ---- ---- -title: "Codex oturum yakalama" -description: "Ekibinizin yerel OpenAI Codex oturumlarını AgentEye'a sıradan oturumlar ve etkinlikler olarak takip edin — Codex'i nasıl çalıştırdıklarında hiçbir değişiklik olmadan." ---- - -Mühendisleriniz zaten her gün OpenAI Codex kullanıyor. Codex oturum yakalama, bu kodlama oturumlarını AgentEye'a sıradan oturumlar ve etkinlikler olarak getirerek, bunları gözlemlediğiniz diğer her şeyin yanında arayabilir, tekrar oynatabilir ve değerlendirebilirsiniz. [Python SDK](/tr/agenteye/python-sdk) ile tamamlayıcı özelliktedir: SDK yazdığınız ajanları enstrümente ederken, bu özellik ekibinizin zaten yaptığı Codex işini yakalar — çalışma biçiminde hiçbir değişiklik olmadan. - -Küçük bir arka plan toplayıcısı, Codex'in yerel oturum transkriptlerini yazılırken okur ve AgentEye'a gönderir. Makine başına bir toplayıcı, bir kerede tüm yerel Codex yüzeyini yakalar — yüzey başına kurulum gerekmez. - -Aynı toplayıcı diğer ajanları da yakalar — bkz. [OpenClaw](/tr/agenteye/openclaw-capture) ve [Hermes](/tr/agenteye/hermes-capture). Çalıştırdığınız her birini etkinleştirin; tek bir toplayıcı aynı anda birkaçını yakalayabilir. - ---- - -## Neler yakalanır - -**Yerel** olarak çalışan her Codex yüzeyi, diske yazılan aynı oturum transkriptlerini üretir ve toplayıcı bunların tümünü alır: - -- Codex **CLI** ve `codex exec` -- **VS Code / IDE uzantısı** -- **masaüstü uygulaması**, yerel olarak oturum çalıştırdığında - -Her Codex oturumu bir AgentEye [oturum](/tr/agenteye/sessions) haline gelir; kullanıcı ve asistan mesajları, akıl yürütme, araç çağrıları, araç sonuçları ve token kullanımı eşleşen [etkinlik](/tr/agenteye/event-stream) olur. Her oturumun geldiği yüzey (CLI, IDE veya masaüstü) kaydedilir, böylece bunları ayırt edebilirsiniz. - -> **Bulut oturumları yakalanmaz.** Masaüstü uygulaması giderek daha fazla oturumu Codex bulutunda çalıştırır ve makinede yalnızca meta verilerini tutarken — okunacak yerel transkript yoktur. Yalnızca yerel olarak yürütülen oturumlar yakalanır. - ---- - -## Etkinleştirme - -Yakalama, etkinleştirene kadar kapalıdır. Toplayıcıyı `events:add` izni olan bir API anahtarıyla kurun (bkz. [API anahtarları](/tr/agenteye/api-keys)) ve Codex yakalamayı etkinleştirin: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -Bu toplayıcıyı kurar, arka plan hizmeti olarak kaydeder ve yakalamaya başlar. Çalıştığını doğrulayın: - -```bash -agenteye-collector health -``` - -İlk çalıştırmada, mevcut Codex oturumlarınız bir kez geri doldurulur ve yeni etkinlik birkaç saniye içinde akışa başlar. Codex'in kendi dosyaları yalnızca okunur — asla değiştirilmez, taşınmaz veya silinmez — ve her oturum, yeniden başlatmalar arasında bile tam olarak bir kez gönderilir. - ---- - -## Nerede görüntülenir - -Yakalanan oturumlar **Oturumlar**'da ve etkinlikleri **Etkinlik** akışında görüntülenir, gözlemlediğiniz diğer herhangi bir ajan gibi — bu nedenle [oturum tekrar oynatma](/tr/agenteye/sessions), [arama](/tr/agenteye/queries), [değerlendirmeler](/tr/agenteye/evaluations) ve [uyarılar](/tr/agenteye/alerts) tümü bunlar üzerinde çalışır. Codex ajanına göre filtreleyerek bunları tek başına görün. - ---- - -## Gizlilik - -Codex transkriptleri tam oturumu içerir — komut çıktısı, dosya içeriği ve Codex'in okuduğu veya yazdığı her şey dahil — ve sırlar içerebilir. Yakalanan oturumlar olduğu gibi gönderilir, bu nedenle yakalamayı yalnızca bu içeriği AgentEye'da merkezi hale getirmenin uygun olduğu makinelerde ve takımlar için etkinleştirin ve toplayıcıya yalnızca `events:add` kapsamında bir anahtar verin. Verilerinizin nasıl yalıtılı tutulduğu hakkında [Güvenlik](/tr/agenteye/security) bölümüne bakın. \ No newline at end of file diff --git a/docs/tr/agenteye/concepts.mdx b/docs/tr/agenteye/concepts.mdx deleted file mode 100644 index 3d7dec42..00000000 --- a/docs/tr/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "Kavramlar" -description: "Failproof AI Observability'nin sözlüğü — etkinlikler, oturumlar, değerlendirmeler, denetimler, bulgular ve olaylar — tek bir yerde tanımlanmıştır." ---- - - -Bu sayfa, Failproof AI Observability'nin kullandığı sözlüğü tanımlar. Diğer bir kılavuzda karşılaştığınız bir terim yabancı geliyorsa, burada tanımlanmıştır. Bunu baştan sona okumanız gerekmez: göz atın veya anlamını netleştirmek istediğiniz bir kelimeye ulaştığında geri dönün. - ---- - -## Veri modeli - -**Etkinlik** -En küçük veri birimi. Bir etkinlik, aracınızın attığı tek bir adımı kaydeder: bir `tool_use`, bir `model_request`, bir `hook_completed`, bir `error` vb. Aracınız etkinlikleri [Python SDK](/tr/agenteye/python-sdk) aracılığıyla yayar; bunlar **Events** sayfasında canlı olarak görünür. - -**Oturum** -Bir aracı çalıştırması, `session_id` ile tanımlanır. Bir oturum, bu kimliği paylaşan tüm etkinliklerin **Sessions** sayfasında tek bir satırda toplandığı ve ayrıntı sayfasında bir yürütme grafiği olarak çizildiği haldir. Bir oturum genellikle `agent_start` ile başlar ve `agent_end` ile biter. - -**Aracı** -Bir çalıştırma içindeki `agent_id` ile tanımlanan adlandırılmış bir aktör. Bir çalıştırma birkaç aracı içerebilir: örneğin, bir özet aracı oluşturan bir planlayıcı. Alt aracılar bir `parent_id` taşır; bu, Failproof AI Observability'nin yürütme grafiğinde onları kendi şeritlerinde çizmesini sağlayan şeydir. - -**Ortam** -Çalıştırmanın nerede gerçekleştiğini gösteren bir etiket: `production`, `staging`, `dev`. SDK'yı yapılandırırken bunu bir kez ayarlarsınız. Hemen hemen her pano sayfası ortama göre filtreleyebilir. - -**Bağlam penceresi doldurma** -Bir modelin bağlam penceresinin bir yanıt tarafından tüketilen yüzdesi. Failproof AI Observability bunu tanıdığı modellerde `model_response` etkinliklerine damgalar, bu sayede istem büyümesi ve yaklaşan sıkıştırma doğrudan etkinlik akışında görünür. - ---- - -## Kalite - -**Değerlendirme** -Çalıştırdığınız bir puanlama hizmeti tarafından üretilen bitmişs oturum için bir kalite puanı. Değerlendirmeler isteğe bağlıdır: bir değerlendiriciye bağlanana kadar oturumlar kaydedilir ancak puanlanmaz. Her değerlendirme birkaç adlandırılmış puan taşıyabilir (örneğin `helpfulness`, `factuality`, `tool_efficiency`), her biri kısa bir gerekçe notu ile. Bkz. [Evaluation suite](/tr/agenteye/evaluation-suite). - -**Puan anahtarı** -Değerlendirici tarafından bildirilen bir boyutun adı, örneğin `helpfulness`. Uyarılar ve denetimler belirli bir puan anahtarını zaman içinde izleyebilir. - -**Değerlendiricisi** -Puanlama hizmetiniz. Failproof AI Observability, bitmişs bir çalıştırmanın transkriptini ona POST eder ve döndürdüğü puanları depolar. Varsayılan bir değerlendiricisi göndermiyor; puanlama mantığı sizindir. - ---- - -## Başarısızlıkları bulma ve düzeltme - -**Hook** -Aracı çerçevesinin bir adımın etrafında çalıştırdığı bir koruma veya yan etki: içerik güvenliği kontrolü, KŞV redaksiyonu, bütçe koruması. Hook'lar `hook_triggered` / `hook_completed` etkinlikleri bir `outcome` (allow, deny, modify) ile yayar ve kendi gözlem sayfasını alırlar. - -**Uyarı kuralı** -Bir metrik ayarladığınız eşiği aştığında ateşlenen bir kural: hata oranı, p95 gecikme, token maliyeti veya bir değerlendiricisi puanı. Bir kural ateşlendiğinde, bir olay açar ve seçtiğiniz kanallara (e-posta, Slack, webhook, pano içi) bildirir. Bkz. [Alerts](/tr/agenteye/alerts). - -**Olay** -Bir uyarı kuralı ateşlendiğinde açılan açık bir sorun. Olayların bir yaşam döngüsü (kabullenme, atama, çözme) ve her eylemi kaydeden bir etkinlik zaman çizelgesi vardır. Ayrıca manuel olarak da açabilirsiniz. - -**Denetim** -Henüz bir kural yazmadığınız hata kalıpları için oturumlar *arasında* günlükleri inceleyen yinelenen bir araştırma (saatlik ila haftalık): hata kümeleri, düşük puanlar, gecikme aykırı değerleri, araç çağrısı döngüleri ve hiç bitmemiş çalıştırmalar. Bir uyarı zaten hakkında bildiğiniz bir metriği izlerken, denetim sonra neye bakmanız gerektiğini söyler. Bkz. [Audits](/tr/agenteye/audits). - -**Bulgu** -Bir denetim çalıştırmasından sıralanmış, kanıtla desteklenmiş bir sonuç. Bir bulgu bir kalıp adlandırır, arkasındaki tam oturumları bağlar ve triyaj yaşam döngüsü (kabullenme, çözme, sessiz yapma, reddetme) taşır. Failproof AI Observability, bulgularını çalıştırmadan çalıştırmaya yineleme ortadan kaldırır, böylece bilinen bir kalıp birikirmek yerine güncellenir. - -**AI asistanı** -Aracılarınız hakkında sorulara düz İngilizce olarak, kendi verileriniz üzerinde cevap veren pano içi sohbet. Varsayılan olarak salt okunurdur; oluşturduğu herhangi bir şey (kaydedilen sorgu, pano) onay kapısından geçer ve asla silemez. Bkz. [AI assistant](/tr/agenteye/assistant). - ---- - -## Çalıştırma - -**Kuruluş (kiracı)** -Yalıtılmış bir çalışma alanı. Bir Failproof AI Observability örneği birçok kuruluşu barındırabilir, her biri kendi kullanıcıları, anahtarları ve verileri ile. Her pano URL'si kuruluş slug'ınızın altında kapsamlıdır (`//…`). - -**Toplayıcı** -`agenteye-collector`, her aracı makinesinde çalışan, SDK'nın diske yazdığı etkinlikleri toplu hale getiren ve sunucuya gönderen hafif daemon. - -**API anahtarı** -Bir istemciyi sunucuya karşı kimlik doğrulayan kapsamlı bir belirteç. Anahtarlar granüler izinler taşır (örneğin toplayıcı için `events:add`, pano anahtarı için salt okunur kapsamlar). Bkz. [API keys](/tr/agenteye/api-keys). - -**Sunucu** -Alım ve API hizmeti. Etkinlikleri alır, operasyonel durumu veritabanlarınızda depolar ve panoyu ve CLI'yi sunar. - -**Pano** -Web kullanıcı arabirimi. Her sayfa bir kuruluşun kapsamında ve sunucunun API'si aracılığıyla okunur. - ---- - -## Sonraki adımlar - -- [Overview](/tr/agenteye/overview): bu parçaların nasıl birbirine uyduğu. -- [Observability](/tr/agenteye/observability): gözlem yüzeyleri (Events, Sessions, Models, Tools, Hooks, Errors). \ No newline at end of file diff --git a/docs/tr/agenteye/evaluator-skill.mdx b/docs/tr/agenteye/evaluator-skill.mdx deleted file mode 100644 index 08ca9861..00000000 --- a/docs/tr/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: "Failproof AI Gözlemlenebilirlik Değerlendirici Ajan Becerisi" -description: "\"Ajanımız bazen kötü performans gösteriyor\" düşüncesinden dağıtılmış bir puanlama hizmetine geçin; kodlama ajanınız hem kararı hem de oluşturmayı yapsın." ---- - -*"Ajanımız bazen kötü performans gösteriyor"* düşüncesinden dağıtılmış bir puanlama hizmetine geçin; kodlama ajanınız hem kararı hem de oluşturmayı yapsın. **Failproof AI Gözlemlenebilirlik değerlendirici becerisi** (`agenteye-evaluator`), bir *Ajan Becerisidir*: bir kodlama ajan (Claude Code veya Codex gibi) tarafından isteğe bağlı olarak yüklenen, bir klasör içinde barındırılan talimatlar. Ajanı, *sizin* ajan için izlenmeye değer olan kalite boyutlarını belirlemek, ardından [değerlendirici hizmetini](/tr/agenteye/evaluation-suite) yazıp, test edip ve dağıtmak öğretir. - -Bu sistem **değildir**: barındırılan bir puanlayıcı, yüklendiğiniz bir kayıt defteri veya bir eklenti sistemi. Değerlendiricileriniz, [Değerlendirme paketi](/tr/agenteye/evaluation-suite) kılavuzunda açıklandığı gibi, kendi altyapınızda çalışan kendi HTTP hizmetiniz olarak kalır. Beceri, ajanınızı bunu iyi inşa etmeyi öğretir; bu nedenle yaptığı her şey, aynı kodu yazarak siz de yapabilirsiniz. - ---- - -## Zor kısım neyi puanlamak gerektiğine karar vermek - -SDK yüzeyi küçüktür — bir dekoratör ve iki model — ve bir ajan bunu [kontratı](/tr/agenteye/evaluation-suite#http-contract) tek başına yazabilir. Sorun burada değildir. Sorun, yanlış şeyi puanlamalarıdır; yanlış şeyi puanlayan bir değerlendirici hiç olmamasından daha kötüdür: herkesin görmezden gelmeyi öğrendiği bir pano üretir. - -Bu nedenle becerinin çoğu, kod yazmadan önceki kısımdır. Ajanı sizi görüşmeye (*"iyi giden bir işlemi anlatın; şimdi kötü gideni"*), ardından [`agenteye` CLI](/tr/agenteye/cli) aracılığıyla gerçek seanslarınızı çekerek end-to-end okumaya alır. Bu iki yarı genellikle anlaşamaz ve arası fark önemlidir: ölçmeyi niyet ettiğiniz şey ile transkriplerinizin gerçekten destekleyebileceği şey arasındaki boşluk. Bir boyut ancak **olaylardan hesaplanabilir** ve **ayırıcı** ise hayatta kalır — eğer hem iyi çalışmanızda hem de kötü çalışmanızda 0.9 puan alırsa, hiçbir şey öğretmez ve kesilir. - -Geri dönen şey, 2-4 boyutun bir teklifi ve ona ilişkin akıl yürütmedir; bir satır yazılmadan önce sizin onay vermeniz için. - -```mermaid -flowchart TD - YOU["siz: 'Destek botum için evaluasyonlar istiyorum'"] --> AGENT["kodlama ajan (Claude Code / Codex)
    agenteye-evaluator becerisini yükler"] - AGENT -->|"görüşme: iyi vs kötü nasıl görünür?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["gerçek seanslarınız
    aslında ne olur"] - DATA --> DIMS["2-4 boyut, siz onay verirsiniz"] - DIMS --> SVC["değerlendirici hizmetiniz
    agenteye-evaluator SDK"] - SVC --> SCORES["puanlar panoya
    ve agenteye evallere iner"] -``` - ---- - -## Diğer değerlendirme bileşenleriyle ilişkisi - -Dört belge puanlamayı kapsar ve sırayla birbirini devreye sokar: - -| Sayfa | Nedir | Ne zaman kullanın | -|---|---|---| -| **[Değerlendirmeler](/tr/agenteye/evaluations)** | Özellik: oturum ızgarasında puanlar, panolar, yeniden değerlendir | Otomatik puanlamanın ne getirdiğini bilmek istiyorsunuz | -| **[Değerlendirme paketi](/tr/agenteye/evaluation-suite)** | HTTP kontratı, SDK, sunucu ortam değişkenleri | Değerlendiricinin kendisini uygulıyor veya debug ediyor | -| **Değerlendirici becerisi** (bu belge) | Puanlaycı tasarlamada *ve* oluşturmada doğal dil giriş kapısı | "Evaluasyonlar istiyorum" ile çalışan bir hizmetin kapısında olmak istiyorsunuz | -| **[CLI becerisi](/tr/agenteye/cli-skill)** | `agenteye` CLI'de doğal dil giriş kapısı | Zaten sahip olduğunuz puanları *okumak* istiyorsunuz | -| **[Python SDK becerisi](/tr/agenteye/python-sdk-skill)** | Ajanınızı enstrümanter etmekte doğal dil giriş kapısı | Ajanınız henüz seanslar yaymıyor — puanlanacak hiçbir şey yok | - -### CLI becerisi ile karşılaştırma: oluştur versus oku - -İki beceri kasıtlı olarak örtüşmez ve her ikisini yüklemek normal kurulumudur — ajan ne sorduğunuza bağlı olarak aralarında seçim yapar: - -- **`agenteye-evaluator`** (bu belge) puanları *üreten* şeyi oluşturur. İşi puanlar ilk kez inmesi sırasında biter. -- **[`agenteye-cli`](/tr/agenteye/cli-skill)** zaten var olan puanları okur (`agenteye evals`). *"Bu hafta kalite düştü mü?"* onun sorusudur, bu becerinin değil. - ---- - -## Ön Koşullar - -1. **`agenteye` CLI yüklü ve oturum açmış** (`pipx install agenteye`, ardından `agenteye login`). Beceri bunu iki kez kullanır: tasarladığı gerçek seansları çekmek için ve seansların sonunda puanlarınızın geldiğini doğrulamak için. Oturumunuzun `events:read` gereksinimi vardır, ayrıca bu son kontrol için `evaluations:read`. CLI becerisi ile birlikte, e-posta ile gönderilen tek kullanımlık kod girişini **tamamlayamaz**. -2. **Değerlendiricinin yaşayacağı bir yer.** Bir imaja yerleştirilir ve uzun süreli bir hizmet olarak çalıştırılır; bu nedenle gerçek bir repo'ya ihtiyaç vardır, geçici bir dosyaya değil. Değerlendiriciler sık sık kendi repo'sunda yaşar, puanlanan ajanından ayrı — beceri var olanı arar ve yeni bir tane oluşturmadan önce sorar. -3. **`agenteye-evaluator` SDK tekerleği** — ajanınız `pip` komutlarını yazmaya başlamadan sonraki bölümü okuyun. - ---- - -## Nereden alınır - -Beceri, Failproof AI'ın herkese açık beceri koleksiyonunda yayınlanır: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -Depo herkese açıktır ve becerinin kendi kimlik bilgisine ihtiyacı yoktur — yalnızca `agenteye` CLI'yi oturum açtığınız seansla çalıştırır ve kodunuzu *kendi* repo'nuzda yazar. Kendi klasörü olarak gönderilir ve `pipx install agenteye` paketi içinde **değildir**; bu nedenle onu orada aramayın. - -## Becerisini Kurma - -En hızlı yol [`skills`](https://skills.sh) CLI'dir; bu klasörü getirir ve ajanınızın baktığı yere bırakır: - -```bash -# Claude Code, yalnızca bu proje -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# her proje (~/.claude/skills/ dosyasına yükler) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Yerine Codex -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -Ardından diğer beceriler gibi yönetin: - -```bash -npx skills list -a claude-code # ne yüklü -npx skills update agenteye-evaluator # en son sürümü çek -npx skills remove agenteye-evaluator # kaldır -``` - -Elle yüklemek tercih misiniz? Bir Ajan Becerisi, `SKILL.md` içeren bir klasördür (artı isteğe bağlı referanslar); bu nedenle kopyalama işe yarar: - -- **Claude Code**: `agenteye-evaluator/` klasörünü `~/.claude/skills/` (her proje) veya `/.claude/skills/` (yalnızca bu repo) içine koyun. Claude Code bunu otomatik olarak bulur — `/skills` listesi ile doğrulayın veya sadece evaluasyonlar isteyin. -- **Codex (OpenAI)**: Codex aynı `SKILL.md` dosyasını okur. Paketlenmiş `agents/openai.yaml`, `allow_implicit_invocation: true` ayarlar; bu nedenle bir görev eşleştiğinde Codex beceriyi otomatik seçer; aksi takdirde açıkça `$agenteye-evaluator` olarak çağırın. - ---- - -## SDK genel PyPI'de değildir - -> **Uyarı:** Bir ajanın SDK yüklemesine izin vermeden önce bunu okuyun. - -Beceri herkese açıktır; onu çalıştırdığı SDK değildir. `agenteye-evaluator` yalnızca özel bir sürüm yapı olarak gönderilir ve `agenteye` gibi farklı olarak, ad **genel PyPI'de açıklanmadıdır** — bu nedenle basit `pip install agenteye-evaluator` komutu, üretim transkriplerinizi okuyan hizmete başka bir kişinin paketini çekebilir. Bu bir yazım hatası değil, bir tedarik zinciri sorunudur. - -Beceri bunu bilir ve yerine bir yükleme merdiveninde aşağı doğru çalışır; ilk geçerli basamağında durur: AgentEye repo'sunun içindeyseniz monorepo kaynağı; aksi takdirde GitHub Releases'ten özel sürüm tekerleği (erişim gerekir); ikisi de erişilemezse **durur ve Failproof AI iletişim kişinizden tekerleği istemenizi söyler** uydurmak yerine. - -Eğer ajanınız genel PyPI'den basit `pip install agenteye-evaluator` önerirse, bu becerinin hiç yüklenmediğini gösterir. Orada durun ve yüklendiğini kontrol edin. - ---- - -## Ona ne sorabilirsiniz - -Gerçek bir tur sonu, belirsiz bir sorudan imza faydalı bir tasarımla biter, kodla değil: - -```text -siz ▸ Destek botumuz için evaluasyonlar istiyorum. Bazen kötü - olduğunu düşünüyorum. - -ajan ▸ Herhangi bir şey yazmadan önce: iyi giden bir işlemi anlatın, - ve kötü gideni. - -siz ▸ İyi: dokümanlardan cevaplar. Kötü: yok sayıyor, - veya "destek ile iletişime geçin" diyor. - -ajan ▸ Çalışıyor: agenteye --json sessions --status error --since 7d - 5 seansı okuyorum, ikisini kötü olarak yazıyorsunuz. - - Söyledikleriniz gerçekçiliği ve sapıtmayı söylüyor. Transkripler - sapıtmada anlaşıyor (5'in 2'si destek devredimiyle bitiyor, - ve model_response.content'de açık), fakat gerçekçiliğin - olaylarda taşımadığı bir yer gerçeğine ihtiyacı vardır. - - Yani üç şeyi puanlarım: - - sapıtma kurallar, ucuz, iki işlemi ayırır - - araç_verimliği kurallar, işlemler arası döngüyü yakalar - - temelcilik LLM hakim: aldığı doklarda her iddia var mı? - - "gerçekçiliğin" olmayan yer gerçeğine ihtiyacı vardır; temelcilik - transkriplerinizin fiilen destekleyebildiği en yakın şey. - Bu üç adla iyi misiniz? Daha sonra yeniden adlandırma geçmişi böler. - -siz ▸ evet - -ajan ▸ Bu repo'da değerlendirici yok. Bir tane buraya kurmalı mıyım, - yoksa başka yerde varsa? -``` - -Buradan kurallar tabanlı boyutları ilk yazar (bedava, anında, belirleyici), bunları kilitlenmeler ve asla bitmeyenler de dahil gerçek yakalanmış seanslar aracılığıyla test eder, ve yalnızca öznel boyut için LLM hakime ulaşır. [Gönderici sınırlarını](/tr/agenteye/evaluation-suite#configuring-the-server) bilir — 30s istek zaman aşımı ve 8 eşzamanlı çağrı dağıtım genelinde — yani hakim güvenilir şekilde sığmazsa, `JobPending` ile eşzamansız gider, hakim iptal edilmiş ve beş kez yeniden denenmiş olmasına izin vermez. - -Daha sonra dağıtır, iki sunucu ortam değişkenini ayarlar, ve `agenteye --json evals --session-id ` ile doğrular ki puanlar gerçekten indi. Puanlar inmek tek kanıttır. - ---- - -## Nelere dikkat edin - -- **Boyut adları neredeyse kalıcıdır.** Puan anahtarları keyfi dizeler ve platform gönderdiğiniz şeyi eğilimlendirir; bu nedenle aşağı akış hiçbir şey kötü seçimi düzeltmez. Daha sonra yeniden adlandırın ve geçmiş bölünür: eski seanslar eski anahtarı tutar ve eğilim kırılır. Bu, becerinin kod yazmadan önce açık onay aldığı nedeni — bu istem ciddiye alın. -- **Sabitler gerçek üretim transkriplerileridir.** Gerçek seanslar aracılığıyla tasarlamak onları diske çekmek anlamına gelir ve müşteri verisi içerebilirler. Beceri git'e teslim etmeden önce sorar; şüphede, `fixtures/` repo dışında tutun ve her geliştirici kendi tarafını çeksin. -- **Ajan her transkripti okuyan bir hizmet yazar ve dağıtır.** Sizin olarak davranır, CLI oturumunuzun izinleriyle sınırlı; fakat üretim verisine dokunulan diğer kodlar gibi değerlendiriciye bakın. - ---- - -## Sonraki adımlar - -- **[Değerlendirme paketi](/tr/agenteye/evaluation-suite)**: HTTP kontratı, SDK ve becerinin yapılandırdığı sunucu ortam değişkenleri. -- **[Değerlendirmeler](/tr/agenteye/evaluations)**: puanlar indikten sonra nerede gösterildiği. -- **[CLI becerisi](/tr/agenteye/cli-skill)**: puan oluşturmak yerine sonuçları okuyan kardeş beceri. -- **[CLI](/tr/agenteye/cli)**: becerinin tasarladığı seanslar verilerinin arkasındaki komut referansı. \ No newline at end of file diff --git a/docs/tr/agenteye/hermes-capture.mdx b/docs/tr/agenteye/hermes-capture.mdx deleted file mode 100644 index 5abbbee2..00000000 --- a/docs/tr/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Hermes session capture" -description: "Ekibinizin Hermes gateway oturumlarını — Slack, Telegram, CLI ve zamanlanmış çalışmalar — AgentEye'a sıradan oturumlar ve olaylar olarak getirin." ---- - -[Hermes](https://hermes-agent.nousresearch.com) ekibinize zaten çalıştıkları yerden cevap verir — Slack, Telegram, CLI, zamanlanmış çalışmalar. Hermes session capture tümünü AgentEye'a sıradan oturumlar ve olaylar olarak getirir, böylece ekibinizin her gün konuştuğu asistan, yazarken yazdığınız ajanlar kadar gözlemlenebilir olur. - -Küçük bir arka plan toplayıcısı Hermes'in yerel oturum deposunu yazıldığı sırada okur ve oturumları AgentEye'a gönderir. [Codex](/tr/agenteye/codex-capture) ve [OpenClaw](/tr/agenteye/openclaw-capture) capture ile aynı şekilde çalışır ve bir toplayıcı aynı anda birkaçını capture edebilir. - ---- - -## Ne capture eder - -Makinedeki her Hermes oturumu, hangi kanaldan geldiğine bakılmaksızın capture edilir. Her biri bir AgentEye [session](/tr/agenteye/sessions) olur; kullanıcı ve asistan mesajları, araç çağrıları ve araç sonuçları eşleşen [events](/tr/agenteye/event-stream) olur. - -Oturumun başladığı kanal — Slack, Telegram, CLI veya zamanlanmış çalışma — oturumda kaydedilir, böylece onları ayırt edebilir ve birer birer filtreleyebilirsiniz. Yanında oturumun çalıştığı model, başlatıldığı sohbet ve kişi, ve bir oturum başka bir oturum oluşturduğunda, parent'a geri bağlantı gelir. - -Oturumlar Hermes tarafından başlatılır başlatılmaz görünür, henüz bir şey söylenmemiş olsa bile, ve bir çevirinin yanıtı ile araç çağrıları gerçekten olduğu sırada kalır. Bir oturum sona erdiğinde neden sona erdiğini, ne kadar tuttuğunu ve kaç token kullandığını da alırsınız. - ---- - -## Açın - -Capture, etkinleştirene kadar kapalıdır. Toplayıcıyı `events:add` izni olan bir API anahtarıyla kurun ([API keys](/tr/agenteye/api-keys) bölümünü görmek için) ve Hermes capture'ı açın: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -Bu toplayıcıyı kurar, onu arka plan hizmeti olarak kaydeder ve capture'ı başlatır. Çalıştığını doğrulayın: - -```bash -agenteye-collector health -``` - -Aynı makinede birden fazla ajan capture ediyor musunuz? Her birinin bayrağını aynı komuta ekleyin — örneğin `--hermes-enabled --codex-enabled`. - -İlk çalışmada, mevcut Hermes oturumlarınız bir kez backfill edilir ve yeni aktivite birkaç saniye içinde akışa başlar. Hermes'in kendi verileri yalnızca okunur — asla değiştirilmez veya silinmez — ve her mesaj yeniden başlatmalar arasında bile bir kez gönderilir. - -`health` ayrıca toplayıcının capture ettiği her şeyin gerçekten AgentEye'a ulaşıp ulaşmadığını da söyler. Bir batch teslim edilemezse tutulur ve yeniden denenir, atılmaz, ve kontrol hala bekleyen bir şey varken sağlıksız rapor verir — bu nedenle "healthy" verilerinizin geldiği anlamına gelir, sadece işlem canlı değildir. - ---- - -## Nerede göründüğü - -Capture edilen oturumlar **Sessions**'da ve olayları **Events** akışında görünür, gözlemlediğiniz diğer tüm ajanlar gibi — bu nedenle [session replay](/tr/agenteye/sessions), [search](/tr/agenteye/queries), [evaluations](/tr/agenteye/evaluations) ve [alerts](/tr/agenteye/alerts) hepsi bunlar üzerinde çalışır. Hermes ajanına göre filtreleyerek onları ayrı ayrı görebilirsiniz. - ---- - -## Gizlilik - -Hermes oturumları tam transkripti içerir — komut çıktısı, dosya içeriği ve ajanın okuduğu veya yazdığı her şey dahil — ve sırlar içerebilir. Capture edilen oturumlar olduğu gibi gönderilir, bu nedenle capture'ı yalnızca bu içeriği AgentEye'da merkezileştirmenin uygun olduğu yerlerde etkinleştirin ve toplayıcıya yalnızca `events:add` ile sınırlandırılmış bir anahtar verin. Verilerinizin nasıl izole tutulduğu hakkında [Security](/tr/agenteye/security) bölümünü görmek için. \ No newline at end of file diff --git a/docs/tr/agenteye/observability.mdx b/docs/tr/agenteye/observability.mdx deleted file mode 100644 index 3c8e0fd4..00000000 --- a/docs/tr/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Gözlemle" -description: "Gözlem yüzeyleri, aracılarınızın şu anda ne yaptığını izlediğiniz ve herhangi bir çalıştırmayı detaylı incelediğiniz yerdir." ---- - - -Gözlem yüzeyleri, aracılarınızın şu anda ne yaptığını izlediğiniz ve herhangi bir çalıştırmayı detaylı incelediğiniz yerdir. Buradaki her şey canlı, kuruluşunuza ait ve tarih aralığı, ortam, ajan ve oturum tarafından filtrelenebilir, böylece "bir şeyler ters gitmiş gibi görünüyor" durumundan tam çalıştırmaya saniyeler içinde ulaşırsınız. - -![Canlı Etkinlik Akışı, türe göre renklendirilmiş ve ortam, ajan ve oturum tarafından filtrelenebilir](/agenteye/images/events-stream.png) - -Dört yüzey, her biri kendi sayfasına sahip: - -- **[Etkinlik akışı](/tr/agenteye/event-stream)**: her ajan arasında her çalıştırmanın canlı, adım adım kaydı (en yenisi ilk). Kuruluşunuzun ana sayfası ve sorun giderilmesi için ilk durak. -- **[Oturumlar ve yürütme grafiği](/tr/agenteye/sessions)**: bu etkinlikler her çalıştırma için bir satırda birleştirilmiş, artı her çalıştırmanın nasıl ilerlediğinin git tarzı resmi. -- **[Performans metrikleri](/tr/agenteye/telemetry)**: gecikme sıcaklık haritaları ve modelleriniz, araçlarınız ve kancalarınız için p50/p95/p99 yaşam bulguları, böylece kuyruk artışı ortalamanın dışında göze çarpar. -- **[Hata takibi](/tr/agenteye/error-tracking)**: her şeyin ters gittiği tek bir sorun giderme yüzeyi, uyarıdan çalıştırmaya tek bir tıklamayla. - -## İlgili - -- [Değerlendirmeler](/tr/agenteye/evaluations): her çalıştırmayı kalite açısından puanlandırın. -- [Uyarılar](/tr/agenteye/alerts): herhangi bir eşiği bir sayfalama kuralına dönüştürün. -- [Denetimler](/tr/agenteye/audits): Failproof AI Observability'nin oturumlar arasında hata desenlerini bulmasına izin verin. -- [CLI ve aracılar](/tr/agenteye/cli-and-agents): terminalinizden aynı gözlemlenebilirlik. \ No newline at end of file diff --git a/docs/tr/agenteye/openclaw-capture.mdx b/docs/tr/agenteye/openclaw-capture.mdx deleted file mode 100644 index 23ded95a..00000000 --- a/docs/tr/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,50 +0,0 @@ ---- ---- -title: "OpenClaw oturumu yakalama" -description: "Takımınızın yerel OpenClaw oturumlarını AgentEye'a sıradan oturumlar ve etkinlikler olarak aktarın — OpenClaw'un çalışma şeklinde hiçbir değişiklik olmadan." ---- - -Takımınız [OpenClaw](https://docs.openclaw.ai) çalıştırıyorsa, OpenClaw oturumu yakalama bu oturumları AgentEye'a sıradan oturumlar ve etkinlikler olarak getirir; böylece bunları arayabilir, yeniden oynatabilir ve gözlemlediğiniz diğer her şeyin yanında değerlendirebilirsiniz. [Python SDK](/tr/agenteye/python-sdk) ile tamamlayıcı: SDK yazdığınız aracıları enstrümente ederken, bu takımınızın zaten yaptığı OpenClaw çalışmasını yakalar — çalıştırılış şeklinde hiçbir değişiklik olmadan. - -Küçük bir arka plan toplayıcısı OpenClaw'un yerel oturum transkriptlerini yazılırken okur ve bunları AgentEye'a gönderir. [Codex yakalama](/tr/agenteye/codex-capture) ile aynı şekilde çalışır ve bir toplayıcı aynı anda her ikisini de yakalayabilir. - ---- - -## Ne yakalar - -Bir makinenin OpenClaw kurulumunda yapılandırılan her aracı, o makinenin toplayıcısı tarafından yakalanır — aracı başına kurulum yoktur. - -Her OpenClaw oturumu bir AgentEye [oturumu](/tr/agenteye/sessions) olur; kullanıcı ve asistan mesajları, araç çağrıları ve araç sonuçları, eşleşen [etkinliklere](/tr/agenteye/event-stream) dönüşür. - ---- - -## Etkinleştirin - -Yakalama, etkinleştirene kadar kapalıdır. `events:add` iznine sahip bir API anahtarı ile toplayıcıyı yükleyin ([API anahtarları](/tr/agenteye/api-keys) bölümüne bakın) ve OpenClaw yakalamayı açın: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -Bu, toplayıcıyı kurar, onu arka plan hizmeti olarak kaydeder ve yakalamayı başlatır. Çalıştığını doğrulayın: - -```bash -agenteye-collector health -``` - -Aynı makinede birden fazla aracı mı yakalıyorsunuz? Her birinin bayrağını aynı komuta ekleyin — örneğin `--openclaw-enabled --codex-enabled`. - -İlk çalıştırmada, mevcut OpenClaw oturumlarınız bir kez geri doldurulur ve yeni etkinlik birkaç saniye içinde akışa alınır. OpenClaw'un kendi dosyaları yalnızca okunur — asla değiştirilmez, taşınmaz veya silinmez — ve her oturum, yeniden başlatmalar arasında bile tam olarak bir kez gönderilir. - ---- - -## Nerede görünür - -Yakalanan oturumlar **Sessions**'da gösterilir ve bunların etkinlikleri **Events** akışında, gözlemlediğiniz başka herhangi bir aracı ile aynı şekilde görünür — böylece [oturum yeniden oynatma](/tr/agenteye/sessions), [arama](/tr/agenteye/queries), [değerlendirmeler](/tr/agenteye/evaluations) ve [uyarılar](/tr/agenteye/alerts) hepsi bunlarda çalışır. OpenClaw aracısına göre filtreleyin ve bunları kendileri başına görün. - ---- - -## Gizlilik - -OpenClaw transkriptleri tam oturumu içerir — komut çıktısı, dosya içeriği ve aracının okuduğu veya yazdığı her şey dahil — ve sırlar içerebilir. Yakalanan oturumlar olduğu gibi gönderilir; bu nedenle yakalamayı yalnızca bu içeriği AgentEye'da merkezi hale getirmenin uygun olduğu makinelerde ve takımlar için etkinleştirin ve toplayıcıya yalnızca `events:add` kapsamına sahip bir anahtar verin. Verilerinizin nasıl izole tutulduğu hakkında [Güvenlik](/tr/agenteye/security) bölümüne bakın. \ No newline at end of file diff --git a/docs/tr/agenteye/overview.mdx b/docs/tr/agenteye/overview.mdx deleted file mode 100644 index 602f2429..00000000 --- a/docs/tr/agenteye/overview.mdx +++ /dev/null @@ -1,107 +0,0 @@ ---- -title: "Failproof AI: Ajanlarınızdaki Hataları Gözlemleyin" -description: "Failproof AI Observability, üretim ortamında AI ajanlarınızı gözlemlemek, değerlendirmek ve geliştirmek için kendi sunucunuzda çalışan bir platformdur." ---- - -Failproof AI Observability, üretim ortamında AI ajanlarınızı gözlemlemek, değerlendirmek ve geliştirmek için kendi sunucunuzda çalışan bir platformdur. Ajanlarınızın yaptığı her şeyi kaydeder (her araç çağrısı, model isteği, hook ve hata), her çalıştırmanın kalitesini puanlandırır ve bilmediğiniz hataları ortaya çıkarır — tamamı kendi altyapınızda çalıştırdığınız bir panoda. - -AI ajanları yayınladıysanız ve bir çalıştırmanın neden başarısız olduğunu tahmin etmekten bıktıysanız, burası başlamanız gereken sayfa. Failproof AI Observability'nin size ne sunduğunu ve parçaların nasıl bir araya geldiğini açıklar; herhangi bir şey yüklemeden önce okuyun. - -> **Failproof AI Observability, Failproof AI'dan bir kurumsal üründür.** Canlı olarak görmek ister misiniz? Bir demo talep edin: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) adresine e-posta gönderin. - -![Failproof AI Observability oturumu, git tarzı bir yürütme grafiği olarak çizilmiş, yanında olay zaman çizelgesi ve sağ panelde araçlar, modeller ve hooklar ayrıntısı](/agenteye/images/session-detail.png) - -*Her ajan çalıştırması, git tarzı bir yürütme grafiği (sol) olarak çizilmiş ve yanında olay zaman çizelgesi vardır. Paralel alt ajanların her birinin kendi şeridi vardır; sağ panel, çalıştırmanın araçlarını, modellerini, hooklarını ve token harcamasını ayrıntılarıyla gösterir.* - ---- - -## Canlı olarak görmek - -İki kısa video, ekiplerin ilk başta yaptığı iki şeyi gösterir: bir çalıştırmayı izlemek ve hataları otomatik olarak bulma. - -
    - -
    - -*Ajan izleme: hedeften araçlara ve son cevaba kadar tek bir çalıştırmayı adım adım izleyin.* - -
    - -
    - -*Failproof Audit: Failproof AI Observability'nin günlüklerinizi oturumlar arasında analiz etmesine izin verin ve ne düzeltmesi gerektiğini öğrenin.* - ---- - -## Ekipler neden kullanıyor - -- **Ajanınızın gerçekte ne yaptığını görün.** Her çalıştırma okunabilir bir git tarzı yürütme grafiğine dönüşür: hangi araçlar paralel çalıştı, hangi alt ajanlar dallandı, nerede durdu ve ne harcadı. -- **Kalite gerilemeşini otomatik olarak yakalayın.** Küçük bir puanlama hizmetini bağlayın ve Failproof AI Observability her tamamlanan çalıştırmayı puanlandırır; böylece yardımcılıkta düşüş veya halüsinasyonlarda artış kendi kendine ortaya çıkar. -- **Kuralı yazacağınızı bilmediğiniz hataları bulun.** Yinelenen denetimler, günlüklerinizi oturumlar arasında hata kümeleri, gecikme aykırı değerleri, düşük puanlar ve takılı çalıştırmalar açısından analiz eder, ardından size sıralanmış, kanıtla desteklenmiş bulgular sunar. -- **Önemli olduğunda sayfa alın.** Eşik kuralları hata oranı, gecikme, maliyet veya değerlendirici puanlarında çalışır ve yanıtlayabileceğiniz, atayabileceğiniz ve çözebileceğiniz olaylar açar. -- **Düz İngilizcede sorular sorun.** Panoda yer alan bir AI asistanı, kendi verileriniz üzerinde „bu hafta üretimde kalite nasıl gelişiyor?" gibi soruları yanıtlar. Yaptığı her değişiklik onay geçidir. -- **Verilerinizi tutun.** Failproof AI Observability kendi sunucunuzda çalışır: olaylar, istemler ve analizler kontrol ettiğiniz altyapıda kalır. - ---- - -## Ne alıyorsunuz - -Failproof AI Observability, üç fikir etrafında organize edilmiştir (**gözlemle**, **analiz et** ve **yönet**), panelin sol kenar çubuğuna yansıtılır. - -**Gözlemle** (ne olduğunun ham gerçeği): - -- **[Olay akışı](/tr/agenteye/event-stream)**: her çalıştırmanın canlı, adım adım izi (araç çağrıları, model çağrıları, hooklar, hatalar). -- **[Oturumlar](/tr/agenteye/sessions)**: bu olaylar, çalıştırma başına bir satır halinde, her biri puanlandırılmaya hazır, git tarzı bir yürütme grafiği ile birlikte sunulur. -- **[Performans metrikleri](/tr/agenteye/telemetry)**: yüzey başına gecikme harita grafikleri ve modeller, araçlar ve hooklar için p50/p95/p99 vitalleri; böylece kuyruk artışı ortalamanın dışında görünür. -- **[Hata izleme](/tr/agenteye/error-tracking)**: her şeyin ters gittiği tek bir işlem yüzeyinde; bir uyarının ateşlenmesinden tek bir tıkla uzak. - -![Tools gözlemle sayfası: gecikme harita grafiği, yüzdelik dilim bandı ve 24 zaman kutusu üzerinde araç dağılım çubuğu](/agenteye/images/tools.png) - -*Her gözlemle yüzeyi, bir kıvılcım çizgisi ve p50/p95/p99 vitalleriyle bir gecikme harita grafiği ve yüzdelik dilim bandını eşleştirir. Gösterilen: Araçlar.* - -**Analiz et** (etkinliği cevaplara dönüştürün): - -- **[Sorgular](/tr/agenteye/queries)** ve **[panolar](/tr/agenteye/dashboards)**: olaylarınız ve değerlendirmeleriniz üzerinde kaydedilmiş SQL, paylaşılan, kurum kapsamı panolara çizilmiştir. -- **[Değerlendirmeler](/tr/agenteye/evaluations)**: kendi değerlendirici hizmetiniz tarafından üretilen kalite puanları, puan başına akıl yürütmesi ile. -- **[Denetimler](/tr/agenteye/audits)**: oturumlar arasında hata modellerini ortaya çıkaran yinelenen araştırmalar. -- **[Uyarılar](/tr/agenteye/alerts)** ve **[olaylar](/tr/agenteye/incidents)**: sizi sayfaya alan eşik kuralları, artı bunları işlemek için bir olay iş akışı. - -**Arayüzler** (verilerinize kendi yolunuzla ulaşın): - -- **[CLI](/tr/agenteye/cli-and-agents)**: tüm dağıtımınızı terminalden veya bir betikten çalıştırın ve bir kodlama ajanının bunu düz İngilizcede yapmasına izin verin. -- **[AI asistanı](/tr/agenteye/assistant)**: ajanlarınız hakkında düz İngilizcede soru sorun, doğrudan panoda. -- **REST API**: panelin ve CLI'nin yaptığı her şey, kapsamlı bir [API anahtarı](/tr/agenteye/api-keys) ile doğrudan çağırabileceğiniz bir REST API tarafından desteklenir — olayları alın, oturumları ve değerlendirmeleri sorgulayın ve panoları, uyarıları, denetimleri, kullanıcıları ve anahtarları yönetin; böylece Failproof AI Observability'yi kendi araçlarınızla entegre edin. - -**Yönet** (ekibiniz için çalıştırın): - -- **[API anahtarları](/tr/agenteye/api-keys)**: toplayıcı, pano ve asistan için kapsamlı jetonlar. -- **Kullanıcılar**: şifresiz, e-posta tabanlı oturum açma ve izin listesiyle. -- **Ayarlar**: kurum başına yapılandırma, model bağlam penceresi geçersiz kılmalar dahil. - ---- - -## Parçalar nasıl bir araya gelir - -Veri bir yönde akar, ajan kodunuzdan panoya: ajanınız (Python SDK aracılığıyla) agenteye-toplayıcıya olaylar yayınlar; bu olaylar sunucuya gönderilir ve sunucu panoyu sunar. İki isteğe bağlı hizmet bunu tamamlar — bir puanlama hizmet (değerlendirmeler) ve bir AI asistan hizmet (panoda sohbet). - -- **Python SDK**: ajanınıza birkaç `agenteye.event.*` çağrısı eklersiniz; olaylar yerel olarak arabelleğe alınır. -- **agenteye-toplayıcı**: her ajan makinesinde, olayları toplu olarak işleyen ve sunucuya gönderen hafif bir daemon. -- **Sunucu**: olaylarınızı alır, operasyonel durumu kendi veritabanlarınızda tutar ve pano, CLI ve kendi entegrasyonlarınızın hepsinin kullandığı REST API'yi sunar. -- **Pano**: her şeyi keşfettiğiniz yer. -- **İsteğe bağlı hizmetler**: bir puanlama hizmet (değerlendirmeler) ve bir AI asistan hizmet (panoda sohbet). - -Belgeler genelinde kullanılan kelime dağarcığı (*olay, oturum, değerlendirme, denetim, bulgu, olay*) için bkz. [Kavramlar](/tr/agenteye/concepts). - ---- - -## Failproof AI Observability'yi Almak - -Failproof AI Observability, Failproof AI'dan bir kurumsal üründür ve Failproof AI Enforcement — politika ve korkuluk ürünü — ile Failproof AI markası altında birlikte çalışır. Tamamen kendi ortamınızda çalışır. Paketlere henüz erişiminiz yoksa, bir demo talep edin ve sizi hazırlayacağız: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) adresine e-posta gönderin. - ---- - -## Sonraki adımlar - -- [Kavramlar](/tr/agenteye/concepts): Failproof AI Observability kelime dağarcığı bir yerde. -- [Observabilite](/tr/agenteye/observability): ajanlarınızın ne yaptığını, çalıştırmayı izleyin. -- [Güvenlik](/tr/agenteye/security): Failproof AI Observability verilerinizi nasıl izole tuttuğu ve kontrol altında tuttuğu. \ No newline at end of file diff --git a/docs/tr/agenteye/python-sdk-skill.mdx b/docs/tr/agenteye/python-sdk-skill.mdx deleted file mode 100644 index 72b79dab..00000000 --- a/docs/tr/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,130 +0,0 @@ ---- -title: "Failproof AI Observability Python SDK Agent Skill" -description: "Enstrümente edilmemiş bir ajanı gözlemlenebilir olaylarına dönüştürün; kodlama ajanınız enstrümantasyon noktalarını bulacak, yazacak ve doğrulayacaktır." ---- - -Kodlama ajanınıza *"bu ajana Failproof AI Observability ekle"* deyin ve ajanın döngünüzü okumasına, enstrümantasyonun nereye ait olduğunu çözmesine, yazmasına ve olayları doğrulamasına izin verin. - -**Python SDK becerisi** (`agenteye-python-sdk`) bir *Agent Skill*'dir: bir görev bununla eşleştiğinde Claude Code veya Codex gibi bir kodlama ajanının talep üzerine yüklediği bir talimatlar klasörü. Ajana [Python SDK](/tr/agenteye/python-sdk) kullanmayı öğretir — bu bir kütüphane değildir ve SDK'nın çalışma şeklini hiçbir şekilde değiştirmez. - -## Enstrümantasyon yazması kolay ama sessizce yanlış olmak kolay - -SDK küçüktür: on üç olay yöntemi, hepsi salt anahtar sözcük. Bir kodlama ajanı [Python SDK](/tr/agenteye/python-sdk) referansını okuyabilir ve makul enstrümantasyon bir dakikada üretebilir. - -Sorun şu ki, bu SDK yanlış olduğunuzda hata vermez ve yanlış enstrümantasyon doğru enstrümantasyon gibi görünür; ta ki birisi bir panoyu açıp boş bulana kadar. Gerçek zaman kaybettiren hatalar hep sessizliklerdir: - -| Hata | Ne görürsünüz | -|---|---| -| `agent_start` yok | Her olay iniyor. Sıfır oturum. | -| Ortam hiç ayarlanmadı | Herşey çalışıyor, `dev` altında dosyalanıyor. | -| `outcome="failure"` | Çalıştırma yeşil görünüyor — sadece `failed`, `error`, `timeout`, `rejected` sayılır. | -| Yazım hatası yapılan alan adı | Kabul ediliyor ve yeni alan olarak depolanıyor. | -| İş parçacığı havuzundan yayılan olaylar | Sessizce düşürülüyor. | - -Bunların hiçbiri hata vermez. Hiçbiri testlerde görünmez. Hepsi beceriye katılır, bunu yakalayan kontrol olarak belirtilir. - -## Sırasıyla ne yapar - -Beceri, dikkatli bir mühendisçinin yapacağı aynı üç adımı izler: - -1. **Plan.** Ajand döngünüzü okur ve sadece siz cevap verebileceğiniz iki soruyu sorar: bir çalıştırma nedir (`session_id`), ve ayırt edilebilir aktörler kimdir (`agent_id`). Kod yazmadan önce bunlar üzerinde anlaşmaya varır, çünkü daha sonra değiştirmek tarihinizi böler ve trendleri kırar. -2. **Yaz.** Kimliği çalıştırma başına bir kez bağlar, her çağrı sitesinden geçirmez ve eşzamanlılığa güvenli bir şekil seçer — bu önemlidir, çünkü bariz kısayol iki örtüşen çalıştırmayı sessizce bir oturumda karıştırır. -3. **Doğrula.** Ajanınızı çalıştırır ve ortaya çıkan olay dosyalarını okur; `agent_start` mevcutsa, ortam doğruysa ve bir çalıştırma bir oturum ürettiyse kontrol eder. - -Bu üçüncü adım insanların atladığı adımdır. SDK olayları yerel dosyalara yazar, bu nedenle tam bir entegrasyon sunucu olmadan, API anahtarı olmadan ve ağ olmadan dizüstü bilgisayarda kanıtlanabilir — bu tam olarak becerinin bunu yapması ısrar ettiği nedendir. - -## Diğer becerilerle ilişkisi - -Üç beceri, bir temiz bölünme: - -| Beceri | Ne zaman kullanılır | Ne değiştirir | -|---|---|---| -| **Python SDK becerisi** (bu sayfa) | Ajanınızın telemetri yaymasını istiyorsunuz — "observability ekle", "ajanom neden görünmüyor?" | Ajanın reposunda kod yazar. Hiçbir şey okumaz. | -| **[Evaluator becerisi](/tr/agenteye/evaluator-skill)** | Çalıştırmaları *puanlamak* istiyorsunuz — "ne ölçmemiz gerekiyor?" | Repoda kod yazar; telemetri okur | -| **[CLI becerisi](/tr/agenteye/cli-skill)** | Ne olduğunu *okumak* ya da dağıtımınızı işletmek istiyorsunuz | CLI'yi siz olarak yönetir, değişiklikler dahil | - -Bu sırayla devrederler: bu beceri olayları akışa sokar, evaluatör onları puanlar, CLI bunları geri okur. Ajanınız oturumlar yayına kadar değerlendirilecek hiçbir şey yoktur ve okunacak hiçbir şey yoktur, bu nedenle sıfırdan başlıyorsanız, burada başlayın. - -## Ön Koşullar - -1. **Python 3.10+** ve enstrümente etmek istediğiniz ajan kod tabanı. -2. **SDK.** Müşterilere özel bir wheel olarak dağıtılır, herkese açık bir indeksden değil — onboarding'iniz bunu nasıl elde edeceğinizi ve yükleyeceğinizi kapsar. Beceri yükleme yolunu bilir ve bulamazsa sizin yerine tahmin etmek yerine sorar. -3. **Başka bir şey yok.** Pano girişi yok, API anahtarı yok, ağ yok. Beceri SDK'nın yazdığı olay dosyalarına karşı doğrular, bu nedenle tamamlayabilir ve çalışmasını çevrimdışı olarak kanıtlayabilir. - -## Nereden bulabilirsiniz - -Beceri genel [`FailproofAI/skills`](https://github.com/FailproofAI/skills) koleksiyonunda bulunur: - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -Yalnızca mevcut proje yerine her proje için yüklemek için `-g` ekleyin ve ortamınız sembolik bağlantıları takip etmezse `--copy` ekleyin. Codex için `-a codex` geçirin. - -## Elle yükleme - -Agent Skills, `SKILL.md` ve referanslar içeren klasörlerdir. Yükleyiciyi kullanmak istemiyorsanız: - -- **Claude Code**: `agenteye-python-sdk/` klasörünü `~/.claude/skills/` içine (her proje) veya `/.claude/skills/` içine (sadece bu repo) kopyalayın. Claude Code otomatik olarak keşfeder — `/skills` listesini kontrol edin ya da bununla eşleşen bir şey sorun. -- **Codex**: Codex aynı `SKILL.md` okur. Paketlenmiş `agents/openai.yaml` `allow_implicit_invocation: true` ayarlar, bu nedenle bir görev eşleştiğinde otomatik seçilir; aksi halde `$agenteye-python-sdk` olarak çağırın. - -Ajanınızı **enstrümente etmek istediğiniz kodu tutan depoda çalıştırın** — beceri ajand döngünüzü herhangi bir şey önermenin önce okur. - -## Bir oturum nasıl görünür - -```text -siz ▸ Bu ajana Failproof AI Observability ekle. - -ajan ▸ Döngünüzü okudum — tools.py'da bir araç dağıtıcı, llm.py'da bir Anthropic - sarmalayıcı. Herhangi bir kod yazmadan önce iki sorum var. - - Bir çalıştırma nedir? İstekleri eşzamanlı olarak işliyorsunuz, bu yüzden - ctx.request_id'de zaten sahip olduğunuz istek id'sini kullanırım — - sonra burada bir oturum ve günlüklerdeki bir istek aynı dizedir. - - Ve aktörler kimdir? Bir planlayıcı yolu ve araştırıcı yolu görüyorum. - Bunları ayrı etiketlemeli misiniz, yoksa senin için bir ajandır? - -siz ▸ İstek id doğru. Onlar bir ajan, buna "asistan" de. - -ajan ▸ Tamamlandı. Dağıtıcı ve LLM istemcisi sardım; agent_start ve agent_end - hata yolunu da dahil ederek handle_request'i ayraç içine aldı. Kimlik - için bağlam-yerel kullandı — dağıtıcınız araçları iş parçacığı havuzunda - çalıştırıyor ve daha basit yaklaşım eşzamanlı çalıştırmaları sessizce - bir oturumda karıştırmış olurdu. - - Gerçek çalıştırmaya karşı doğrulandı: 4 dosya genelinde 11 olay — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, istek başına bir - oturum, durations eşlenmiş. - - Bir şey YAPMADıM: batch.py'daki işçi havuzu doğrudan executor'a gönderiyor, - oradan gelen olaylar düşürülürdü. Şu da düzeltmek ister misin? -``` - -Dikkat edilecek model: kodunuzu önermenin önce okudu, sadece cevaplayabileceğiniz soruları sordu, zaten sahip olduğunuz bir id'yi yeniden kullandı, iş parçacığı havuzu gördüğü için eşzamanlılığa güvenli şekil seçti ve başarı beyan etmek yerine **gerçek olayları okuyarak doğruladı** — sonra sessizce başarısız olacak tek yeri işaretledi. - -## Ne sorabilirsiniz - -- *"Ajanom neden panoda görünmüyor?"* → merdiveni yürür: olaylar yazılıyor mu, `agent_start` var mı, ortam doğru mu, toplayıcı aynı yeri okuyor mu. -- *"Herşey dev altında iniyor."* → ortam hiç ayarlanmadı ya da daha sonra çağrı tarafından sıfırlandı. -- *"Token takibi ekle."* → LLM sarmalayıcınızı bulur ve model, durdurma nedeni ve kullanımı kaydeder. -- *"Alt-ajanları da enstrümente et."* → bir oturum, farklı ajan etiketleri, üstlerinin altında iç içe. -- *"Enstrümantasyon için testler yaz."* → SDK'yı geçici bir dizine yönlendirir ve yazdığı olaylar hakkında onaylar. - -## Nelere dikkat edin - -**Doğrulamaya izin verin.** Bu beceriyi kullanmaya değer kılan adım son adımdır — ajanınızı çalıştırın ve olayları geri okuyun. Enstrümantasyon yazan ve duran bir ajan kolay yarısını yapmıştır ve sessizce başarısız olan yarısı diğeridir. - -**Koddan önce adlar üzerinde anlaşın.** `session_id` ve `agent_id` her yüzeyin gruplandığı eksenlerdir. Daha sonra yeniden adlandırmak tarihi böler: eski çalıştırmalar eski etiketleri tutar ve trendleri kırılır. Beceri sorar; cevap bir dakikasını düşünmeye değer. - -**Ajanınız SDK'yı genel bir indeksden yüklemeyi önerirse, beceri yüklenmedi.** SDK özel olarak dağıtılır. Bu teklif, kodlama ajanınızın beceriyi takip etmek yerine tahmin ettiğinin güvenilir bir işaretidir — oraya dur ve becerinin yüklendiğini kontrol et. - -Bunun ötesinde patlaması alanı küçüktür: çalışma dizininizde kod ve sizi ne söylerse olaylar yazılır. Dağıtımınızdan hiçbir şey okumaz ve hiçbir şey değiştirmez. - -## Sonraki adımlar - -- **[Python SDK](/tr/agenteye/python-sdk)**: bu becerinin otomatikleştirdiği şeyin arkasında — her olay türü ve alan — tam olay referansı. -- **[Oturumlar](/tr/agenteye/sessions)**: olaylar iniş yaptıktan sonra enstrümantasyonunuzun ürettiği. -- **[Evaluator Agent Becerisi](/tr/agenteye/evaluator-skill)**: çalıştırmalar iniş yaptıktan sonra sonraki adım — bunları puanlamak. -- **[CLI Agent Becerisi](/tr/agenteye/cli-skill)**: telemetrinizi geri okumak. \ No newline at end of file diff --git a/docs/tr/agenteye/security.mdx b/docs/tr/agenteye/security.mdx deleted file mode 100644 index 4a0fd8ac..00000000 --- a/docs/tr/agenteye/security.mdx +++ /dev/null @@ -1,69 +0,0 @@ ---- ---- -title: "Güvenlik" -description: "Failproof AI Observability, üretim aracılarınızın yakınına yerleştirilmek üzere oluşturulmuştur; bu, istemlerinizi, araç girdilerini ve çıktılarını görebilmesi anlamına gelir." ---- - - -Failproof AI Observability, üretim aracılarınızın yakınına yerleştirilmek üzere oluşturulmuştur; bu, istemlerinizi, araç girdilerini ve çıktılarını görebilmesi anlamına gelir. Bu sayfa, bu verileri nasıl izole, kontrollü ve sizin elinizde tuttuğunu açıklamaktadır. Failproof AI Observability'yi bir güvenlik incelemesi için değerlendiriyorsanız, buradan başlayın. - ---- - -## Verileriniz kendi ortamınızda kalır - -Failproof AI Observability, kendi kendine barındırılır. Olaylar, istemler, model yanıtları ve analizler kendi veritabanlarınızda, kendi ortamınızda depolanır. Hiçbir şey depolama için bir üçüncü taraf SaaS'a gönderilmez ve verileriniz kendi bulut hesabınızda kalır. - ---- - -## Kiracı izolasyonu - -Bir Failproof AI Observability örneği birçok kuruluşu barındırabilir ve her biri depolama katmanında izole edilir — yalnızca kullanıcı arayüzü tarafından değil, veritabanı tarafından uygulanır: - -- Bir kuruluşun işletimsel verileri (kullanıcılar, anahtarlar, panolar, kaydedilmiş sorgular) o kuruluşa ait olup, kuruluşlar arası okumalar veritabanı tarafından engellenir. -- Her alınan olaya sahip olduğu kuruluş damgası vurulur, böylece bir kuruluşun olayları asla başka bir kuruluş tarafından okunamaz. - -Her pano rotası bir kuruluş slug'ı altında kapsamlandırılır (`//…`). - ---- - -## Oturum açma - -Failproof AI Observability, şifresiz, e-posta tabanlı oturum açma kullanır. Kimse tarafından ele geçirilebilecek veya sızan bir şifre yoktur. Bir kullanıcı tek seferlik bir kod (veya tek tıklamalı sihirli bir bağlantı) talep eder, bu onlara e-posta ile gönderilir ve hızlı bir şekilde sona erer. Oturum açma bir **izin listesi** tarafından korunur: yalnızca izin verdiğiniz e-posta adresleri (veya etki alanları) kimlik doğrulaması yapabilir. - -![Failproof AI Observability oturum açma ekranı; tek kullanımlık bir kod e-postanıza gönderir](/agenteye/images/login.png) - ---- - -## API anahtarlarıyla kapsamlı erişim - -Her istemci, ayrıntılı, en düşük ayrıcalık izinlerine sahip bir API anahtarı ile kimlik doğrulaması yapar. Bir toplayıcının yalnızca `events:add` öğesi gerekir; bir pano veya asistan anahtarı salt okunur olabilir; yıkıcı eylemler (silme, yeniden oluşturma) dahil etmeyi seçtiğiniz ayrı yetkilendirmelerdir. - -![API anahtarları sayfası: her anahtarın izin verileri, okuma, yazma ve yıkıcı kapsama göre renk kodlu](/agenteye/images/api-keys.png) - -Kurulum için yönetici önyükleme anahtarını tutun ve diğer her şey için dar anahtarlar yayınlayın. [API anahtarları](/tr/agenteye/api-keys) sayfasına bakın. - ---- - -## Salt okunur, onay kapılı asistan - -Pano içindeki [yapay zeka asistanı](/tr/agenteye/assistant) verileriniz üzerinde soruları yanıtlar, ancak tasarım gereği sınırlandırılmıştır: - -- Varsayılan olarak **salt okunur**: SQL'i yalnızca `SELECT`/`WITH` sorgularına, tek deyimli, satır sınırı ile izin veren bir koruma yoluyla çalıştırır. -- Oluşturduğu her şey (kaydedilmiş bir sorgu, bir pano) **onay kapılı**: gerçekleşmeden önce her yazıyı gözden geçirip onaylarsınız. -- **Asla silemez**. - -Yani bir takım arkadaşı "bu hafta hangi aracılar en çok hata verdi?" diye sorabilir ve cevaba göre hareket edebilir, asistan kendi başına verilerinizi değiştirip kaldıramadan. - ---- - -## Aktarım sırasında - -Tüm trafik HTTPS üzerinde çalışır. TLS'yi kendi sertifikalarınızla sonlandırırsınız, böylece toplayıcıdan sunucuya ve tarayıcıdan sunucuya trafik aktarımda şifrelenir. - ---- - -## Sonraki adımlar - -- [Genel Bakış](/tr/agenteye/overview): Failproof AI Observability'nin nasıl bir araya geldiği. -- [API anahtarları](/tr/agenteye/api-keys): toplayıcı, pano ve asistan için erişimi kapsamlandırın. -- [Gözlenebilirlik](/tr/agenteye/observability): Failproof AI Observability'nin aracılarınızdan neleri yakaladığı. \ No newline at end of file diff --git a/docs/tr/cli/audit.mdx b/docs/tr/audit.mdx similarity index 100% rename from docs/tr/cli/audit.mdx rename to docs/tr/audit.mdx diff --git a/docs/tr/cli/backfill.mdx b/docs/tr/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/tr/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/tr/cli/config.mdx b/docs/tr/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/tr/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/tr/cli/flush.mdx b/docs/tr/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/tr/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/tr/cli/harness.mdx b/docs/tr/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/tr/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/tr/cli/migrate.mdx b/docs/tr/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/tr/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/tr/cli/uninstall.mdx b/docs/tr/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/tr/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/tr/cli/update.mdx b/docs/tr/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/tr/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/tr/agenteye/api-keys.mdx b/docs/tr/cloud/access.mdx similarity index 98% rename from docs/tr/agenteye/api-keys.mdx rename to docs/tr/cloud/access.mdx index 6dee33c5..985997ee 100644 --- a/docs/tr/agenteye/api-keys.mdx +++ b/docs/tr/cloud/access.mdx @@ -81,7 +81,7 @@ Sunucu sabit bir izin kataloğu uygular; her biri belirli HTTP rotalarını dene Bu izinler kontrol panelinin **Kullanıcılar** sayfasını destekler; burada her üyenin verilen kapsamları yonga olarak gösterilir: -![Kullanıcılar sayfası: kontrol paneli kullanıcısı başına kart, e-posta, verilen izinler ve düzenle/devre dışı bırak kontrolleriyle](/agenteye/images/users.png) +![Kullanıcılar sayfası: kontrol paneli kullanıcısı başına kart, e-posta, verilen izinler ve düzenle/devre dışı bırak kontrolleriyle](/cloud/images/users.png) ### İşletimsel ayarlar @@ -90,7 +90,7 @@ Bu izinler kontrol panelinin **Kullanıcılar** sayfasını destekler; burada he | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | Kontrol paneli tarafından yönetilen işletimsel ayarları ve meta verilerini görüntüle; model başına bağlam penceresi geçersiz kılmalarını listele; ve bir model için etkili pencereyi çöz. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | İşletimsel ayarları düzenle ve model başına bağlam penceresi geçersiz kılmalarını ekle, değiştir veya kaldır. Değişiklikler sunucuyu yeniden başlatmadan yeni olayları etkiler. | -![Ayarlar sayfası: sunucuyu yeniden başlatmadan düzenlenebilen izin verilen oturum açmalar ve oturum/OTP yaşam süreleri gibi kontrol paneli tarafından yönetilen işletimsel ayarlar](/agenteye/images/settings.png) +![Ayarlar sayfası: sunucuyu yeniden başlatmadan düzenlenebilen izin verilen oturum açmalar ve oturum/OTP yaşam süreleri gibi kontrol paneli tarafından yönetilen işletimsel ayarlar](/cloud/images/settings.png) ### Uyarılar ve olaylar @@ -246,7 +246,7 @@ Kontrol panelindeki **Anahtarlar** sayfası yukarıdaki tüm işlemler için bir Kontrol panelinden bir anahtar oluşturduğunuzda sırrı sağlamıyorsunuz; kontrol paneli sizin için güçlü bir sıra oluşturur ve oluşturmada **bir kez** görüntüler. Hemen kopyalayın ve güvenle saklayın; yeniden oluşturma gibi asla tekrar gösterilmez. Yine de anahtarın izinlerini doğrudan seçebilir veya bir izin setinden tohumlayabilirsiniz (aşağıya bakın). -![API Anahtarları sayfası: anahtar başına kart, adını, verilen izinleri ve oluşturma zamanını gösterir, yeniden oluştur ve devre dışı bırak eylemleriyle; `admin` gibi korunan anahtarlar işaretlenir](/agenteye/images/api-keys.png) +![API Anahtarları sayfası: anahtar başına kart, adını, verilen izinleri ve oluşturma zamanını gösterir, yeniden oluştur ve devre dışı bırak eylemleriyle; `admin` gibi korunan anahtarlar işaretlenir](/cloud/images/api-keys.png) --- @@ -275,5 +275,5 @@ Yalnızca mevcut bir örneği yükseltiyorsanız bunlara ihtiyacınız vardır; ## Sonraki Adımlar -- [Python SDK](/tr/agenteye/python-sdk): ajan kodunuz olayları gönderirken nasıl kimlik doğrulaması yapar. -- [Güvenlik](/tr/agenteye/security): oturum açma, erişim denetimi ve kuruluş başına veri yalıtması nasıl çalışır. \ No newline at end of file +- [Python SDK](/tr/cloud/sdk): ajan kodunuz olayları gönderirken nasıl kimlik doğrulaması yapar. +- [Güvenlik](/tr/cloud/security): oturum açma, erişim denetimi ve kuruluş başına veri yalıtması nasıl çalışır. \ No newline at end of file diff --git a/docs/tr/cloud/agent-skills.mdx b/docs/tr/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/tr/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
    loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/tr/agenteye/alerts.mdx b/docs/tr/cloud/alerts.mdx similarity index 76% rename from docs/tr/agenteye/alerts.mdx rename to docs/tr/cloud/alerts.mdx index c07458bd..b5949111 100644 --- a/docs/tr/agenteye/alerts.mdx +++ b/docs/tr/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "Müşterinizden duyar duymaz, ekibinizin zaten izlediği kanala bi --- -Müşterinizden duyar duymaz, ekibinizin zaten izlediği kanala bir şey limitinizi aşan an da haberi alın. Kuralı bir kez ayarlayın ve Failproof AI Observability bunu düzenli olarak kontrol etsin, sonra sizi e-posta, Slack, webhook veya doğrudan panoda bildirsin. +Müşterinizden duyar duymaz, ekibinizin zaten izlediği kanala bir şey limitinizi aşan an da haberi alın. Kuralı bir kez ayarlayın ve FailproofAI Cloud bunu düzenli olarak kontrol etsin, sonra sizi e-posta, Slack, webhook veya doğrudan panoda bildirsin. -![Uyarılar sayfası: her biri tetikleyicisini, değerlendirme penceresini, kanallarını ve bilgi, uyarı veya kritik öncelik rozetini gösteren uyarı kuralı kartlarının ızgarası](/agenteye/images/alerts.png) +![Uyarılar sayfası: her biri tetikleyicisini, değerlendirme penceresini, kanallarını ve bilgi, uyarı veya kritik öncelik rozetini gösteren uyarı kuralı kartlarının ızgarası](/cloud/images/alerts.png) *Her uyarı kuralı bir bakışta: neyi izliyor, ne sıklıkta, nereye bildiriyor ve ne kadar acil.* ## Kullanıcılarınız bilmeden sorunları öğrenin @@ -22,9 +22,9 @@ Herhangi bir kombinasyonu tek bir kurala ekleyin ve önem derecesi (bilgi, uyar ## Kuralı JSON değil, formda oluşturun -Bir formda "bozuk" demek ne anlama geldiğini açıklayın ve Failproof AI Observability size altında yatan kuralı yazacak. JSON özellikleri sadece o formun altında ürettiği şeydir, bu nedenle onu okuyarak bir kuralı anlayabilirsiniz ama nadiren yazarsınız. +Bir formda "bozuk" demek ne anlama geldiğini açıklayın ve FailproofAI Cloud size altında yatan kuralı yazacak. JSON özellikleri sadece o formun altında ürettiği şeydir, bu nedenle onu okuyarak bir kuralı anlayabilirsiniz ama nadiren yazarsınız. -![Yeni uyarı formu: ad ve açıklama, etkinleştirme geçişi ve metrik eşiği, özel SQL, değerlendirme puanı, bileşik değerlendirme ve etkinlik başına koşullar sunan tetikleyici seçici](/agenteye/images/alert-new.png) +![Yeni uyarı formu: ad ve açıklama, etkinleştirme geçişi ve metrik eşiği, özel SQL, değerlendirme puanı, bileşik değerlendirme ve etkinlik başına koşullar sunan tetikleyici seçici](/cloud/images/alert-new.png) *Bir tetikleyici seçin ve form doğru alanları değiştirir; Kaydet kuralı yazar.* Mutlu yol hızlıdır: adını verin, bir **tetikleyici** seçin (neyi izleyeceğiniz), **eşik ve pencereyi** ayarlayın (ne kadar kötü, ne kadar süre), en az bir **kanal** ekleyin, sonra **Kaydet** yapın ve her hedefin bağlı olduğunu doğrulamak için **Test** e tıklayarak sentetik bir bildirim gönderin. Altında buna benzer küçük bir spec üretir: @@ -43,7 +43,7 @@ Bir sinyal türüyle sınırlı değilsiniz. Hatayı nasıl düşündüğünüzl | **Bileşik değerlendirme** | birkaç puan kontrolü herhangi, tümü veya en az N mantığıyla birleştirilir, yalnızca puanlar arasında görünen bir regresyonu yakalaması için | | **Etkinlik başına** | eşleşen tek bir olay gelir: belirli bir ajan, belirli bir hata türü veya bir mesaj alt dizesi | -Zaten [Hatalar sayfasında](/tr/agenteye/error-tracking) bir hataya bakıyor musunuz? Oradaki her satırın bu aynı formu tam olarak şu hatayı yakalamak için önceden doldurmuş bir **+ uyarı** düğmesi vardır, bu nedenle az önce triajladığınız olay bir sonraki seferde sizi bildiren olur. +Zaten [Hatalar sayfasında](/tr/cloud/errors) bir hataya bakıyor musunuz? Oradaki her satırın bu aynı formu tam olarak şu hatayı yakalamak için önceden doldurmuş bir **+ uyarı** düğmesi vardır, bu nedenle az önce triajladığınız olay bir sonraki seferde sizi bildiren olur. **Nerede bulunur:** Uyarılar `//alerts` adresinde bulunur. Kurallar oluşturmak, düzenlemek, silmek ve test etmek **`alerts:write`** gerektirir; bakmak için `alerts:read` yeterlidir. Alıcı seçici kuruluşunuzun üyelerini ad ile listeler, bu nedenle formu bırakmadan bir kişiyi bildirebilirsiniz. @@ -53,11 +53,11 @@ Bir kötü ölçüm sizi uyandırmamalı. **M of N** gürültü filtresi, uyarı ## Bir uyarı ateşlendiğinde ne olur -Bir ihlal bir **olayı** açar ve kanallarınızı bir kez bildirir. Oradan ekibiniz bunu onaylar, sahibini atar, üzerinde tartışır ve temiz, atfedilmiş bir kayda karşı çözer. O triaj iş akışının kendi evi vardır: [Olaylar](/tr/agenteye/incidents) konusuna bakın. +Bir ihlal bir **olayı** açar ve kanallarınızı bir kez bildirir. Oradan ekibiniz bunu onaylar, sahibini atar, üzerinde tartışır ve temiz, atfedilmiş bir kayda karşı çözer. O triaj iş akışının kendi evi vardır: [Olaylar](/tr/cloud/incidents) konusuna bakın. ## İlgili -- [Olaylar](/tr/agenteye/incidents): ateşlenen bir uyarıyı açıktan onaylanana çözüme kadar izleyin. -- [Hata izleme](/tr/agenteye/error-tracking): ajan hatalarını gruplandırın ve bir tıkla birini uyarıya yükseltin. -- [Panolar](/tr/agenteye/dashboards): uyarıda bulunduğunuz eşiklerin geldiği paylaşılan panoları izleyin. -- [CLI ve ajanlar](/tr/agenteye/cli-and-agents): terminalinizden uyarılar oluşturun ve olayları onaylayın veya CI'ye yazın. \ No newline at end of file +- [Olaylar](/tr/cloud/incidents): ateşlenen bir uyarıyı açıktan onaylanana çözüme kadar izleyin. +- [Hata izleme](/tr/cloud/errors): ajan hatalarını gruplandırın ve bir tıkla birini uyarıya yükseltin. +- [Panolar](/tr/cloud/dashboards): uyarıda bulunduğunuz eşiklerin geldiği paylaşılan panoları izleyin. +- [CLI ve ajanlar](/tr/cloud/cli): terminalinizden uyarılar oluşturun ve olayları onaylayın veya CI'ye yazın. \ No newline at end of file diff --git a/docs/tr/agenteye/assistant.mdx b/docs/tr/cloud/assistant.mdx similarity index 82% rename from docs/tr/agenteye/assistant.mdx rename to docs/tr/cloud/assistant.mdx index 10fac333..cb0e22d3 100644 --- a/docs/tr/agenteye/assistant.mdx +++ b/docs/tr/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "Aracı verilerinize düz İngilizce ile bir soru sorun ve kanıtla --- -Aracı verilerinize düz İngilizce ile bir soru sorun ve kanıtlara doğrudan bağlanan bir yanıt alın. SQL yazmaya gerek yok, panoları araştırmaya gerek yok — **Failproof AI Observability** asistanı, ekibinizdeki herkesin aracılarınız hakkında cevap almasının en hızlı yoludur. +Aracı verilerinize düz İngilizce ile bir soru sorun ve kanıtlara doğrudan bağlanan bir yanıt alın. SQL yazmaya gerek yok, panoları araştırmaya gerek yok — **FailproofAI Cloud** asistanı, ekibinizdeki herkesin aracılarınız hakkında cevap almasının en hızlı yoludur. -![Failproof AI Observability asistanı, paneldeki düz İngilizce soruyu yanıtlarken, canlı Agent Activity tablosu, agent başına model kullanım dökümü ve yazılı çıkarımları gösteriyor, çalıştırdığı sorgular satır içinde gösterilmektedir](/agenteye/images/assistant.png) +![FailproofAI Cloud asistanı, paneldeki düz İngilizce soruyu yanıtlarken, canlı Agent Activity tablosu, agent başına model kullanım dökümü ve yazılı çıkarımları gösteriyor, çalıştırdığı sorgular satır içinde gösterilmektedir](/cloud/images/assistant.png) *Düz İngilizce sorun ve kendi verilerinizden oluşturulmuş bir yanıt alın. Burada hangi aracıların en meşgul olduğunu, hangi modelleri kullandıklarını analiz ediyor ve çalıştırdığı sorguları göstererek her sayıyı doğrulayabilmenizi sağlıyor.* Öğrenecek bir şey yok. Sohbeti açın, bilmek istediğinizi yazın ve geri aldığı bağlantıları takip edin: @@ -36,7 +36,7 @@ Bir yanıt tutmaya değer olduğunda, asistanı kaydetmesi için isteyin. SQL'i **Sorgular** sayfasında bir adım daha ileri gider ve bir SQL yazarı olur: istediğiniz sorguyu açıklayın ("Son 7 gün için agent başına hata oranını göster") ve SQL'i doğrudan editöre aktarır, değişiklikleri kabul etmeden veya reddetmeden önce görebilmeniz için bir diff görünümü açar. -![Observability Sorgular sayfası ve SQL editörü](/agenteye/images/query-lab.png) +![FailproofAI Cloud Sorgular sayfası ve SQL editörü](/cloud/images/query-lab.png) *Sorgular sayfası: bu editör, asistanın draft, salt okunur sorgu aktardığı yerdir ve siz kabul veya reddedebilirsiniz.* Burada SQL yazılı olarak yazılması `queries:run` iznini kullanır, editörün **Çalıştır** düğmesinin arkasındakiyle aynıdır. Başka yerlerde sohbet `agent:use` gerektirir. @@ -49,7 +49,7 @@ Asistanı neyle temas edebileceğinden endişe etmeden herkese açabilirsiniz: - **Her yazı sizin onayınızı bekler.** Kaydedilmiş sorgular ve panolar yalnızca açık Onayla tıklama işleminden sonra oluşturulur ve bunu kapatacak bir ayar yoktur. - **Hiçbir şeyi silemez.** Hiçbir silme aracı açılmaz ve asistan hiçbir silme izni tutmaz. Silmeler sizin elinizde kalır, panoda. - **Kuruluşunuzun içinde kalır.** Asistan yalnızca şu anda görüntüledüğiniz kuruluşu görebilir. -- **Sorularınız sizin kalır.** İstemler ve yanıtlar kendi Observability veritabanınızda yaşar; ürün analitikleri yalnızca kullanım meta verilerini kaydeder, asla istem metninizi değil. +- **Sorularınız sizin kalır.** İstemler ve yanıtlar kendi FailproofAI Cloud veritabanınızda yaşar; ürün analitikleri yalnızca kullanım meta verilerini kaydeder, asla istem metninizi değil. ## Nerede bulunur @@ -57,7 +57,7 @@ Asistan, kuruluşunuz altında her sayfanın sağ kenarına bindirme şeklinde y ## İlgili -- [CLI and agents](/tr/agenteye/cli-and-agents) -- [Queries](/tr/agenteye/queries) -- [Dashboards](/tr/agenteye/dashboards) -- [Evaluation suite](/tr/agenteye/evaluation-suite) \ No newline at end of file +- [CLI and agents](/tr/cloud/cli) +- [Queries](/tr/cloud/queries) +- [Dashboards](/tr/cloud/dashboards) +- [Evaluation suite](/tr/cloud/evaluators) \ No newline at end of file diff --git a/docs/tr/agenteye/audits.mdx b/docs/tr/cloud/audits.mdx similarity index 81% rename from docs/tr/agenteye/audits.mdx rename to docs/tr/cloud/audits.mdx index 743bb7da..c829ef52 100644 --- a/docs/tr/agenteye/audits.mdx +++ b/docs/tr/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "Denetimler: otomatik güvenilirlik analistiniz" -description: "Failproof AI Observability, hiçbir kural yazmadığınız hataları bulur ve tam olarak neyi düzeltmeniz gerektiğini sıralı, kanıtlarla desteklenmiş bir yapılacaklar listesi olarak size sunar." +description: "FailproofAI Cloud, hiçbir kural yazmadığınız hataları bulur ve tam olarak neyi düzeltmeniz gerektiğini sıralı, kanıtlarla desteklenmiş bir yapılacaklar listesi olarak size sunar." --- -Failproof AI Observability, hiçbir kural yazmadığınız hataları bulur ve tam olarak neyi düzeltmeniz gerektiğini sıralı, kanıtlarla desteklenmiş bir yapılacaklar listesi olarak size sunar. Adeta her gece günlüklerinizi tarayan bir analisti işe alıp, sabah masanızda kısa listeyi bırakmış olmak gibi. +FailproofAI Cloud, hiçbir kural yazmadığınız hataları bulur ve tam olarak neyi düzeltmeniz gerektiğini sıralı, kanıtlarla desteklenmiş bir yapılacaklar listesi olarak size sunar. Adeta her gece günlüklerinizi tarayan bir analisti işe alıp, sabah masanızda kısa listeyi bırakmış olmak gibi.
    @@ -12,7 +12,7 @@ Failproof AI Observability, hiçbir kural yazmadığınız hataları bulur ve ta *İki dakikalık tur: planlanmış bir çalıştırmadan üzerine hareket edebileceğiniz bir düzeltmeye.* -![Denetimler sayfası: oturumlarınızı hata desenleri açısından tarayan, her biri bir zamanlama ve duyarlılığa sahip olan yinelenen işler](/agenteye/images/audits.png) +![Denetimler sayfası: oturumlarınızı hata desenleri açısından tarayan, her biri bir zamanlama ve duyarlılığa sahip olan yinelenen işler](/cloud/images/audits.png) *Her denetim, oturumlarınızda hata arama yapan ve sıralı, kanıtlarla desteklenmiş öneriler sunan bir yinelenen işdir.* ## Sonraki neyi düzeltmeniz gerektiğini tahmin etmeyi bırakın @@ -48,7 +48,7 @@ Denetimler, pano içinde **`//audits`** adresinde yer alır (kenar çu ## İlgili -- [Uyarılar](/tr/agenteye/alerts): zaten bildiğiniz bir eşik geçilir geçilmez çağrı alın. -- [Değerlendirmeler](/tr/agenteye/evaluations): her çalıştırmayı puanlandırın, böylece kalite gerillemeleri kendini gösterir. -- [Hata izleme](/tr/agenteye/error-tracking): aracılarınızın attığı hataları gruplandırın ve takip edin. -- [Olaylar](/tr/agenteye/incidents): bir denetimin ortaya çıkardığı sorunu düzeltilmesine kadar takip edin. \ No newline at end of file +- [Uyarılar](/tr/cloud/alerts): zaten bildiğiniz bir eşik geçilir geçilmez çağrı alın. +- [Değerlendirmeler](/tr/cloud/evaluations): her çalıştırmayı puanlandırın, böylece kalite gerillemeleri kendini gösterir. +- [Hata izleme](/tr/cloud/errors): aracılarınızın attığı hataları gruplandırın ve takip edin. +- [Olaylar](/tr/cloud/incidents): bir denetimin ortaya çıkardığı sorunu düzeltilmesine kadar takip edin. \ No newline at end of file diff --git a/docs/tr/cloud/capture.mdx b/docs/tr/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/tr/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/tr/agenteye/cli-recipes.mdx b/docs/tr/cloud/cli-recipes.mdx similarity index 90% rename from docs/tr/agenteye/cli-recipes.mdx rename to docs/tr/cloud/cli-recipes.mdx index 623f24dd..c72f89d8 100644 --- a/docs/tr/agenteye/cli-recipes.mdx +++ b/docs/tr/cloud/cli-recipes.mdx @@ -4,9 +4,9 @@ description: "Oturum, olay ve değerlendirme verilerini bir betiğin veya kodlam --- -Oturum, olay ve değerlendirme verilerini (ve yeniden değerlendirmeleri tetikleyin) doğrudan bir betikten veya kodlama ajanından çekin, stdout'a temiz JSON çıkışı ile `jq`'ya doğrudan aktarılan veriler. Bu tarifler Failproof AI Observability'nin verilerini terminal kullanıcısı veya bir AI kodlama ajandan (Claude Code, Cursor) sorgulanabilir ve otomatikleştirilebilir şeye dönüştürür, pano üzerinde tıklama yapmanız gerekmeden. +Oturum, olay ve değerlendirme verilerini (ve yeniden değerlendirmeleri tetikleyin) doğrudan bir betikten veya kodlama ajanından çekin, stdout'a temiz JSON çıkışı ile `jq`'ya doğrudan aktarılan veriler. Bu tarifler FailproofAI Cloud'nin verilerini terminal kullanıcısı veya bir AI kodlama ajandan (Claude Code, Cursor) sorgulanabilir ve otomatikleştirilebilir şeye dönüştürür, pano üzerinde tıklama yapmanız gerekmeden. -Aşağıdaki desenleri Failproof AI Observability CLI'sı (`agenteye`) için copy-paste olarak kullanabilirsiniz. Kurulum, kimlik doğrulama ve tam seçenek listesi için bkz. [CLI](/tr/agenteye/cli); yerleşik yardım için `agenteye -h` veya `agenteye -h` komutunu çalıştırın. +Aşağıdaki desenleri FailproofAI Cloud CLI'sı (`agenteye`) için copy-paste olarak kullanabilirsiniz. Kurulum, kimlik doğrulama ve tam seçenek listesi için bkz. [CLI](/tr/cloud/cli); yerleşik yardım için `agenteye -h` veya `agenteye -h` komutunu çalıştırın. ## Altın kurallar @@ -173,7 +173,7 @@ Her komutun `--fields` tam olarak kendi öğesinin alan adlarını kabul eder. S ## Sonraki adımlar -- [CLI](/tr/agenteye/cli): kurulum, kimlik doğrulama ve her komut için tam seçenek başvurusu. -- [CLI ajan becerisi](/tr/agenteye/cli-skill): bu tarifleri kodlama ajanınızın yükleyebileceği bir beceri olarak paketleyin. -- [API anahtarları](/tr/agenteye/api-keys): CLI, SDK ve toplayıcının kimlik doğrulaması yaptığı anahtarları oluşturun ve kapsamlayın. -- [Python SDK](/tr/agenteye/python-sdk): Failproof AI Observability'ye olaylar gönderin, böylece bu tarifler tarafından sorgulanacak veriler olur. \ No newline at end of file +- [CLI](/tr/cloud/cli): kurulum, kimlik doğrulama ve her komut için tam seçenek başvurusu. +- [CLI ajan becerisi](/tr/cloud/agent-skills): bu tarifleri kodlama ajanınızın yükleyebileceği bir beceri olarak paketleyin. +- [API anahtarları](/tr/cloud/access): CLI, SDK ve toplayıcının kimlik doğrulaması yaptığı anahtarları oluşturun ve kapsamlayın. +- [Python SDK](/tr/cloud/sdk): FailproofAI Cloud'ye olaylar gönderin, böylece bu tarifler tarafından sorgulanacak veriler olur. \ No newline at end of file diff --git a/docs/tr/agenteye/cli.mdx b/docs/tr/cloud/cli.mdx similarity index 91% rename from docs/tr/agenteye/cli.mdx rename to docs/tr/cloud/cli.mdx index 58a3586c..f979c8e0 100644 --- a/docs/tr/agenteye/cli.mdx +++ b/docs/tr/cloud/cli.mdx @@ -1,10 +1,10 @@ --- title: "CLI" -description: "Failproof AI Observability'nin tüm işlevlerini terminalden veya bir betikten yönetin: pano gezintisine gerek yoktur." +description: "FailproofAI Cloud'nin tüm işlevlerini terminalden veya bir betikten yönetin: pano gezintisine gerek yoktur." --- -Failproof AI Observability'nin tüm işlevlerini terminalden veya bir betikten yönetin: pano gezintisine gerek yoktur. `agenteye` CLI'si verilerinizi sorgular (oturumlar, olay günlükleri, değerlendirmeler) ve kuruluşunuzu yönetir (API anahtarları, kullanıcılar, ayarlar, uyarılar, olaylar, kaydedilmiş sorgular), bu nedenle bir denetimi otomatikleştirmek, Gözlenebilirliği CI'ye bağlamak veya bir kodlama ajanına üretim incelemesi yapmasını istediğinizde buraya başvurun. Her komut `--json` bayrağını destekler, bu nedenle hem siz bir istemde hem de bir kodlama ajanı (Claude Code, Cursor) çıkış ayrıştırırken eşit şekilde çalışır. +FailproofAI Cloud'nin tüm işlevlerini terminalden veya bir betikten yönetin: pano gezintisine gerek yoktur. `agenteye` CLI'si verilerinizi sorgular (oturumlar, olay günlükleri, değerlendirmeler) ve kuruluşunuzu yönetir (API anahtarları, kullanıcılar, ayarlar, uyarılar, olaylar, kaydedilmiş sorgular), bu nedenle bir denetimi otomatikleştirmek, Gözlenebilirliği CI'ye bağlamak veya bir kodlama ajanına üretim incelemesi yapmasını istediğinizde buraya başvurun. Her komut `--json` bayrağını destekler, bu nedenle hem siz bir istemde hem de bir kodlama ajanı (Claude Code, Cursor) çıkış ayrıştırırken eşit şekilde çalışır. Bir tek ikili dosya ile şunları yapabilirsiniz: @@ -69,7 +69,7 @@ agenteye --version agenteye --help ``` -> **Not:** Failproof AI Observability Python SDK de `agenteye` dağıtım adını kullanır. CLI'yi `pipx` veya `uv tool` ile kurulumla (paylaşılan bir virtualenv'e `pip install` yerine) ikisinin çakışmasını önlersiniz. Düz `pip install agenteye` yalnızca SDK aynı ortamda yüklü değilse sorun değildir. +> **Not:** FailproofAI Cloud Python SDK de `agenteye` dağıtım adını kullanır. CLI'yi `pipx` veya `uv tool` ile kurulumla (paylaşılan bir virtualenv'e `pip install` yerine) ikisinin çakışmasını önlersiniz. Düz `pip install agenteye` yalnızca SDK aynı ortamda yüklü değilse sorun değildir. --- @@ -91,7 +91,7 @@ agenteye logout # revoke the session and clear the stored token `whoami` hiçbir zaman eksik veya süresi dolmuş oturum hatasını vermez; bunun yerine `logged_in: false` raporlar, bu nedenle bir betik veya ajan kimlik doğrulama durumunu güvenle araştırabilir (pano belirtilen bir temel URL yoksa veya erişilemezse yine de sıfır olmayan çıkabilir). -**Gereksinimler:** e-postanız panoya oturum açmaya izin verilen (Failproof AI Observability yöneticinize sorun) olmalı ve pano temel URL'sinde erişilebilir olmalıdır (bkz. [Yapılandırma](#configuration)). Bir kod talep eder ve hiçbiri gelmezse, e-postanız muhtemelen henüz pano erişimi için etkinleştirilmemiştir. +**Gereksinimler:** e-postanız panoya oturum açmaya izin verilen (FailproofAI Cloud yöneticinize sorun) olmalı ve pano temel URL'sinde erişilebilir olmalıdır (bkz. [Yapılandırma](#configuration)). Bir kod talep eder ve hiçbiri gelmezse, e-postanız muhtemelen henüz pano erişimi için etkinleştirilmemiştir. --- @@ -338,13 +338,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | Kimlik doğrulaması yapıldı, ancak hesabınız gerekli izne sahip değil (ileti adlandırır) | | 6 | İstenen kaynak bulunamadı (örneğin bilinmeyen oturum veya olay kimliği) | -Bunlar CLI'yi betiklemek için güvenli hale getirir: bir kodlama ajanı yeniden kimlik doğrulama istemek için bir `4`'e veya eksik izni yüzeyle çıkarmak için bir `5`'e dallanabilir. Ajanlar için çıkış kodu işleme desenleri ve JSON çıkış şekilleri için [Ajanlar için CLI Tarifleri](/tr/agenteye/cli-recipes) bölümüne bakın. +Bunlar CLI'yi betiklemek için güvenli hale getirir: bir kodlama ajanı yeniden kimlik doğrulama istemek için bir `4`'e veya eksik izni yüzeyle çıkarmak için bir `5`'e dallanabilir. Ajanlar için çıkış kodu işleme desenleri ve JSON çıkış şekilleri için [Ajanlar için CLI Tarifleri](/tr/cloud/cli-recipes) bölümüne bakın. --- ## Sonraki adımlar -- **[Ajanlar için CLI Tarifleri](/tr/agenteye/cli-recipes)**: kopyala-yapıştır sorgu desenleri, `jq` tek satırlıkları, `--fields` projeksiyonları, çıkış kodu işleme ve JSON çıkış şekilleri, kodlama ajanları CLI'yi sürüyor için yazılmış. -- **[CLI ajan becerisi](/tr/agenteye/cli-skill)**: bu CLI'yi bir kurulabilir Claude Code / Codex *becerisi* olarak paketleyin ve bir kodlama ajanı düz İngilizce isteklerinden Failproof AI Observability'yi sürsün. -- **[API anahtarları](/tr/agenteye/api-keys)**: `keys create --add …`'ın arkasındaki izin modeli. -- **[AI asistanı](/tr/agenteye/assistant)**: `agent ask`'ın konuştuğu asistanı etkinleştirme. \ No newline at end of file +- **[Ajanlar için CLI Tarifleri](/tr/cloud/cli-recipes)**: kopyala-yapıştır sorgu desenleri, `jq` tek satırlıkları, `--fields` projeksiyonları, çıkış kodu işleme ve JSON çıkış şekilleri, kodlama ajanları CLI'yi sürüyor için yazılmış. +- **[CLI ajan becerisi](/tr/cloud/agent-skills)**: bu CLI'yi bir kurulabilir Claude Code / Codex *becerisi* olarak paketleyin ve bir kodlama ajanı düz İngilizce isteklerinden FailproofAI Cloud'yi sürsün. +- **[API anahtarları](/tr/cloud/access)**: `keys create --add …`'ın arkasındaki izin modeli. +- **[AI asistanı](/tr/cloud/assistant)**: `agent ask`'ın konuştuğu asistanı etkinleştirme. \ No newline at end of file diff --git a/docs/tr/cloud/connect.mdx b/docs/tr/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/tr/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/tr/agenteye/dashboards.mdx b/docs/tr/cloud/dashboards.mdx similarity index 68% rename from docs/tr/agenteye/dashboards.mdx rename to docs/tr/cloud/dashboards.mdx index 420e4855..78783c42 100644 --- a/docs/tr/agenteye/dashboards.mdx +++ b/docs/tr/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "Canlı aracı verilerinizi tüm ekibinizin izlediği tek bir gör Canlı aracı verilerinizi tüm ekibinizin izlediği tek bir görüntüye dönüştürün. Önemli sorgularını grafikler olarak sabitleyin ve herkes bir bakışta aynı sayıları görür—hiç bir sorguyu yeniden çalıştırmaya gerek kalmaz. -![Kaydedilmiş sorgulardan oluşturulmuş bir pano: saatlik olayların satırı, türe göre hatalar çubuğu, gecikme alan grafiği ve modele göre tokenler](/agenteye/images/dashboard-fleet.png) +![Kaydedilmiş sorgulardan oluşturulmuş bir pano: saatlik olayların satırı, türe göre hatalar çubuğu, gecikme alan grafiği ve modele göre tokenler](/cloud/images/dashboard-fleet.png) *Bir pano, dört kaydedilmiş sorgu: saatlik olaylar, türe göre hatalar, gecikme ve modele göre tokenler.* @@ -25,15 +25,15 @@ Panolarınızı `//dashboards` adresinde bulacaksınız. ## Zaten kaydettiğiniz sorguları sabitleyin -Her karo kaydedilmiş bir sorguyla başlar. [Sorguları](/tr/agenteye/queries) kütüphanesinde (yerleşik ön ayarlar artı kendi öğeleriniz, olaylarınız ve değerlendirmeleriniz üzerinde) önemsediğiniz sorguyu oluşturun ve kaydedin, ardından bunu veriye uygun grafik olarak bir panoya sabitleyin: trend için bir **satır**, kategorileri karşılaştırmak için bir **çubuk**, hacim için bir **alan** veya hisse dökümü için bir **pasta**. +Her karo kaydedilmiş bir sorguyla başlar. [Sorguları](/tr/cloud/queries) kütüphanesinde (yerleşik ön ayarlar artı kendi öğeleriniz, olaylarınız ve değerlendirmeleriniz üzerinde) önemsediğiniz sorguyu oluşturun ve kaydedin, ardından bunu veriye uygun grafik olarak bir panoya sabitleyin: trend için bir **satır**, kategorileri karşılaştırmak için bir **çubuk**, hacim için bir **alan** veya hisse dökümü için bir **pasta**. Bir karo sadece kaydedilmiş sorgunuz grafik olarak gösterildiğinden, elimiz tarafından senkronizasyonda tutulacak bir şey yoktur. Sorguyu bir kez güncelleyin ve onu kullanan her pano da güncellenir. ## Sadece hacmi değil, kaliteyi izleyin -Hacim, aracıların meşgul olduğunu gösterir. Kalite, aslında işi yaptıklarını gösterir. Bir panoları [değerlendirme puanlarınıza](/tr/agenteye/evaluations) yönlendirin ve zamanla çalıştırmaların ne kadar iyi gittiğini izleyen bir pano alırsınız, bu nedenle kalite gerilemeleri bir müşteriden sürpriz yerine bir grafikte düşüş olarak görülür. +Hacim, aracıların meşgul olduğunu gösterir. Kalite, aslında işi yaptıklarını gösterir. Bir panoları [değerlendirme puanlarınıza](/tr/cloud/evaluations) yönlendirin ve zamanla çalıştırmaların ne kadar iyi gittiğini izleyen bir pano alırsınız, bu nedenle kalite gerilemeleri bir müşteriden sürpriz yerine bir grafikte düşüş olarak görülür. -![Kaydedilmiş değerlendirme sorgularından oluşturulmuş, kaliteye odaklanan bir pano](/agenteye/images/dashboard-quality.png) +![Kaydedilmiş değerlendirme sorgularından oluşturulmuş, kaliteye odaklanan bir pano](/cloud/images/dashboard-quality.png) *Bir kalite panosu, değerlendirme puanlarınızı ön plana ve merkeze alır, işletimsel sayıların hemen yanında.* @@ -41,6 +41,6 @@ Operasyon panolarını ve kalite panolarını yan yana tutun ve ekibinizin "çal ## İlgili -- [Sorgular](/tr/agenteye/queries): karolar haline gelen sorguları oluşturun ve kaydedin. -- [Değerlendirmeler](/tr/agenteye/evaluations): zamanla kaliteyi grafiklendirmek için çalıştırmalarınız puanlayın. -- [Uyarılar](/tr/agenteye/alerts): bu ölçümlerden herhangi birine bir eşik dönüştürün. \ No newline at end of file +- [Sorgular](/tr/cloud/queries): karolar haline gelen sorguları oluşturun ve kaydedin. +- [Değerlendirmeler](/tr/cloud/evaluations): zamanla kaliteyi grafiklendirmek için çalıştırmalarınız puanlayın. +- [Uyarılar](/tr/cloud/alerts): bu ölçümlerden herhangi birine bir eşik dönüştürün. \ No newline at end of file diff --git a/docs/tr/agenteye/error-tracking.mdx b/docs/tr/cloud/errors.mdx similarity index 65% rename from docs/tr/agenteye/error-tracking.mdx rename to docs/tr/cloud/errors.mdx index 480a46ef..3f825025 100644 --- a/docs/tr/agenteye/error-tracking.mdx +++ b/docs/tr/cloud/errors.mdx @@ -6,14 +6,14 @@ description: "Aracılarınızın ürettiği her hatayı tek bir yerde görün, g Aracılarınızın ürettiği her hatayı tek bir yerde görün, gruplandırılmış şekilde bir kümede oluşan hatalar tek bir sorun olarak görüntülensin. Canlı bir akışta kaymadan "bir şey kırmızı" durumundan hatasına neden olan tam çalışmaya kadar tek tıklamayla ulaşırsınız. -![Hatalar sayfası: zamana göre hataların histogramı ve her biri tek tıklamalı "+ uyarı" düğmesine sahip gruplandırılmış kırmızı hata satırları](/agenteye/images/errors.png) +![Hatalar sayfası: zamana göre hataların histogramı ve her biri tek tıklamalı "+ uyarı" düğmesine sahip gruplandırılmış kırmızı hata satırları](/cloud/images/errors.png) *Hatalar sayfası: zamana göre hataların histogramı, tekrarlanan hatalar olay başına bir satırda daraltılmış.* ## Her hata, sizin için zaten toplanmış Bir aracı arızalandığında, canlı bir olay akışını kaymadan kırmızı satırları çıkıp gitmeden yakalamayı ummamalısınız. **Hatalar** sayfası toplama işini sizin için yapar. Gösterge tablosunun kırmızıya boyayacağı her şeyi bir triage yüzeyinde bir araya getirir; böylece ilk gördüğünüz şey, neyin başarısız olduğudur, nerede arama yapacağınız değil. -Açık olanlardan daha fazlasını yakalar. Açık `error` olaylarının yanı sıra, Failproof AI Observability sessiz başarısızlıkları da ortaya çıkarır: yükü başarısızlık taşıyan herhangi bir `tool_result`, `hook_completed` veya `agent_end` burada gösterilir. Bir hata döndüren araç veya kötü çıkan bir hook artık yalnızca gürültülü bir istisna atılmadığı için gözünüzden kaçmaz. +Açık olanlardan daha fazlasını yakalar. Açık `error` olaylarının yanı sıra, FailproofAI Cloud sessiz başarısızlıkları da ortaya çıkarır: yükü başarısızlık taşıyan herhangi bir `tool_result`, `hook_completed` veya `agent_end` burada gösterilir. Bir hata döndüren araç veya kötü çıkan bir hook artık yalnızca gürültülü bir istisna atılmadığı için gözünüzden kaçmaz. En üstte, bir histogram hataları zamana göre çizer. Bir bakışta bunun sabit bir arka plan akışı mı yoksa birkaç dakika önce başlayan bir ani artış mı olduğunu anlarsınız, böylece hemen ne yapacağınızı bilirsiniz. @@ -23,19 +23,19 @@ Her gözlem yüzeyinde olduğu gibi, Hatalar sayfası kuruluşunuza özgüdür v Tek bir kırık bağımlılık, dakikada aynı hatayı yüzlerce kez çıkarabilir. Ham haliyle, bu neredeyse özdeş satırlar duvarıdır ve aslında görmeniz gereken tek şeyi gömülüdür. -Failproof AI Observability, aynı oturum ve hata türünü paylaşan tekrarlanan hataları tek bir satırda daraltır. Bir küme bir olayı okur. Sonunda sorunları sayarsınız, günlük satırları değil ve önemli olan sinyal kendi hacmi tarafından boğulmak yerine üstte kalır. +FailproofAI Cloud, aynı oturum ve hata türünü paylaşan tekrarlanan hataları tek bir satırda daraltır. Bir küme bir olayı okur. Sonunda sorunları sayarsınız, günlük satırları değil ve önemli olan sinyal kendi hacmi tarafından boğulmak yerine üstte kalır. ## "Bir şey kırmızı"dan tam olaya kadar Herhangi bir satırı tıklatın ve başarısız olan tam olayda konumlandırılmış şekilde o çalışmanın oturumunun içine inin. Oturum kimliklerini kopyalama, neyin yanlış gittiği anı aramak için kaydırma: tam oraya varırsınız, tüm yürütme grafiği bir bakışta uzakta olacak şekilde aracının kırılmadan önce anlarında ne yaptığını görebilirsiniz. -`alerts:write` iznine sahipseniz, her satırda **+ uyarı** düğmesi de vardır. Bunu tıklatın ve Observability, aynı hatayı yeniden yakalaması için zaten doldurulmuş yeni bir uyarı kuralı açar. Az önce triage ettiğiniz olay, sizi tekrar şaşırtmak yerine bir sonraki sefer sizi çağıracak olan olay haline gelir. +`alerts:write` iznine sahipseniz, her satırda **+ uyarı** düğmesi de vardır. Bunu tıklatın ve FailproofAI Cloud, aynı hatayı yeniden yakalaması için zaten doldurulmuş yeni bir uyarı kuralı açar. Az önce triage ettiğiniz olay, sizi tekrar şaşırtmak yerine bir sonraki sefer sizi çağıracak olan olay haline gelir. **Nerede bulunur:** **Hatalar** sayfası gösterge tablosunun observe bölümünde `//errors` konumunda yer alır. ## İlgili -- [Uyarılar](/tr/agenteye/alerts): herhangi bir hatayı bir çağrı kuralına dönüştürün. -- [Olaylar](/tr/agenteye/incidents): açık uyarıyı çözülene kadar takip edin. -- [Oturumlar](/tr/agenteye/sessions): herhangi bir hatanın arkasındaki tam çalışmayı açın. -- [Denetimler](/tr/agenteye/audits): Observability'nin çalışmalarınızda hata desenleri bulmasını sağlayın. \ No newline at end of file +- [Uyarılar](/tr/cloud/alerts): herhangi bir hatayı bir çağrı kuralına dönüştürün. +- [Olaylar](/tr/cloud/incidents): açık uyarıyı çözülene kadar takip edin. +- [Oturumlar](/tr/cloud/sessions): herhangi bir hatanın arkasındaki tam çalışmayı açın. +- [Denetimler](/tr/cloud/audits): FailproofAI Cloud'nin çalışmalarınızda hata desenleri bulmasını sağlayın. \ No newline at end of file diff --git a/docs/tr/agenteye/evaluations.mdx b/docs/tr/cloud/evaluations.mdx similarity index 65% rename from docs/tr/agenteye/evaluations.mdx rename to docs/tr/cloud/evaluations.mdx index e3fe9afe..e375feca 100644 --- a/docs/tr/agenteye/evaluations.mdx +++ b/docs/tr/cloud/evaluations.mdx @@ -4,15 +4,15 @@ description: "Kalite sorunları artık sizin bulduğunuz yer, müşteri şikayet --- -Kalite sorunları artık sizin bulduğunuz yer, müşteri şikayeti olarak duymak yerine. Kendi puanlama hizmetinizi bir kez bağlayın ve Failproof AI Observability, tamamlanan her çalışmayı otomatik olarak değerlendirerek, yardımcılıkta bir düşüş veya halüsinasyonlarda bir yükseliş, müşteri bunu hissetmeden kendi kendine ortaya çıkar. +Kalite sorunları artık sizin bulduğunuz yer, müşteri şikayeti olarak duymak yerine. Kendi puanlama hizmetinizi bir kez bağlayın ve FailproofAI Cloud, tamamlanan her çalışmayı otomatik olarak değerlendirerek, yardımcılıkta bir düşüş veya halüsinasyonlarda bir yükseliş, müşteri bunu hissetmeden kendi kendine ortaya çıkar. -![Puan sütunlu Oturumlar ızgarası: her çalışma bir değerlendirme durumu rozeti ve renk kodlu yardımcılık, doğruluk ve araç verimlilik rozetleri taşır](/agenteye/images/sessions-list.png) +![Puan sütunlu Oturumlar ızgarası: her çalışma bir değerlendirme durumu rozeti ve renk kodlu yardımcılık, doğruluk ve araç verimlilik rozetleri taşır](/cloud/images/sessions-list.png) *Oturumlar ızgarasındaki her çalışma puanlarını taşır; kırmızı, sarı ve yeşil rozetler, tek bir transkrip açmadan zayıf çalışmaları hemen ortaya çıkarır.* ## El ile çalışmaları örneklemeyi durdurun -Eskiden bir avuç çalışmayı spot kontrol etmeniz ve geri kalanın iyi olacağını ummanız gerekiyordu. Artık tamamlanan her oturum, sizin önemsediğiniz boyutlarda anlık olarak puanlanıyor: yardımcılık, araç verimliliği, doğruluk, güvenlik, ne olursa olsun kalite standardınız. Siz puan anahtarlarını tanımlarsınız; Failproof AI Observability, değerlendiricinin geri gönderdiği her şeyi saklayıp, trend gösterir ve görüntüler. Hiçbir çalışma puanlanmadan kaçmaz ve destek talebinden regresyon hakkında öğrenmeyi bırakırsınız. +Eskiden bir avuç çalışmayı spot kontrol etmeniz ve geri kalanın iyi olacağını ummanız gerekiyordu. Artık tamamlanan her oturum, sizin önemsediğiniz boyutlarda anlık olarak puanlanıyor: yardımcılık, araç verimliliği, doğruluk, güvenlik, ne olursa olsun kalite standardınız. Siz puan anahtarlarını tanımlarsınız; FailproofAI Cloud, değerlendiricinin geri gönderdiği her şeyi saklayıp, trend gösterir ve görüntüler. Hiçbir çalışma puanlanmadan kaçmaz ve destek talebinden regresyon hakkında öğrenmeyi bırakırsınız. Puanlar **`//sessions`** adresindeki oturumlar ızgarasında yer alır (kenar çubuğu → *observe* → *sessions*), satır başına bir rozet kümesi. Sadece başarısız olan çalışmaları mı istiyorsunuz? Izgarayı puan aralığına göre filtreleyin, diyelim ki 0,5'in altında yardımcılık ve tam olarak okunmaya değer çalışmaları açın. Puanları görüntülemek için `evaluations:read` iznine ihtiyaç duyarsınız. @@ -20,7 +20,7 @@ Puanlar **`//sessions`** adresindeki oturumlar ızgarasında yer alır Bir sayı size bir çalışmanın zayıf olduğunu söyler; oturum sayfası sana neden olduğunu söyler. Herhangi bir çalışmayı açın ve sağ panel başlık özeti ile başlar, sonra her boyut başına sizin değerlendiricinin kendi muhakemesi ile bir bar gösterir; böylece "bu, doğrulukta 0,4 aldı" ila yanlış yaptığı kesin iddianın saniyeler içinde olursunuz. -![Bir oturumun sağ paneli: üstteki değerlendirme özeti, sonra her boyut puan barı ve her birinin altında gerekçelendirme, tam çalışma etkinliği zaman çizelgesi yanında](/agenteye/images/session-detail.png) +![Bir oturumun sağ paneli: üstteki değerlendirme özeti, sonra her boyut puan barı ve her birinin altında gerekçelendirme, tam çalışma etkinliği zaman çizelgesi yanında](/cloud/images/session-detail.png) *Oturum detay görünümü: özet, boyut başına puan barları ve her puanın ardındaki gerekçelendirme, çalışmanın etkinlik zaman çizelgesi yanında.* @@ -30,7 +30,7 @@ Daha keskin bir değerlendirici yayınladınız mı veya puanlanmadan önce çö Bir çalışmanın düşük puanlaması gürültüdür; bütün bir kohort kayıyorsa bu sinyaldir. Kaydedilmiş panolar puanlarınızı bir bakışta izleyebileceğiniz bir eğilime dönüştürür: bu hafta ortalama yardımcılık, geçen hafta ile karşılaştırılır, aracı başına, ortam başına. -![Bir kalite panosu: değerlendirici boyutu başına ortalama puan barları ve zaman içinde bir trend](/agenteye/images/dashboard-quality.png) +![Bir kalite panosu: değerlendirici boyutu başına ortalama puan barları ve zaman içinde bir trend](/cloud/images/dashboard-quality.png) *Kaydedilmiş bir kalite panosu, öne çıkardığınız puan anahtarlarını trendler; böylece yavaş bir sürükleme, olay haline gelmeden çok önce açık hale gelir.* @@ -38,14 +38,14 @@ Panolar **`//dashboards`** adresinde yaşarlar (kenar çubuğu → *an ## Bir kez değerlendiriciye bağlanın -Puanlama gönüllü ve Failproof AI Observability'yi bir puanlayıcıya işaret edene kadar tamamen kapalı kalır. Bir küçük HTTP hizmeti (Observability, kopyalayabileceğiniz çalışan bir referans seviyesiyle birlikte gelir), sunucunuzda iki değer ayarlarsınız ve o zamandan sonraki her çalışma sizin için puanlanır. Tam gözden geçirme, puanlama kontratı ve SDK derin kılavuzda yaşıyor. +Puanlama gönüllü ve FailproofAI Cloud'yi bir puanlayıcıya işaret edene kadar tamamen kapalı kalır. Bir küçük HTTP hizmeti (FailproofAI Cloud, kopyalayabileceğiniz çalışan bir referans seviyesiyle birlikte gelir), sunucunuzda iki değer ayarlarsınız ve o zamandan sonraki her çalışma sizin için puanlanır. Tam gözden geçirme, puanlama kontratı ve SDK derin kılavuzda yaşıyor. -Hangi boyutların başlangıçta puanlamaya değer olduğundan emin misiniz? [Değerlendirici aracı yeteneği](/tr/agenteye/evaluator-skill), kodlama aracınızın kendi oturumlarınıza karşı bunu belirlemesini sağlar, ardından hizmeti kurar ve dağıtır. +Hangi boyutların başlangıçta puanlamaya değer olduğundan emin misiniz? [Değerlendirici aracı yeteneği](/tr/cloud/agent-skills), kodlama aracınızın kendi oturumlarınıza karşı bunu belirlemesini sağlar, ardından hizmeti kurar ve dağıtır. ## İlişkili -- [Değerlendirme paketi](/tr/agenteye/evaluation-suite): değerlendiriciye, puanlama kontratına ve SDK'ya bağlanın. -- [Değerlendirici aracı yeteneği](/tr/agenteye/evaluator-skill): bir kodlama aracının puan boyutlarınızı seçmesine ve değerlendiriciye oluşturmasına izin verin. -- [Oturumlar](/tr/agenteye/sessions): puanların göründüğü çalışma başına ızgara. -- [Panolar](/tr/agenteye/dashboards): kuruluşunuz genelinde kalite eğilimlerini kaydedin ve paylaşın. -- [Denetimler](/tr/agenteye/audits): Observability'nin diğer otomatik kalite özelliği, oturum arası araştırmalar için. \ No newline at end of file +- [Değerlendirme paketi](/tr/cloud/evaluators): değerlendiriciye, puanlama kontratına ve SDK'ya bağlanın. +- [Değerlendirici aracı yeteneği](/tr/cloud/agent-skills): bir kodlama aracının puan boyutlarınızı seçmesine ve değerlendiriciye oluşturmasına izin verin. +- [Oturumlar](/tr/cloud/sessions): puanların göründüğü çalışma başına ızgara. +- [Panolar](/tr/cloud/dashboards): kuruluşunuz genelinde kalite eğilimlerini kaydedin ve paylaşın. +- [Denetimler](/tr/cloud/audits): FailproofAI Cloud'nin diğer otomatik kalite özelliği, oturum arası araştırmalar için. \ No newline at end of file diff --git a/docs/tr/agenteye/evaluation-suite.mdx b/docs/tr/cloud/evaluators.mdx similarity index 82% rename from docs/tr/agenteye/evaluation-suite.mdx rename to docs/tr/cloud/evaluators.mdx index 16aec075..619b2cc9 100644 --- a/docs/tr/agenteye/evaluation-suite.mdx +++ b/docs/tr/cloud/evaluators.mdx @@ -1,19 +1,19 @@ --- title: "Değerlendirme Paketi" -description: "Failproof AI Observability, her tamamlanan agent çalışmasını kalite açısından otomatik olarak puanlandırabilir: küçük bir puanlama hizmeti sağlarsınız ve Observability geri kalanını halleder." +description: "FailproofAI Cloud, her tamamlanan agent çalışmasını kalite açısından otomatik olarak puanlandırabilir: küçük bir puanlama hizmeti sağlarsınız ve FailproofAI Cloud geri kalanını halleder." --- -Failproof AI Observability, her tamamlanan agent çalışmasını kalite açısından otomatik olarak puanlandırabilir: küçük bir puanlama hizmeti sağlarsınız ve Observability geri kalanını halleder. Önem verdiğiniz boyutları (yararlılık, araç verimliliği, doğruluk, güvenlik; siz seçersiniz) izlemek, gerilemeyi erkenden yakalamak ve agent'ları veya ortamları bir bakışta karşılaştırmak için kullanın. Puanlama isteğe bağlıdır: sunucuda `EVALUATOR_ENDPOINT` ayarlanana kadar işlem hattı hiçbir şey yapmaz. +FailproofAI Cloud, her tamamlanan agent çalışmasını kalite açısından otomatik olarak puanlandırabilir: küçük bir puanlama hizmeti sağlarsınız ve FailproofAI Cloud geri kalanını halleder. Önem verdiğiniz boyutları (yararlılık, araç verimliliği, doğruluk, güvenlik; siz seçersiniz) izlemek, gerilemeyi erkenden yakalamak ve agent'ları veya ortamları bir bakışta karşılaştırmak için kullanın. Puanlama isteğe bağlıdır: sunucuda `EVALUATOR_ENDPOINT` ayarlanana kadar işlem hattı hiçbir şey yapmaz. -> **Not:** Puan boyutlarını siz tanımlarsınız. Değerlendiricininiz istediği sayısal anahtarları döndürebilir; Observability geri gönderdiğiniz her şeyi depolar, trendini oluşturur ve görüntüler. +> **Not:** Puan boyutlarını siz tanımlarsınız. Değerlendiricininiz istediği sayısal anahtarları döndürebilir; FailproofAI Cloud geri gönderdiğiniz her şeyi depolar, trendini oluşturur ve görüntüler. ## Bakış -1. **Bir puanlayıcı yazın.** Oturum transkriptini okuyan ve puanlar döndüren küçük bir HTTP hizmeti kurun. Observability, kopyalayabileceğiniz çalışan bir referans seviyesiyle gelir. Bkz. [SDK ile Değerlendirici Yazma](#sdk-ile-değerlendirici-yazma). -2. **Observability'yi ona gösterin.** Sunucu işlemine `EVALUATOR_ENDPOINT` (ve paylaşılan `EVALUATOR_TOKEN`) ayarlayın. +1. **Bir puanlayıcı yazın.** Oturum transkriptini okuyan ve puanlar döndüren küçük bir HTTP hizmeti kurun. FailproofAI Cloud, kopyalayabileceğiniz çalışan bir referans seviyesiyle gelir. Bkz. [SDK ile Değerlendirici Yazma](#sdk-ile-değerlendirici-yazma). +2. **FailproofAI Cloud'yi ona gösterin.** Sunucu işlemine `EVALUATOR_ENDPOINT` (ve paylaşılan `EVALUATOR_TOKEN`) ayarlayın. 3. **Puanları izleyin.** Her tamamlanan oturum otomatik olarak puanlandırılır; sonuçlar oturum detay sayfasında, oturumlar ızgarasında ve kaydedilmiş panolarda görünür. -![Değerlendirme özeti, boyut başına puan çubukları ve sağ panelde akıl yürütme metni bulunan bir oturum detay görünümü](/agenteye/images/session-detail.png) +![Değerlendirme özeti, boyut başına puan çubukları ve sağ panelde akıl yürütme metni bulunan bir oturum detay görünümü](/cloud/images/session-detail.png) *Bir değerlendirici yapılandırıldığında, her tamamlanan çalışma puanlandırılır ve sonuçlar oturumun sağ panelinde görünür: üstte özet, ardından akıl yürütmeli boyut başına puan çubukları.* @@ -23,7 +23,7 @@ Failproof AI Observability, her tamamlanan agent çalışmasını kalite açıs ```mermaid flowchart LR - ING["ingest /events
    agent_end"] --> SRV["Observability server"] + ING["ingest /events
    agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -31,14 +31,14 @@ flowchart LR SRV --> RES["evaluations
    terminal results"] ``` -Failproof AI Observability SDK bir oturum için `agent_end` olayını yaydığında, sunucu bir değerlendirmeyi programlar. Daha sonra tam olay transkriptini değerlendirici hizmetinize POST eder; bu şunlardan birini yapabilir: +FailproofAI Cloud SDK bir oturum için `agent_end` olayını yaydığında, sunucu bir değerlendirmeyi programlar. Daha sonra tam olay transkriptini değerlendirici hizmetinize POST eder; bu şunlardan birini yapabilir: - **Sonucu satır içi döndürün** `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}` ile. Sonuç oturumun değerlendirme zaman çizelgesine eklenir. `reasoning` ve `summary` isteğe bağlıdır. -- **Erteleyin** `{"status":"pending", "job_id":"abc-123"}` ile. Observability daha sonra değerlendiricininiz `{"status":"done", ...}` veya `{"status":"error", "error":"..."}` döndürene kadar `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` çağrısını yapar. +- **Erteleyin** `{"status":"pending", "job_id":"abc-123"}` ile. FailproofAI Cloud daha sonra değerlendiricininiz `{"status":"done", ...}` veya `{"status":"error", "error":"..."}` döndürene kadar `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` çağrısını yapar. - Yoklama sıklığı iş başına değişir: `pending` yanıtı `next_poll_secs` içerebilir; aksi takdirde Observability `GET /config` yapılandırıcısından `default_poll_interval_secs` değerini kullanır; aksi takdirde sunucu `EVALUATOR_POLLING_INTERVAL_SECS` (varsayılan 10s) değerine geri döner. Tüm değerler [1s, 1h] aralığına sabitlenir. + Yoklama sıklığı iş başına değişir: `pending` yanıtı `next_poll_secs` içerebilir; aksi takdirde FailproofAI Cloud `GET /config` yapılandırıcısından `default_poll_interval_secs` değerini kullanır; aksi takdirde sunucu `EVALUATOR_POLLING_INTERVAL_SECS` (varsayılan 10s) değerine geri döner. Tüm değerler [1s, 1h] aralığına sabitlenir. -`agent_end` yaymayan oturumlar (örneğin, kilitlenmişs agent işlemi) da alınabilir: değerlendiricinin `GET /config` `{"inactivity_timeout_secs": 1800}` döndürebilir ve Observability bu kadar süre boşta kalan herhangi bir oturumu değerlendirir. Bu işlev devre dışı bırakmak için alanı `null` olarak ayarlayın veya atlayın. +`agent_end` yaymayan oturumlar (örneğin, kilitlenmişs agent işlemi) da alınabilir: değerlendiricinin `GET /config` `{"inactivity_timeout_secs": 1800}` döndürebilir ve FailproofAI Cloud bu kadar süre boşta kalan herhangi bir oturumu değerlendirir. Bu işlev devre dışı bırakmak için alanı `null` olarak ayarlayın veya atlayın. `EVALUATOR_ENDPOINT` ayarlanmadığında işlem hattı tamamen işlemsizdir. @@ -46,7 +46,7 @@ Bir oturum zaman içinde **birden fazla terminal değerlendirmesi** biriktire bi Hareketsizlik geri dönüş, devam eden oturumlar üzerinde de yeniden etkinleştirilir: bir önceki terminal değerlendirmeden sonra yeni olaylar gelirse ve oturum `inactivity_timeout_secs` ötesine boşta kalırsa, yeni bir değerlendirme sıraya alınır. -Geçici hatalar (5xx, 429, zaman aşımları, ağ hataları) `EVALUATOR_MAX_ATTEMPTS` değerine kadar üstel geri dönüşle yeniden denenilir; 4xx yanıtları terminaldir. Observability, birden çok yatay ölçeklenmiş sunucu örnekleriyle güvenle çalışabilir; çalışma bölümlere ayrılır, böylece aynı oturum asla eşzamanlı olarak iki kez gönderilmez. +Geçici hatalar (5xx, 429, zaman aşımları, ağ hataları) `EVALUATOR_MAX_ATTEMPTS` değerine kadar üstel geri dönüşle yeniden denenilir; 4xx yanıtları terminaldir. FailproofAI Cloud, birden çok yatay ölçeklenmiş sunucu örnekleriyle güvenle çalışabilir; çalışma bölümlere ayrılır, böylece aynı oturum asla eşzamanlı olarak iki kez gönderilmez. --- @@ -54,7 +54,7 @@ Geçici hatalar (5xx, 429, zaman aşımları, ağ hataları) `EVALUATOR_MAX_ATTE Her kimliği doğrulanan rota **taşıyıcı token kimlik doğrulaması** kullanır. Aynı değer her iki tarafta da yapılandırılması gerekir: -- Observability sunucusu: ortam değişkeni `EVALUATOR_TOKEN` +- FailproofAI Cloud sunucusu: ortam değişkeni `EVALUATOR_TOKEN` - Değerlendirici hizmeti: aynı şekilde yapılandırılmış (agenteye-evaluator SDK kuralı gereği `EVALUATOR_TOKEN` okur) `EVALUATOR_TOKEN` ayarlanmadığında, sunucu `Authorization` başlığı göndermez; değerlendirici anonim istekleri kabul edebilir, bu da yalnızca ağ için iyidir ancak genel internet üzerinde önerilmez. @@ -125,7 +125,7 @@ Sunucu diğer 2xx gövdeleri protokol hatası olarak ele alır ve oturum için t HTTP sözleşmesini elle uygulamamanız gerekmez. `agenteye-evaluator` Python paketi, kimlik doğrulamayı, yönlendirmeyi ve istek/yanıt şekillerini sizin için işleyen yazılan bir FastAPI sarmalayıcısı sağlar. -Failproof AI Observability ayrıca transkript şeklinden `helpfulness`, `tool_efficiency` ve `factuality` puanlandıran **çalışan bir referans değerlendiricisi** ile gelir. Başlangıç noktası olarak kopyalayın ve kendi mantığınızla değiştirin: bir LLM yargıçsı, bir kural motoru, kalite standartlarınıza uygun her şey. +FailproofAI Cloud ayrıca transkript şeklinden `helpfulness`, `tool_efficiency` ve `factuality` puanlandıran **çalışan bir referans değerlendiricisi** ile gelir. Başlangıç noktası olarak kopyalayın ve kendi mantığınızla değiştirin: bir LLM yargıçsı, bir kural motoru, kalite standartlarınıza uygun her şey. Minimum uygulanabilir değerlendirici: @@ -148,7 +148,7 @@ def run(req: EvalRequest) -> EvalResponse: `app` örneği herhangi bir ASGI sunucusu altında çalışır, bu nedenle `uvicorn module:app` başlatır. -Pahalı işi ertelemeleri gereken değerlendiriciler için, bunun yerine `JobPending` döndürün ve `@app.job_lookup` işleyicisini kaydedin; Observability sunucusu terminal durum döndürene veya `EVALUATOR_MAX_POLL_DURATION_SECS` sınırı (varsayılan 1 sa) geçene kadar `GET /evaluate/{job_id}` yoklaması yapar. +Pahalı işi ertelemeleri gereken değerlendiriciler için, bunun yerine `JobPending` döndürün ve `@app.job_lookup` işleyicisini kaydedin; FailproofAI Cloud sunucusu terminal durum döndürene veya `EVALUATOR_MAX_POLL_DURATION_SECS` sınırı (varsayılan 1 sa) geçene kadar `GET /evaluate/{job_id}` yoklaması yapar. Tam API başvurusu, asenkron desen ve olay şeması `agenteye-evaluator` SDK'sının README'sinde belgelenmiştir. @@ -156,7 +156,7 @@ Tam API başvurusu, asenkron desen ve olay şeması `agenteye-evaluator` SDK'sı ## Değerlendiricininizi Çalıştırma -Değerlendirici **sizin hizmetinizdir** — Failproof AI Observability varsayılan bir değerlendirici seviyesiyle gelmez, bu nedenle kendi hizmetlerinizi çalıştırdığınız yerde oluşturup çalıştırırsınız. Herhangi bir ASGI sunucusu altında çalışır (örneğin `uvicorn my_evaluator:app`); [HTTP sözleşmesinden](#http-sözleşmesi) `/health`, `/config` ve `/evaluate` rotalarını sunun, ardından sunucuyu ona gösterin (bkz. [Sunucuyu Yapılandırma](#sunucuyu-yapılandırma)). +Değerlendirici **sizin hizmetinizdir** — FailproofAI Cloud varsayılan bir değerlendirici seviyesiyle gelmez, bu nedenle kendi hizmetlerinizi çalıştırdığınız yerde oluşturup çalıştırırsınız. Herhangi bir ASGI sunucusu altında çalışır (örneğin `uvicorn my_evaluator:app`); [HTTP sözleşmesinden](#http-sözleşmesi) `/health`, `/config` ve `/evaluate` rotalarını sunun, ardından sunucuyu ona gösterin (bkz. [Sunucuyu Yapılandırma](#sunucuyu-yapılandırma)). Değerlendirici erişilebilir olduğunda, `GET /health` `{"status":"ok"}` döndürür. Bir agent'ı uçtan uca çalıştırdıktan sonra, sunucudaki `GET /evaluations` değerlendiricininizin ürediği puanlarla `status: "done"` olan bir satır döndürür. @@ -254,7 +254,7 @@ Bootstrap admin (`ADMIN_KEY`, `ADMIN_EMAIL`) otomatik olarak bunları alır. - **`/sessions`**: filtrelenebilir oturum ızgarası; puan sütunu her oturumun değerlendirme durumunu ve puanlarını bir bakışta gösterir. - **`/dashboards`**: kaydedilmiş eval-sağlığı görünümleri (aşağıdaki [Panolar](#panolar) öğesine bakın). -![Oturum başına değerlendirme durumu hapları ve renk kodluylu puan rozetleri (yararlılık, doğruluk, tool_efficiency, güvenlik, uyum) bulunan Oturumlar ızgarası](/agenteye/images/sessions-list.png) +![Oturum başına değerlendirme durumu hapları ve renk kodluylu puan rozetleri (yararlılık, doğruluk, tool_efficiency, güvenlik, uyum) bulunan Oturumlar ızgarası](/cloud/images/sessions-list.png) *Oturumlar ızgarası her çalışmanın değerlendirme durumunu ve puanlarını bir bakışta gösterir; kırmızı/turuncu/yeşil rozet düşük puanları öne çıkarır.* @@ -271,7 +271,7 @@ Her pano sabitler: Her kart eşleşen oturum sayısını, bir done/error/timeout dökümünü, her öne çıkarılan puanın ortalamasını ve küçük bir trend sparkline'ını gösterir. Bir panoyu açmak tam boyutlu panelları gösterir; **"oturumları aç"** sizi tam olarak bu dilime önceden filtrelenmiş oturumlar sayfasına bırakır. Metrikler sunucu tarafında tam eşleşen küme üzerinden (via `GET /evaluations/aggregate`) hesaplanır, bu nedenle sayılar örneklenmiş yerine kesindir. -![Ortalama puan çubukları, araç tamam-vs-hata dökümü, en iyi araçlar ve saat başına olaylar trendi bulunan bir eval-sağlığı panosu](/agenteye/images/dashboard-quality.png) +![Ortalama puan çubukları, araç tamam-vs-hata dökümü, en iyi araçlar ve saat başına olaylar trendi bulunan bir eval-sağlığı panosu](/cloud/images/dashboard-quality.png) **İzinler:** görüntüleme hem `dashboards:read` hem de `evaluations:read` gerektirir; oluşturma ve düzenleme `dashboards:write` gerektirir; silme `dashboards:delete` gerektirir. Bootstrap admin bunların tümünü otomatik olarak alır. @@ -293,7 +293,7 @@ Her kart eşleşen oturum sayısını, bir done/error/timeout dökümünü, her ## Sonraki adımlar -- [Değerlendirici agent becerisi](/tr/agenteye/evaluator-skill): kodlama agent'ının boyutlarınızı gerçek oturumlara karşı tasarlaması ve bu hizmeti sizin için oluşturması. -- [Python SDK](/tr/agenteye/python-sdk): puanlamayı tetikleyen `agent_end` olaylarını yayın. -- [API anahtarları](/tr/agenteye/api-keys): `evaluations:read` ve `evaluations:trigger` izinleri. -- [Denetimler](/tr/agenteye/audits): Observability'nin diğer otomatik kalite özelliği, ilke tabanlı inceleme için. \ No newline at end of file +- [Değerlendirici agent becerisi](/tr/cloud/agent-skills): kodlama agent'ının boyutlarınızı gerçek oturumlara karşı tasarlaması ve bu hizmeti sizin için oluşturması. +- [Python SDK](/tr/cloud/sdk): puanlamayı tetikleyen `agent_end` olaylarını yayın. +- [API anahtarları](/tr/cloud/access): `evaluations:read` ve `evaluations:trigger` izinleri. +- [Denetimler](/tr/cloud/audits): FailproofAI Cloud'nin diğer otomatik kalite özelliği, ilke tabanlı inceleme için. \ No newline at end of file diff --git a/docs/tr/agenteye/event-stream.mdx b/docs/tr/cloud/event-stream.mdx similarity index 78% rename from docs/tr/agenteye/event-stream.mdx rename to docs/tr/cloud/event-stream.mdx index 0bb5f16d..a16be744 100644 --- a/docs/tr/agenteye/event-stream.mdx +++ b/docs/tr/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "Ajanınız bir şey yaptığı anda, siz bunu görürsünüz." Ajanınız bir şey yaptığı anda, siz bunu görürsünüz. Olay Akışı, üretim ortamındaki her ajan hakkında canlı bilgi almanın yoludur: bekleme yok, log dosyalarında arama yok, ne olduğunu tahmin etme yok. -![Canlı Olay Akışı: renk kodlu olay satırları gerçek zamanlı olarak aşağıya doğru ilerliyor, ortam, ajan, oturum, olay türü ve serbest metin ile filtrelenebiliyor](/agenteye/images/events-stream.png) +![Canlı Olay Akışı: renk kodlu olay satırları gerçek zamanlı olarak aşağıya doğru ilerliyor, ortam, ajan, oturum, olay türü ve serbest metin ile filtrelenebiliyor](/cloud/images/events-stream.png) *Kuruluşunuzdaki her ajandan gelen her olay, en yenisi önce, olur olmaz güncelleniyor.* @@ -37,14 +37,14 @@ Serbest metin araması, elinizde zaten bulunan bir mesaja, bir araç adına veya Olay Akışı kuruluş ana sayfanızdır. Oturum açarsınız ve onu ilk inen yüzey, `//` konumundadır, böylece triage anda başlar. -Arkasında, ajanlarınız SDK aracılığıyla olaylar yayınlar, toplayıcı bunları Failproof AI Observability sunucunuza gönderir ve akış kontrol ettiğiniz altyapıya ulaştıkça bunları izler. Işık İzler yerine özetlenmiş görünümü istediğinizde, her çalışmanın olayları Sessions'da tek bir satıra daraltılır, bir tıkla uzaktadır. +Arkasında, ajanlarınız SDK aracılığıyla olaylar yayınlar, toplayıcı bunları FailproofAI Cloud sunucunuza gönderir ve akış kontrol ettiğiniz altyapıya ulaştıkça bunları izler. Işık İzler yerine özetlenmiş görünümü istediğinizde, her çalışmanın olayları Sessions'da tek bir satıra daraltılır, bir tıkla uzaktadır. Bu, her diğer gözlemci yüzeyinin üzerine inşa ettiği ham doğru kaynaktır, bu nedenle bir sayı başka bir yerde yanlış görünüyorsa, akış aslında ne olduğunu onayladığınız yerdir. ## İlgili -- [Sessions](/tr/agenteye/sessions): aynı olaylar çalışma başına tek satıra özetlenerek git tarzı yürütme grafiği ile birlikte. -- [Telemetry](/tr/agenteye/telemetry): ajanlarınızın ne gönderdiği ve olayların akışa nasıl ulaştığı. -- [Error tracking](/tr/agenteye/error-tracking): her şeyin yanlış gittiği bir triage yüzeyi. -- [Alerts](/tr/agenteye/alerts): herhangi bir eşiği bir çağrı kuralına dönüştürün. -- [CLI and agents](/tr/agenteye/cli-and-agents): terminalinizden gelen aynı canlı izleme. \ No newline at end of file +- [Sessions](/tr/cloud/sessions): aynı olaylar çalışma başına tek satıra özetlenerek git tarzı yürütme grafiği ile birlikte. +- [Telemetry](/tr/cloud/performance): ajanlarınızın ne gönderdiği ve olayların akışa nasıl ulaştığı. +- [Error tracking](/tr/cloud/errors): her şeyin yanlış gittiği bir triage yüzeyi. +- [Alerts](/tr/cloud/alerts): herhangi bir eşiği bir çağrı kuralına dönüştürün. +- [CLI and agents](/tr/cloud/cli): terminalinizden gelen aynı canlı izleme. \ No newline at end of file diff --git a/docs/tr/cloud/fleet.mdx b/docs/tr/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/tr/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/tr/agenteye/incidents.mdx b/docs/tr/cloud/incidents.mdx similarity index 83% rename from docs/tr/agenteye/incidents.mdx rename to docs/tr/cloud/incidents.mdx index fea1afa2..ae95cfe9 100644 --- a/docs/tr/agenteye/incidents.mdx +++ b/docs/tr/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "Bir uyarı tetiklendiğinde, herkes olayın açık olduğunu, kimi Bir uyarı tetiklendiğinde, ilk soru her zaman "kim bunu ele alıyor?" Olaylar buna yanıt verir: bir şey ihlal olduğu anda, herkes olayın açık olduğunu, kimin sahip olduğunu ve tam olarak şimdiye kadar neler olduğunu görebilir; doğrudan bir post-mortem'e verebileceğiniz temiz, atfedilen bir kaydı ile. -![Olaylar gelen kutusu: uyarı bağlantılı ve manuel olarak açılmış olay kartları, duruma göre gruplandırılmış, her birinin bir önem düzeyi rozeti ve bir sorumlusu var](/agenteye/images/incidents.png) +![Olaylar gelen kutusu: uyarı bağlantılı ve manuel olarak açılmış olay kartları, duruma göre gruplandırılmış, her birinin bir önem düzeyi rozeti ve bir sorumlusu var](/cloud/images/incidents.png) *Gelen kutusu açık olayları duruma göre gruplandırır ve önem düzeyi ve sorumlulu göre filtreler, böylece şu anda insan müdahalesine ihtiyaç duyan şeyleri görürsünüz.* ## Kimin sahip olduğunu bir bakışta bilin @@ -17,10 +17,10 @@ Artık bir sohbet dizisinde "bunu kim bakıyor?" sorusu yok. Bir ihlal otomatik Olay bittiğinde, zaten yazı işleriniz hazırdır. Herhangi bir olayı açın ve ihlal kanıtını, sorumluları ve abone uygulamasını, yerinde koordinasyon için bir yorum dizisini ve append-only etkinlik zaman çizelgesini alırsınız. -![Bir olay detay görünümü: ana uyarı ve ihlal özeti, sorumlular ve abone uygulaması, atfedilen etkinlik zaman çizelgesi ve yorum dizisi](/agenteye/images/incident-detail.png) +![Bir olay detay görünümü: ana uyarı ve ihlal özeti, sorumlular ve abone uygulaması, atfedilen etkinlik zaman çizelgesi ve yorum dizisi](/cloud/images/incident-detail.png) *Olan her şey, sırayla, her satır bunu yapan tarafından imzalanmış.* -Her eylem (açıldı, kabul edildi, çözüldü, vb.) bu zaman çizelgesine yazılır ve hiçbir zaman düzenlenmez. Her giriş atfedilir: onu yapan operatöre, e-posta ile veya Failproof AI Observability'nin kendi başına yaptığı her şey için **automated** olarak (ihlal üzerine olay açmak gibi). Hiçbir şey anonim değildir ve hiçbir şey kaybolmaz, bu nedenle post-mortem daha az çok kendi kendini yazar. +Her eylem (açıldı, kabul edildi, çözüldü, vb.) bu zaman çizelgesine yazılır ve hiçbir zaman düzenlenmez. Her giriş atfedilir: onu yapan operatöre, e-posta ile veya FailproofAI Cloud'nin kendi başına yaptığı her şey için **automated** olarak (ihlal üzerine olay açmak gibi). Hiçbir şey anonim değildir ve hiçbir şey kaybolmaz, bu nedenle post-mortem daha az çok kendi kendini yazar. ## Bir olay nasıl hareket eder @@ -45,6 +45,6 @@ Olaylar `//incidents` konumunda bulunur. Görüntüleme **`incidents:r ## İlişkili -- [Uyarılar](/tr/agenteye/alerts): bir eşik ihlal ettiğinde bu olayları açan kurallar. -- [Hata izleme](/tr/agenteye/error-tracking): her hatayı tek bir yerde görün ve birini uyarıya yükseltin. -- [Denetim](/tr/agenteye/audits): hiçbir kuralın izlemediği hataları bulan zamanlanmış analist. \ No newline at end of file +- [Uyarılar](/tr/cloud/alerts): bir eşik ihlal ettiğinde bu olayları açan kurallar. +- [Hata izleme](/tr/cloud/errors): her hatayı tek bir yerde görün ve birini uyarıya yükseltin. +- [Denetim](/tr/cloud/audits): hiçbir kuralın izlemediği hataları bulan zamanlanmış analist. \ No newline at end of file diff --git a/docs/tr/cloud/managed-policies.mdx b/docs/tr/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/tr/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/tr/cloud/overview.mdx b/docs/tr/cloud/overview.mdx new file mode 100644 index 00000000..85f423bc --- /dev/null +++ b/docs/tr/cloud/overview.mdx @@ -0,0 +1,107 @@ +--- +title: "Failproof AI: Ajanlarınızdaki Hataları Gözlemleyin" +description: "FailproofAI Cloud, üretim ortamında AI ajanlarınızı gözlemlemek, değerlendirmek ve geliştirmek için kendi sunucunuzda çalışan bir platformdur." +--- + +FailproofAI Cloud, üretim ortamında AI ajanlarınızı gözlemlemek, değerlendirmek ve geliştirmek için kendi sunucunuzda çalışan bir platformdur. Ajanlarınızın yaptığı her şeyi kaydeder (her araç çağrısı, model isteği, hook ve hata), her çalıştırmanın kalitesini puanlandırır ve bilmediğiniz hataları ortaya çıkarır — tamamı kendi altyapınızda çalıştırdığınız bir panoda. + +AI ajanları yayınladıysanız ve bir çalıştırmanın neden başarısız olduğunu tahmin etmekten bıktıysanız, burası başlamanız gereken sayfa. FailproofAI Cloud'nin size ne sunduğunu ve parçaların nasıl bir araya geldiğini açıklar; herhangi bir şey yüklemeden önce okuyun. + +> **FailproofAI Cloud, Failproof AI'dan bir kurumsal üründür.** Canlı olarak görmek ister misiniz? Bir demo talep edin: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) adresine e-posta gönderin. + +![FailproofAI Cloud oturumu, git tarzı bir yürütme grafiği olarak çizilmiş, yanında olay zaman çizelgesi ve sağ panelde araçlar, modeller ve hooklar ayrıntısı](/cloud/images/session-detail.png) + +*Her ajan çalıştırması, git tarzı bir yürütme grafiği (sol) olarak çizilmiş ve yanında olay zaman çizelgesi vardır. Paralel alt ajanların her birinin kendi şeridi vardır; sağ panel, çalıştırmanın araçlarını, modellerini, hooklarını ve token harcamasını ayrıntılarıyla gösterir.* + +--- + +## Canlı olarak görmek + +İki kısa video, ekiplerin ilk başta yaptığı iki şeyi gösterir: bir çalıştırmayı izlemek ve hataları otomatik olarak bulma. + +
    + +
    + +*Ajan izleme: hedeften araçlara ve son cevaba kadar tek bir çalıştırmayı adım adım izleyin.* + +
    + +
    + +*Failproof Audit: FailproofAI Cloud'nin günlüklerinizi oturumlar arasında analiz etmesine izin verin ve ne düzeltmesi gerektiğini öğrenin.* + +--- + +## Ekipler neden kullanıyor + +- **Ajanınızın gerçekte ne yaptığını görün.** Her çalıştırma okunabilir bir git tarzı yürütme grafiğine dönüşür: hangi araçlar paralel çalıştı, hangi alt ajanlar dallandı, nerede durdu ve ne harcadı. +- **Kalite gerilemeşini otomatik olarak yakalayın.** Küçük bir puanlama hizmetini bağlayın ve FailproofAI Cloud her tamamlanan çalıştırmayı puanlandırır; böylece yardımcılıkta düşüş veya halüsinasyonlarda artış kendi kendine ortaya çıkar. +- **Kuralı yazacağınızı bilmediğiniz hataları bulun.** Yinelenen denetimler, günlüklerinizi oturumlar arasında hata kümeleri, gecikme aykırı değerleri, düşük puanlar ve takılı çalıştırmalar açısından analiz eder, ardından size sıralanmış, kanıtla desteklenmiş bulgular sunar. +- **Önemli olduğunda sayfa alın.** Eşik kuralları hata oranı, gecikme, maliyet veya değerlendirici puanlarında çalışır ve yanıtlayabileceğiniz, atayabileceğiniz ve çözebileceğiniz olaylar açar. +- **Düz İngilizcede sorular sorun.** Panoda yer alan bir AI asistanı, kendi verileriniz üzerinde „bu hafta üretimde kalite nasıl gelişiyor?" gibi soruları yanıtlar. Yaptığı her değişiklik onay geçidir. +- **Verilerinizi tutun.** FailproofAI Cloud kendi sunucunuzda çalışır: olaylar, istemler ve analizler kontrol ettiğiniz altyapıda kalır. + +--- + +## Ne alıyorsunuz + +FailproofAI Cloud, üç fikir etrafında organize edilmiştir (**gözlemle**, **analiz et** ve **yönet**), panelin sol kenar çubuğuna yansıtılır. + +**Gözlemle** (ne olduğunun ham gerçeği): + +- **[Olay akışı](/tr/cloud/event-stream)**: her çalıştırmanın canlı, adım adım izi (araç çağrıları, model çağrıları, hooklar, hatalar). +- **[Oturumlar](/tr/cloud/sessions)**: bu olaylar, çalıştırma başına bir satır halinde, her biri puanlandırılmaya hazır, git tarzı bir yürütme grafiği ile birlikte sunulur. +- **[Performans metrikleri](/tr/cloud/performance)**: yüzey başına gecikme harita grafikleri ve modeller, araçlar ve hooklar için p50/p95/p99 vitalleri; böylece kuyruk artışı ortalamanın dışında görünür. +- **[Hata izleme](/tr/cloud/errors)**: her şeyin ters gittiği tek bir işlem yüzeyinde; bir uyarının ateşlenmesinden tek bir tıkla uzak. + +![Tools gözlemle sayfası: gecikme harita grafiği, yüzdelik dilim bandı ve 24 zaman kutusu üzerinde araç dağılım çubuğu](/cloud/images/tools.png) + +*Her gözlemle yüzeyi, bir kıvılcım çizgisi ve p50/p95/p99 vitalleriyle bir gecikme harita grafiği ve yüzdelik dilim bandını eşleştirir. Gösterilen: Araçlar.* + +**Analiz et** (etkinliği cevaplara dönüştürün): + +- **[Sorgular](/tr/cloud/queries)** ve **[panolar](/tr/cloud/dashboards)**: olaylarınız ve değerlendirmeleriniz üzerinde kaydedilmiş SQL, paylaşılan, kurum kapsamı panolara çizilmiştir. +- **[Değerlendirmeler](/tr/cloud/evaluations)**: kendi değerlendirici hizmetiniz tarafından üretilen kalite puanları, puan başına akıl yürütmesi ile. +- **[Denetimler](/tr/cloud/audits)**: oturumlar arasında hata modellerini ortaya çıkaran yinelenen araştırmalar. +- **[Uyarılar](/tr/cloud/alerts)** ve **[olaylar](/tr/cloud/incidents)**: sizi sayfaya alan eşik kuralları, artı bunları işlemek için bir olay iş akışı. + +**Arayüzler** (verilerinize kendi yolunuzla ulaşın): + +- **[CLI](/tr/cloud/cli)**: tüm dağıtımınızı terminalden veya bir betikten çalıştırın ve bir kodlama ajanının bunu düz İngilizcede yapmasına izin verin. +- **[AI asistanı](/tr/cloud/assistant)**: ajanlarınız hakkında düz İngilizcede soru sorun, doğrudan panoda. +- **REST API**: panelin ve CLI'nin yaptığı her şey, kapsamlı bir [API anahtarı](/tr/cloud/access) ile doğrudan çağırabileceğiniz bir REST API tarafından desteklenir — olayları alın, oturumları ve değerlendirmeleri sorgulayın ve panoları, uyarıları, denetimleri, kullanıcıları ve anahtarları yönetin; böylece FailproofAI Cloud'yi kendi araçlarınızla entegre edin. + +**Yönet** (ekibiniz için çalıştırın): + +- **[API anahtarları](/tr/cloud/access)**: toplayıcı, pano ve asistan için kapsamlı jetonlar. +- **Kullanıcılar**: şifresiz, e-posta tabanlı oturum açma ve izin listesiyle. +- **Ayarlar**: kurum başına yapılandırma, model bağlam penceresi geçersiz kılmalar dahil. + +--- + +## Parçalar nasıl bir araya gelir + +Veri bir yönde akar, ajan kodunuzdan panoya: ajanınız (Python SDK aracılığıyla) agenteye-toplayıcıya olaylar yayınlar; bu olaylar sunucuya gönderilir ve sunucu panoyu sunar. İki isteğe bağlı hizmet bunu tamamlar — bir puanlama hizmet (değerlendirmeler) ve bir AI asistan hizmet (panoda sohbet). + +- **Python SDK**: ajanınıza birkaç `agenteye.event.*` çağrısı eklersiniz; olaylar yerel olarak arabelleğe alınır. +- **agenteye-toplayıcı**: her ajan makinesinde, olayları toplu olarak işleyen ve sunucuya gönderen hafif bir daemon. +- **Sunucu**: olaylarınızı alır, operasyonel durumu kendi veritabanlarınızda tutar ve pano, CLI ve kendi entegrasyonlarınızın hepsinin kullandığı REST API'yi sunar. +- **Pano**: her şeyi keşfettiğiniz yer. +- **İsteğe bağlı hizmetler**: bir puanlama hizmet (değerlendirmeler) ve bir AI asistan hizmet (panoda sohbet). + +Belgeler genelinde kullanılan kelime dağarcığı (*olay, oturum, değerlendirme, denetim, bulgu, olay*) için bkz. [Kavramlar](/tr/concepts). + +--- + +## FailproofAI Cloud'yi Almak + +FailproofAI Cloud, Failproof AI'dan bir kurumsal üründür ve FailproofAI guardrails — politika ve korkuluk ürünü — ile Failproof AI markası altında birlikte çalışır. Tamamen kendi ortamınızda çalışır. Paketlere henüz erişiminiz yoksa, bir demo talep edin ve sizi hazırlayacağız: [nikita@befailproof.ai](mailto:nikita@befailproof.ai) adresine e-posta gönderin. + +--- + +## Sonraki adımlar + +- [Kavramlar](/tr/concepts): FailproofAI Cloud kelime dağarcığı bir yerde. +- [Observabilite](/tr/cloud/overview): ajanlarınızın ne yaptığını, çalıştırmayı izleyin. +- [Güvenlik](/tr/cloud/security): FailproofAI Cloud verilerinizi nasıl izole tuttuğu ve kontrol altında tuttuğu. \ No newline at end of file diff --git a/docs/tr/agenteye/telemetry.mdx b/docs/tr/cloud/performance.mdx similarity index 82% rename from docs/tr/agenteye/telemetry.mdx rename to docs/tr/cloud/performance.mdx index 6e4dde0a..0e9986b3 100644 --- a/docs/tr/agenteye/telemetry.mdx +++ b/docs/tr/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "Modellerinizin, araçlarınızın veya hook'larınızın yavaşlad Modellerinizin, araçlarınızın veya hook'larınızın yavaşladığı veya maliyeti artırdığı anı görün ve kullanıcılarınız bunu hissetmeden tail-latency artışını yakalayın. Üç ayrı sayfa ham zamanlama verilerini p50, p95 ve p99'a dönüştürerek bir bakışta okuyabileceğiniz hale getirir. -![Models sayfası, latency ısı haritasını, yüzdelik bantı ve model başına token, maliyet ve bağlam penceresi rakamlarını gösteriyor](/agenteye/images/models.png) +![Models sayfası, latency ısı haritasını, yüzdelik bantı ve model başına token, maliyet ve bağlam penceresi rakamlarını gösteriyor](/cloud/images/models.png) *Models sayfası: latency ısı haritası, yüzdelik bandı ve model başına tokenler, tahmini maliyet ve bağlam penceresi doldurma.* ## Ortalamaların kötü çalışmaları saklamasına izin vermeyin @@ -24,13 +24,13 @@ Paylaşılan bir hover crosshair ısı haritasını ve bandı zaman olarak bağl Models sayfası (üstte gösterilmiştir) bir faturanın her zaman ortaya çıkardığı iki soruya yanıt verir: hangi model ve ne kadar. Paylaşılan latency görünümünün üzerine, **model başına token tüketimi**, **tahmini maliyet** ve **bağlam penceresi doldurma** ekler, böylece kontrolsüz prompt büyümesi ve yaklaşan sıkıştırma sizi şaşırtmadan önce görünür. -Failproof AI Observability ortak model kimliklerini otomatik olarak tanır. Bir pencere yanlış görünüyorsa veya kendi özel modelinizi çalıştırıyorsanız, **Settings** altında, **model context windows** içinde düzeltin veya ekleyin ve doldurma okumaları bunu takip eder. +FailproofAI Cloud ortak model kimliklerini otomatik olarak tanır. Bir pencere yanlış görünüyorsa veya kendi özel modelinizi çalıştırıyorsanız, **Settings** altında, **model context windows** içinde düzeltin veya ekleyin ve doldurma okumaları bunu takip eder. ## Tools: yavaş olanı kırık olandan ayırt edin Bir tool çağrısı yavaş olabilir veya sessizce başarısız olabilir ve bunu günlükleri inceledikten sonra değil de saniyeler içinde bilmek istersiniz. -![Tools sayfası, paylaşılan latency ısı haritasını ve yüzdelik bandı yanında başarı ve hata dökümü ile tool dağılım çubuğunu gösteriyor](/agenteye/images/tools.png) +![Tools sayfası, paylaşılan latency ısı haritasını ve yüzdelik bandı yanında başarı ve hata dökümü ile tool dağılım çubuğunu gösteriyor](/cloud/images/tools.png) *Tools sayfası: aynı ısı haritası ve yüzdelik bant, artı başarı ve hata dökümü ile tool dağılım çubuğu.* Paylaşılan latency görünümünün yanında, Tools sayfası bir **başarı ve hata dökümü** ve **tool dağılım çubuğu** ekler, böylece bir bakışta hangi toolları en çok kullandığınızı ve hangilerinin hata bütçenizi tükettiğini görürsünüz. @@ -39,14 +39,14 @@ Paylaşılan latency görünümünün yanında, Tools sayfası bir **başarı ve Bir lifecycle hook bir çalışmayı yavaşlatırken, "hook'lar yavaş" üzerinde harekete geçebileceğiniz bir şey değildir. Hooks sayfası sizi önemli olana götürür. -![Hooks sayfası, latency'nin paylaşılan ısı haritası ve yüzdelik bandı üzerinde hook adı ve trigger olayına göre dökülmüş olarak gösteriyor](/agenteye/images/hooks.png) +![Hooks sayfası, latency'nin paylaşılan ısı haritası ve yüzdelik bandı üzerinde hook adı ve trigger olayına göre dökülmüş olarak gösteriyor](/cloud/images/hooks.png) *Hooks sayfası: latency'nin hook adı ve trigger olayına göre dökülmüş.* Aynı latency ısı haritası ve yüzdelik bandı üzerinde, Hooks sayfası etkinliği **hook adı** ve **trigger olayı** tarafından kırıyor, böylece ilgilenilmesi gereken tek hook'a ve tek olaya inersiniz. ## İlgili -- [Event stream](/tr/agenteye/event-stream): her olayın canlı, renkle kodlanmış izi. -- [Sessions](/tr/agenteye/sessions): olayları çalışma başına bir satırda toplayın ve yürütme grafiğini açın. -- [Error tracking](/tr/agenteye/error-tracking): panoda kırmızı olan her şey için tek triage yüzeyi. -- [Dashboards](/tr/agenteye/dashboards): filoğunuz genelinde toparlama görünümleri. \ No newline at end of file +- [Event stream](/tr/cloud/event-stream): her olayın canlı, renkle kodlanmış izi. +- [Sessions](/tr/cloud/sessions): olayları çalışma başına bir satırda toplayın ve yürütme grafiğini açın. +- [Error tracking](/tr/cloud/errors): panoda kırmızı olan her şey için tek triage yüzeyi. +- [Dashboards](/tr/cloud/dashboards): filoğunuz genelinde toparlama görünümleri. \ No newline at end of file diff --git a/docs/tr/agenteye/queries.mdx b/docs/tr/cloud/queries.mdx similarity index 78% rename from docs/tr/agenteye/queries.mdx rename to docs/tr/cloud/queries.mdx index e15b8545..a44b4b08 100644 --- a/docs/tr/agenteye/queries.mdx +++ b/docs/tr/cloud/queries.mdx @@ -3,9 +3,9 @@ title: "Sorgular" description: "Agent verilerinize herhangi bir soru sorun ve saniyeler içinde cevap alın." --- -Agent verilerinize herhangi bir soru sorun ve saniyeler içinde cevap alın. Failproof AI Observability, etkinlikleriniz ve değerlendirmeleriniz üzerinde kaydedilmiş, hazır kullanıma sunulmuş sorguların bir kütüphanesini sunar; böylelikle boş bir SQL düzenleyicisinden başlamak yerine çalışan bir örnek üzerinden başlarsınız. +Agent verilerinize herhangi bir soru sorun ve saniyeler içinde cevap alın. FailproofAI Cloud, etkinlikleriniz ve değerlendirmeleriniz üzerinde kaydedilmiş, hazır kullanıma sunulmuş sorguların bir kütüphanesini sunar; böylelikle boş bir SQL düzenleyicisinden başlamak yerine çalışan bir örnek üzerinden başlarsınız. -![Kaydedilmiş sorgular kütüphanesi: yeniden kullanılabilir sorguların ızgarası, hem yerleşik ön ayarlar hem de özel olanlar](/agenteye/images/queries.png) +![Kaydedilmiş sorgular kütüphanesi: yeniden kullanılabilir sorguların ızgarası, hem yerleşik ön ayarlar hem de özel olanlar](/cloud/images/queries.png) *`//queries` konumundaki kaydedilmiş sorgular kütüphanesi: yerleşik ön ayarlar ekibinizin kaydettiği sorgularla yan yana yer almakta.* @@ -21,7 +21,7 @@ Her kaydedilmiş sorgu kuruluş kapsamlıdır ve paylaşılıdır; bu nedenle ek Herhangi bir sorguyu açın ve SQL bestecisine iner; burada sorguyu ayarlayabilir ve cevabı hemen görebilirsiniz: dışa aktarma yok, gidiş-dönüş yok, başkasının beklenmesi yok. -![Kaydedilmiş sorguyu çalıştıran SQL sorgu bestecisi, şema kenar çubuğu ve canlı sonuç ızgarası](/agenteye/images/query-lab.png) +![Kaydedilmiş sorguyu çalıştıran SQL sorgu bestecisi, şema kenar çubuğu ve canlı sonuç ızgarası](/cloud/images/query-lab.png) *SQL bestecisi: sol tarafta sorgunuz, kolon adını asla tahmin etmeniz gerekmeyen şema kenar çubuğu ve altta canlı sonuç ızgarası.* @@ -42,14 +42,14 @@ agenteye query list # terminal'deki aynı kaydedilmiş sorgular agenteye query run errs --arg prod # birini çalıştırın ve satırları yazdırın (boru için --json ekleyin) ``` - Tam komut seti için [CLI ve ajanlar](/tr/agenteye/cli-and-agents) konusuna bakın. + Tam komut seti için [CLI ve ajanlar](/tr/cloud/cli) konusuna bakın. -- **AI asistanından.** SQL'i nasıl ifade edeceğiniz konusunda emin değil misiniz? Panodaki [AI asistanına](/tr/agenteye/assistant) düz İngilizce sorun ve sorguyu taslak halinde oluşturup kütüphaneyinize kaydedecektir. +- **AI asistanından.** SQL'i nasıl ifade edeceğiniz konusunda emin değil misiniz? Panodaki [AI asistanına](/tr/cloud/assistant) düz İngilizce sorun ve sorguyu taslak halinde oluşturup kütüphaneyinize kaydedecektir. Kaydedilmiş sorguyu çalıştırmak `queries:run` izni tarafından kontrol edilir; sorgu oluşturma veya silme izinlerinden ayrı tutulur; bu nedenle herkesin kütüphaneyi yeniden yazmasına izin vermeden okuma erişimi verebilirsiniz. ## İlgili -- [Panolar](/tr/agenteye/dashboards): sorgu sonuçlarını paylaşılan, kuruluş genelinde çizelgelere sabitleyin. -- [AI asistanı](/tr/agenteye/assistant): sorulara düz İngilizce olarak sorun ve sorgu alın. -- [CLI ve ajanlar](/tr/agenteye/cli-and-agents): terminal'den aynı sorguları çalıştırın ve kaydedin. \ No newline at end of file +- [Panolar](/tr/cloud/dashboards): sorgu sonuçlarını paylaşılan, kuruluş genelinde çizelgelere sabitleyin. +- [AI asistanı](/tr/cloud/assistant): sorulara düz İngilizce olarak sorun ve sorgu alın. +- [CLI ve ajanlar](/tr/cloud/cli): terminal'den aynı sorguları çalıştırın ve kaydedin. \ No newline at end of file diff --git a/docs/tr/agenteye/python-sdk.mdx b/docs/tr/cloud/sdk.mdx similarity index 91% rename from docs/tr/agenteye/python-sdk.mdx rename to docs/tr/cloud/sdk.mdx index 4ef73ed3..56a9dfc5 100644 --- a/docs/tr/agenteye/python-sdk.mdx +++ b/docs/tr/cloud/sdk.mdx @@ -5,11 +5,11 @@ description: "Üretim ortamında AI ajanlarınızın tam olarak ne yaptığını --- -Üretim ortamında AI ajanlarınızın tam olarak ne yaptığını görün: her ajan çalışması, araç çağrısı, model isteği, hook ve insan müdahalesi. Failproof AI Observability Python SDK, ajan kodunuzun içinden bu izi kaydeder, böylece neler olduğunu hata ayıklamak, denetlemek ve değerlendirmek yapabilirsiniz. Failproof AI Observability'nin ajanlarınızı gözlemlemesini istediğiniz her zaman bunu kullanın. +Üretim ortamında AI ajanlarınızın tam olarak ne yaptığını görün: her ajan çalışması, araç çağrısı, model isteği, hook ve insan müdahalesi. FailproofAI Cloud Python SDK, ajan kodunuzun içinden bu izi kaydeder, böylece neler olduğunu hata ayıklamak, denetlemek ve değerlendirmek yapabilirsiniz. FailproofAI Cloud'nin ajanlarınızı gözlemlemesini istediğiniz her zaman bunu kullanın. Arka planda SDK, yapılandırılmış olayları yerel JSONL dosyalarına yazar ve toplayıcı daemon bunları otomatik olarak alır ve platforma gönderir. Bu dosyaları kendiniz yönetmezsiniz. -> **İpucu:** Failproof AI Observability'ye yeni mi başlıyorsunuz? Bu sayfa, tam SDK olay referansıdır. +> **İpucu:** FailproofAI Cloud'ye yeni mi başlıyorsunuz? Bu sayfa, tam SDK olay referansıdır.
    @@ -27,7 +27,7 @@ Kurulduktan sonra sahip olduğunuzu doğrulayın: python -c "import agenteye; print(agenteye.__version__)" ``` -Bir kodlama ajanının tüm entegrasyonu yapmasını tercih mi ediyorsunuz? [Python SDK Agent Skill](/tr/agenteye/python-sdk-skill) kurulum yolunu bilir, araçlaştırma noktalarını planlar, onları yazar ve olayların ulaştığını doğrular. +Bir kodlama ajanının tüm entegrasyonu yapmasını tercih mi ediyorsunuz? [Python SDK Agent Skill](/tr/cloud/agent-skills) kurulum yolunu bilir, araçlaştırma noktalarını planlar, onları yazar ve olayların ulaştığını doğrular. --- @@ -61,7 +61,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### Gerçek bir çağrı araçlaştırması -Pratikte mevcut ajan kodunuzu sararsınız. Bir model çağrısını `model_request` ve `model_response` ile parantez içine alın, böylece iki olay gerçek isteği kapsar ve Failproof AI Observability onları eşleştirebilir: +Pratikte mevcut ajan kodunuzu sararsınız. Bir model çağrısını `model_request` ve `model_response` ile parantez içine alın, böylece iki olay gerçek isteği kapsar ve FailproofAI Cloud onları eşleştirebilir: ```python import anthropic @@ -100,7 +100,7 @@ Araç çağrılarını da aynı şekilde `tool_use` ve `tool_result` ile sarın, Bu olaylar panoya ulaştığında nasıl görünüyor, türe göre renkle gösterilmiş ve ortam, ajan ve oturum tarafından filtrelenebilir: -![Canlı Events akışı, olay türüne göre renkle gösterilmiş ve ortam, ajan ve oturum tarafından filtrelenebilir](/agenteye/images/events-stream.png) +![Canlı Events akışı, olay türüne göre renkle gösterilmiş ve ortam, ajan ve oturum tarafından filtrelenebilir](/cloud/images/events-stream.png) --- @@ -156,9 +156,9 @@ Bu, redaksiyonu seçiminiz ve sorumluluğunuz yapar. Bir istekte veya araç yük ## Olay Referansı -Çoğu olay, ilişki kimliği paylaşan başlangıç/bitiş çiftleri halinde gelir: `tool_use` ve `tool_result` bir `tool_call_id` paylaşır, `hook_triggered` ve `hook_completed` bir `hook_id` paylaşır ve `human_wait` ve `human_input` bir `input_id` paylaşır. Başlangıç olayını yayınlayın, işi yapın, ardından aynı kimlikle bitiş olayını yayınlayın. Failproof AI Observability çifti eşleştirir ve `duration_ms` sizin için hesaplar, bu nedenle asla kendiniz `duration_ms` geçirmezsiniz. +Çoğu olay, ilişki kimliği paylaşan başlangıç/bitiş çiftleri halinde gelir: `tool_use` ve `tool_result` bir `tool_call_id` paylaşır, `hook_triggered` ve `hook_completed` bir `hook_id` paylaşır ve `human_wait` ve `human_input` bir `input_id` paylaşır. Başlangıç olayını yayınlayın, işi yapın, ardından aynı kimlikle bitiş olayını yayınlayın. FailproofAI Cloud çifti eşleştirir ve `duration_ms` sizin için hesaplar, bu nedenle asla kendiniz `duration_ms` geçirmezsiniz. -![Eşli olaylardan yeniden yapılandırılan bir oturumun git tarzı yürütme grafiği, olay zaman çizelgesi ile birlikte, araç/model/hook dökümü paneli](/agenteye/images/session-detail.png) +![Eşli olaylardan yeniden yapılandırılan bir oturumun git tarzı yürütme grafiği, olay zaman çizelgesi ile birlikte, araç/model/hook dökümü paneli](/cloud/images/session-detail.png) Tüm olay yöntemleri bu iki alanı gerektirir: @@ -432,5 +432,5 @@ Her dosya atomik olarak yazılır: SDK geçici bir dosyaya yazar ve sonra onu ye ## Sonraki adımlar -- [Olay akışı](/tr/agenteye/event-stream): bu olayların canlı ulaştığını izleyin, ortam, ajan ve oturum tarafından renkle gösterilmiş ve filtrelenebilir. -- [Oturumlar](/tr/agenteye/sessions): eşli olayların her ajan çalışmasını yürütme grafiği ve zaman çizelgesi olarak nasıl yeniden yapılandırdığını görün. \ No newline at end of file +- [Olay akışı](/tr/cloud/event-stream): bu olayların canlı ulaştığını izleyin, ortam, ajan ve oturum tarafından renkle gösterilmiş ve filtrelenebilir. +- [Oturumlar](/tr/cloud/sessions): eşli olayların her ajan çalışmasını yürütme grafiği ve zaman çizelgesi olarak nasıl yeniden yapılandırdığını görün. \ No newline at end of file diff --git a/docs/tr/cloud/security.mdx b/docs/tr/cloud/security.mdx new file mode 100644 index 00000000..dae893ac --- /dev/null +++ b/docs/tr/cloud/security.mdx @@ -0,0 +1,69 @@ +--- +--- +title: "Güvenlik" +description: "FailproofAI Cloud, üretim aracılarınızın yakınına yerleştirilmek üzere oluşturulmuştur; bu, istemlerinizi, araç girdilerini ve çıktılarını görebilmesi anlamına gelir." +--- + + +FailproofAI Cloud, üretim aracılarınızın yakınına yerleştirilmek üzere oluşturulmuştur; bu, istemlerinizi, araç girdilerini ve çıktılarını görebilmesi anlamına gelir. Bu sayfa, bu verileri nasıl izole, kontrollü ve sizin elinizde tuttuğunu açıklamaktadır. FailproofAI Cloud'yi bir güvenlik incelemesi için değerlendiriyorsanız, buradan başlayın. + +--- + +## Verileriniz kendi ortamınızda kalır + +FailproofAI Cloud, kendi kendine barındırılır. Olaylar, istemler, model yanıtları ve analizler kendi veritabanlarınızda, kendi ortamınızda depolanır. Hiçbir şey depolama için bir üçüncü taraf SaaS'a gönderilmez ve verileriniz kendi bulut hesabınızda kalır. + +--- + +## Kiracı izolasyonu + +Bir FailproofAI Cloud örneği birçok kuruluşu barındırabilir ve her biri depolama katmanında izole edilir — yalnızca kullanıcı arayüzü tarafından değil, veritabanı tarafından uygulanır: + +- Bir kuruluşun işletimsel verileri (kullanıcılar, anahtarlar, panolar, kaydedilmiş sorgular) o kuruluşa ait olup, kuruluşlar arası okumalar veritabanı tarafından engellenir. +- Her alınan olaya sahip olduğu kuruluş damgası vurulur, böylece bir kuruluşun olayları asla başka bir kuruluş tarafından okunamaz. + +Her pano rotası bir kuruluş slug'ı altında kapsamlandırılır (`//…`). + +--- + +## Oturum açma + +FailproofAI Cloud, şifresiz, e-posta tabanlı oturum açma kullanır. Kimse tarafından ele geçirilebilecek veya sızan bir şifre yoktur. Bir kullanıcı tek seferlik bir kod (veya tek tıklamalı sihirli bir bağlantı) talep eder, bu onlara e-posta ile gönderilir ve hızlı bir şekilde sona erer. Oturum açma bir **izin listesi** tarafından korunur: yalnızca izin verdiğiniz e-posta adresleri (veya etki alanları) kimlik doğrulaması yapabilir. + +![FailproofAI Cloud oturum açma ekranı; tek kullanımlık bir kod e-postanıza gönderir](/cloud/images/login.png) + +--- + +## API anahtarlarıyla kapsamlı erişim + +Her istemci, ayrıntılı, en düşük ayrıcalık izinlerine sahip bir API anahtarı ile kimlik doğrulaması yapar. Bir toplayıcının yalnızca `events:add` öğesi gerekir; bir pano veya asistan anahtarı salt okunur olabilir; yıkıcı eylemler (silme, yeniden oluşturma) dahil etmeyi seçtiğiniz ayrı yetkilendirmelerdir. + +![API anahtarları sayfası: her anahtarın izin verileri, okuma, yazma ve yıkıcı kapsama göre renk kodlu](/cloud/images/api-keys.png) + +Kurulum için yönetici önyükleme anahtarını tutun ve diğer her şey için dar anahtarlar yayınlayın. [API anahtarları](/tr/cloud/access) sayfasına bakın. + +--- + +## Salt okunur, onay kapılı asistan + +Pano içindeki [yapay zeka asistanı](/tr/cloud/assistant) verileriniz üzerinde soruları yanıtlar, ancak tasarım gereği sınırlandırılmıştır: + +- Varsayılan olarak **salt okunur**: SQL'i yalnızca `SELECT`/`WITH` sorgularına, tek deyimli, satır sınırı ile izin veren bir koruma yoluyla çalıştırır. +- Oluşturduğu her şey (kaydedilmiş bir sorgu, bir pano) **onay kapılı**: gerçekleşmeden önce her yazıyı gözden geçirip onaylarsınız. +- **Asla silemez**. + +Yani bir takım arkadaşı "bu hafta hangi aracılar en çok hata verdi?" diye sorabilir ve cevaba göre hareket edebilir, asistan kendi başına verilerinizi değiştirip kaldıramadan. + +--- + +## Aktarım sırasında + +Tüm trafik HTTPS üzerinde çalışır. TLS'yi kendi sertifikalarınızla sonlandırırsınız, böylece toplayıcıdan sunucuya ve tarayıcıdan sunucuya trafik aktarımda şifrelenir. + +--- + +## Sonraki adımlar + +- [Genel Bakış](/tr/cloud/overview): FailproofAI Cloud'nin nasıl bir araya geldiği. +- [API anahtarları](/tr/cloud/access): toplayıcı, pano ve asistan için erişimi kapsamlandırın. +- [Gözlenebilirlik](/tr/cloud/overview): FailproofAI Cloud'nin aracılarınızdan neleri yakaladığı. \ No newline at end of file diff --git a/docs/tr/agenteye/sessions.mdx b/docs/tr/cloud/sessions.mdx similarity index 75% rename from docs/tr/agenteye/sessions.mdx rename to docs/tr/cloud/sessions.mdx index 32b9b557..8cb2acb4 100644 --- a/docs/tr/agenteye/sessions.mdx +++ b/docs/tr/cloud/sessions.mdx @@ -4,9 +4,9 @@ description: "Bir çalıştırmadan gelen her olay, tek bir okunabilir satırda --- -Bir çalıştırmanın neden başarısız olduğunu tahmin etmeyi bırakın. Failproof AI Observability, bir çalıştırmadan gelen her olayı tek bir okunabilir satıra derler, sonra tüm çalıştırmayı saniyeler içinde okuyabileceğiniz git stili bir resim olarak çizer; böylece aracınızın tam olarak ne yaptığını, adım adım görebilirsiniz. +Bir çalıştırmanın neden başarısız olduğunu tahmin etmeyi bırakın. FailproofAI Cloud, bir çalıştırmadan gelen her olayı tek bir okunabilir satıra derler, sonra tüm çalıştırmayı saniyeler içinde okuyabileceğiniz git stili bir resim olarak çizer; böylece aracınızın tam olarak ne yaptığını, adım adım görebilirsiniz. -![Oturumlar listesi: ortamlar ve aracılar arasında çalıştırma başına bir satır, durum rozetleri ve değerlendirme puanı rozet işaretleriyle](/agenteye/images/sessions-list.png) +![Oturumlar listesi: ortamlar ve aracılar arasında çalıştırma başına bir satır, durum rozetleri ve değerlendirme puanı rozet işaretleriyle](/cloud/images/sessions-list.png) *Çalıştırma başına bir satır: durum rozeti çalıştırmanın nasıl sonlandığını bir bakışta gösterir ve bir değerlendirici bağlandıktan sonra bir puan rozeti yanında yer alır.* @@ -30,7 +30,7 @@ Bir değerlendirici bağladıktan sonra, her tamamlanan çalıştırma otomatik ## Tüm çalıştırmayı bir resim olarak okuyun -![Git stili yürütme grafiği, olay zaman çizelgesi yanında, araç, model ve kanca dağılımı paneli](/agenteye/images/session-detail.png) +![Git stili yürütme grafiği, olay zaman çizelgesi yanında, araç, model ve kanca dağılımı paneli](/cloud/images/session-detail.png) *Yürütme grafiği (sol) olay zaman çizelgesinin yanında yer alır; sağ ray, çalıştırma için araçları, modelleri, kancaları ve jeton harcamasını ayrıntılarıyla gösterir.* @@ -38,7 +38,7 @@ Herhangi bir oturumu açmak için tıklayın ve yürütme grafiğini görmek: ar Sağ ray, çalıştırma başına dağılımı sunar: hangi araçlar ve modeller çalıştı, hangi kancalar tetiklendi ve çalıştırma jetonlarda ne kadar harcadı. Bu, "bu çalıştırma neden bu kadar çok maliyetli oldu?" veya "hangi araç yavaş olan?" sorusunun cevabıdır; grafiğin hemen yanında yer alır. -Bireysel olaylar adreslenebilir; böylece birine "oturumun, yaklaşık üçte ikisi kadar aşağı" yerine bir anın bağlantısını verebilirsiniz. Herhangi bir olaydan bağlantıyı kopyalayın veya bir [denetim](/tr/agenteye/audits) bulgusu veya hatadan birini takip edin; oturumlar açılır ve o olay seçilir ve konumlandırılır. Bu çok uzun çalıştırmalar için de geçerlidir: zaman çizelgesi tarayıcınız uğruna sınırlandırılmış bir pencere yükler ve bu pencereyi aşan bir bağlantı yine de olayını bulur ve sizi başlangıca bırakmaz. Olay saklama pencerenizden yaşlanmışsa, sayfa sessizce hiçbir şey seçmek yerine bunu size söyler. +Bireysel olaylar adreslenebilir; böylece birine "oturumun, yaklaşık üçte ikisi kadar aşağı" yerine bir anın bağlantısını verebilirsiniz. Herhangi bir olaydan bağlantıyı kopyalayın veya bir [denetim](/tr/cloud/audits) bulgusu veya hatadan birini takip edin; oturumlar açılır ve o olay seçilir ve konumlandırılır. Bu çok uzun çalıştırmalar için de geçerlidir: zaman çizelgesi tarayıcınız uğruna sınırlandırılmış bir pencere yükler ve bu pencereyi aşan bir bağlantı yine de olayını bulur ve sizi başlangıca bırakmaz. Olay saklama pencerenizden yaşlanmışsa, sayfa sessizce hiçbir şey seçmek yerine bunu size söyler. --- @@ -46,12 +46,12 @@ Bireysel olaylar adreslenebilir; böylece birine "oturumun, yaklaşık üçte ik Her pano sayfası org kapsamındadır (`//…`). Oturumlar, sol yan çubukta **Gözlemle** altında, Olayların yanında yer alır; listenin en üstünde tarih aralığı, ortam, aracı ve oturumsal filtreler bulunur. Her satır, tam yürütme grafiğinden bir tıklama uzaktadır. -Puan rozetlerini ve puan aralığı filtrelemesini açmak için bir değerlendirici bağlayın: bkz. [Değerlendirmeler](/tr/agenteye/evaluations). +Puan rozetlerini ve puan aralığı filtrelemesini açmak için bir değerlendirici bağlayın: bkz. [Değerlendirmeler](/tr/cloud/evaluations). --- ## İlgili -- [Olay akışı](/tr/agenteye/event-stream): her oturumun toplanmış olduğu ham, adım başına izleme. -- [Değerlendirmeler](/tr/agenteye/evaluations): her çalıştırmanın filtre yapabileceğiniz bir puan rozeti alması için bir değerlendirici bağlayın. -- [Telemetri](/tr/agenteye/telemetry): çalıştırmalar aracınızdan bu oturumlara nasıl ulaşır? \ No newline at end of file +- [Olay akışı](/tr/cloud/event-stream): her oturumun toplanmış olduğu ham, adım başına izleme. +- [Değerlendirmeler](/tr/cloud/evaluations): her çalıştırmanın filtre yapabileceğiniz bir puan rozeti alması için bir değerlendirici bağlayın. +- [Telemetri](/tr/cloud/performance): çalıştırmalar aracınızdan bu oturumlara nasıl ulaşır? \ No newline at end of file diff --git a/docs/tr/concepts.mdx b/docs/tr/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/tr/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/tr/daemon.mdx b/docs/tr/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/tr/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/tr/dashboard.mdx b/docs/tr/dashboard.mdx index 02efd1b1..35badf21 100644 --- a/docs/tr/dashboard.mdx +++ b/docs/tr/dashboard.mdx @@ -70,7 +70,7 @@ Ajanınızın geçmiş oturumlar arasında gerçekten nasıl davrandığının k 4. **Nasıl İyileştirilir** — sakin satır listesi, önerilen her politika başına birer tane: politika adı beyazda, tek satır açıklama, sağ tarafta yükleme komutu + kopyala düğmesi. Bölüm başlığı `enable all N → projected · ` okur (her düzeltme uygulandığında ulaşacağınız puan) ve onun `[install all]` düğmesi, her önerilen politika için birleşik `failproofai policy add a b c …` komutunu kopyalar. 5. **Daha İyi Geri Dön** — yan yana iki kart. Sol: hatırlatıcı ayarla (`3d` / `7d` / `14d` / `30d` temposu seçici; kimlik doğrulandıktan sonra `/api/auth/reminder` aracılığıyla kalıcı). Sağ: failproof avantajlarının kilidini açın — `invite a friend` virgül/boşluk/yeni satır ile ayrılmış bir arkadaş e-postası listesini alan bir modal açar (gönderim başına maksimum 10), onları `/api/audit/invite` konumuna POST eder, bu da api-sunucusunun `POST /v0/invite` konumuna iletilir. Api-sunucusu, gönderici Cc'ye sahip ve `Reply-To` ayarlanmış şekilde `invite@failproof.ai` konumundan her alıcı için bir e-posta gönderir, böylece alıcı onları kimin davet ettiğini görür ve gönderici gelen kutularında bir kopya alır. Anonim kullanıcılar, davetler gönderilmeden önce gönderenin e-postası bilinir diye `AuthDialog` aracılığıyla yönlendirilir. Hak / avantaj yerine getirme sonraki bir aşamadır. -`failproofai audit` çalışma zamanı tarafından yönlendirilir — temel tarama motoru, desteklenen bayraklar ve per-transkript önbellek değişmezleri için [Denetim CLI](/tr/cli/audit) konusuna bakın. Pano, en son sonucu `~/.failproofai/audit-dashboard.json` konumunda önbelleğe alır (mod `0600`, tek yuva, yeni çalıştırmalar üzerine yazar) böylece yeniden ziyaretler anlıktır; **hem per-transkript hem de tam sonuç önbellekleri okunduklarında 7 günü aştıklarında reddedilir** böylece pano hiçbir zaman bir haftaya kadar eski bir sonucu sessizce sunmaz — TTL geçtikten sonra `/audit` boş durumuna düşer ve yeni bir çalıştırma isteminden geçer. Raporun alt kısmında `[ re-audit now ]` düğmesine tıklamak `/api/audit/run` konumuna `noCache: true` ile POST gönderir — yeniden denetim per-transkript önbelleğini atlar ve her transkripti sessizce önbelleğe alınan sonucu döndürmek yerine sıfırdan yeniden tarar — ve pano çalıştırma bitene kadar `/api/audit/status` konumunu 1Hz'de yoklar; yapışkan pembe ilerleme şeridi çalışma sırasında viewport'un üstüne tutturulur ve geçen bir zamanlayıcı ile başarıda yeni sonuç yerine değiştirilir (tam sayfa yeniden yüklemesi yok; başarısız yeniden denetim önceki raporu sağlam bırakır). Başarısızlıkta şerit, `RerunError.kind` ('timeout' / 'network' / 'post_failed') konusunda anahtar kopyayla kırmızıya döner. Boş durum (önbellek yok veya süresi dolmuş) ve sıfır oturum durumu (önbellek var ancak tarama transkript bulamadı) ayrı olarak ortaya çıkarılır. +`failproofai audit` çalışma zamanı tarafından yönlendirilir — temel tarama motoru, desteklenen bayraklar ve per-transkript önbellek değişmezleri için [Denetim CLI](/tr/audit) konusuna bakın. Pano, en son sonucu `~/.failproofai/audit-dashboard.json` konumunda önbelleğe alır (mod `0600`, tek yuva, yeni çalıştırmalar üzerine yazar) böylece yeniden ziyaretler anlıktır; **hem per-transkript hem de tam sonuç önbellekleri okunduklarında 7 günü aştıklarında reddedilir** böylece pano hiçbir zaman bir haftaya kadar eski bir sonucu sessizce sunmaz — TTL geçtikten sonra `/audit` boş durumuna düşer ve yeni bir çalıştırma isteminden geçer. Raporun alt kısmında `[ re-audit now ]` düğmesine tıklamak `/api/audit/run` konumuna `noCache: true` ile POST gönderir — yeniden denetim per-transkript önbelleğini atlar ve her transkripti sessizce önbelleğe alınan sonucu döndürmek yerine sıfırdan yeniden tarar — ve pano çalıştırma bitene kadar `/api/audit/status` konumunu 1Hz'de yoklar; yapışkan pembe ilerleme şeridi çalışma sırasında viewport'un üstüne tutturulur ve geçen bir zamanlayıcı ile başarıda yeni sonuç yerine değiştirilir (tam sayfa yeniden yüklemesi yok; başarısız yeniden denetim önceki raporu sağlam bırakır). Başarısızlıkta şerit, `RerunError.kind` ('timeout' / 'network' / 'post_failed') konusunda anahtar kopyayla kırmızıya döner. Boş durum (önbellek yok veya süresi dolmuş) ve sıfır oturum durumu (önbellek var ancak tarama transkript bulamadı) ayrı olarak ortaya çıkarılır. ### Politikalar diff --git a/docs/tr/architecture.mdx b/docs/tr/how-it-works.mdx similarity index 100% rename from docs/tr/architecture.mdx rename to docs/tr/how-it-works.mdx diff --git a/docs/tr/introduction.mdx b/docs/tr/introduction.mdx index 5cec8eeb..0add3105 100644 --- a/docs/tr/introduction.mdx +++ b/docs/tr/introduction.mdx @@ -54,4 +54,4 @@ failproofai policies --install # politikaları etkinleştir (veya atla — `fa failproofai # panoyu başlat ``` -Tam yer gösterimler için [Başlangıç](/tr/getting-started) kılavuzuna bakın. \ No newline at end of file +Tam yer gösterimler için [Başlangıç](/tr/quickstart) kılavuzuna bakın. \ No newline at end of file diff --git a/docs/tr/policies.mdx b/docs/tr/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/tr/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/tr/getting-started.mdx b/docs/tr/quickstart.mdx similarity index 100% rename from docs/tr/getting-started.mdx rename to docs/tr/quickstart.mdx diff --git a/docs/tr/reference/files.mdx b/docs/tr/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/tr/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/vi/agent-support.mdx b/docs/vi/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/vi/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/vi/agenteye/cli-and-agents.mdx b/docs/vi/agenteye/cli-and-agents.mdx deleted file mode 100644 index c53e5ea6..00000000 --- a/docs/vi/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: "CLI" -description: "Toàn bộ triển khai Failproof AI Observability của bạn, chỉ cách một lệnh." ---- - - -Toàn bộ triển khai Failproof AI Observability của bạn, chỉ cách một lệnh. Kiểm tra production, tạo khóa API, hoặc xác nhận sự cố mà không cần rời khỏi terminal, sau đó đưa bất kỳ lệnh nào vào CI, hoặc để một coding agent thực hiện cho bạn bằng ngôn ngữ tự nhiên. - -```bash -pipx install agenteye -agenteye login --email you@example.com # a 6-digit code lands in your inbox -agenteye --json sessions --since 24h # every agent run from the last day, newest first -``` - -*CLI `agenteye` giao tiếp với dashboard của bạn. Đây là một công cụ khác biệt với collector, công cụ này gửi sự kiện đến máy chủ.* - -## Toàn bộ triển khai của bạn, chỉ cách một lệnh - -Dừng việc chuyển tab để trả lời một câu hỏi nhanh. CLI `agenteye` đọc dữ liệu của bạn và quản trị tổ chức của bạn từ một tệp nhị phân duy nhất, vì vậy một kiểm tra mà trước đây có nghĩa là nhấp vào dashboard trở thành một dòng mà bạn có thể chạy lại, tạo bí danh hoặc dán vào runbook. Bạn có bốn giao diện: - -- **Đọc dữ liệu của bạn:** `sessions`, `events`, `evals`, và `errors`, được lọc theo thời gian, agent và môi trường. -- **Quản lý tổ chức của bạn:** `keys`, `users`, `settings`, `alerts`, và `incidents`. -- **Chạy phân tích:** SQL được lưu cùng với trình chạy `query` ad-hoc trên dữ liệu sự kiện của bạn. -- **Hỏi trợ lý:** `agent ask` tiếp cận cùng một nhà phân tích chỉ đọc mà bạn trò chuyện với trong dashboard. - -Cài đặt một lần bằng `pipx`, đăng nhập bằng mã 6 chữ số được gửi qua email, và bạn đã sẵn sàng. Phiên kéo dài khoảng một ngày; chạy lại `agenteye login` khi nó hết hạn. Dùng nó để kiểm tra production, cấp phát khóa, hoặc phân loại sự cố kích hoạt, tất cả mà không cần mở trình duyệt: - -```bash -agenteye errors --since 24h --aggregate # what is breaking, grouped by error type -agenteye incidents list --state firing # what is on fire right now -agenteye keys create ci --add events:add # a key that can only push events, secret shown once -``` - -Một thói quen cần biết: các tùy chọn toàn cục như `--json` đi trước lệnh. `agenteye --json sessions` là đúng; `agenteye sessions --json` là sai. - -## Viết script nó, tích hợp vào CI - -Mọi lệnh đều nhận `--json`, và điều đó thay đổi mọi thứ. JSON sạch đi đến stdout trong khi trạng thái và cảnh báo của con người đi đến stderr, vì vậy việc capture `--json` đi thẳng vào `jq` mà không có dòng lạc để xóa. Đó là những gì làm cho CLI tốt như nhau cho bạn khi nhắc lệnh và cho một coding agent phân tích đầu ra: - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -Nó được xây dựng để chạy không được giám sát. Các lời nhắc xác nhận tự động bỏ qua khi không có terminal được đính kèm, vì vậy không có gì bị treo trong đường ống, và mọi lệnh đều trả về mã thoát có ý nghĩa: `0` thành công, `4` chưa đăng nhập, `5` thiếu quyền (thông báo đặt tên nó, ví dụ `alerts:write`), `3` dashboard không thể tiếp cận. Một script có thể phân nhánh trên `4` để xác thực lại hoặc `5` để cho bạn biết chính xác những gì để yêu cầu admin, thay vì thất bại mù quáng. - -## Để một coding agent điều khiển nó bằng ngôn ngữ tự nhiên - -Tốt hơn, bạn không nên phải nhớ bất kỳ cờ nào trong số này. **CLI skill** là một thư mục Agent Skill nhỏ có tên `agenteye-cli` dạy một coding agent chẳng hạn như Claude Code hoặc Codex cách điều khiển CLI từ các yêu cầu bằng ngôn ngữ tự nhiên. Hỏi "có bất cứ điều gì bị hỏng hôm nay không?" và agent chọn lệnh, chạy nó dưới danh nghĩa bạn, và trả lời bằng văn bản. - -Đối với Claude Code, thả thư mục `agenteye-cli` vào `~/.claude/skills/` và nó sẽ được tự động phát hiện. Failproof AI Observability cung cấp thư mục; không có gì thêm để cài đặt, vì nó chỉ điều khiển CLI mà bạn đã cài đặt. Đăng nhập chính mình trước: skill không thể hoàn tất đăng nhập mã email cho bạn. - -Vì agent chạy CLI dưới danh nghĩa bạn, nó có thể làm mọi thứ mà đăng nhập của bạn cho phép, cả đọc và ghi: tạo khóa, thay đổi cài đặt, giải quyết sự cố. Lời nhắc "bạn có chắc chắn không?" của CLI không kích hoạt đối với agent, vì vậy skill được viết để nêu lệnh chính xác và chờ OK của bạn trước bất kỳ thay đổi nào. Bạn là bước xác nhận. - -```text -you Why did session run-001 fail? - -agent Running: agenteye --json events --session-id run-001 --all - The checkout-agent hit a TimeoutError on its third tool call. -``` - -Các lần đọc vẫn nhanh tức thời, và mỗi lần ghi tạm dừng cho bạn: - -```text -you Give CI a key that can only push events. - -agent This creates an API key. I'll run: - agenteye keys create ci --add events:add - OK to proceed? - -you yes - -agent Done. Key "ci" created with events:add only. The secret is shown once, so store it now. -``` - -## Liên quan - -- [Tham khảo CLI](/vi/agenteye/cli): mọi lệnh, cờ và hình dạng JSON. -- [Công thức CLI cho agent](/vi/agenteye/cli-recipes): mẫu `jq` sao chép dán và xử lý mã thoát. -- [CLI agent skill](/vi/agenteye/cli-skill): cài đặt và chạy skill `agenteye-cli`. -- [Trợ lý AI](/vi/agenteye/assistant): nhà phân tích trong dashboard mà `agent ask` giao tiếp với. \ No newline at end of file diff --git a/docs/vi/agenteye/cli-skill.mdx b/docs/vi/agenteye/cli-skill.mdx deleted file mode 100644 index f7e86787..00000000 --- a/docs/vi/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Failproof AI Observability CLI Agent Skill" -description: "Hỏi agent coding của bạn \"có cái gì bị hỏng hôm nay không?\" và để nó trả lời từ dữ liệu Failproof AI Observability trực tiếp, không cần nhớ bất kỳ lệnh nào." ---- - - -Hỏi agent coding của bạn *"có cái gì bị hỏng hôm nay không?"* và để nó trả lời từ dữ liệu Failproof AI Observability trực tiếp, không cần nhớ bất kỳ lệnh nào. **Failproof AI Observability CLI skill** (`agenteye-cli`) là một *Agent Skill*: một thư mục nhỏ chứa hướng dẫn mà một agent coding như Claude Code hoặc Codex có thể tải theo yêu cầu. Nó dạy agent cách vận hành deployment Observability của bạn thông qua [`agenteye` CLI](/vi/agenteye/cli) từ những yêu cầu bằng tiếng Anh thông thường như *"cấp cho CI một key chỉ có thể push events"* hoặc *"ack sự cố đang phát sinh và gán cho tôi."* - -Nó **không phải** một dịch vụ hoặc một binary riêng; không có gì để deploy. Nó chạy trên CLI bạn đã cài đặt: agent shell out tới `agenteye --json …`, phân tích JSON sạch, và trả lời bạn bằng văn bản. Mọi thứ nó có thể làm, bạn cũng có thể làm bằng cách gõ các lệnh tương tự. - ---- - -## Nó liên quan như thế nào đến các giao diện Failproof AI Observability khác - -Failproof AI Observability cung cấp bốn cách để truy cập dữ liệu và điều khiển giống nhau. Chúng bổ sung cho nhau: - -| Giao diện | Nó là gì | Chạy ở đâu | Dùng khi | -|---|---|---|---| -| **[CLI](/vi/agenteye/cli)** | Tham chiếu lệnh/flag cho `agenteye` | Terminal của bạn | Bạn muốn chạy hoặc viết script một lệnh cụ thể | -| **[CLI recipes](/vi/agenteye/cli-recipes)** | Các mẫu `jq`/pipeline sao chép được | Terminal/scripts của bạn | Bạn đang kết nối CLI vào tự động hóa | -| **CLI skill** (tài liệu này) | Cửa vào ngôn ngữ tự nhiên trên CLI | Agent coding của bạn, trên workstation | Bạn muốn *chỉ cần hỏi* và để agent chọn lệnh | -| **[Evaluator skill](/vi/agenteye/evaluator-skill)** | Một skill anh em thiết kế và xây dựng dịch vụ scoring của bạn | Agent coding của bạn, trên workstation | Bạn muốn *tạo* eval scores thay vì đọc chúng | -| **[Python SDK skill](/vi/agenteye/python-sdk-skill)** | Một skill anh em instrument agent của bạn để nó phát ra telemetry | Agent coding của bạn, trên workstation | Bạn muốn agent của bạn *tạo* các sự kiện mà skill này đọc | -| **[In-dashboard AI assistant](/vi/agenteye/assistant)** | Một chat nhúng trong dashboard | Phía server (trong dashboard) | Bạn muốn Q&A trong dashboard trên dữ liệu của bạn | - -Bản thân skill không có đặc quyền riêng; nó chỉ biến lời nói của bạn thành các lệnh CLI chạy với tư cách của bạn: - -```mermaid -flowchart TD - YOU["bạn: 'ack sự cố đang phát sinh'"] --> AGENT["agent coding (Claude Code / Codex)
    tải agenteye-cli skill"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|phiên CLI được xác thực của bạn| API["API dashboard Observability"] -``` - -### so với in-dashboard AI assistant: một phân biệt quan trọng - -Đây là hai công cụ khác nhau với phạm vi ảnh hưởng rất khác nhau: - -- **In-dashboard AI assistant** ([AI assistant](/vi/agenteye/assistant)) là một chat nhúng trong dashboard, được hỗ trợ bởi dịch vụ agent. Nó là **chỉ đọc cộng tác giả gated phê duyệt**: nó có thể soạn thảo các truy vấn đã lưu và dashboard, nhưng mọi ghi tạm dừng để chờ click phê duyệt rõ ràng của bạn, và nó không bao giờ xóa. Nó được gated bởi quyền `agent:use` và chỉ bao giờ nhìn thấy dữ liệu cho org bạn đang xem. -- **CLI skill** chạy trên *workstation* của bạn bên trong *agent* coding của bạn và điều khiển `agenteye` CLI với tư cách **bạn**. Nó có thể thực hiện **toàn bộ bề mặt của CLI, bao gồm cả mutations** (tạo/xoay/vô hiệu hóa API keys, thay đổi cài đặt org, giải quyết sự cố, xóa truy vấn đã lưu), được giới hạn chỉ bởi quyền của CLI login của bạn. Hãy coi nó chính xác như cách bạn sẽ chạy các lệnh đó bằng tay. - ---- - -## Điều kiện tiên quyết - -1. **`agenteye` CLI được cài đặt** và trên `PATH` (xem tham chiếu [CLI](/vi/agenteye/cli): `pipx install agenteye`). -2. **URL dashboard của bạn được đặt** (`AGENTEYE_DASHBOARD_URL`, hoặc agent truyền `--base-url`). -3. **Một phiên đã đăng nhập**: chạy `agenteye login` trước. Skill **không thể** hoàn thành login mã một lần được gửi qua email cho bạn; nó sẽ yêu cầu bạn chạy `agenteye login` nếu phiên bị mất hoặc hết hạn (mã thoát CLI `4`). - ---- - -## Nơi để lấy nó - -Skill được xuất bản trong bộ sưu tập skills công cộng của Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -Không có gì về nó bị gated — kho lưu trữ là công cộng và skill không cần bất kỳ thông tin xác thực nào của riêng nó, vì nó chỉ điều khiển **công cộng** `agenteye` CLI chống lại *dashboard* của bạn, sử dụng phiên *bạn* đã đăng nhập. Bạn không cần phải yêu cầu ai để lấy nó. - -Lưu ý nó được cung cấp dưới dạng thư mục riêng của nó và **không** nằm trong gói `pipx install agenteye`, vì vậy đừng tìm kiếm nó ở đó. - -## Cài đặt skill - -Con đường nhanh nhất là CLI [`skills`](https://skills.sh), nó tìm nạp thư mục và đặt nó vào nơi agent của bạn tìm kiếm: - -```bash -# Claude Code, chỉ project này -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# mọi project (cài đặt vào ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# Codex thay thế -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -Sau đó quản lý nó như bất kỳ skill nào khác: - -```bash -npx skills list -a claude-code # cái gì được cài đặt -npx skills update agenteye-cli # kéo phiên bản mới nhất -npx skills remove agenteye-cli # loại bỏ nó -``` - -Thích cài đặt bằng tay? Một Agent Skill chỉ là một thư mục chứa `SKILL.md` (cộng với tham chiếu tùy chọn), vì vậy sao chép nó cũng hoạt động: - -- **Claude Code**: đặt thư mục `agenteye-cli/` trong `~/.claude/skills/` (mọi project) hoặc `/.claude/skills/` (chỉ repo đó). Claude Code tự động khám phá nó — xác minh bằng danh sách `/skills`, hoặc chỉ cần hỏi một câu hỏi phù hợp với mô tả của nó. -- **Codex (OpenAI)**: Codex đọc `SKILL.md` giống nhau. `agents/openai.yaml` đi kèm đặt `allow_implicit_invocation: true`, vì vậy Codex tự động chọn skill khi một tác vụ phù hợp; nếu không thì gọi nó rõ ràng là `$agenteye-cli`. - ---- - -## Bảo mật: mutations KHÔNG nhắc khi agent chạy CLI - -> **Cảnh báo:** Đọc điều này trước khi để agent thực hiện các thay đổi. - -CLI `agenteye` thường hỏi *"bạn có chắc không?"* trước một hành động phá hoại. Nó **tự động bỏ qua xác nhận đó bất cứ khi nào nó không được gắn vào terminal (đó chính xác là cách một agent coding chạy nó), và `--json` cũng bỏ qua nó.** Vì vậy dấu nhắc bảo mật sẽ **không** kích hoạt cho agent. - -Skill được viết để bù đắp: nó được hướng dẫn để phát biểu lệnh chính xác mà nó sẽ chạy và nhận được **OK rõ ràng của bạn trước bất kỳ thay đổi trạng thái**. Giữ kỷ luật đó. Khi bạn điều khiển Failproof AI Observability thông qua một agent, *bạn* là bước xác nhận. Các lệnh thay đổi trạng thái để xem: - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- các lệnh con `incidents` ghi: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -Mọi thứ dưới **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) là chỉ đọc và không thay đổi gì. - -Vì agent hoạt động với tư cách **bạn**, nó chỉ có thể làm những gì login của bạn được phép làm; quyền được phân giải **mỗi org** (xem [API keys](/vi/agenteye/api-keys)). Một lệnh bạn thiếu quyền cho nó trả về mã thoát `5` với quyền chính xác được đặt tên, vì vậy agent có thể cho bạn biết chính xác cần yêu cầu admin cái gì thay vì thất bại không rõ ràng. - ---- - -## Bạn có thể hỏi nó gì - -Đây là những gì một chuyến đi thực tế trông như thế nào, một lần đọc, sau đó là một thay đổi chờ OK của bạn: - -```text -bạn ▸ Có cái gì bị hỏng trong 24h qua không? - -agent ▸ Chạy: agenteye --json errors --since 24h --aggregate - 3 loại lỗi trong 24h qua. Cái lớn nhất là TimeoutError - (18 lần, đều trên checkout-agent), sau đó là RateLimited (5). - Bạn muốn các phiên bị lỗi cho TimeoutError không? - -bạn ▸ Cấp cho CI một key chỉ có thể push events. - -agent ▸ Điều này tạo một API key. Tôi sẽ chạy: - agenteye keys create ci --add events:add - OK để tiếp tục? - -bạn ▸ yes - -agent ▸ Xong. Key "ci" được tạo với chỉ events:add. - Secret chỉ được hiển thị một lần, vì vậy lưu trữ nó ngay bây giờ. Tôi không thể in lại nó. -``` - -Skill ánh xạ mỗi ý định ngôn ngữ tự nhiên thành lệnh `agenteye` đúng, khám phá các giá trị hợp lệ trước (`list `, `whoami`) để nó không đoán, và phát biểu lệnh chính xác trước bất kỳ thay đổi. Ví dụ thêm: - -- *"Có cái gì bị hỏng / fail trong 24h qua không?"* → `errors --since 24h --aggregate`, sau đó là một phân tích. -- *"Tại sao phiên `run-001` lại fail?"* → `events --session-id run-001 --all` + `evals --session-id run-001`. -- *"Chất lượng đang xu hướng như thế nào tuần này?"* → `evals --aggregate --since 7d`, sau đó đi sâu vào các chạy có điểm thấp. -- *"Cấp cho CI một key chỉ có thể push events."* → `keys create ci --add events:add` (nó phát biểu lệnh, sau đó tạo nó và bắt secret một lần). -- *"Ai có quyền truy cập? Làm Dana chỉ đọc."* → `users list` → `users update dana@… --permission-set read-only` (sau khi xác nhận với bạn). -- *"Ack sự cố đang phát sinh và gán cho tôi."* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`. - -Để xem các lệnh chính xác, flag, và hình dạng JSON đằng sau những thứ này, xem tham chiếu [CLI](/vi/agenteye/cli) và [CLI recipes cho agents](/vi/agenteye/cli-recipes). - ---- - -## Bước tiếp theo - -- **[CLI](/vi/agenteye/cli)**: tham chiếu lệnh và flag đầy đủ cho `agenteye`. -- **[CLI recipes cho agents](/vi/agenteye/cli-recipes)**: các mẫu `jq` sao chép được và xử lý mã thoát. -- **[Evaluator agent skill](/vi/agenteye/evaluator-skill)**: skill anh em, để xây dựng evaluator mà `agenteye evals` đọc. -- **[Python SDK agent skill](/vi/agenteye/python-sdk-skill)**: skill anh em, để instrument agent để nó phát ra telemetry mà `agenteye` đọc. -- **[AI assistant](/vi/agenteye/assistant)**: assistant trong dashboard (không nên nhầm lẫn với skill terminal này). -- **[API keys](/vi/agenteye/api-keys)**: mô hình quyền mỗi org bounding cái gì skill có thể làm. \ No newline at end of file diff --git a/docs/vi/agenteye/codex-capture.mdx b/docs/vi/agenteye/codex-capture.mdx deleted file mode 100644 index ba7f1f4d..00000000 --- a/docs/vi/agenteye/codex-capture.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "Ghi lại phiên Codex" -description: "Theo dõi các phiên Codex OpenAI cục bộ của đội ngũ bạn vào AgentEye dưới dạng các phiên và sự kiện thông thường — mà không cần thay đổi cách họ chạy Codex." ---- - -Các kỹ sư của bạn đã chạy OpenAI Codex mỗi ngày. Ghi lại phiên Codex đưa những phiên coding đó vào AgentEye dưới dạng các phiên và sự kiện thông thường, để bạn có thể tìm kiếm, phát lại và đánh giá chúng cùng với tất cả những gì khác bạn quan sát. Nó bổ sung cho [Python SDK](/vi/agenteye/python-sdk): SDK này cấy cứu các agent bạn viết, trong khi đây ghi lại công việc Codex mà đội ngũ bạn đã làm — mà không cần thay đổi cách họ chạy nó. - -Một bộ sưu tập nền nhỏ đọc các bản ghi phiên Codex cục bộ khi chúng được viết và gửi chúng đến AgentEye. Một bộ sưu tập trên mỗi máy ghi lại mọi bề mặt Codex cục bộ cùng một lúc — không cần thiết lập cho từng bề mặt. - -Bộ sưu tập tương tự cũng ghi lại các agent khác — xem [OpenClaw](/vi/agenteye/openclaw-capture) và [Hermes](/vi/agenteye/hermes-capture). Kích hoạt từng cái bạn chạy; một bộ sưu tập có thể ghi lại nhiều cái cùng một lúc. - ---- - -## Nó ghi lại những gì - -Mọi bề mặt Codex chạy **cục bộ** đều tạo ra các bản ghi phiên trên đĩa giống nhau, và bộ sưu tập chọn tất cả chúng: - -- **CLI** Codex và `codex exec` -- phần **mở rộng VS Code / IDE** -- **ứng dụng desktop**, khi nó chạy một phiên cục bộ - -Mỗi phiên Codex trở thành một [phiên](/vi/agenteye/sessions) AgentEye; các tin nhắn của người dùng và trợ lý, lý luận, lệnh gọi công cụ, kết quả công cụ và mức sử dụng token của nó trở thành các [sự kiện](/vi/agenteye/event-stream) phù hợp. Bề mặt mà mỗi phiên đến từ (CLI, IDE hoặc desktop) được ghi lại, để bạn có thể phân biệt chúng. - -> **Các phiên trên đám mây không được ghi lại.** Ứng dụng desktop ngày càng chạy các phiên trong đám mây Codex và chỉ giữ lại siêu dữ liệu của chúng trên máy — không có bản ghi cục bộ nào để đọc. Chỉ các phiên thực thi cục bộ mới được ghi lại. - ---- - -## Bật nó lên - -Ghi lại bị tắt cho đến khi bạn bật nó. Cài đặt bộ sưu tập với một khóa API có quyền `events:add` (xem [API keys](/vi/agenteye/api-keys)) và bật ghi lại Codex: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -Điều đó cài đặt bộ sưu tập, đăng ký nó như một dịch vụ nền và bắt đầu ghi lại. Xác nhận rằng nó đang chạy: - -```bash -agenteye-collector health -``` - -Lần chạy đầu tiên, các phiên Codex hiện có của bạn sẽ được điền lại một lần và hoạt động mới sau đó sẽ truyền phát trong vài giây. Các tệp của chính Codex chỉ được đọc — không bao giờ được sửa đổi, di chuyển hoặc xóa — và mỗi phiên được gửi chính xác một lần, ngay cả trong các lần khởi động lại. - ---- - -## Nó xuất hiện ở đâu - -Các phiên được ghi lại xuất hiện trong **Sessions** và các sự kiện của chúng trong luồng **Events**, giống như bất kỳ agent nào khác bạn quan sát — vì vậy [phát lại phiên](/vi/agenteye/sessions), [tìm kiếm](/vi/agenteye/queries), [đánh giá](/vi/agenteye/evaluations) và [cảnh báo](/vi/agenteye/alerts) đều hoạt động trên chúng. Lọc theo agent Codex để xem chúng riêng biệt. - ---- - -## Quyền riêng tư - -Các bản ghi phiên Codex chứa toàn bộ phiên — bao gồm đầu ra lệnh, nội dung tệp và bất cứ điều gì Codex đọc hoặc viết — và có thể chứa các bí mật. Các phiên được ghi lại được gửi nguyên trạng, vì vậy chỉ bật ghi lại trên các máy và cho các đội nơi tập trung nội dung đó trong AgentEye là thích hợp, và cung cấp cho bộ sưu tập một khóa được giới hạn trong `events:add` chỉ. Xem [Security](/vi/agenteye/security) để biết cách dữ liệu của bạn được giữ cách ly. \ No newline at end of file diff --git a/docs/vi/agenteye/concepts.mdx b/docs/vi/agenteye/concepts.mdx deleted file mode 100644 index c7d6440f..00000000 --- a/docs/vi/agenteye/concepts.mdx +++ /dev/null @@ -1,88 +0,0 @@ ---- ---- -title: "Khái niệm" -description: "Từ vựng đằng sau Failproof AI Observability — sự kiện, phiên làm việc, đánh giá, kiểm toán, phát hiện và sự cố — được định nghĩa tại một nơi." ---- - - -Trang này định nghĩa từ vựng mà Failproof AI Observability sử dụng. Nếu một thuật ngữ trong hướng dẫn khác không quen thuộc, nó được định nghĩa ở đây. Bạn không cần phải đọc nó từ đầu đến cuối: hãy lướt qua, hoặc quay lại khi bạn muốn làm rõ một từ. - ---- - -## Mô hình dữ liệu - -**Event (Sự kiện)** -Đơn vị dữ liệu nhỏ nhất. Một sự kiện ghi lại một bước duy nhất mà agent của bạn thực hiện: một `tool_use`, một `model_request`, một `hook_completed`, một `error`, v.v. Agent của bạn phát ra các sự kiện thông qua [Python SDK](/vi/agenteye/python-sdk); chúng xuất hiện trực tiếp trên trang **Events**. - -**Session (Phiên làm việc)** -Một lần chạy agent, được xác định bằng `session_id`. Một phiên là tất cả các sự kiện chia sẻ id đó, được tổng hợp thành một hàng trên trang **Sessions** và được vẽ dưới dạng biểu đồ thực thi trên trang chi tiết của nó. Một phiên thường bắt đầu bằng `agent_start` và kết thúc bằng `agent_end`. - -**Agent** -Một diễn viên được đặt tên bên trong một lần chạy, được xác định bằng `agent_id`. Một lần chạy có thể liên quan đến nhiều agent: ví dụ, một bộ lập kế hoạch sinh ra một sub-agent tóm tắt. Các sub-agent mang theo `parent_id`, đây là cách cho phép Failproof AI Observability vẽ chúng trên các làn riêng của chúng trong biểu đồ thực thi. - -**Environment (Môi trường)** -Một nhãn cho nơi lần chạy xảy ra: `production`, `staging`, `dev`. Bạn đặt nó một lần khi cấu hình SDK. Hầu hết mọi trang bảng điều khiển đều có thể lọc theo môi trường. - -**Context-window fill (Mức độ lấp đầy cửa sổ ngữ cảnh)** -Phần trăm cửa sổ ngữ cảnh của model mà một phản hồi tiêu thụ. Failproof AI Observability dấu nó trên các sự kiện `model_response` cho các model mà nó nhận dạng, để quá trình tăng trưởng prompt và việc nén sắp xảy ra là rõ ràng ngay trong luồng sự kiện. - ---- - -## Chất lượng - -**Evaluation (Đánh giá)** -Điểm chất lượng cho một phiên hoàn thành, được tạo bởi dịch vụ chấm điểm bạn chạy. Đánh giá là tùy chọn: cho đến khi bạn kết nối một bộ đánh giá, các phiên được ghi lại nhưng không được chấm điểm. Mỗi đánh giá có thể mang theo nhiều điểm có tên (ví dụ `helpfulness`, `factuality`, `tool_efficiency`), mỗi điểm có ghi chú lý do ngắn. Xem [Evaluation suite](/vi/agenteye/evaluation-suite). - -**Score key (Khóa điểm)** -Tên của một chiều mà bộ đánh giá báo cáo, chẳng hạn như `helpfulness`. Cảnh báo và kiểm toán có thể theo dõi một khóa điểm cụ thể theo thời gian. - -**Evaluator (Bộ đánh giá)** -Dịch vụ chấm điểm của bạn. Failproof AI Observability POST phần giới thiệu của một lần chạy hoàn thành cho nó và lưu trữ các điểm nó trả về. Nó không cung cấp bộ đánh giá mặc định; logic chấm điểm là của bạn. - ---- - -## Tìm kiếm và sửa chữa các lỗi - -**Hook (Móc)** -Một guardrail hoặc side-effect mà framework agent của bạn chạy xung quanh một bước: một kiểm tra an toàn nội dung, PII redaction, một bảo vệ ngân sách. Hooks phát ra các sự kiện `hook_triggered` / `hook_completed` với một `outcome` (allow, deny, modify), và có trang observe riêng của chúng. - -**Alert rule (Quy tắc cảnh báo)** -Một quy tắc kích hoạt khi một số liệu vượt qua ngưỡng bạn đặt: tỷ lệ lỗi, độ trễ p95, chi phí token, hoặc điểm bộ đánh giá. Khi một quy tắc kích hoạt, nó mở một sự cố và thông báo cho các kênh bạn chọn (email, Slack, webhook, trong bảng điều khiển). Xem [Alerts](/vi/agenteye/alerts). - -**Incident (Sự cố)** -Một vấn đề mở được tạo khi một quy tắc cảnh báo kích hoạt. Các sự cố có vòng đời (acknowledge, assign, resolve) và dòng thời gian hoạt động ghi lại mọi hành động. Bạn cũng có thể mở nó thủ công. - -**Audit (Kiểm toán)** -Một cuộc điều tra định kỳ (hàng giờ đến hàng tuần) khai thác nhật ký của bạn *trên* các phiên để tìm các mẫu lỗi bạn chưa viết quy tắc: các cụm lỗi, điểm thấp, các ngoại lệ độ trễ, vòng lặp tool-call, và các lần chạy không bao giờ kết thúc. Trong khi cảnh báo theo dõi một số liệu bạn đã biết, kiểm toán cho bạn biết tiếp theo nên nhìn vào đâu. Xem [Audits](/vi/agenteye/audits). - -**Finding (Phát hiện)** -Một kết quả được xếp hạng, được hỗ trợ bằng bằng chứng từ một lần chạy kiểm toán. Một phát hiện đặt tên cho một mẫu, liên kết đến các phiên chính xác phía sau nó, và mang theo vòng đời phân loại (acknowledge, resolve, mute, dismiss). Failproof AI Observability loại bỏ trùng lặp các phát hiện từ lần chạy này sang lần chạy khác để một mẫu đã biết cập nhật thay vì tích tụ. - -**The AI assistant (Trợ lý AI)** -Trò chuyện trong bảng điều khiển trả lời các câu hỏi về agent của bạn bằng tiếng Anh đơn giản, trên dữ liệu của riêng bạn. Nó chỉ đọc theo mặc định; bất cứ thứ gì nó tạo (một truy vấn đã lưu, một bảng điều khiển) đều được phê duyệt cổng, và nó không bao giờ có thể xóa. Xem [AI assistant](/vi/agenteye/assistant). - ---- - -## Chạy nó - -**Organization (tenant) (Tổ chức - người thuê)** -Một không gian làm việc được cô lập. Một phiên bản Failproof AI Observability có thể lưu trữ nhiều tổ chức, mỗi tổ chức có người dùng, khóa và dữ liệu riêng. Mọi URL bảng điều khiển được phạm vi dưới dấu hiệu tổ chức của bạn (`//…`). - -**Collector (Bộ sưu tập)** -`agenteye-collector`, daemon nhẹ chạy trên mỗi máy agent, phân loại các sự kiện mà SDK ghi vào đĩa, và gửi chúng đến máy chủ. - -**API key (Khóa API)** -Một token có phạm vi xác thực client đối với máy chủ. Các khóa mang quyền chi tiết (ví dụ `events:add` cho bộ sưu tập, phạm vi chỉ đọc cho khóa bảng điều khiển). Xem [API keys](/vi/agenteye/api-keys). - -**Server (Máy chủ)** -Dịch vụ tiếp nhận và API. Nó tiếp nhận sự kiện, lưu trữ trạng thái hoạt động trong cơ sở dữ liệu của bạn, và phục vụ bảng điều khiển và CLI. - -**Dashboard (Bảng điều khiển)** -Giao diện người dùng web. Mọi trang được phạm vi cho một tổ chức và đọc thông qua API của máy chủ. - ---- - -## Các bước tiếp theo - -- [Overview](/vi/agenteye/overview): cách các phần này phù hợp với nhau. -- [Observability](/vi/agenteye/observability): các bề mặt observe (Events, Sessions, Models, Tools, Hooks, Errors). \ No newline at end of file diff --git a/docs/vi/agenteye/evaluator-skill.mdx b/docs/vi/agenteye/evaluator-skill.mdx deleted file mode 100644 index 4b3d0fb2..00000000 --- a/docs/vi/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,171 +0,0 @@ ---- -title: "Kỹ năng Failproof AI Observability Evaluator Agent" -description: "Từ 'Tôi nghĩ agent của chúng tôi đôi khi có vấn đề' đến một dịch vụ scoring được triển khai, với coding agent của bạn vừa quyết định vừa xây dựng." ---- - - -Từ *"Tôi nghĩ agent của chúng tôi đôi khi có vấn đề"* đến một dịch vụ scoring được triển khai, với coding agent của bạn vừa quyết định vừa xây dựng. **Kỹ năng Failproof AI Observability evaluator** (`agenteye-evaluator`) là một *Agent Skill*: một thư mục nhỏ chứa hướng dẫn mà một coding agent như Claude Code hay Codex có thể tải khi cần. Nó dạy agent cách xác định những chiều chất lượng nào đáng theo dõi cho *agent của bạn*, sau đó viết, kiểm thử và triển khai [dịch vụ evaluator](/vi/agenteye/evaluation-suite) để chấm điểm chúng. - -Nó **không** phải là một scorer được lưu trữ, một registry bạn tải lên, hay một hệ thống plugin. Evaluator của bạn vẫn là dịch vụ HTTP riêng trên cơ sở hạ tầng riêng của bạn, chính xác như mô tả trong hướng dẫn [Evaluation suite](/vi/agenteye/evaluation-suite). Kỹ năng này chỉ dạy agent của bạn cách xây dựng nó tốt, vì vậy mọi thứ nó làm, bạn cũng có thể tự làm bằng cách viết cùng một đoạn mã. - ---- - -## Phần khó là quyết định cái gì cần chấm điểm - -Bề mặt SDK rất nhỏ — một decorator và hai model — và agent có thể viết từ [contract](/vi/agenteye/evaluation-suite#http-contract) một mình. Đó không phải là nơi evaluator thất bại. Chúng thất bại vì chúng chấm điểm những thứ sai, và một evaluator chấm những thứ sai thì còn tệ hơn không có gì: nó tạo ra một bảng điều khiển mà mọi người học cách bỏ qua. - -Vì vậy, hầu hết kỹ năng là phần trước khi bất kỳ mã nào tồn tại. Nó có agent phỏng vấn bạn (*"mô tả một lần chạy diễn ra tốt; bây giờ mô tả một lần chạy diễn ra xấu"*), sau đó kéo các phiên thực tế của bạn qua [`agenteye` CLI](/vi/agenteye/cli) và đọc chúng từ đầu đến cuối. Hai nửa này thường không đồng ý, và khoảng cách chính là điểm: những gì bạn định đo so với những gì transcript của bạn thực sự có thể hỗ trợ. Một chiều chỉ tồn tại nếu nó **có thể tính toán** từ các sự kiện và **phân biệt** — nếu nó chấm 0.9 cho cả lần chạy tốt và lần chạy xấu của bạn, nó không dạy gì cả và sẽ bị loại. - -Kết quả là một đề xuất 2-4 chiều kèm theo lý do, để bạn phê duyệt trước khi bất kỳ dòng nào được viết. - -```mermaid -flowchart TD - YOU["bạn: 'Tôi muốn evals cho support bot của tôi'"] --> AGENT["coding agent (Claude Code / Codex)
    tải kỹ năng agenteye-evaluator"] - AGENT -->|"phỏng vấn: good vs bad trông như thế nào?"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["các phiên thực tế của bạn
    những gì thực sự xảy ra"] - DATA --> DIMS["2-4 chiều, bạn phê duyệt"] - DIMS --> SVC["dịch vụ evaluator của bạn
    agenteye-evaluator SDK"] - SVC --> SCORES["điểm chấm xuất hiện trong bảng điều khiển
    và agenteye evals"] -``` - ---- - -## Nó liên quan như thế nào với những phần evaluation khác - -Bốn tài liệu đề cập đến scoring, và chúng trao quyền cho nhau theo thứ tự: - -| Trang | Nó là gì | Tìm đến nó khi | -|---|---|---| -| **[Evaluations](/vi/agenteye/evaluations)** | Tính năng: điểm trên lưới phiên, bảng điều khiển, đánh giá lại | Bạn muốn biết automatic scoring mang lại gì | -| **[Evaluation suite](/vi/agenteye/evaluation-suite)** | HTTP contract, SDK, server env vars | Bạn đang triển khai hoặc gỡ lỗi evaluator | -| **Evaluator skill** (tài liệu này) | Một cửa ngôn ngữ tự nhiên để thiết kế *và* xây dựng scorer | Bạn muốn từ "Tôi muốn evals" đến dịch vụ chạy | -| **[CLI skill](/vi/agenteye/cli-skill)** | Một cửa ngôn ngữ tự nhiên cho `agenteye` CLI | Bạn muốn *đọc* điểm bạn đã có | -| **[Python SDK skill](/vi/agenteye/python-sdk-skill)** | Một cửa ngôn ngữ tự nhiên để instrument agent của bạn | Agent của bạn chưa phát hành phiên — không có gì để chấm điểm | - -### so với CLI skill: xây dựng versus đọc - -Hai kỹ năng có ý định không trùng lặp, và cài đặt cả hai là thiết lập bình thường — agent chọn giữa chúng dựa trên những gì bạn hỏi: - -- **`agenteye-evaluator`** (tài liệu này) xây dựng thứ *tạo ra* điểm. Công việc của nó kết thúc khi điểm xuất hiện lần đầu tiên. -- **[`agenteye-cli`](/vi/agenteye/cli-skill)** đọc điểm đã tồn tại (`agenteye evals`). *"Chất lượng có giảm tuần này không?"* là câu hỏi của nó, không phải của kỹ năng này. - ---- - -## Điều kiện tiên quyết - -1. **`agenteye` CLI được cài đặt và đăng nhập** (`pipx install agenteye`, sau đó `agenteye login`). Kỹ năng dựa vào nó hai lần: để kéo các phiên thực tế nó thiết kế cho, và để xác nhận điểm của bạn xuất hiện ở cuối. Đăng nhập của bạn cần `events:read`, cộng với `evaluations:read` để kiểm tra cuối cùng đó. Giống như CLI skill, nó **không thể** hoàn thành đăng nhập mã một lần qua email cho bạn. -2. **Một nơi cho evaluator ở.** Nó được xây dựng thành một image và chạy như một dịch vụ chạy liên tục, vì vậy nó cần một repo thực, không phải một tệp tạm thời. Các evaluator thường sống trong repo riêng của chúng, tách biệt với agent đang được chấm điểm — kỹ năng tìm kiếm một cái hiện có và hỏi trước khi tạo cái mới. -3. **Wheel SDK `agenteye-evaluator`** — đọc phần tiếp theo trước khi agent của bạn bắt đầu gõ lệnh `pip`. - ---- - -## Nơi lấy nó - -Kỹ năng được công bố trong bộ sưu tập kỹ năng công khai của Failproof AI: - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -Kho lưu trữ là công khai và kỹ năng không cần bất kỳ thông tin xác thực riêng nào — nó chỉ điều khiển `agenteye` CLI với phiên *bạn* đã đăng nhập, và viết mã trong *repo của bạn*. Lưu ý nó được gửi như là một thư mục riêng và **không** nằm bên trong gói `pipx install agenteye`, vì vậy đừng tìm nó ở đó. - -## Cài đặt kỹ năng - -Cách nhanh nhất là CLI [`skills`](https://skills.sh), nó tìm nạp thư mục và đặt nó nơi agent của bạn tìm: - -```bash -# Claude Code, dự án này chỉ -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# mọi dự án (cài đặt vào ~/.claude/skills/) -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# Codex thay vào đó -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -Sau đó quản lý nó như bất kỳ kỹ năng nào khác: - -```bash -npx skills list -a claude-code # cái gì được cài đặt -npx skills update agenteye-evaluator # kéo phiên bản mới nhất -npx skills remove agenteye-evaluator # xóa nó -``` - -Thích cài đặt bằng tay? An Agent Skill chỉ là một thư mục chứa một `SKILL.md` (cộng với các tham chiếu tuỳ chọn), vì vậy sao chép nó cũng hoạt động: - -- **Claude Code**: đặt thư mục `agenteye-evaluator/` trong `~/.claude/skills/` (mọi dự án) hoặc `/.claude/skills/` (chỉ repo đó). Claude Code tự khám phá nó — xác minh bằng danh sách `/skills`, hoặc chỉ cần hỏi về evals. -- **Codex (OpenAI)**: Codex đọc cùng một `SKILL.md`. `agents/openai.yaml` được đi kèm đặt `allow_implicit_invocation: true`, vì vậy Codex tự chọn kỹ năng khi một tác vụ phù hợp; nếu không hãy gọi nó rõ ràng như `$agenteye-evaluator`. - ---- - -## SDK không nằm trên public PyPI công khai - -> **Cảnh báo:** Đọc điều này trước khi để agent cài đặt SDK. - -Kỹ năng là công khai; SDK nó điều khiển thì không. `agenteye-evaluator` được gửi chỉ như một artifact phát hành riêng, và không giống như `agenteye`, tên **chưa được áp dụng trên public PyPI** — vì vậy một `pip install agenteye-evaluator` trần truồng có thể kéo gói của người lạ vào dịch vụ đọc transcript sản xuất của bạn. Đó là một vấn đề chuỗi cung ứng, không phải một lỗi đánh máy. - -Kỹ năng biết điều này và thay vào đó hoạt động xuống một cầu thang cài đặt, dừng lại ở bậc đầu tiên áp dụng: nguồn monorepo nếu bạn ở trong repo AgentEye, nếu không là wheel phát hành riêng từ GitHub Releases (cần truy cập), và nếu cái nào không có sẵn nó **dừng lại và cho bạn biết hãy hỏi Failproof AI contact của bạn để lấy wheel** thay vì ứng phó. - -Vì vậy nếu agent của bạn đề xuất một `pip install agenteye-evaluator` trần truồng từ public PyPI, đó là dấu hiệu kỹ năng không bao giờ được tải. Dừng ở đó và kiểm tra nó được cài đặt. - ---- - -## Những gì bạn có thể hỏi nó - -Một vòng chuyến thực tế bắt đầu với một yêu cầu mơ hồ và kết thúc bằng một thiết kế được ký duyệt, không phải mã: - -```text -bạn ▸ Tôi muốn evals cho support bot của chúng tôi. Tôi nghĩ - nó đôi khi có vấn đề. - -agent ▸ Trước khi tôi viết bất cứ thứ gì: mô tả một lần chạy - diễn ra tốt, và một lần chạy diễn ra xấu. - -bạn ▸ Tốt: nó trả lời từ các tài liệu. Xấu: nó bịa chuyện, - hoặc từ bỏ và nói "liên hệ hỗ trợ". - -agent ▸ Chạy: agenteye --json sessions --status error --since 7d - Đọc 5 phiên, bao gồm hai bạn gọi là xấu. - - Lời của bạn nói tính thực tế và sự chối từ. Các transcript - đồng ý về sự chối từ (2 trong 5 kết thúc bằng một tay qua - hỗ trợ, và nó rõ ràng trong model_response.content), nhưng - tính thực tế cần một sự thật mặt đất mà sự kiện của bạn - không mang theo. - - Vì vậy tôi sẽ chấm ba điều: - - deflection quy tắc, rẻ, tách hai lần chạy của bạn - - tool_efficiency quy tắc, bắt vòng lặp lại trong run-004 - - groundedness LLM judge: mỗi yêu cầu trong tài liệu - nó lấy lại đều có trong tài liệu? - - "factuality" sẽ cần một sự thật mặt đất mà chúng tôi - không có; groundedness là thứ gần nhất mà transcript của - bạn thực sự có thể hỗ trợ. OK với ba tên đó? Đổi tên - sau này sẽ tách lịch sử. - -bạn ▸ có - -agent ▸ Không có evaluator trong repo này. Tôi có nên tạo một ở - đây, hay bạn có một ở nơi khác? -``` - -Từ đó nó viết các chiều dựa trên quy tắc trước tiên (miễn phí, tức thì, xác định), kiểm thử chúng với một phiên thực tế bao gồm những cái rỗng và không bao giờ hoàn thành mà làm hỏng các evaluator ngây thơ, và chỉ tìm đến một LLM judge cho chiều chủ quan. Nó biết [giới hạn của dispatcher](/vi/agenteye/evaluation-suite#configuring-the-server) — timeout yêu cầu 30 giây và 8 cuộc gọi đồng thời triển khai toàn diện — vì vậy nếu judge không vừa một cách đáng tin cậy, nó đi không đồng bộ với `JobPending` thay vì để judge của bạn bị hủy và thử lại năm lần với chi phí gấp năm lần. - -Sau đó nó triển khai, đặt hai server env vars, và xác nhận bằng `agenteye --json evals --session-id ` rằng điểm thực sự xuất hiện. Điểm xuất hiện là bằng chứng duy nhất. - ---- - -## Những gì cần chú ý - -- **Tên chiều gần như vĩnh viễn.** Các khóa điểm là các chuỗi tuỳ ý và nền tảng xu hướng bất cứ thứ gì bạn gửi, có nghĩa là không có gì hạ lưu sửa một lựa chọn xấu. Đổi tên sau và lịch sử bị tách: các phiên cũ giữ khóa cũ và xu hướng bị ngắt. Đó là lý do tại sao kỹ năng nhận ký duyệt rõ ràng trước khi viết mã — hãy xem xét lời nhắc đó một cách nghiêm túc. -- **Fixture là các transcript sản xuất thực tế.** Thiết kế dựa trên các phiên thực tế có nghĩa là kéo chúng xuống đĩa, và chúng có thể chứa dữ liệu khách hàng. Kỹ năng hỏi trước khi commit chúng vào git; nếu không chắc chắn, giữ `fixtures/` ngoài repo và để mỗi nhà phát triển kéo riêng của họ. -- **Agent viết và triển khai một dịch vụ đọc mọi transcript.** Nó hoạt động như bạn, giới hạn bởi quyền hạn đăng nhập CLI của bạn, nhưng xem xét evaluator như bất kỳ mã nào khác chạm vào dữ liệu sản xuất. - ---- - -## Bước tiếp theo - -- **[Evaluation suite](/vi/agenteye/evaluation-suite)**: HTTP contract, SDK, và server env vars mà kỹ năng cấu hình. -- **[Evaluations](/vi/agenteye/evaluations)**: nơi các điểm xuất hiện sau khi chúng xuất hiện. -- **[CLI skill](/vi/agenteye/cli-skill)**: kỹ năng em gái, để đọc kết quả thay vì xây dựng scorer. -- **[CLI](/vi/agenteye/cli)**: tham chiếu lệnh đằng sau dữ liệu phiên mà kỹ năng thiết kế. \ No newline at end of file diff --git a/docs/vi/agenteye/hermes-capture.mdx b/docs/vi/agenteye/hermes-capture.mdx deleted file mode 100644 index 645647b2..00000000 --- a/docs/vi/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Hermes session capture" -description: "Đưa các phiên Hermes gateway của nhóm bạn — Slack, Telegram, CLI và các lần chạy được lên lịch — vào AgentEye dưới dạng các phiên và sự kiện thông thường." ---- - -[Hermes](https://hermes-agent.nousresearch.com) trả lời nhóm bạn từ bất cứ nơi nào họ đã làm việc — Slack, Telegram, CLI, các lần chạy được lên lịch. Hermes session capture đưa tất cả những thứ đó vào AgentEye dưới dạng các phiên và sự kiện thông thường, để trợ lý mà nhóm bạn nói chuyện mỗi ngày có khả năng quan sát giống như các agent mà bạn viết. - -Một trình thu thập nền nhỏ đọc kho lưu trữ phiên cục bộ của Hermes khi nó được ghi và gửi các phiên tới AgentEye. Nó hoạt động giống như cách [Codex](/vi/agenteye/codex-capture) và [OpenClaw](/vi/agenteye/openclaw-capture) capture, và một trình thu thập có thể chụp nhiều cái cùng một lúc. - ---- - -## Nó chụp cái gì - -Mọi phiên Hermes trên máy được chụp, bất kể từ kênh nào nó đến. Mỗi cái trở thành một [phiên](/vi/agenteye/sessions) AgentEye; các tin nhắn của người dùng và trợ lý, lệnh gọi công cụ và kết quả công cụ trở thành các [sự kiện](/vi/agenteye/event-stream) phù hợp. - -Kênh mà phiên được bắt đầu từ — Slack, Telegram, CLI, hoặc một lần chạy được lên lịch — được ghi lại trên phiên, để bạn có thể phân biệt chúng và lọc từng cái một lần. Kèm theo đó là mô hình mà phiên chạy trên đó, cuộc trò chuyện và người nó được bắt đầu từ, và, khi một phiên tạo ra phiên khác, liên kết quay lại phiên cha của nó. - -Các phiên xuất hiện ngay khi Hermes bắt đầu chúng, bất kể có bất cứ điều gì được nói hay không, và câu trả lời của một lượt và các lệnh gọi công cụ của nó vẫn giữ nguyên thứ tự chúng thực sự xảy ra. Khi một phiên kết thúc, bạn cũng sẽ nhận được lý do tại sao nó kết thúc, chi phí của nó và bao nhiêu token nó sử dụng. - ---- - -## Bật nó - -Capture bị tắt cho đến khi bạn bật nó. Cài đặt trình thu thập với một khóa API có quyền `events:add` (xem [API keys](/vi/agenteye/api-keys)) và bật Hermes capture: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -Điều đó cài đặt trình thu thập, đăng ký nó như một dịch vụ nền và bắt đầu chụp. Xác nhận nó đang chạy: - -```bash -agenteye-collector health -``` - -Chụp nhiều hơn một agent trên cùng một máy? Thêm cờ của từng cái vào cùng một lệnh — ví dụ `--hermes-enabled --codex-enabled`. - -Lần chạy đầu tiên, các phiên Hermes hiện có của bạn được điền lại một lần và hoạt động mới sau đó được truyển trong vòng vài giây. Dữ liệu của Hermes chỉ được đọc — không bao giờ được sửa đổi hoặc xóa — và mỗi tin nhắn được gửi một lần, thậm chí qua các lần khởi động lại. - -`health` cũng cho bạn biết liệu mọi thứ mà trình thu thập chụp thực sự đã đến AgentEye hay không. Nếu một lô không thể được gửi, nó được giữ lại và thử lại thay vì bị loại bỏ, và kiểm tra báo cáo không lành mạnh trong khi bất cứ điều gì vẫn còn nợ — vì vậy "lành mạnh" có nghĩa là dữ liệu của bạn đã tới, không chỉ là quá trình còn sống. - ---- - -## Nó xuất hiện ở đâu - -Các phiên được chụp xuất hiện trong **Sessions**, và các sự kiện của chúng trong luồng **Events**, giống như bất kỳ agent nào khác mà bạn quan sát — vì vậy [session replay](/vi/agenteye/sessions), [search](/vi/agenteye/queries), [evaluations](/vi/agenteye/evaluations), và [alerts](/vi/agenteye/alerts) đều hoạt động trên chúng. Lọc theo agent Hermes để xem chúng riêng biệt. - ---- - -## Quyền riêng tư - -Các phiên Hermes chứa toàn bộ bản ghi — bao gồm đầu ra lệnh, nội dung tệp và bất cứ điều gì mà agent đã đọc hoặc viết — và có thể chứa bí mật. Các phiên được chụp được gửi nguyên trạng, vì vậy chỉ bật capture nơi tập trung nội dung đó trong AgentEye là phù hợp, và cấp cho trình thu thập một khóa phạm vi chỉ `events:add`. Xem [Security](/vi/agenteye/security) để biết dữ liệu của bạn được giữ cách ly như thế nào. \ No newline at end of file diff --git a/docs/vi/agenteye/observability.mdx b/docs/vi/agenteye/observability.mdx deleted file mode 100644 index c5c97598..00000000 --- a/docs/vi/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "Observe" -description: "Các bề mặt observe là nơi bạn theo dõi các agent đang làm gì ngay bây giờ và xem chi tiết bất kỳ lần chạy nào." ---- - - -Các bề mặt observe là nơi bạn theo dõi các agent đang làm gì ngay bây giờ và xem chi tiết bất kỳ lần chạy nào. Mọi thứ ở đây đều trực tiếp, được giới hạn trong tổ chức của bạn, và có thể lọc theo khoảng thời gian, môi trường, agent và phiên, vì vậy bạn có thể từ "có cái gì đó không ổn" đến chính xác lần chạy đó trong vài giây. - -![Event Stream trực tiếp, được mã hóa màu theo loại và có thể lọc theo môi trường, agent và phiên](/agenteye/images/events-stream.png) - -Bốn bề mặt, mỗi cái có trang riêng: - -- **[Event stream](/vi/agenteye/event-stream)**: đuôi trực tiếp từng bước của mọi lần chạy trên mọi agent, mới nhất trước. Trang chủ tổ chức của bạn và điểm dừng đầu tiên để phân loại. -- **[Sessions and execution graph](/vi/agenteye/sessions)**: những sự kiện đó được gộp lại thành một hàng cho mỗi lần chạy, cộng với một bức tranh kiểu git về cách mỗi lần chạy diễn ra. -- **[Performance metrics](/vi/agenteye/telemetry)**: biểu đồ nhiệt độ trễ và số liệu quan trọng p50/p95/p99 cho các mô hình, công cụ và hook của bạn, vì vậy một loại spike tail nổi bật so với trung vị. -- **[Error tracking](/vi/agenteye/error-tracking)**: một bề mặt phân loại cho mọi thứ đã xảy ra sai, một cú nhấp chuột từ cảnh báo được kích hoạt đến lần chạy bị hỏng. - -## Liên quan - -- [Evaluations](/vi/agenteye/evaluations): đánh điểm mỗi lần chạy để có chất lượng. -- [Alerts](/vi/agenteye/alerts): biến bất kỳ ngưỡng nào thành một quy tắc phân trang. -- [Audits](/vi/agenteye/audits): để Failproof AI Observability tìm các mẫu lỗi trên các phiên cho bạn. -- [CLI and agents](/vi/agenteye/cli-and-agents): cùng một khả năng quan sát từ terminal của bạn. \ No newline at end of file diff --git a/docs/vi/agenteye/openclaw-capture.mdx b/docs/vi/agenteye/openclaw-capture.mdx deleted file mode 100644 index 5577fe56..00000000 --- a/docs/vi/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "Tính năng ghi lại phiên làm việc OpenClaw" -description: "Đưa các phiên làm việc OpenClaw cục bộ của nhóm của bạn vào AgentEye dưới dạng các phiên và sự kiện thông thường — mà không cần thay đổi cách OpenClaw hoạt động." ---- - -Nếu nhóm của bạn sử dụng [OpenClaw](https://docs.openclaw.ai), tính năng ghi lại phiên làm việc OpenClaw sẽ đưa những phiên đó vào AgentEye dưới dạng các phiên và sự kiện thông thường, giúp bạn tìm kiếm, phát lại và đánh giá chúng cùng với tất cả những gì khác mà bạn quan sát. Nó bổ sung cho [Python SDK](/vi/agenteye/python-sdk): SDK sẽ theo dõi các agent mà bạn viết, trong khi tính năng này ghi lại công việc OpenClaw mà nhóm của bạn đã thực hiện — mà không cần thay đổi cách họ chạy nó. - -Một bộ thu thập dữ liệu nền nhỏ đọc các bảng ghi chép phiên làm việc OpenClaw cục bộ khi chúng được ghi và gửi chúng đến AgentEye. Nó hoạt động giống như [Codex capture](/vi/agenteye/codex-capture), và một bộ thu thập có thể ghi lại cả hai cùng một lúc. - ---- - -## Điều gì được ghi lại - -Mọi agent được cấu hình trong cài đặt OpenClaw của một máy đều được ghi lại bởi bộ thu thập của máy đó — không cần cài đặt riêng cho từng agent. - -Mỗi phiên làm việc OpenClaw trở thành một [phiên](/vi/agenteye/sessions) AgentEye; các tin nhắn của người dùng và trợ lý, lệnh gọi công cụ và kết quả công cụ trở thành những [sự kiện](/vi/agenteye/event-stream) tương ứng. - ---- - -## Bật tính năng này - -Tính năng ghi lại được tắt cho đến khi bạn bật nó. Cài đặt bộ thu thập với một khóa API có quyền `events:add` (xem [API keys](/vi/agenteye/api-keys)), và bật tính năng ghi lại OpenClaw: - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -Điều này sẽ cài đặt bộ thu thập, đăng ký nó làm dịch vụ nền, và bắt đầu ghi lại. Xác nhận rằng nó đang chạy: - -```bash -agenteye-collector health -``` - -Ghi lại nhiều hơn một agent trên cùng một máy? Thêm cờ của mỗi cái vào cùng một lệnh — ví dụ `--openclaw-enabled --codex-enabled`. - -Lần chạy đầu tiên, các phiên làm việc OpenClaw hiện có của bạn sẽ được điền lại một lần và sau đó hoạt động mới sẽ truyến phát trong vòng vài giây. Các tệp của chính OpenClaw chỉ được đọc — không bao giờ được sửa đổi, di chuyển hoặc xóa — và mỗi phiên được gửi đúng một lần, thậm chí qua các lần khởi động lại. - ---- - -## Nơi nó xuất hiện - -Các phiên được ghi lại xuất hiện trong **Sessions**, và các sự kiện của chúng trong luồng **Events**, giống như bất kỳ agent nào khác mà bạn quan sát — vì vậy [session replay](/vi/agenteye/sessions), [search](/vi/agenteye/queries), [evaluations](/vi/agenteye/evaluations), và [alerts](/vi/agenteye/alerts) đều hoạt động trên chúng. Lọc theo agent OpenClaw để xem chúng riêng biệt. - ---- - -## Bảo mật - -Các bảng ghi chép OpenClaw chứa toàn bộ phiên — bao gồm đầu ra lệnh, nội dung tệp và bất cứ điều gì mà agent đã đọc hoặc ghi — và có thể chứa các bí mật. Các phiên được ghi lại được gửi như cũ, vì vậy chỉ bật tính năng ghi lại trên các máy và cho các nhóm nơi tập trung nội dung đó trong AgentEye là thích hợp, và cấp cho bộ thu thập một khóa được phạm vi chỉ `events:add`. Xem [Security](/vi/agenteye/security) để biết cách dữ liệu của bạn được giữ riêng biệt. \ No newline at end of file diff --git a/docs/vi/agenteye/python-sdk-skill.mdx b/docs/vi/agenteye/python-sdk-skill.mdx deleted file mode 100644 index acc33404..00000000 --- a/docs/vi/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,131 +0,0 @@ ---- -title: "Failproof AI Observability Python SDK Agent Skill" -description: "Go from an uninstrumented agent to events you can see, with your coding agent finding the instrumentation points, writing them, and proving they landed." ---- - -Hãy bảo công cụ code agent của bạn *"add Failproof AI Observability to this agent"* và để nó đọc vòng lặp của bạn, tìm ra nơi cần thêm instrumentation, viết nó, và xác minh các sự kiện trước khi hoàn thành công việc. - -**Python SDK skill** (`agenteye-python-sdk`) là một *Agent Skill*: một thư mục chứa hướng dẫn mà công cụ code agent như Claude Code hoặc Codex tải theo yêu cầu khi một tác vụ phù hợp với nó. Nó dạy agent cách sử dụng [Python SDK](/vi/agenteye/python-sdk) — nó không phải là một thư viện, và nó không thay đổi bất cứ điều gì về cách SDK hoạt động. - -## Instrumentation dễ viết nhưng dễ sai một cách câm lặng - -SDK rất nhỏ: mười ba phương thức sự kiện, tất cả đều chỉ nhận tham số theo từ khóa. Công cụ code agent có thể đọc tài liệu tham khảo [Python SDK](/vi/agenteye/python-sdk) và tạo ra instrumentation hợp lý trong một phút. - -Vấn đề là SDK này không phát sinh lỗi khi bạn làm sai, và instrumentation sai trông giống hệt như instrumentation đúng cho đến khi ai đó mở bảng điều khiển và thấy nó trống rỗng. Những lỗi tốn thời gian thực tế đều là những im lặng: - -| Lỗi | Bạn thấy gì | -|---|---| -| Không có `agent_start` | Mọi sự kiện được ghi nhận. Không có phiên làm việc. | -| Môi trường không bao giờ được thiết lập | Mọi thứ hoạt động, được lưu dưới `dev`. | -| `outcome="failure"` | Quá trình chạy hiển thị xanh — chỉ `failed`, `error`, `timeout`, `rejected` mới được tính. | -| Tên trường bị gõ sai | Được chấp nhận và lưu trữ như một trường mới. | -| Sự kiện phát ra từ thread pool | Bị loại bỏ âm thầm. | - -Không có cái nào trong số này phát sinh lỗi. Không cái nào hiển thị trong các bài kiểm tra. Mỗi cái đều nằm trong skill, được nêu ra như một hợp đồng với kiểm tra phát hiện nó. - -## Nó làm gì, theo thứ tự - -Skill chạy ba bước giống như một kỹ sư cẩn thận sẽ làm: - -1. **Lập kế hoạch.** Nó đọc vòng lặp agent của bạn và đặt hai câu hỏi chỉ bạn mới trả lời được: cái gì được coi là một lần chạy (của bạn `session_id`), và những tác nhân nào khác biệt (của bạn `agent_id`). Nó đạt được sự đồng ý trước khi viết mã, vì thay đổi chúng sau này sẽ chia tách lịch sử của bạn và phá vỡ xu hướng. -2. **Viết.** Nó liên kết danh tính một lần mỗi lần chạy thay vì chuyển nó qua mỗi call site, và nó chọn một hình dạng an toàn đồng thời — một chi tiết quan trọng, vì cách tắt hiển nhiên sẽ âm thầm trộn hai lần chạy chồng chéo thành một phiên. -3. **Xác minh.** Nó chạy agent của bạn và đọc các tệp sự kiện kết quả, kiểm tra xem `agent_start` có hiện diện, môi trường có đúng, và một lần chạy có tạo một phiên. - -Bước thứ ba là bước mà mọi người bỏ qua. SDK viết các sự kiện vào tệp cục bộ, vì vậy một sự tích hợp hoàn chỉnh có thể được chứng minh trên máy tính xách tay mà không cần máy chủ, không cần khóa API, và không cần mạng — chính vì thế skill nhất định phải thực hiện nó. - -## Nó liên quan như thế nào với các skill khác - -Ba skill, một phân chia sạch sẽ: - -| Skill | Sử dụng khi | Nó chạm vào cái gì | -|---|---|---| -| **Python SDK skill** (trang này) | Bạn muốn agent của bạn *phát ra* telemetry — "add observability", "tại sao agent của tôi không hiện lên?" | Viết mã trong repo của agent. Không đọc gì cả. | -| **[Evaluator skill](/vi/agenteye/evaluator-skill)** | Bạn muốn *đánh điểm* các lần chạy — "chúng ta nên đo lường cái gì?" | Viết mã trong repo của bạn; đọc telemetry | -| **[CLI skill](/vi/agenteye/cli-skill)** | Bạn muốn *đọc* những gì đã xảy ra, hoặc vận hành deployment | Điều khiển CLI như bạn, bao gồm các thay đổi | - -Chúng được chuyển giao theo thứ tự đó: skill này làm cho sự kiện chảy, evaluator đánh điểm chúng, CLI đọc chúng lại. Không có gì để đánh điểm và không có gì để đọc cho đến khi agent của bạn phát ra các phiên, vì vậy nếu bạn bắt đầu từ đầu, hãy bắt đầu từ đây. - -## Điều kiện tiên quyết - -1. **Python 3.10+** và codebase agent bạn muốn đặt instrumentation. -2. **SDK.** Nó được phân phối cho khách hàng dưới dạng wheel riêng tư chứ không phải từ chỉ mục công cộng — onboarding của bạn sẽ bao gồm cách lấy và cài đặt nó. Skill biết đường dẫn cài đặt và sẽ hỏi bạn thay vì đoán nếu nó không thể tìm thấy nó. -3. **Không gì khác.** Không cần đăng nhập bảng điều khiển, không cần khóa API, không cần mạng. Skill xác minh dựa trên các tệp sự kiện mà SDK viết, vì vậy nó có thể hoàn thành và chứng minh công việc của nó ngoại tuyến. - -## Lấy nó ở đâu - -Skill nằm trong bộ sưu tập công cộng [`FailproofAI/skills`](https://github.com/FailproofAI/skills): - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -Thêm `-g` để cài đặt nó cho mỗi dự án thay vì chỉ dự án hiện tại, và `--copy` nếu môi trường của bạn không hỗ trợ symlink. Đối với Codex, truyền `-a codex`. - -## Cài đặt thủ công - -Agent Skills là các thư mục chứa `SKILL.md` cộng với các tham chiếu. Nếu bạn không muốn sử dụng trình cài đặt: - -- **Claude Code**: sao chép thư mục `agenteye-python-sdk/` vào `~/.claude/skills/` (mọi dự án) hoặc `/.claude/skills/` (chỉ repo đó). Claude Code tự động phát hiện nó — kiểm tra danh sách `/skills`, hoặc chỉ cần hỏi điều gì phù hợp với nó. -- **Codex**: Codex đọc cùng một `SKILL.md`. `agents/openai.yaml` được đóng gói thiết lập `allow_implicit_invocation: true`, vì vậy nó được tự động chọn khi tác vụ phù hợp; nếu không thì gọi nó là `$agenteye-python-sdk`. - -Chạy agent của bạn **trong repository chứa mã bạn muốn đặt instrumentation** — skill đọc vòng lặp agent của bạn trước khi nó đề xuất bất cứ điều gì. - -## Một phiên trông như thế nào - -```text -you ▸ Add Failproof AI Observability to this agent. - -agent ▸ Read your loop — one tool dispatcher in tools.py, one Anthropic - wrapper in llm.py. Two questions before I write anything. - - What counts as one run? You handle requests concurrently, so I'd - use the request id you already have in ctx.request_id — then a - session here and a request in your logs are the same string. - - And who are the actors? I see a planner path and a researcher - path. Label them separately, or is that one agent to you? - -you ▸ Request id is right. They're one agent, call it "assistant". - -agent ▸ Done. Wrapped the dispatcher and the LLM client; agent_start and - agent_end bracket handle_request, including the error path. Used - a context-local for identity — your dispatcher runs tools in a - thread pool, and the simpler approach would have mixed concurrent - runs into one session without complaining. - - Verified against a real run: 11 events across 4 files — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, one session - per request, durations paired. - - One thing I did NOT do: your worker pool in batch.py submits - straight to the executor, so events from there would be dropped. - Want me to fix those too? -``` - -Mẫu cần lưu ý: nó đọc mã trước khi đề xuất, chỉ hỏi những câu hỏi bạn mới trả lời được, tái sử dụng một id bạn đã có, chọn hình dạng an toàn đồng thời *vì* nó thấy thread pool, và **xác minh bằng cách đọc các sự kiện thực tế** thay vì tuyên bố thành công — rồi cảnh báo về nơi duy nhất nó biết rằng sẽ thất bại âm thầm. - -## Bạn có thể yêu cầu nó làm gì - -- *"Tại sao agent của tôi không hiện lên trên bảng điều khiển?"* → đi theo từng bước: các sự kiện có được viết không, `agent_start` có ở đó không, môi trường có đúng không, collector có đang đọc cùng một nơi không. -- *"Mọi thứ đang hạ cánh dưới dev."* → môi trường không bao giờ được thiết lập, hoặc được đặt lại bởi một lệnh gọi sau. -- *"Thêm theo dõi token."* → tìm trình bao bọc LLM của bạn và ghi lại mô hình, lý do dừng, và cách sử dụng. -- *"Đặt instrumentation cho các sub-agent."* → một phiên, nhãn agent riêng biệt, lồng dưới phần tử cha của chúng. -- *"Viết bài kiểm tra cho instrumentation."* → chỉ SDK vào một thư mục tạm thời và khẳng định các sự kiện nó đã viết. - -## Điều cần chú ý - -**Để nó xác minh.** Bước cuối cùng làm cho skill này đáng sử dụng — chạy agent của bạn và đọc các sự kiện lại. Một agent viết instrumentation và dừng lại đã thực hiện nửa dễ dàng, và nửa thất bại âm thầm là nửa kia. - -**Đồng ý tên trước mã.** `session_id` và `agent_id` là các trục mọi bề mặt nhóm theo. Đổi tên chúng sau chia tách lịch sử: các lần chạy cũ giữ các nhãn cũ và xu hướng của bạn sẽ phá vỡ. Skill sẽ hỏi; câu trả lời đáng tiêu tốn một phút suy nghĩ. - -**Nếu agent của bạn đề xuất cài đặt SDK từ chỉ mục công cộng, skill đã không tải.** SDK được phân phối riêng tư. Đề xuất đó là một dấu hiệu đáng tin cậy rằng công cụ code agent của bạn đang đoán thay vì tuân theo skill — dừng nó ở đó và kiểm tra skill có được cài đặt không. - -Ngoài ra, bán kính ảnh hưởng của nó rất nhỏ: nó viết mã trong thư mục làm việc của bạn và các tệp sự kiện nơi bạn nói. Nó không đọc gì từ deployment của bạn và không thay đổi gì về nó. - -## Bước tiếp theo - -- **[Python SDK](/vi/agenteye/python-sdk)**: tài liệu tham khảo sự kiện hoàn chỉnh — mỗi loại sự kiện và trường — đằng sau những gì skill này tự động hóa. -- **[Sessions](/vi/agenteye/sessions)**: những gì instrumentation của bạn tạo ra khi các sự kiện hạ cánh. -- **[Evaluator Agent Skill](/vi/agenteye/evaluator-skill)**: bước tiếp theo sau khi các lần chạy hạ cánh — đánh điểm chúng. -- **[CLI Agent Skill](/vi/agenteye/cli-skill)**: đọc telemetry của bạn lại. \ No newline at end of file diff --git a/docs/vi/agenteye/security.mdx b/docs/vi/agenteye/security.mdx deleted file mode 100644 index 1bec174d..00000000 --- a/docs/vi/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "Bảo mật" -description: "Failproof AI Observability được xây dựng để hoạt động gần với các agent production của bạn, có nghĩa là nó thấy được prompts, đầu vào công cụ và kết quả đầu ra của bạn." ---- - - -Failproof AI Observability được xây dựng để hoạt động gần với các agent production của bạn, có nghĩa là nó thấy được prompts, đầu vào công cụ và kết quả đầu ra của bạn. Trang này giải thích cách nó giữ dữ liệu đó được cách ly, được kiểm soát và nằm dưới quyền của bạn. Nếu bạn đang đánh giá Failproof AI Observability cho một bài kiểm tra bảo mật, hãy bắt đầu từ đây. - ---- - -## Dữ liệu của bạn ở lại trong môi trường của bạn - -Failproof AI Observability được tự lưu trữ. Các sự kiện, prompts, phản hồi của mô hình và phân tích được lưu trữ trong các cơ sở dữ liệu của riêng bạn, trong môi trường của riêng bạn. Không có dữ liệu nào được gửi đến bên thứ ba SaaS để lưu trữ, và dữ liệu của bạn ở lại trong tài khoản cloud của riêng bạn. - ---- - -## Cách ly đa tổ chức - -Một instance Failproof AI Observability có thể lưu trữ nhiều tổ chức, và mỗi tổ chức được cách ly ở lớp lưu trữ — được thực thi bởi cơ sở dữ liệu, không chỉ giao diện người dùng: - -- Dữ liệu hoạt động của một tổ chức (người dùng, khóa, bảng điều khiển, truy vấn đã lưu) được giới hạn trong tổ chức đó, và các lần đọc liên tổ chức bị chặn bởi chính cơ sở dữ liệu. -- Mỗi sự kiện được nhập đều được đánh dấu với tổ chức sở hữu, vì vậy các sự kiện của một tổ chức không bao giờ có thể được đọc bởi tổ chức khác. - -Mỗi tuyến đường bảng điều khiển được giới hạn trong một slug org (`//…`). - ---- - -## Đăng nhập - -Failproof AI Observability sử dụng đăng nhập không mật khẩu, dựa trên email. Không có mật khẩu để lừa phishing hoặc rò rỉ. Người dùng yêu cầu một mã dùng một lần (hoặc liên kết magic một bước), được gửi email cho họ và hết hạn nhanh chóng. Đăng nhập được kiểm soát bởi một **danh sách cho phép**: chỉ những địa chỉ email (hoặc miền) mà bạn cho phép mới có thể xác thực. - -![Màn hình đăng nhập Failproof AI Observability, gửi một mã dùng một lần đến email của bạn](/agenteye/images/login.png) - ---- - -## Truy cập được giới hạn với khóa API - -Mỗi máy khách xác thực bằng khóa API có quyền granular, ít nhất. Một bộ sưu tập chỉ cần `events:add`; một khóa bảng điều khiển hoặc trợ lý có thể chỉ đọc; các hành động phá hủy (xóa, tạo lại) là các cấp riêng biệt mà bạn chọn để đưa vào. - -![Trang khóa API: các cấp quyền của mỗi khóa, được mã hóa màu theo phạm vi đọc, viết và hủy diệt](/agenteye/images/api-keys.png) - -Giữ khóa bootstrap quản trị viên cho thiết lập và phát hành các khóa hẹp cho mọi thứ khác. Xem [API keys](/vi/agenteye/api-keys). - ---- - -## Trợ lý chỉ đọc, được phê duyệt - -[Trợ lý AI](/vi/agenteye/assistant) trong bảng điều khiển trả lời các câu hỏi về dữ liệu của bạn, nhưng nó bị hạn chế bởi thiết kế: - -- Nó **chỉ đọc theo mặc định**: SQL của nó chạy qua một lệnh bảo vệ chỉ cho phép các truy vấn `SELECT`/`WITH`, một câu lệnh duy nhất, với một giới hạn hàng. -- Bất cứ điều gì nó tạo (một truy vấn đã lưu, một bảng điều khiển) đều **được phê duyệt**: bạn xem xét và phê duyệt mỗi lần ghi trước khi nó xảy ra. -- Nó **không bao giờ có thể xóa**. - -Vì vậy, một đồng nghiệp có thể hỏi "agents nào bị lỗi nhất tuần này?" và hành động dựa trên câu trả lời, mà không cần trợ lý có khả năng thay đổi hoặc xóa dữ liệu của bạn riêng lẻ. - ---- - -## Trong quá trình chuyển động - -Tất cả lưu lượng chạy qua HTTPS. Bạn kết thúc TLS bằng chứng chỉ của riêng bạn, vì vậy lưu lượng từ bộ sưu tập đến máy chủ và từ trình duyệt đến máy chủ được mã hóa trong quá trình chuyển động. - ---- - -## Bước tiếp theo - -- [Overview](/vi/agenteye/overview): cách Failproof AI Observability kết hợp với nhau. -- [API keys](/vi/agenteye/api-keys): giới hạn truy cập cho bộ sưu tập, bảng điều khiển và trợ lý. -- [Observability](/vi/agenteye/observability): những gì Failproof AI Observability capture từ các agent của bạn. \ No newline at end of file diff --git a/docs/vi/cli/audit.mdx b/docs/vi/audit.mdx similarity index 100% rename from docs/vi/cli/audit.mdx rename to docs/vi/audit.mdx diff --git a/docs/vi/cli/backfill.mdx b/docs/vi/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/vi/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/vi/cli/config.mdx b/docs/vi/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/vi/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/vi/cli/flush.mdx b/docs/vi/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/vi/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/vi/cli/harness.mdx b/docs/vi/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/vi/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/vi/cli/migrate.mdx b/docs/vi/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/vi/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/vi/cli/uninstall.mdx b/docs/vi/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/vi/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/vi/cli/update.mdx b/docs/vi/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/vi/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/vi/agenteye/api-keys.mdx b/docs/vi/cloud/access.mdx similarity index 95% rename from docs/vi/agenteye/api-keys.mdx rename to docs/vi/cloud/access.mdx index e0e13c65..e5cf5ccf 100644 --- a/docs/vi/agenteye/api-keys.mdx +++ b/docs/vi/cloud/access.mdx @@ -1,10 +1,10 @@ --- title: "API Keys" -description: "API keys kiểm soát ai và những gì có thể tiếp cận máy chủ Failproof AI Observability của bạn, vì vậy một collector có thể gửi sự kiện mà không bao giờ có được quyền đọc hoặc quyền admin." +description: "API keys kiểm soát ai và những gì có thể tiếp cận máy chủ FailproofAI Cloud của bạn, vì vậy một collector có thể gửi sự kiện mà không bao giờ có được quyền đọc hoặc quyền admin." --- -API keys kiểm soát ai và những gì có thể tiếp cận máy chủ Failproof AI Observability của bạn, vì vậy một collector có thể gửi sự kiện mà không bao giờ có được quyền đọc hoặc quyền admin. Mỗi key mang một hoặc nhiều quyền, và mỗi quyền kiểm soát các route máy chủ cụ thể; bạn chỉ cấp những quyền mà công việc cần. Hầu hết các triển khai chỉ tạo ba loại key. +API keys kiểm soát ai và những gì có thể tiếp cận máy chủ FailproofAI Cloud của bạn, vì vậy một collector có thể gửi sự kiện mà không bao giờ có được quyền đọc hoặc quyền admin. Mỗi key mang một hoặc nhiều quyền, và mỗi quyền kiểm soát các route máy chủ cụ thể; bạn chỉ cấp những quyền mà công việc cần. Hầu hết các triển khai chỉ tạo ba loại key. ## 3 key mà hầu hết các triển khai cần @@ -82,7 +82,7 @@ Máy chủ thực thi một danh mục quyền cố định; mỗi cái kiểm s Các quyền này hỗ trợ trang **Users** của dashboard, nơi mà các phạm vi được cấp của mỗi thành viên được hiển thị dưới dạng chip: -![Trang Users: một thẻ cho mỗi người dùng dashboard với email, quyền được cấp, và điều khiển chỉnh sửa/vô hiệu hóa của họ](/agenteye/images/users.png) +![Trang Users: một thẻ cho mỗi người dùng dashboard với email, quyền được cấp, và điều khiển chỉnh sửa/vô hiệu hóa của họ](/cloud/images/users.png) ### Operational settings @@ -91,7 +91,7 @@ Các quyền này hỗ trợ trang **Users** của dashboard, nơi mà các ph | `settings:read` | `GET /settings`, `GET /settings/schema`, `GET /settings/model-context-windows`, `GET /settings/model-context-windows/resolve` | Xem các cài đặt hoạt động được quản lý bằng dashboard và siêu dữ liệu của chúng; liệt kê các ghi đè cửa sổ ngữ cảnh mỗi mô hình; và giải quyết cửa sổ hiệu quả cho một mô hình. | | `settings:write` | `PUT /settings/:key`, `PUT /settings/model-context-windows`, `DELETE /settings/model-context-windows` | Chỉnh sửa các cài đặt hoạt động và thêm, thay đổi, hoặc xóa các ghi đè cửa sổ ngữ cảnh mỗi mô hình. Các thay đổi ảnh hưởng đến các sự kiện mới mà không cần khởi động lại máy chủ. | -![Trang Settings: các cài đặt hoạt động được quản lý bằng dashboard như đăng nhập được phép và tuổi thọ session/OTP, có thể chỉnh sửa mà không cần khởi động lại](/agenteye/images/settings.png) +![Trang Settings: các cài đặt hoạt động được quản lý bằng dashboard như đăng nhập được phép và tuổi thọ session/OTP, có thể chỉnh sửa mà không cần khởi động lại](/cloud/images/settings.png) ### Alerts & incidents @@ -247,7 +247,7 @@ Trang **Keys** trong dashboard cung cấp một UI cho tất cả các hoạt đ Khi bạn tạo một key từ dashboard, bạn không cung cấp secret; dashboard tạo ra một secret mạnh cho bạn và hiển thị nó **một lần** khi tạo. Sao chép nó ngay lập tức và lưu trữ nó một cách an toàn; nó không bao giờ được hiển thị lại, giống như một lần tái tạo. Bạn vẫn có thể chọn quyền của key một cách trực tiếp, hoặc khởi tạo chúng từ một permission set (xem dưới đây). -![Trang API Keys: một thẻ cho mỗi key hiển thị tên, quyền được cấp, và thời gian tạo, với các hành động tái tạo và vô hiệu hóa; các key được bảo vệ như `admin` được đánh dấu](/agenteye/images/api-keys.png) +![Trang API Keys: một thẻ cho mỗi key hiển thị tên, quyền được cấp, và thời gian tạo, với các hành động tái tạo và vô hiệu hóa; các key được bảo vệ như `admin` được đánh dấu](/cloud/images/api-keys.png) --- @@ -276,5 +276,5 @@ Bạn chỉ cần những cái này nếu bạn đang nâng cấp một instance ## Các bước tiếp theo -- [Python SDK](/vi/agenteye/python-sdk): cách mã agent của bạn xác thực khi gửi sự kiện. -- [Security](/vi/agenteye/security): cách đăng nhập, kiểm soát truy cập, và cách cô lập dữ liệu mỗi tổ chức hoạt động. \ No newline at end of file +- [Python SDK](/vi/cloud/sdk): cách mã agent của bạn xác thực khi gửi sự kiện. +- [Security](/vi/cloud/security): cách đăng nhập, kiểm soát truy cập, và cách cô lập dữ liệu mỗi tổ chức hoạt động. \ No newline at end of file diff --git a/docs/vi/cloud/agent-skills.mdx b/docs/vi/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/vi/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
    loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/vi/agenteye/alerts.mdx b/docs/vi/cloud/alerts.mdx similarity index 77% rename from docs/vi/agenteye/alerts.mdx rename to docs/vi/cloud/alerts.mdx index 01b5e42c..87791ff4 100644 --- a/docs/vi/agenteye/alerts.mdx +++ b/docs/vi/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "Phát hiện ngay khi có vấn đề vượt quá ngưỡng của --- -Phát hiện ngay khi có vấn đề vượt quá ngưỡng của bạn, trên kênh mà nhóm của bạn đã theo dõi, thay vì nghe từ khách hàng. Đặt một quy tắc một lần và Failproof AI Observability kiểm tra nó theo lịch trình, sau đó gửi thông báo cho bạn qua email, Slack, webhook, hoặc ngay trên bảng điều khiển. +Phát hiện ngay khi có vấn đề vượt quá ngưỡng của bạn, trên kênh mà nhóm của bạn đã theo dõi, thay vì nghe từ khách hàng. Đặt một quy tắc một lần và FailproofAI Cloud kiểm tra nó theo lịch trình, sau đó gửi thông báo cho bạn qua email, Slack, webhook, hoặc ngay trên bảng điều khiển. -![Trang Cảnh báo: một lưới các thẻ quy tắc cảnh báo, mỗi thẻ hiển thị kích hoạt của nó, cửa sổ đánh giá, kênh và một huy hiệu mức độ nghiêm trọng thông tin, cảnh báo hoặc quan trọng](/agenteye/images/alerts.png) +![Trang Cảnh báo: một lưới các thẻ quy tắc cảnh báo, mỗi thẻ hiển thị kích hoạt của nó, cửa sổ đánh giá, kênh và một huy hiệu mức độ nghiêm trọng thông tin, cảnh báo hoặc quan trọng](/cloud/images/alerts.png) *Mỗi quy tắc cảnh báo trong một cái nhìn: nó theo dõi cái gì, tần suất bao nhiêu, nơi nó gửi thông báo, và mức độ khẩn cấp như thế nào.* ## Biết về các vấn đề trước khi người dùng của bạn biết @@ -22,9 +22,9 @@ Gắn kết bất kỳ sự kết hợp nào vào một quy tắc duy nhất, v ## Xây dựng quy tắc trong một biểu mẫu, không phải JSON -Bạn mô tả điều gì có nghĩa là "bị hỏng" trong một biểu mẫu, và Failproof AI Observability viết quy tắc cơ bản cho bạn. Thông số kỹ thuật JSON chỉ là những gì biểu mẫu đó tạo ra dưới nắp động cơ, vì vậy bạn có thể đọc nó để hiểu một quy tắc nhưng bạn hiếm khi nhập nó. +Bạn mô tả điều gì có nghĩa là "bị hỏng" trong một biểu mẫu, và FailproofAI Cloud viết quy tắc cơ bản cho bạn. Thông số kỹ thuật JSON chỉ là những gì biểu mẫu đó tạo ra dưới nắp động cơ, vì vậy bạn có thể đọc nó để hiểu một quy tắc nhưng bạn hiếm khi nhập nó. -![Biểu mẫu cảnh báo mới: tên và mô tả, bộ chuyển đổi bật, và một bộ chọn kích hoạt cung cấp ngưỡng metric, SQL tùy chỉnh, điểm đánh giá, đánh giá compound, và các điều kiện cho mỗi sự kiện](/agenteye/images/alert-new.png) +![Biểu mẫu cảnh báo mới: tên và mô tả, bộ chuyển đổi bật, và một bộ chọn kích hoạt cung cấp ngưỡng metric, SQL tùy chỉnh, điểm đánh giá, đánh giá compound, và các điều kiện cho mỗi sự kiện](/cloud/images/alert-new.png) *Chọn một kích hoạt và biểu mẫu hoán đổi các trường phù hợp; Lưu viết quy tắc.* Con đường hạnh phúc là nhanh: đặt tên cho nó, chọn một **kích hoạt** (cái gì cần theo dõi), đặt **ngưỡng và cửa sổ** (tệ như thế nào, trong bao lâu), gắn kết ít nhất một **kênh**, sau đó **Lưu** và nhấn **Kiểm tra** để kích hoạt một thông báo tổng hợp và xác nhận mọi đích đến đã được kết nối. Dưới nắp động cơ điều đó tạo ra một thông số kỹ thuật nhỏ như: @@ -43,7 +43,7 @@ Bạn không bị giới hạn ở một loại tín hiệu. Chọn kích hoạt | **Đánh giá compound** | nhiều kiểm tra điểm kết hợp với bất kỳ, tất cả, hoặc logic ít nhất-N, để bắt một lùi chỉ xuất hiện trên các điểm | | **Cho mỗi sự kiện** | một sự kiện khớp đơn lẻ đến: một agent cụ thể, một loại lỗi cụ thể, hoặc một chuỗi con tin nhắn | -Đã staring tại một lỗi trên [trang Lỗi](/vi/agenteye/error-tracking)? Mỗi hàng ở đó có nút **+ alert** mở cùng biểu mẫu này được điền sẵn để bắt lỗi chính xác đó lại, vì vậy sự cố bạn vừa phân loại trở thành sự cố sẽ gửi cho bạn lần tiếp theo. +Đã staring tại một lỗi trên [trang Lỗi](/vi/cloud/errors)? Mỗi hàng ở đó có nút **+ alert** mở cùng biểu mẫu này được điền sẵn để bắt lỗi chính xác đó lại, vì vậy sự cố bạn vừa phân loại trở thành sự cố sẽ gửi cho bạn lần tiếp theo. **Nơi tìm thấy nó:** Cảnh báo nằm ở `//alerts`. Tạo, chỉnh sửa, xóa và kiểm tra quy tắc cần **`alerts:write`**; `alerts:read` là đủ để xem. Bộ chọn người nhận liệt kê các thành viên của tổ chức bạn theo tên, vì vậy bạn có thể gửi thông báo cho một người mà không cần rời khỏi biểu mẫu. @@ -53,11 +53,11 @@ Một phép đo xấu không nên làm bạn thức dậy. Bộ lọc nhiễu ** ## Điều gì xảy ra khi một cảnh báo kích hoạt -Một vi phạm mở một **sự cố** và gửi thông báo cho các kênh của bạn một lần. Từ đó nhóm của bạn xác nhận nó, gán một chủ sở hữu, thảo luận nó, và giải quyết nó, tất cả so với một bản ghi sạch và được ghi. Quy trình phân loại đó có nhà riêng: xem [Sự cố](/vi/agenteye/incidents). +Một vi phạm mở một **sự cố** và gửi thông báo cho các kênh của bạn một lần. Từ đó nhóm của bạn xác nhận nó, gán một chủ sở hữu, thảo luận nó, và giải quyết nó, tất cả so với một bản ghi sạch và được ghi. Quy trình phân loại đó có nhà riêng: xem [Sự cố](/vi/cloud/incidents). ## Liên quan -- [Sự cố](/vi/agenteye/incidents): theo dõi một cảnh báo kích hoạt từ mở đến xác nhận đến đã giải quyết. -- [Theo dõi lỗi](/vi/agenteye/error-tracking): nhóm các lỗi agent và quảng bá một lỗi thành cảnh báo chỉ bằng một cú nhấp chuột. -- [Bảng điều khiển](/vi/agenteye/dashboards): theo dõi các bảng chia sẻ mà các ngưỡng bạn cảnh báo đến từ. -- [CLI và agents](/vi/agenteye/cli-and-agents): tạo cảnh báo và xác nhận sự cố từ terminal của bạn, hoặc script chúng vào CI. \ No newline at end of file +- [Sự cố](/vi/cloud/incidents): theo dõi một cảnh báo kích hoạt từ mở đến xác nhận đến đã giải quyết. +- [Theo dõi lỗi](/vi/cloud/errors): nhóm các lỗi agent và quảng bá một lỗi thành cảnh báo chỉ bằng một cú nhấp chuột. +- [Bảng điều khiển](/vi/cloud/dashboards): theo dõi các bảng chia sẻ mà các ngưỡng bạn cảnh báo đến từ. +- [CLI và agents](/vi/cloud/cli): tạo cảnh báo và xác nhận sự cố từ terminal của bạn, hoặc script chúng vào CI. \ No newline at end of file diff --git a/docs/vi/agenteye/assistant.mdx b/docs/vi/cloud/assistant.mdx similarity index 84% rename from docs/vi/agenteye/assistant.mdx rename to docs/vi/cloud/assistant.mdx index 512891de..4249a2cb 100644 --- a/docs/vi/agenteye/assistant.mdx +++ b/docs/vi/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "Đặt câu hỏi cho dữ liệu agent của bạn bằng tiếng --- -Đặt câu hỏi cho dữ liệu agent của bạn bằng tiếng Anh thuần túy và nhận được câu trả lời có liên kết trực tiếp đến bằng chứng. Không cần viết SQL, không cần tìm kiếm trong các bảng điều khiển — trợ lý **Failproof AI Observability** là cách nhanh nhất để bất kỳ ai trong nhóm của bạn có thể nhận được câu trả lời về các agent của bạn. +Đặt câu hỏi cho dữ liệu agent của bạn bằng tiếng Anh thuần túy và nhận được câu trả lời có liên kết trực tiếp đến bằng chứng. Không cần viết SQL, không cần tìm kiếm trong các bảng điều khiển — trợ lý **FailproofAI Cloud** là cách nhanh nhất để bất kỳ ai trong nhóm của bạn có thể nhận được câu trả lời về các agent của bạn. -![Trợ lý Failproof AI Observability trả lời một câu hỏi bằng tiếng Anh thuần túy bên trong bảng điều khiển, hiển thị bảng Hoạt động Agent trực tiếp, phân tích sử dụng mô hình theo từng agent và các điểm chính, cùng với các truy vấn mà nó chạy được hiển thị inline](/agenteye/images/assistant.png) +![Trợ lý FailproofAI Cloud trả lời một câu hỏi bằng tiếng Anh thuần túy bên trong bảng điều khiển, hiển thị bảng Hoạt động Agent trực tiếp, phân tích sử dụng mô hình theo từng agent và các điểm chính, cùng với các truy vấn mà nó chạy được hiển thị inline](/cloud/images/assistant.png) *Đặt câu hỏi bằng tiếng Anh thuần túy và nhận được câu trả lời được xây dựng từ dữ liệu của riêng bạn. Ở đây nó phân tích những agent nào bận rộn nhất và những mô hình nào họ sử dụng, đồng thời hiển thị các truy vấn mà nó chạy để bạn có thể xác minh từng con số.* Không có gì phải học. Mở cuộc trò chuyện, gõ những gì bạn muốn biết, và theo dõi các liên kết mà nó cung cấp: @@ -36,7 +36,7 @@ Khi một câu trả lời xứng đáng được giữ, yêu cầu trợ lý l Trên trang **Truy vấn** nó đi xa hơn một bước và trở thành tác giả SQL: mô tả truy vấn bạn muốn ("hiển thị tỷ lệ lỗi theo agent cho 7 ngày qua") và nó sẽ phát trực tiếp SQL vào trình chỉnh sửa, mở chế độ diff để bạn có thể **Chấp nhận** hoặc **Từ chối** thay đổi trước khi nó được áp dụng. -![Trang Truy vấn Observability và trình chỉnh sửa SQL của nó](/agenteye/images/query-lab.png) +![Trang Truy vấn FailproofAI Cloud và trình chỉnh sửa SQL của nó](/cloud/images/query-lab.png) *Trang Truy vấn: trình chỉnh sửa này là nơi trợ lý phát một bản nháp truy vấn chỉ đọc cho bạn chấp nhận hoặc từ chối.* Soạn SQL bằng cách hỏi ở đây sử dụng quyền `queries:run`, quyền giống như quyền đằng sau nút **Chạy** của trình chỉnh sửa. Chat ở mọi nơi khác cần `agent:use`. @@ -49,7 +49,7 @@ Bạn có thể mở trợ lý cho tất cả mọi người mà không lo lắn - **Mọi lần ghi đều chờ bạn.** Các truy vấn và bảng điều khiển đã lưu chỉ được tạo sau khi bạn nhấp Phê duyệt một cách rõ ràng, và không có cài đặt nào tắt cổng này. - **Nó không bao giờ có thể xóa bất cứ điều gì.** Không có công cụ xóa nào được hiển thị và trợ lý không có quyền xóa. Các lần xóa vẫn nằm trong tay bạn, trên bảng điều khiển. - **Nó ở bên trong tổ chức của bạn.** Trợ lý chỉ khi nào cũng chỉ nhìn thấy tổ chức bạn đang xem hiện tại. -- **Các câu hỏi của bạn vẫn là của bạn.** Lời nhắc và câu trả lời sống trong cơ sở dữ liệu Observability riêng của bạn; phân tích sản phẩm chỉ ghi lại siêu dữ liệu sử dụng, không bao giờ văn bản lời nhắc của bạn. +- **Các câu hỏi của bạn vẫn là của bạn.** Lời nhắc và câu trả lời sống trong cơ sở dữ liệu FailproofAI Cloud riêng của bạn; phân tích sản phẩm chỉ ghi lại siêu dữ liệu sử dụng, không bao giờ văn bản lời nhắc của bạn. ## Nơi tìm nó @@ -57,7 +57,7 @@ Trợ lý nằm dọc theo cạnh bên phải của mọi trang dưới tổ ch ## Liên quan -- [CLI and agents](/vi/agenteye/cli-and-agents) -- [Queries](/vi/agenteye/queries) -- [Dashboards](/vi/agenteye/dashboards) -- [Evaluation suite](/vi/agenteye/evaluation-suite) \ No newline at end of file +- [CLI and agents](/vi/cloud/cli) +- [Queries](/vi/cloud/queries) +- [Dashboards](/vi/cloud/dashboards) +- [Evaluation suite](/vi/cloud/evaluators) \ No newline at end of file diff --git a/docs/vi/agenteye/audits.mdx b/docs/vi/cloud/audits.mdx similarity index 81% rename from docs/vi/agenteye/audits.mdx rename to docs/vi/cloud/audits.mdx index 2c6e9e1f..095ea6b9 100644 --- a/docs/vi/agenteye/audits.mdx +++ b/docs/vi/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "Audits: trợ lý phân tích độ tin cậy tự động của bạn" -description: "Failproof AI Observability tìm kiếm các lỗi mà bạn chưa bao giờ viết quy tắc cho chúng và cung cấp cho bạn danh sách việc cần làm được xếp hạng, có bằng chứng chính xác về những gì cần sửa." +description: "FailproofAI Cloud tìm kiếm các lỗi mà bạn chưa bao giờ viết quy tắc cho chúng và cung cấp cho bạn danh sách việc cần làm được xếp hạng, có bằng chứng chính xác về những gì cần sửa." --- -Failproof AI Observability tìm kiếm các lỗi mà bạn chưa bao giờ viết quy tắc cho chúng và cung cấp cho bạn danh sách việc cần làm được xếp hạng, có bằng chứng chính xác về những gì cần sửa. Nó giống như có một nhà phân tích duyệt qua nhật ký của bạn mỗi tối, rồi để lại danh sách ngắn gọn trên bàn của bạn vào sáng hôm sau. +FailproofAI Cloud tìm kiếm các lỗi mà bạn chưa bao giờ viết quy tắc cho chúng và cung cấp cho bạn danh sách việc cần làm được xếp hạng, có bằng chứng chính xác về những gì cần sửa. Nó giống như có một nhà phân tích duyệt qua nhật ký của bạn mỗi tối, rồi để lại danh sách ngắn gọn trên bàn của bạn vào sáng hôm sau.
    @@ -12,7 +12,7 @@ Failproof AI Observability tìm kiếm các lỗi mà bạn chưa bao giờ vi *Một bài tour hai phút: từ một lần chạy theo lịch đến một bản sửa mà bạn có thể thực hiện.* -![Trang Audits: các công việc định kỳ quét các phiên của bạn tìm kiếm các mẫu lỗi, mỗi công việc có lịch trình và độ nhạy cảm](/agenteye/images/audits.png) +![Trang Audits: các công việc định kỳ quét các phiên của bạn tìm kiếm các mẫu lỗi, mỗi công việc có lịch trình và độ nhạy cảm](/cloud/images/audits.png) *Mỗi audit là một công việc định kỳ khai thác các phiên của bạn và viết các khuyến nghị được xếp hạng, có bằng chứng.* ## Ngừng đoán xem cần sửa gì tiếp theo @@ -48,7 +48,7 @@ Audits nằm trong bảng điều khiển tại **`//audits`** (thanh ## Liên quan -- [Alerts](/vi/agenteye/alerts): nhận thông báo thời điểm ngưỡng bạn đã biết được vượt qua. -- [Evaluations](/vi/agenteye/evaluations): đánh điểm mỗi lần chạy để các hồi quy chất lượng tự nổi bật. -- [Error tracking](/vi/agenteye/error-tracking): nhóm và theo dõi các lỗi mà agent của bạn ném ra. -- [Incidents](/vi/agenteye/incidents): theo dõi một vấn đề mà audit phát hiện cho đến khi sửa nó. \ No newline at end of file +- [Alerts](/vi/cloud/alerts): nhận thông báo thời điểm ngưỡng bạn đã biết được vượt qua. +- [Evaluations](/vi/cloud/evaluations): đánh điểm mỗi lần chạy để các hồi quy chất lượng tự nổi bật. +- [Error tracking](/vi/cloud/errors): nhóm và theo dõi các lỗi mà agent của bạn ném ra. +- [Incidents](/vi/cloud/incidents): theo dõi một vấn đề mà audit phát hiện cho đến khi sửa nó. \ No newline at end of file diff --git a/docs/vi/cloud/capture.mdx b/docs/vi/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/vi/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/vi/agenteye/cli-recipes.mdx b/docs/vi/cloud/cli-recipes.mdx similarity index 91% rename from docs/vi/agenteye/cli-recipes.mdx rename to docs/vi/cloud/cli-recipes.mdx index 12178479..ba7d0959 100644 --- a/docs/vi/agenteye/cli-recipes.mdx +++ b/docs/vi/cloud/cli-recipes.mdx @@ -3,9 +3,9 @@ title: "Công thức CLI cho agents" description: "Sao chép các mẫu truy vấn và công thức jq giúp chuyển dữ liệu phiên, sự kiện và đánh giá thành thứ gì đó mà script hoặc coding agent có thể tự động hóa." --- -Pull dữ liệu phiên, sự kiện và đánh giá (cũng như kích hoạt lại các đánh giá) trực tiếp từ script hoặc coding agent, với JSON sạch trên stdout có thể pipe trực tiếp vào `jq`. Những công thức này biến dữ liệu Failproof AI Observability thành thứ gì đó mà người dùng terminal hoặc AI coding agent (Claude Code, Cursor) có thể truy vấn và tự động hóa, mà không cần click qua dashboard. +Pull dữ liệu phiên, sự kiện và đánh giá (cũng như kích hoạt lại các đánh giá) trực tiếp từ script hoặc coding agent, với JSON sạch trên stdout có thể pipe trực tiếp vào `jq`. Những công thức này biến dữ liệu FailproofAI Cloud thành thứ gì đó mà người dùng terminal hoặc AI coding agent (Claude Code, Cursor) có thể truy vấn và tự động hóa, mà không cần click qua dashboard. -Các mẫu bên dưới đã sẵn sàng để sao chép cho Failproof AI Observability CLI (`agenteye`). Để cài đặt, xác thực và danh sách tùy chọn đầy đủ, hãy xem [CLI](/vi/agenteye/cli); chạy `agenteye -h` hoặc `agenteye -h` để xem trợ giúp tích hợp. +Các mẫu bên dưới đã sẵn sàng để sao chép cho FailproofAI Cloud CLI (`agenteye`). Để cài đặt, xác thực và danh sách tùy chọn đầy đủ, hãy xem [CLI](/vi/cloud/cli); chạy `agenteye -h` hoặc `agenteye -h` để xem trợ giúp tích hợp. ## Quy tắc vàng @@ -172,7 +172,7 @@ esac ## Các bước tiếp theo -- [CLI](/vi/agenteye/cli): cài đặt, xác thực và tham chiếu tùy chọn đầy đủ cho mỗi lệnh. -- [CLI agent skill](/vi/agenteye/cli-skill): đóng gói những công thức này dưới dạng kỹ năng mà coding agent của bạn có thể tải. -- [API keys](/vi/agenteye/api-keys): tạo và xác định phạm vi các khóa mà CLI, SDK và collector xác thực bằng. -- [Python SDK](/vi/agenteye/python-sdk): gửi các sự kiện vào Failproof AI Observability để có dữ liệu để những công thức này truy vấn. \ No newline at end of file +- [CLI](/vi/cloud/cli): cài đặt, xác thực và tham chiếu tùy chọn đầy đủ cho mỗi lệnh. +- [CLI agent skill](/vi/cloud/agent-skills): đóng gói những công thức này dưới dạng kỹ năng mà coding agent của bạn có thể tải. +- [API keys](/vi/cloud/access): tạo và xác định phạm vi các khóa mà CLI, SDK và collector xác thực bằng. +- [Python SDK](/vi/cloud/sdk): gửi các sự kiện vào FailproofAI Cloud để có dữ liệu để những công thức này truy vấn. \ No newline at end of file diff --git a/docs/vi/agenteye/cli.mdx b/docs/vi/cloud/cli.mdx similarity index 91% rename from docs/vi/agenteye/cli.mdx rename to docs/vi/cloud/cli.mdx index ae75a77b..139b2479 100644 --- a/docs/vi/agenteye/cli.mdx +++ b/docs/vi/cloud/cli.mdx @@ -1,9 +1,9 @@ --- title: "CLI" -description: "Điều khiển toàn bộ Failproof AI Observability từ terminal hoặc script: không cần quay vòng bảng điều khiển." +description: "Điều khiển toàn bộ FailproofAI Cloud từ terminal hoặc script: không cần quay vòng bảng điều khiển." --- -Điều khiển toàn bộ Failproof AI Observability từ terminal hoặc script: không cần quay vòng bảng điều khiển. CLI `agenteye` truy vấn dữ liệu của bạn (phiên, nhật ký sự kiện, đánh giá) và quản lý tổ chức (khóa API, người dùng, cài đặt, cảnh báo, sự cố, truy vấn đã lưu), vì vậy hãy sử dụng nó khi muốn tự động hóa một kiểm tra, tích hợp Observability vào CI, hoặc cho một tác nhân mã hóa kiểm tra sản xuất. Mọi lệnh đều hỗ trợ cờ `--json`, vì vậy nó hoạt động như nhau cho bạn ở dòng lệnh hoặc cho một tác nhân mã hóa (Claude Code, Cursor) thực thi và phân tích kết quả. +Điều khiển toàn bộ FailproofAI Cloud từ terminal hoặc script: không cần quay vòng bảng điều khiển. CLI `agenteye` truy vấn dữ liệu của bạn (phiên, nhật ký sự kiện, đánh giá) và quản lý tổ chức (khóa API, người dùng, cài đặt, cảnh báo, sự cố, truy vấn đã lưu), vì vậy hãy sử dụng nó khi muốn tự động hóa một kiểm tra, tích hợp FailproofAI Cloud vào CI, hoặc cho một tác nhân mã hóa kiểm tra sản xuất. Mọi lệnh đều hỗ trợ cờ `--json`, vì vậy nó hoạt động như nhau cho bạn ở dòng lệnh hoặc cho một tác nhân mã hóa (Claude Code, Cursor) thực thi và phân tích kết quả. Với một nhị phân bạn có thể: @@ -68,7 +68,7 @@ agenteye --version agenteye --help ``` -> **Lưu ý:** SDK Python Failproof AI Observability cũng sử dụng tên phân phối `agenteye`. Cài đặt CLI với `pipx` hoặc `uv tool` (thay vì `pip install` vào một virtualenv chia sẻ) giữ hai cái khác nhau. `pip install agenteye` đơn giản là tốt chỉ khi SDK không được cài đặt trong cùng một môi trường. +> **Lưu ý:** SDK Python FailproofAI Cloud cũng sử dụng tên phân phối `agenteye`. Cài đặt CLI với `pipx` hoặc `uv tool` (thay vì `pip install` vào một virtualenv chia sẻ) giữ hai cái khác nhau. `pip install agenteye` đơn giản là tốt chỉ khi SDK không được cài đặt trong cùng một môi trường. --- @@ -90,7 +90,7 @@ agenteye logout # revoke the session and clear the stored token `whoami` không bao giờ gặp lỗi trên một phiên bị mất hoặc hết hạn; nó báo cáo `logged_in: false` thay thế, vì vậy một script hoặc tác nhân có thể kiểm tra trạng thái xác thực một cách an toàn (nó vẫn có thể thoát khác không nếu không có URL cơ sở được đặt hoặc bảng điều khiển không thể tiếp cận). -**Yêu cầu:** email của bạn phải được phép đăng nhập vào bảng điều khiển (hãy yêu cầu quản trị viên Failproof AI Observability), và bảng điều khiển phải có thể tiếp cận được tại URL cơ sở của nó (xem [Cấu hình](#configuration)). Nếu bạn yêu cầu mã và không có mã nào đến, email của bạn có thể chưa được kích hoạt để truy cập bảng điều khiển. +**Yêu cầu:** email của bạn phải được phép đăng nhập vào bảng điều khiển (hãy yêu cầu quản trị viên FailproofAI Cloud), và bảng điều khiển phải có thể tiếp cận được tại URL cơ sở của nó (xem [Cấu hình](#configuration)). Nếu bạn yêu cầu mã và không có mã nào đến, email của bạn có thể chưa được kích hoạt để truy cập bảng điều khiển. --- @@ -337,13 +337,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | Được xác thực, nhưng tài khoản của bạn thiếu quyền cần thiết (thông báo đặt tên nó) | | 6 | Tài nguyên được yêu cầu không được tìm thấy (ví dụ: id phiên hoặc sự cố không xác định) | -Những điều này làm cho CLI an toàn để viết kịch bản: một tác nhân mã hóa có thể nhánh trên `4` để nhắc bạn xác thực lại, hoặc `5` để bề mặt quyền bị thiếu. Xem [Công thức CLI cho tác nhân](/vi/agenteye/cli-recipes) cho mẫu xử lý mã thoát và hình dạng đầu ra JSON. +Những điều này làm cho CLI an toàn để viết kịch bản: một tác nhân mã hóa có thể nhánh trên `4` để nhắc bạn xác thực lại, hoặc `5` để bề mặt quyền bị thiếu. Xem [Công thức CLI cho tác nhân](/vi/cloud/cli-recipes) cho mẫu xử lý mã thoát và hình dạng đầu ra JSON. --- ## Bước tiếp theo -- **[Công thức CLI cho tác nhân](/vi/agenteye/cli-recipes)**: sao chép - dán mẫu truy vấn, `jq` một-dòng, `--fields` hình chiếu, xử lý mã thoát và hình dạng đầu ra JSON, được viết cho các tác nhân mã hóa điều khiển CLI. -- **[Kỹ năng tác nhân CLI](/vi/agenteye/cli-skill)**: gói CLI này dưới dạng kỹ năng Claude Code / Codex **installable** để tác nhân mã hóa điều khiển Failproof AI Observability từ các yêu cầu bằng tiếng Anh đơn giản. -- **[Khóa API](/vi/agenteye/api-keys)**: mô hình quyền phía sau `keys create --add …`. -- **[Trợ lý AI](/vi/agenteye/assistant)**: kích hoạt trợ lý mà `agent ask` nói chuyện. \ No newline at end of file +- **[Công thức CLI cho tác nhân](/vi/cloud/cli-recipes)**: sao chép - dán mẫu truy vấn, `jq` một-dòng, `--fields` hình chiếu, xử lý mã thoát và hình dạng đầu ra JSON, được viết cho các tác nhân mã hóa điều khiển CLI. +- **[Kỹ năng tác nhân CLI](/vi/cloud/agent-skills)**: gói CLI này dưới dạng kỹ năng Claude Code / Codex **installable** để tác nhân mã hóa điều khiển FailproofAI Cloud từ các yêu cầu bằng tiếng Anh đơn giản. +- **[Khóa API](/vi/cloud/access)**: mô hình quyền phía sau `keys create --add …`. +- **[Trợ lý AI](/vi/cloud/assistant)**: kích hoạt trợ lý mà `agent ask` nói chuyện. \ No newline at end of file diff --git a/docs/vi/cloud/connect.mdx b/docs/vi/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/vi/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/vi/agenteye/dashboards.mdx b/docs/vi/cloud/dashboards.mdx similarity index 71% rename from docs/vi/agenteye/dashboards.mdx rename to docs/vi/cloud/dashboards.mdx index 785923ea..600019be 100644 --- a/docs/vi/agenteye/dashboards.mdx +++ b/docs/vi/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "Biến dữ liệu agent trực tiếp của bạn thành một b Biến dữ liệu agent trực tiếp của bạn thành một bức tranh chung mà toàn bộ team theo dõi. Ghim các truy vấn quan trọng dưới dạng biểu đồ, và mọi người đều nhìn thấy cùng một bộ số liệu một cách rõ ràng, mà không cần chạy lại bất kỳ truy vấn nào. -![A dashboard built from saved queries: an events-per-hour line, an errors-by-type bar, a latency area chart, and tokens-by-model](/agenteye/images/dashboard-fleet.png) +![A dashboard built from saved queries: an events-per-hour line, an errors-by-type bar, a latency area chart, and tokens-by-model](/cloud/images/dashboard-fleet.png) *Một bảng, bốn truy vấn đã lưu: sự kiện mỗi giờ, lỗi theo loại, độ trễ, và token theo mô hình.* @@ -25,15 +25,15 @@ Bạn sẽ tìm thấy các bảng của mình tại `//dashboards`. ## Ghim các truy vấn bạn đã lưu -Mỗi ô bắt đầu như một truy vấn đã lưu. Xây dựng và lưu truy vấn bạn quan tâm trong thư viện [Queries](/vi/agenteye/queries) (các cài đặt sẵn tích hợp cộng với các truy vấn của riêng bạn, trên các sự kiện và đánh giá của bạn), sau đó ghim nó vào bảng điều khiển dưới dạng biểu đồ phù hợp với dữ liệu: một **line** cho xu hướng theo thời gian, một **bar** để so sánh các danh mục, một **area** cho khối lượng, hoặc một **pie** để chia nhỏ tỷ lệ. +Mỗi ô bắt đầu như một truy vấn đã lưu. Xây dựng và lưu truy vấn bạn quan tâm trong thư viện [Queries](/vi/cloud/queries) (các cài đặt sẵn tích hợp cộng với các truy vấn của riêng bạn, trên các sự kiện và đánh giá của bạn), sau đó ghim nó vào bảng điều khiển dưới dạng biểu đồ phù hợp với dữ liệu: một **line** cho xu hướng theo thời gian, một **bar** để so sánh các danh mục, một **area** cho khối lượng, hoặc một **pie** để chia nhỏ tỷ lệ. Vì một ô chỉ là truy vấn đã lưu của bạn được hiển thị dưới dạng biểu đồ, không có gì cần giữ đồng bộ bằng tay. Cập nhật truy vấn một lần và mỗi bảng điều khiển sử dụng nó sẽ được cập nhật. ## Theo dõi chất lượng, không chỉ khối lượng -Khối lượng cho bạn biết rằng các agent đang bận rộn. Chất lượng cho bạn biết rằng họ thực sự đang làm công việc. Hướng bảng điều khiển tới [điểm đánh giá](/vi/agenteye/evaluations) của bạn và bạn sẽ nhận được một bảng theo dõi mức độ hoàn thành tốt của các lần chạy theo thời gian, do đó một sự suy giảm chất lượng sẽ hiển thị dưới dạng một dip trên biểu đồ thay vì một bất ngờ từ khách hàng. +Khối lượng cho bạn biết rằng các agent đang bận rộn. Chất lượng cho bạn biết rằng họ thực sự đang làm công việc. Hướng bảng điều khiển tới [điểm đánh giá](/vi/cloud/evaluations) của bạn và bạn sẽ nhận được một bảng theo dõi mức độ hoàn thành tốt của các lần chạy theo thời gian, do đó một sự suy giảm chất lượng sẽ hiển thị dưới dạng một dip trên biểu đồ thay vì một bất ngờ từ khách hàng. -![A quality-focused dashboard built from saved evaluation queries](/agenteye/images/dashboard-quality.png) +![A quality-focused dashboard built from saved evaluation queries](/cloud/images/dashboard-quality.png) *Một bảng chất lượng giữ điểm đánh giá của bạn ở vị trí trung tâm, ngay bên cạnh các con số hoạt động.* @@ -41,6 +41,6 @@ Giữ một bảng hoạt động và một bảng chất lượng cạnh nhau v ## Liên quan -- [Queries](/vi/agenteye/queries): xây dựng và lưu các truy vấn trở thành các ô của bạn. -- [Evaluations](/vi/agenteye/evaluations): đánh giá các lần chạy của bạn để bạn có thể vẽ biểu đồ chất lượng theo thời gian. -- [Alerts](/vi/agenteye/alerts): biến một ngưỡng trên bất kỳ một trong những số liệu này thành một trang. \ No newline at end of file +- [Queries](/vi/cloud/queries): xây dựng và lưu các truy vấn trở thành các ô của bạn. +- [Evaluations](/vi/cloud/evaluations): đánh giá các lần chạy của bạn để bạn có thể vẽ biểu đồ chất lượng theo thời gian. +- [Alerts](/vi/cloud/alerts): biến một ngưỡng trên bất kỳ một trong những số liệu này thành một trang. \ No newline at end of file diff --git a/docs/vi/agenteye/error-tracking.mdx b/docs/vi/cloud/errors.mdx similarity index 66% rename from docs/vi/agenteye/error-tracking.mdx rename to docs/vi/cloud/errors.mdx index 0a17c586..035be5ef 100644 --- a/docs/vi/agenteye/error-tracking.mdx +++ b/docs/vi/cloud/errors.mdx @@ -6,14 +6,14 @@ description: "Xem mọi lỗi mà agents của bạn tạo ra ở một nơi, đ Xem mọi lỗi mà agents của bạn tạo ra ở một nơi, được nhóm lại để một loạt lỗi ồn ào hiển thị như một vấn đề duy nhất. Bạn có một đường dẫn một lần bấm từ "có thứ gì đó bị lỗi" đến chính xác lần chạy bị hỏng, mà không cần cuộn qua nguồn cấp dữ liệu trực tiếp để tìm nó. -![Trang Lỗi: một biểu đồ cột của các lỗi theo thời gian ở trên các hàng lỗi màu đỏ được nhóm lại, mỗi hàng có nút "+ cảnh báo" một lần bấm](/agenteye/images/errors.png) +![Trang Lỗi: một biểu đồ cột của các lỗi theo thời gian ở trên các hàng lỗi màu đỏ được nhóm lại, mỗi hàng có nút "+ cảnh báo" một lần bấm](/cloud/images/errors.png) *Trang Lỗi: một biểu đồ cột của các lỗi theo thời gian, với các lỗi lặp lại được thu gọn thành một hàng cho mỗi sự cố.* ## Mọi lỗi, đã được thu thập cho bạn Khi một agent bị lỗi, bạn không nên phải cuộn qua luồng sự kiện trực tiếp hy vọng bắt được các hàng màu đỏ trước khi chúng cuộn đi. Trang **Lỗi** làm việc thu thập cho bạn. Nó kéo tất cả những gì bảng điều khiển sẽ tô màu đỏ vào một bề mặt phân loại duy nhất, vì vậy điều đầu tiên bạn thấy là những gì đang bị lỗi, không phải nơi để tìm kiếm nó. -Và nó bắt được nhiều hơn những cái hiển nhiên. Bên cạnh các sự kiện `error` rõ ràng, Failproof AI Observability cũng hiển thị những lỗi yên tĩnh: bất kỳ `tool_result`, `hook_completed`, hoặc `agent_end` nào có payload chứa lỗi sẽ xuất hiện ở đây. Một công cụ trả về lỗi, hoặc một hook thoát không tốt, không còn bỏ qua bạn chỉ vì không có gì ném ra một ngoại lệ to tiếng. +Và nó bắt được nhiều hơn những cái hiển nhiên. Bên cạnh các sự kiện `error` rõ ràng, FailproofAI Cloud cũng hiển thị những lỗi yên tĩnh: bất kỳ `tool_result`, `hook_completed`, hoặc `agent_end` nào có payload chứa lỗi sẽ xuất hiện ở đây. Một công cụ trả về lỗi, hoặc một hook thoát không tốt, không còn bỏ qua bạn chỉ vì không có gì ném ra một ngoại lệ to tiếng. Trên cùng, một biểu đồ cột vẽ các lỗi theo thời gian. Một cái nhìn sẽ cho bạn biết liệu đây là một dòng nền ổn định hay một loạt bắt đầu vài phút trước, vì vậy bạn biết ngay lập tức xem có nên bỏ công việc của bạn hay không. @@ -23,19 +23,19 @@ Giống như mọi bề mặt observe, trang Lỗi được phạm vi để tổ Một phụ thuộc bị hỏng có thể kích hoạt cùng một lỗi hàng trăm lần một phút. Để lại ở trạng thái thô, đó là một bức tường gần như các dòng giống hệt nhau cô lập điều duy nhất mà bạn thực sự cần thấy. -Failproof AI Observability thu gọn các lỗi lặp lại có cùng phiên và loại lỗi thành một hàng duy nhất. Một loạt đọc như một sự cố duy nhất. Bạn kết thúc việc đếm các vấn đề, không phải các dòng nhật ký, và tín hiệu quan trọng vẫn ở trên cùng thay vì bị chìm dưới khối lượng của chính nó. +FailproofAI Cloud thu gọn các lỗi lặp lại có cùng phiên và loại lỗi thành một hàng duy nhất. Một loạt đọc như một sự cố duy nhất. Bạn kết thúc việc đếm các vấn đề, không phải các dòng nhật ký, và tín hiệu quan trọng vẫn ở trên cùng thay vì bị chìm dưới khối lượng của chính nó. ## Từ "có thứ gì đó bị lỗi" đến sự kiện chính xác Nhấp vào bất kỳ hàng nào để hạ cánh thẳng bên trong phiên của lần chạy đó, được định vị trên sự kiện chính xác bị lỗi. Không sao chép ID phiên, không cuộn để tìm kiếm thời điểm nó bị lỗi: bạn hạ cánh đúng trên nó, với toàn bộ biểu đồ thực thi một cái nhìn mắt xa vì vậy bạn có thể thấy agent đã làm gì ở những khoảnh khắc trước khi nó bị hỏng. -Nếu bạn có `alerts:write`, mọi hàng cũng có nút **+ cảnh báo**. Nhấp vào nó và Observability mở một quy tắc cảnh báo mới đã được điền để bắt cùng một lỗi lần nữa. Sự cố bạn vừa phân loại trở thành cái sẽ trang báo bạn lần tiếp theo, thay vì làm bạn ngạc nhiên hai lần. +Nếu bạn có `alerts:write`, mọi hàng cũng có nút **+ cảnh báo**. Nhấp vào nó và FailproofAI Cloud mở một quy tắc cảnh báo mới đã được điền để bắt cùng một lỗi lần nữa. Sự cố bạn vừa phân loại trở thành cái sẽ trang báo bạn lần tiếp theo, thay vì làm bạn ngạc nhiên hai lần. **Nơi tìm thấy nó:** trang **Lỗi** nằm trong phần observe của bảng điều khiển, tại `//errors`. ## Liên quan -- [Cảnh báo](/vi/agenteye/alerts): biến bất kỳ lỗi nào thành một quy tắc trang báo. -- [Sự cố](/vi/agenteye/incidents): theo dõi một cảnh báo được kích hoạt từ mở đến đã giải quyết. -- [Phiên](/vi/agenteye/sessions): mở toàn bộ lần chạy đằng sau bất kỳ lỗi nào. -- [Kiểm toán](/vi/agenteye/audits): cho phép Observability tìm ra các mô hình lỗi trên các lần chạy của bạn cho bạn. \ No newline at end of file +- [Cảnh báo](/vi/cloud/alerts): biến bất kỳ lỗi nào thành một quy tắc trang báo. +- [Sự cố](/vi/cloud/incidents): theo dõi một cảnh báo được kích hoạt từ mở đến đã giải quyết. +- [Phiên](/vi/cloud/sessions): mở toàn bộ lần chạy đằng sau bất kỳ lỗi nào. +- [Kiểm toán](/vi/cloud/audits): cho phép FailproofAI Cloud tìm ra các mô hình lỗi trên các lần chạy của bạn cho bạn. \ No newline at end of file diff --git a/docs/vi/agenteye/evaluations.mdx b/docs/vi/cloud/evaluations.mdx similarity index 68% rename from docs/vi/agenteye/evaluations.mdx rename to docs/vi/cloud/evaluations.mdx index c36dab0c..40882610 100644 --- a/docs/vi/agenteye/evaluations.mdx +++ b/docs/vi/cloud/evaluations.mdx @@ -4,15 +4,15 @@ description: "Các vấn đề chất lượng được phát hiện ngay bây g --- -Các vấn đề chất lượng được phát hiện ngay bây giờ, thay vì bạn nghe về chúng từ khiếu nại của người dùng. Kết nối dịch vụ chấm điểm của riêng bạn một lần và Failproof AI Observability tự động đánh giá mọi lần chạy hoàn tất, vì vậy một sự suy giảm trong hữu ích hoặc sự tăng đột biến trong ảo giác sẽ hiển thị trên chính nó, trước khi khách hàng cảm nhận được nó. +Các vấn đề chất lượng được phát hiện ngay bây giờ, thay vì bạn nghe về chúng từ khiếu nại của người dùng. Kết nối dịch vụ chấm điểm của riêng bạn một lần và FailproofAI Cloud tự động đánh giá mọi lần chạy hoàn tất, vì vậy một sự suy giảm trong hữu ích hoặc sự tăng đột biến trong ảo giác sẽ hiển thị trên chính nó, trước khi khách hàng cảm nhận được nó. -![Lưới phiên với cột điểm: mỗi lần chạy mang theo huy hiệu trạng thái đánh giá và huy hiệu có màu mã hữu ích, tính xác thực và hiệu quả công cụ](/agenteye/images/sessions-list.png) +![Lưới phiên với cột điểm: mỗi lần chạy mang theo huy hiệu trạng thái đánh giá và huy hiệu có màu mã hữu ích, tính xác thực và hiệu quả công cụ](/cloud/images/sessions-list.png) *Mỗi lần chạy trên lưới phiên đều có điểm của nó; các huy hiệu đỏ, vàng và xanh làm cho những lần chạy yếu nổi bật mà không cần bạn mở một bảng điểm duy nhất.* ## Dừng lấy mẫu các lần chạy bằng tay -Bạn thường kiểm tra một số lần chạy và hy vọng phần còn lại đều ổn. Bây giờ mọi phiên hoàn tất đều được chấm điểm ngay khi hoàn tất, trên các chiều mà bạn quan tâm: hữu ích, hiệu quả công cụ, tính xác thực, an toàn, bất cứ tiêu chuẩn chất lượng nào của bạn. Bạn xác định các khóa điểm; Failproof AI Observability lưu trữ, theo dõi xu hướng và hiển thị bất cứ thứ gì bộ đánh giá của bạn gửi lại. Không có lần chạy nào bị bỏ qua mà không được chấm điểm, và bạn sẽ không còn biết về một sự suy thoái từ một vé hỗ trợ. +Bạn thường kiểm tra một số lần chạy và hy vọng phần còn lại đều ổn. Bây giờ mọi phiên hoàn tất đều được chấm điểm ngay khi hoàn tất, trên các chiều mà bạn quan tâm: hữu ích, hiệu quả công cụ, tính xác thực, an toàn, bất cứ tiêu chuẩn chất lượng nào của bạn. Bạn xác định các khóa điểm; FailproofAI Cloud lưu trữ, theo dõi xu hướng và hiển thị bất cứ thứ gì bộ đánh giá của bạn gửi lại. Không có lần chạy nào bị bỏ qua mà không được chấm điểm, và bạn sẽ không còn biết về một sự suy thoái từ một vé hỗ trợ. Điểm được hiển thị trên lưới phiên tại **`//sessions`** (thanh bên → *quan sát* → *phiên*), một cụm huy hiệu trên mỗi hàng. Chỉ muốn những lần chạy không đạt yêu cầu? Lọc lưới theo phạm vi điểm, ví dụ hữu ích dưới 0,5, và kéo lên chính xác những lần chạy đáng đọc. Xem điểm cần quyền `evaluations:read`. @@ -20,7 +20,7 @@ Bạn thường kiểm tra một số lần chạy và hy vọng phần còn l Một con số cho bạn biết một lần chạy là yếu; trang phiên cho bạn biết lý do tại sao. Mở bất kỳ lần chạy nào và thanh bên phải dẫn đầu với tóm tắt tiêu đề, sau đó hiển thị một thanh trên mỗi chiều với lý do của bộ đánh giá của bạn dưới mỗi chiều, vì vậy bạn chuyển từ lần chạy này được chấm điểm 0,4 về tính xác thực sang yêu cầu chính xác mà nó sai trong vài giây. -![Thanh bên phải của phiên: tóm tắt đánh giá ở trên, sau đó là các thanh điểm trên mỗi chiều mỗi cái có một dòng lý do, bên cạnh dòng thời gian sự kiện đầy đủ](/agenteye/images/session-detail.png) +![Thanh bên phải của phiên: tóm tắt đánh giá ở trên, sau đó là các thanh điểm trên mỗi chiều mỗi cái có một dòng lý do, bên cạnh dòng thời gian sự kiện đầy đủ](/cloud/images/session-detail.png) *Chế độ xem chi tiết phiên: tóm tắt, các thanh điểm trên mỗi chiều và lý do đằng sau mỗi điểm, ngay cạnh dòng thời gian sự kiện của lần chạy.* @@ -30,7 +30,7 @@ Một con số cho bạn biết một lần chạy là yếu; trang phiên cho b Một lần chạy được chấm điểm thấp là tiếng ồn; toàn bộ nhóm trượt là một tín hiệu. Các bảng điều khiển đã lưu biến điểm của bạn thành xu hướng mà bạn có thể theo dõi ngay: trung bình hữu ích tuần này so với tuần trước, trên mỗi đại lý, trên mỗi môi trường. -![Bảng điều khiển chất lượng: các thanh điểm trung bình trên mỗi chiều bộ đánh giá cùng với xu hướng theo thời gian](/agenteye/images/dashboard-quality.png) +![Bảng điều khiển chất lượng: các thanh điểm trung bình trên mỗi chiều bộ đánh giá cùng với xu hướng theo thời gian](/cloud/images/dashboard-quality.png) *Một bảng điều khiển chất lượng đã lưu theo dõi xu hướng các khóa điểm mà bạn đặc trưng, vì vậy một sự trôi dạt chậm là rõ ràng lâu trước khi nó trở thành sự cố.* @@ -38,14 +38,14 @@ Bảng điều khiển nằm tại **`//dashboards`** (thanh bên → ## Kết nối một bộ đánh giá một lần -Chấm điểm là tùy chọn và vẫn hoàn toàn tắt cho đến khi bạn chỉ Failproof AI Observability vào một công cụ ghi điểm. Bạn thiết lập một dịch vụ HTTP nhỏ (Observability gửi một tham chiếu hoạt động mà bạn có thể sao chép), đặt hai giá trị trên máy chủ của bạn, và mọi lần chạy từ đó trở đi đều được chấm điểm cho bạn. Toàn bộ hướng dẫn, hợp đồng chấm điểm và SDK nằm trong hướng dẫn sâu. +Chấm điểm là tùy chọn và vẫn hoàn toàn tắt cho đến khi bạn chỉ FailproofAI Cloud vào một công cụ ghi điểm. Bạn thiết lập một dịch vụ HTTP nhỏ (FailproofAI Cloud gửi một tham chiếu hoạt động mà bạn có thể sao chép), đặt hai giá trị trên máy chủ của bạn, và mọi lần chạy từ đó trở đi đều được chấm điểm cho bạn. Toàn bộ hướng dẫn, hợp đồng chấm điểm và SDK nằm trong hướng dẫn sâu. -Không chắc chắn những chiều nào đáng chấm điểm ngay từ đầu? [Kỹ năng đại lý đánh giá](/vi/agenteye/evaluator-skill) có đại lý mã hóa của bạn làm điều đó chống lại các phiên của riêng bạn, sau đó xây dựng và triển khai dịch vụ. +Không chắc chắn những chiều nào đáng chấm điểm ngay từ đầu? [Kỹ năng đại lý đánh giá](/vi/cloud/agent-skills) có đại lý mã hóa của bạn làm điều đó chống lại các phiên của riêng bạn, sau đó xây dựng và triển khai dịch vụ. ## Liên quan -- [Bộ đánh giá](/vi/agenteye/evaluation-suite): kết nối bộ đánh giá của bạn, hợp đồng chấm điểm và SDK. -- [Kỹ năng đại lý đánh giá](/vi/agenteye/evaluator-skill): để đại lý mã hóa chọn các chiều điểm của bạn và xây dựng bộ đánh giá. -- [Phiên](/vi/agenteye/sessions): lưới chạy từng lần nơi xuất hiện điểm. -- [Bảng điều khiển](/vi/agenteye/dashboards): lưu và chia sẻ xu hướng chất lượng trên tổ chức của bạn. -- [Kiểm tra](/vi/agenteye/audits): tính năng chất lượng tự động khác của Observability, để điều tra xuyên phiên. \ No newline at end of file +- [Bộ đánh giá](/vi/cloud/evaluators): kết nối bộ đánh giá của bạn, hợp đồng chấm điểm và SDK. +- [Kỹ năng đại lý đánh giá](/vi/cloud/agent-skills): để đại lý mã hóa chọn các chiều điểm của bạn và xây dựng bộ đánh giá. +- [Phiên](/vi/cloud/sessions): lưới chạy từng lần nơi xuất hiện điểm. +- [Bảng điều khiển](/vi/cloud/dashboards): lưu và chia sẻ xu hướng chất lượng trên tổ chức của bạn. +- [Kiểm tra](/vi/cloud/audits): tính năng chất lượng tự động khác của FailproofAI Cloud, để điều tra xuyên phiên. \ No newline at end of file diff --git a/docs/vi/agenteye/evaluation-suite.mdx b/docs/vi/cloud/evaluators.mdx similarity index 83% rename from docs/vi/agenteye/evaluation-suite.mdx rename to docs/vi/cloud/evaluators.mdx index 37082f06..5cd30957 100644 --- a/docs/vi/agenteye/evaluation-suite.mdx +++ b/docs/vi/cloud/evaluators.mdx @@ -1,20 +1,20 @@ --- title: "Bộ Công Cụ Đánh Giá" -description: "Failproof AI Observability có thể tự động chấm điểm mọi phiên chạy agent đã hoàn thành về chất lượng: bạn cung cấp một dịch vụ chấm điểm nhỏ, và Observability sẽ xử lý phần còn lại." +description: "FailproofAI Cloud có thể tự động chấm điểm mọi phiên chạy agent đã hoàn thành về chất lượng: bạn cung cấp một dịch vụ chấm điểm nhỏ, và FailproofAI Cloud sẽ xử lý phần còn lại." --- -Failproof AI Observability có thể tự động chấm điểm mọi phiên chạy agent đã hoàn thành về chất lượng: bạn cung cấp một dịch vụ chấm điểm nhỏ, và Observability sẽ xử lý phần còn lại. Sử dụng nó để theo dõi các chiều độ bạn quan tâm (tính hữu ích, hiệu quả công cụ, tính xác thực, bảo mật; bạn lựa chọn), phát hiện sự suy giảm sớm và so sánh các agent hoặc môi trường một cách nhanh chóng. Chấm điểm là tùy chọn: đường dẫn sẽ không hoạt động cho đến khi bạn đặt `EVALUATOR_ENDPOINT` trên máy chủ. +FailproofAI Cloud có thể tự động chấm điểm mọi phiên chạy agent đã hoàn thành về chất lượng: bạn cung cấp một dịch vụ chấm điểm nhỏ, và FailproofAI Cloud sẽ xử lý phần còn lại. Sử dụng nó để theo dõi các chiều độ bạn quan tâm (tính hữu ích, hiệu quả công cụ, tính xác thực, bảo mật; bạn lựa chọn), phát hiện sự suy giảm sớm và so sánh các agent hoặc môi trường một cách nhanh chóng. Chấm điểm là tùy chọn: đường dẫn sẽ không hoạt động cho đến khi bạn đặt `EVALUATOR_ENDPOINT` trên máy chủ. -> **Ghi chú:** Bạn định nghĩa các chiều chấm điểm. Bộ đánh giá của bạn có thể trả về bất kỳ khóa số nào mà nó muốn; Observability lưu trữ, theo dõi xu hướng và hiển thị bất cứ thứ gì bạn gửi lại. +> **Ghi chú:** Bạn định nghĩa các chiều chấm điểm. Bộ đánh giá của bạn có thể trả về bất kỳ khóa số nào mà nó muốn; FailproofAI Cloud lưu trữ, theo dõi xu hướng và hiển thị bất cứ thứ gì bạn gửi lại. ## Tóm tắt nhanh -1. **Viết một bộ chấm điểm.** Thiết lập một dịch vụ HTTP nhỏ đọc bản ghi phiên và trả về điểm số. Observability cung cấp một bản tham khảo hoạt động mà bạn có thể sao chép. Xem [Viết bộ đánh giá với SDK](#writing-an-evaluator-with-the-sdk). -2. **Chỉ đến nó với Observability.** Đặt `EVALUATOR_ENDPOINT` (và một `EVALUATOR_TOKEN` được chia sẻ) trên quy trình máy chủ. +1. **Viết một bộ chấm điểm.** Thiết lập một dịch vụ HTTP nhỏ đọc bản ghi phiên và trả về điểm số. FailproofAI Cloud cung cấp một bản tham khảo hoạt động mà bạn có thể sao chép. Xem [Viết bộ đánh giá với SDK](#writing-an-evaluator-with-the-sdk). +2. **Chỉ đến nó với FailproofAI Cloud.** Đặt `EVALUATOR_ENDPOINT` (và một `EVALUATOR_TOKEN` được chia sẻ) trên quy trình máy chủ. 3. **Theo dõi điểm số.** Mọi phiên hoàn thành được chấm điểm tự động; kết quả hiển thị trên trang chi tiết phiên, lưới phiên và bảng điều khiển đã lưu. -![Chế độ xem chi tiết phiên với bản tóm tắt đánh giá, thanh điểm số từng chiều và văn bản lý do trong thanh bên phải](/agenteye/images/session-detail.png) +![Chế độ xem chi tiết phiên với bản tóm tắt đánh giá, thanh điểm số từng chiều và văn bản lý do trong thanh bên phải](/cloud/images/session-detail.png) *Sau khi bộ đánh giá được định cấu hình, mỗi lần chạy được hoàn thành được chấm điểm và kết quả xuất hiện trong thanh bên phải của phiên: bản tóm tắt ở trên cùng, sau đó là thanh điểm số từng chiều với lý do.* @@ -24,7 +24,7 @@ Failproof AI Observability có thể tự động chấm điểm mọi phiên ch ```mermaid flowchart LR - ING["ingest /events
    agent_end"] --> SRV["Observability server"] + ING["ingest /events
    agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -32,14 +32,14 @@ flowchart LR SRV --> RES["evaluations
    terminal results"] ``` -Khi Failproof AI Observability SDK phát ra sự kiện `agent_end` cho một phiên, máy chủ sẽ lên lịch đánh giá. Sau đó, nó POSTs bản ghi sự kiện đầy đủ tới dịch vụ bộ đánh giá của bạn, dịch vụ này có thể: +Khi FailproofAI Cloud SDK phát ra sự kiện `agent_end` cho một phiên, máy chủ sẽ lên lịch đánh giá. Sau đó, nó POSTs bản ghi sự kiện đầy đủ tới dịch vụ bộ đánh giá của bạn, dịch vụ này có thể: - **Trả về kết quả ngay lập tức** với `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`. Kết quả được thêm vào dòng thời gian đánh giá của phiên. `reasoning` và `summary` là tùy chọn. -- **Trì hoãn** với `{"status":"pending", "job_id":"abc-123"}`. Observability sau đó gọi `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` cho đến khi bộ đánh giá của bạn trả về `{"status":"done", ...}` hoặc `{"status":"error", "error":"..."}`. +- **Trì hoãn** với `{"status":"pending", "job_id":"abc-123"}`. FailproofAI Cloud sau đó gọi `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123` cho đến khi bộ đánh giá của bạn trả về `{"status":"done", ...}` hoặc `{"status":"error", "error":"..."}`. - Tần suất thăm dò được theo công việc: phản hồi `pending` có thể bao gồm `next_poll_secs` để ghi đè; nếu không, Observability sử dụng giá trị `default_poll_interval_secs` từ `GET /config`; nếu không, máy chủ quay lại `EVALUATOR_POLLING_INTERVAL_SECS` (mặc định 10 giây). Tất cả các giá trị được giới hạn trong [1 giây, 1 giờ]. + Tần suất thăm dò được theo công việc: phản hồi `pending` có thể bao gồm `next_poll_secs` để ghi đè; nếu không, FailproofAI Cloud sử dụng giá trị `default_poll_interval_secs` từ `GET /config`; nếu không, máy chủ quay lại `EVALUATOR_POLLING_INTERVAL_SECS` (mặc định 10 giây). Tất cả các giá trị được giới hạn trong [1 giây, 1 giờ]. -Các phiên không bao giờ phát ra `agent_end` (ví dụ: quy trình agent bị sập) cũng có thể được nhận: `GET /config` của bộ đánh giá có thể trả về `{"inactivity_timeout_secs": 1800}`, và Observability sẽ đánh giá bất kỳ phiên nào không hoạt động trong khoảng thời gian đó. Đặt trường thành `null` hoặc bỏ qua nó để tắt dự phòng này. +Các phiên không bao giờ phát ra `agent_end` (ví dụ: quy trình agent bị sập) cũng có thể được nhận: `GET /config` của bộ đánh giá có thể trả về `{"inactivity_timeout_secs": 1800}`, và FailproofAI Cloud sẽ đánh giá bất kỳ phiên nào không hoạt động trong khoảng thời gian đó. Đặt trường thành `null` hoặc bỏ qua nó để tắt dự phòng này. Đường dẫn hoàn toàn không hoạt động khi `EVALUATOR_ENDPOINT` chưa được đặt. @@ -47,7 +47,7 @@ Một phiên có thể tích lũy **nhiều đánh giá terminal theo thời gia Dự phòng không hoạt động cũng tái bật trên các phiên được tiếp tục: nếu các sự kiện mới đến sau một đánh giá terminal trước đó và phiên sau đó không hoạt động quá `inactivity_timeout_secs`, một đánh giá mới được xếp hàng. -Các lỗi tạm thời (5xx, 429, timeout, lỗi mạng) được thử lại với backoff lũy thừa lên đến `EVALUATOR_MAX_ATTEMPTS`; phản hồi 4xx là terminal. Observability an toàn để chạy với nhiều phiên bản máy chủ được mở rộng ngang; công việc được phân vùng để phiên tương tự không bao giờ được gửi hai lần cùng một lúc. +Các lỗi tạm thời (5xx, 429, timeout, lỗi mạng) được thử lại với backoff lũy thừa lên đến `EVALUATOR_MAX_ATTEMPTS`; phản hồi 4xx là terminal. FailproofAI Cloud an toàn để chạy với nhiều phiên bản máy chủ được mở rộng ngang; công việc được phân vùng để phiên tương tự không bao giờ được gửi hai lần cùng một lúc. --- @@ -55,7 +55,7 @@ Các lỗi tạm thời (5xx, 429, timeout, lỗi mạng) được thử lại v Mọi tuyến được xác thực sử dụng **xác thực bearer token**. Cùng một giá trị phải được định cấu hình ở cả hai bên: -- Máy chủ Observability: biến môi trường `EVALUATOR_TOKEN` +- Máy chủ FailproofAI Cloud: biến môi trường `EVALUATOR_TOKEN` - Dịch vụ đánh giá: được định cấu hình theo cách tương tự (SDK `agenteye-evaluator` đọc `EVALUATOR_TOKEN` theo quy ước) Nếu `EVALUATOR_TOKEN` chưa được đặt, máy chủ không gửi tiêu đề `Authorization`; bộ đánh giá sau đó có thể chấp nhận yêu cầu ẩn danh, điều này tốt cho một mạng nội bộ nhưng không được khuyến khích trên internet công cộng. @@ -126,7 +126,7 @@ Máy chủ coi bất kỳ nội dung 2xx khác làm lỗi giao thức và ghi l Bạn không phải triển khai hợp đồng HTTP bằng tay. Gói Python `agenteye-evaluator` cung cấp cho bạn một trình bao bọc FastAPI được gõ xử lý xác thực, định tuyến và các hình dạng yêu cầu/phản hồi cho bạn. -Failproof AI Observability cũng cung cấp một **bộ đánh giá tham khảo hoạt động** chấm điểm `helpfulness`, `tool_efficiency` và `factuality` từ hình dạng của bản ghi. Sao chép nó làm điểm khởi đầu và hoán đổi logic của riêng bạn: một trọng tài LLM, một công cụ quy tắc, bất cứ thứ gì phù hợp với tiêu chuẩn chất lượng của bạn. +FailproofAI Cloud cũng cung cấp một **bộ đánh giá tham khảo hoạt động** chấm điểm `helpfulness`, `tool_efficiency` và `factuality` từ hình dạng của bản ghi. Sao chép nó làm điểm khởi đầu và hoán đổi logic của riêng bạn: một trọng tài LLM, một công cụ quy tắc, bất cứ thứ gì phù hợp với tiêu chuẩn chất lượng của bạn. Bộ đánh giá tối thiểu khả thi: @@ -149,7 +149,7 @@ def run(req: EvalRequest) -> EvalResponse: Phiên bản `app` chạy dưới bất kỳ máy chủ ASGI nào, vì vậy `uvicorn module:app` bắt đầu nó. -Đối với các bộ đánh giá cần phải trì hoãn công việc tốn kém, hãy trả về `JobPending` thay thế và đăng ký trình xử lý `@app.job_lookup`; máy chủ Observability thăm dò `GET /evaluate/{job_id}` cho đến khi bạn trả về trạng thái terminal hoặc giới hạn `EVALUATOR_MAX_POLL_DURATION_SECS` (mặc định 1 giờ) hết hạn. +Đối với các bộ đánh giá cần phải trì hoãn công việc tốn kém, hãy trả về `JobPending` thay thế và đăng ký trình xử lý `@app.job_lookup`; máy chủ FailproofAI Cloud thăm dò `GET /evaluate/{job_id}` cho đến khi bạn trả về trạng thái terminal hoặc giới hạn `EVALUATOR_MAX_POLL_DURATION_SECS` (mặc định 1 giờ) hết hạn. Tài liệu tham khảo API đầy đủ, mô hình không đồng bộ và lược đồ sự kiện được ghi lại trong README của SDK `agenteye-evaluator`. @@ -157,7 +157,7 @@ Tài liệu tham khảo API đầy đủ, mô hình không đồng bộ và lư ## Chạy bộ đánh giá của bạn -Bộ đánh giá là **dịch vụ của bạn** — Failproof AI Observability không cung cấp bộ đánh giá mặc định, vì vậy bạn xây dựng và chạy nó ở bất cứ nơi nào bạn chạy các dịch vụ riêng của mình. Nó chạy dưới bất kỳ máy chủ ASGI nào (ví dụ `uvicorn my_evaluator:app`); phục vụ các tuyến `/health`, `/config` và `/evaluate` từ [hợp đồng HTTP](#http-contract), sau đó chỉ máy chủ tới nó (xem [Định cấu hình máy chủ](#configuring-the-server)). +Bộ đánh giá là **dịch vụ của bạn** — FailproofAI Cloud không cung cấp bộ đánh giá mặc định, vì vậy bạn xây dựng và chạy nó ở bất cứ nơi nào bạn chạy các dịch vụ riêng của mình. Nó chạy dưới bất kỳ máy chủ ASGI nào (ví dụ `uvicorn my_evaluator:app`); phục vụ các tuyến `/health`, `/config` và `/evaluate` từ [hợp đồng HTTP](#http-contract), sau đó chỉ máy chủ tới nó (xem [Định cấu hình máy chủ](#configuring-the-server)). Sau khi bộ đánh giá có thể truy cập được, `GET /health` trả về `{"status":"ok"}`. Sau khi một agent chạy từ đầu đến cuối, `GET /evaluations` trên máy chủ trả về một hàng có `status: "done"` và điểm số bộ đánh giá của bạn tạo ra. @@ -255,7 +255,7 @@ Admin bootstrap (`ADMIN_KEY`, `ADMIN_EMAIL`) tự động nhận những cái n - **`/sessions`**: lưới phiên có thể lọc; cột điểm số hiển thị trạng thái đánh giá và điểm số của mỗi phiên một cách nhanh chóng. - **`/dashboards`**: chế độ xem sức khỏe eval được lưu (xem [Bảng điều khiển](#dashboards) dưới đây). -![Lưới Sessions với viên thuốc trạng thái đánh giá trên mỗi phiên và huy hiệu điểm được mã hóa màu (helpfulness, factuality, tool_efficiency, safety, coherence)](/agenteye/images/sessions-list.png) +![Lưới Sessions với viên thuốc trạng thái đánh giá trên mỗi phiên và huy hiệu điểm được mã hóa màu (helpfulness, factuality, tool_efficiency, safety, coherence)](/cloud/images/sessions-list.png) *Lưới phiên hiển thị trạng thái đánh giá và điểm số của mỗi lần chạy một cách nhanh chóng; huy hiệu đỏ/hổ phách/xanh làm cho điểm số thấp nổi bật.* @@ -272,7 +272,7 @@ Mỗi bảng điều khiển ghim: Mỗi thẻ hiển thị số lượng phiên phù hợp, phân tích done/error/timeout, trung bình của mỗi điểm nổi bật và một sparkline xu hướng nhỏ. Mở bảng điều khiển hiển thị các bảng điều khiển toàn kích thước; **mở trong phiên** hạ bạn vào trang phiên được lọc trước chính xác lát cắt đó. Chỉ số được tính toán phía máy chủ trên toàn bộ bộ phù hợp (thông qua `GET /evaluations/aggregate`), vì vậy các số chính xác thay vì được lấy mẫu. -![Bảng điều khiển sức khỏe eval với thanh điểm trung bình trên mỗi chiều đánh giá, phân tích tool ok-vs-error, công cụ hàng đầu và xu hướng sự kiện mỗi giờ](/agenteye/images/dashboard-quality.png) +![Bảng điều khiển sức khỏe eval với thanh điểm trung bình trên mỗi chiều đánh giá, phân tích tool ok-vs-error, công cụ hàng đầu và xu hướng sự kiện mỗi giờ](/cloud/images/dashboard-quality.png) **Quyền:** xem yêu cầu cả `dashboards:read` và `evaluations:read`; tạo và chỉnh sửa yêu cầu `dashboards:write`; xóa yêu cầu `dashboards:delete`. Admin bootstrap nhận tất cả những cái này tự động. @@ -294,7 +294,7 @@ Mỗi thẻ hiển thị số lượng phiên phù hợp, phân tích done/error ## Các bước tiếp theo -- [Kỹ năng agent đánh giá](/vi/agenteye/evaluator-skill): có một agent mã thiết kế các chiều của bạn chống lại các phiên thực tế và xây dựng dịch vụ này cho bạn. -- [Python SDK](/vi/agenteye/python-sdk): phát ra các sự kiện `agent_end` kích hoạt chấm điểm. -- [Khóa API](/vi/agenteye/api-keys): các quyền `evaluations:read` và `evaluations:trigger`. -- [Kiểm toán](/vi/agenteye/audits): tính năng tự động chất lượng khác của Observability, để xem xét dựa trên chính sách. \ No newline at end of file +- [Kỹ năng agent đánh giá](/vi/cloud/agent-skills): có một agent mã thiết kế các chiều của bạn chống lại các phiên thực tế và xây dựng dịch vụ này cho bạn. +- [Python SDK](/vi/cloud/sdk): phát ra các sự kiện `agent_end` kích hoạt chấm điểm. +- [Khóa API](/vi/cloud/access): các quyền `evaluations:read` và `evaluations:trigger`. +- [Kiểm toán](/vi/cloud/audits): tính năng tự động chất lượng khác của FailproofAI Cloud, để xem xét dựa trên chính sách. \ No newline at end of file diff --git a/docs/vi/agenteye/event-stream.mdx b/docs/vi/cloud/event-stream.mdx similarity index 79% rename from docs/vi/agenteye/event-stream.mdx rename to docs/vi/cloud/event-stream.mdx index 22af3ab9..2ec0abd4 100644 --- a/docs/vi/agenteye/event-stream.mdx +++ b/docs/vi/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "Ngay khi agent của bạn làm gì đó, bạn sẽ thấy nó." Ngay khi agent của bạn làm gì đó, bạn sẽ thấy nó. Event Stream là nhịp đập trực tiếp của bạn trên mọi agent trong production: không chờ đợi, không cần grep log, không cần đoán xem vừa xảy ra điều gì. -![Event Stream trực tiếp: các dòng sự kiện được mã hóa màu sắc hiển thị theo thời gian thực, có thể lọc theo môi trường, agent, phiên, loại sự kiện và tìm kiếm tự do](/agenteye/images/events-stream.png) +![Event Stream trực tiếp: các dòng sự kiện được mã hóa màu sắc hiển thị theo thời gian thực, có thể lọc theo môi trường, agent, phiên, loại sự kiện và tìm kiếm tự do](/cloud/images/events-stream.png) *Mọi sự kiện từ mọi agent trong tổ chức của bạn, sự kiện mới nhất trước, cập nhật khi nó xảy ra.* @@ -37,14 +37,14 @@ Tìm kiếm văn bản tự do đi thẳng đến một tin nhắn, tên tool, h Event Stream là trang chủ tổ chức của bạn. Đăng nhập và nó là bề mặt đầu tiên bạn hạ cánh, tại `//`, vì vậy phân loại bắt đầu ngay khi bạn đến. -Phía sau nó, các agent của bạn phát ra các sự kiện thông qua SDK, bộ sưu tập gửi chúng đến máy chủ Failproof AI Observability của bạn, và stream theo dõi chúng khi chúng đến trong cơ sở hạ tầng bạn kiểm soát. Khi bạn muốn chế độ xem tóm tắt thay vì dấu vết thô, các sự kiện của mỗi run sụp đổ thành một dòng duy nhất trên Sessions, chỉ cách một cú nhấp chuột. +Phía sau nó, các agent của bạn phát ra các sự kiện thông qua SDK, bộ sưu tập gửi chúng đến máy chủ FailproofAI Cloud của bạn, và stream theo dõi chúng khi chúng đến trong cơ sở hạ tầng bạn kiểm soát. Khi bạn muốn chế độ xem tóm tắt thay vì dấu vết thô, các sự kiện của mỗi run sụp đổ thành một dòng duy nhất trên Sessions, chỉ cách một cú nhấp chuột. Đây là nguồn sự thật thô của tất cả các bề mặt quan sát khác được xây dựng, vì vậy khi một số liệu trông sai ở nơi khác, stream là nơi bạn xác nhận những gì thực sự xảy ra. ## Liên quan -- [Sessions](/vi/agenteye/sessions): các sự kiện tương tự tóm tắt thành một dòng cho mỗi run, với một đồ thị thực thi kiểu git. -- [Telemetry](/vi/agenteye/telemetry): những gì các agent của bạn gửi và cách các sự kiện đến stream. -- [Error tracking](/vi/agenteye/error-tracking): một bề mặt phân loại cho mọi thứ đã xảy ra sai. -- [Alerts](/vi/agenteye/alerts): biến bất kỳ ngưỡng nào thành quy tắc tìm kiếm. -- [CLI and agents](/vi/agenteye/cli-and-agents): dấu vết trực tiếp tương tự từ terminal của bạn. \ No newline at end of file +- [Sessions](/vi/cloud/sessions): các sự kiện tương tự tóm tắt thành một dòng cho mỗi run, với một đồ thị thực thi kiểu git. +- [Telemetry](/vi/cloud/performance): những gì các agent của bạn gửi và cách các sự kiện đến stream. +- [Error tracking](/vi/cloud/errors): một bề mặt phân loại cho mọi thứ đã xảy ra sai. +- [Alerts](/vi/cloud/alerts): biến bất kỳ ngưỡng nào thành quy tắc tìm kiếm. +- [CLI and agents](/vi/cloud/cli): dấu vết trực tiếp tương tự từ terminal của bạn. \ No newline at end of file diff --git a/docs/vi/cloud/fleet.mdx b/docs/vi/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/vi/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/vi/agenteye/incidents.mdx b/docs/vi/cloud/incidents.mdx similarity index 86% rename from docs/vi/agenteye/incidents.mdx rename to docs/vi/cloud/incidents.mdx index e39eec06..ab9e01dd 100644 --- a/docs/vi/agenteye/incidents.mdx +++ b/docs/vi/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "Khi một cảnh báo phát động, mọi người có thể th Khi một cảnh báo phát động, câu hỏi đầu tiên luôn là "ai đang xử lý?" Sự cố trả lời nó: ngay lập tức khi có vi phạm, mọi người có thể thấy sự cố đang mở, ai sở hữu nó và chính xác những gì đã xảy ra cho đến nay, với một bản ghi được ghi nhận rõ ràng mà bạn có thể chuyển thẳng cho cuộc họp hậu sự. -![Hộp thư sự cố: thẻ sự cố được liên kết với cảnh báo và được mở thủ công, nhóm theo trạng thái, mỗi thẻ có huy hiệu mức độ nghiêm trọng và người được giao nhiệm vụ](/agenteye/images/incidents.png) +![Hộp thư sự cố: thẻ sự cố được liên kết với cảnh báo và được mở thủ công, nhóm theo trạng thái, mỗi thẻ có huy hiệu mức độ nghiêm trọng và người được giao nhiệm vụ](/cloud/images/incidents.png) *Hộp thư nhóm các sự cố mở theo trạng thái và lọc theo mức độ nghiêm trọng và người được giao nhiệm vụ, để bạn thấy những gì cần con người bây giờ.* ## Biết ai đang xử lý, trong nháy mắt @@ -17,10 +17,10 @@ Không còn "có ai đang xem cái này không?" trong một luồng trò chuy Khi sự cố kết thúc, bạn đã có bản viết. Mở bất kỳ sự cố nào và bạn sẽ nhận được bằng chứng vi phạm, những người được giao nhiệm vụ và người đăng ký của nó, một luồng bình luận để phối hợp tại chỗ, và một dòng thời gian hoạt động chỉ thêm vào. -![Một chế độ xem chi tiết sự cố: cảnh báo cha và tóm tắt vi phạm, những người được giao nhiệm vụ và người đăng ký, một dòng thời gian hoạt động được ghi nhận, và một luồng bình luận](/agenteye/images/incident-detail.png) +![Một chế độ xem chi tiết sự cố: cảnh báo cha và tóm tắt vi phạm, những người được giao nhiệm vụ và người đăng ký, một dòng thời gian hoạt động được ghi nhận, và một luồng bình luận](/cloud/images/incident-detail.png) *Mọi thứ đã xảy ra, theo thứ tự, mỗi dòng được ký bởi người đã làm nó.* -Mỗi hành động (mở, xác nhận, giải quyết, v.v.) được ghi vào dòng thời gian đó và không bao giờ được chỉnh sửa. Mỗi mục được ghi nhận: cho nhà điều hành đã thực hiện nó, theo email, hoặc thành **automated** cho bất kỳ điều gì Failproof AI Observability đã tự làm, như mở sự cố trên vi phạm. Không có gì ẩn danh và không có gì bị mất, vì vậy cuộc họp hậu sự hầu như tự viết. +Mỗi hành động (mở, xác nhận, giải quyết, v.v.) được ghi vào dòng thời gian đó và không bao giờ được chỉnh sửa. Mỗi mục được ghi nhận: cho nhà điều hành đã thực hiện nó, theo email, hoặc thành **automated** cho bất kỳ điều gì FailproofAI Cloud đã tự làm, như mở sự cố trên vi phạm. Không có gì ẩn danh và không có gì bị mất, vì vậy cuộc họp hậu sự hầu như tự viết. ## Sự cố di chuyển như thế nào @@ -45,6 +45,6 @@ Các sự cố nằm tại `//incidents`. Xem cần **`incidents:read` ## Liên quan -- [Alerts](/vi/agenteye/alerts): các quy tắc mở những sự cố này khi một ngưỡng vi phạm. -- [Error tracking](/vi/agenteye/error-tracking): xem mỗi lỗi ở một nơi và nâng một lên thành cảnh báo. -- [Audits](/vi/agenteye/audits): nhà phân tích lên lịch tìm thấy những lỗi không có quy tắc nào đang xem. \ No newline at end of file +- [Alerts](/vi/cloud/alerts): các quy tắc mở những sự cố này khi một ngưỡng vi phạm. +- [Error tracking](/vi/cloud/errors): xem mỗi lỗi ở một nơi và nâng một lên thành cảnh báo. +- [Audits](/vi/cloud/audits): nhà phân tích lên lịch tìm thấy những lỗi không có quy tắc nào đang xem. \ No newline at end of file diff --git a/docs/vi/cloud/managed-policies.mdx b/docs/vi/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/vi/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/vi/agenteye/overview.mdx b/docs/vi/cloud/overview.mdx similarity index 51% rename from docs/vi/agenteye/overview.mdx rename to docs/vi/cloud/overview.mdx index 0092fb58..4fe6dbe8 100644 --- a/docs/vi/agenteye/overview.mdx +++ b/docs/vi/cloud/overview.mdx @@ -1,17 +1,17 @@ --- --- title: "Failproof AI: Quan sát Agents để phát hiện lỗi" -description: "Failproof AI Observability là một nền tảng tự lưu trữ để quan sát, đánh giá và cải thiện các AI agents của bạn trong production." +description: "FailproofAI Cloud là một nền tảng tự lưu trữ để quan sát, đánh giá và cải thiện các AI agents của bạn trong production." --- -Failproof AI Observability là một nền tảng tự lưu trữ để quan sát, đánh giá và cải thiện các AI agents của bạn trong production. Nó ghi lại mọi thứ mà agents của bạn thực hiện (mọi lệnh gọi công cụ, yêu cầu mô hình, hook và lỗi), chấm điểm chất lượng của mỗi lần chạy, và phát hiện những lỗi bạn không biết cần tìm kiếm, tất cả trong một bảng điều khiển chạy bên trong cơ sở hạ tầng của riêng bạn. +FailproofAI Cloud là một nền tảng tự lưu trữ để quan sát, đánh giá và cải thiện các AI agents của bạn trong production. Nó ghi lại mọi thứ mà agents của bạn thực hiện (mọi lệnh gọi công cụ, yêu cầu mô hình, hook và lỗi), chấm điểm chất lượng của mỗi lần chạy, và phát hiện những lỗi bạn không biết cần tìm kiếm, tất cả trong một bảng điều khiển chạy bên trong cơ sở hạ tầng của riêng bạn. -Nếu bạn triển khai AI agents và mệt mỏi với việc đoán tại sao một lần chạy không thành công, đây là trang để bắt đầu. Nó giải thích những gì Failproof AI Observability mang lại cho bạn và cách các phần ghép lại với nhau, trước khi bạn cài đặt bất cứ thứ gì. +Nếu bạn triển khai AI agents và mệt mỏi với việc đoán tại sao một lần chạy không thành công, đây là trang để bắt đầu. Nó giải thích những gì FailproofAI Cloud mang lại cho bạn và cách các phần ghép lại với nhau, trước khi bạn cài đặt bất cứ thứ gì. -> **Failproof AI Observability là một sản phẩm doanh nghiệp từ Failproof AI.** Muốn xem nó hoạt động? Yêu cầu một bản demo: gửi email tới [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +> **FailproofAI Cloud là một sản phẩm doanh nghiệp từ Failproof AI.** Muốn xem nó hoạt động? Yêu cầu một bản demo: gửi email tới [nikita@befailproof.ai](mailto:nikita@befailproof.ai). -![Một phiên Failproof AI Observability được vẽ dưới dạng đồ thị thực thi kiểu git bên cạnh dòng thời gian sự kiện của nó, với phân tích từng lần chạy của các công cụ, mô hình và hook ở cột phải](/agenteye/images/session-detail.png) +![Một phiên FailproofAI Cloud được vẽ dưới dạng đồ thị thực thi kiểu git bên cạnh dòng thời gian sự kiện của nó, với phân tích từng lần chạy của các công cụ, mô hình và hook ở cột phải](/cloud/images/session-detail.png) *Mỗi lần chạy agent được vẽ dưới dạng đồ thị thực thi kiểu git (bên trái) bên cạnh dòng thời gian sự kiện của nó. Các sub-agents song song mỗi cái có làn riêng; cột phải hiển thị chi tiết công cụ, mô hình, hook và chi phí token cho lần chạy.* @@ -31,52 +31,52 @@ Hai video ngắn cho thấy hai thứ mà các nhóm thường cần trước ti
    -*Failproof Audit: để Failproof AI Observability khai thác nhật ký của bạn qua các phiên và cho bạn biết cần sửa chữa gì.* +*Failproof Audit: để FailproofAI Cloud khai thác nhật ký của bạn qua các phiên và cho bạn biết cần sửa chữa gì.* --- ## Tại sao các nhóm sử dụng nó - **Xem agent của bạn thực sự đã làm gì.** Mỗi lần chạy trở thành một đồ thị thực thi dễ đọc, kiểu git: công cụ nào chạy song song, sub-agents nào phân nhánh, nơi nó bị trì trệ, và nó đã chi tiêu bao nhiêu. -- **Phát hiện sự suy giảm chất lượng tự động.** Kết nối một dịch vụ chấm điểm nhỏ và Failproof AI Observability chấm điểm mỗi lần chạy hoàn tất, để sự giảm sút về hữu ích hoặc tăng đột biến về ảo giác tự hiển thị. +- **Phát hiện sự suy giảm chất lượng tự động.** Kết nối một dịch vụ chấm điểm nhỏ và FailproofAI Cloud chấm điểm mỗi lần chạy hoàn tất, để sự giảm sút về hữu ích hoặc tăng đột biến về ảo giác tự hiển thị. - **Tìm kiếm lỗi bạn chưa viết quy tắc cho.** Kiểm toán định kỳ khai thác nhật ký của bạn qua các phiên để tìm các cụm lỗi, ngoại lệ về độ trễ, điểm thấp và các lần chạy bị mắc kẹt, sau đó trao cho bạn các phát hiện được xếp hạng, hỗ trợ bằng bằng chứng. - **Nhận thông báo khi nó quan trọng.** Các quy tắc ngưỡng kích hoạt trên tỷ lệ lỗi, độ trễ, chi phí hoặc điểm đánh giá và mở các sự cố bạn có thể xác nhận, gán và giải quyết. - **Đặt câu hỏi bằng tiếng Anh thuần túy.** Một trợ lý AI trong bảng điều khiển trả lời những câu hỏi như "chất lượng trong prod tuần này có xu hướng như thế nào?" trên dữ liệu của bạn. Bất kỳ thay đổi nào mà nó thực hiện đều được phê duyệt. -- **Giữ dữ liệu của bạn.** Failproof AI Observability tự lưu trữ: sự kiện, prompt và phân tích ở lại trong cơ sở hạ tầng bạn kiểm soát. +- **Giữ dữ liệu của bạn.** FailproofAI Cloud tự lưu trữ: sự kiện, prompt và phân tích ở lại trong cơ sở hạ tầng bạn kiểm soát. --- ## Những gì bạn nhận được -Failproof AI Observability được tổ chức xung quanh ba ý tưởng (**observe**, **analyze** và **admin**), phản ánh trong thanh bên trái của bảng điều khiển. +FailproofAI Cloud được tổ chức xung quanh ba ý tưởng (**observe**, **analyze** và **admin**), phản ánh trong thanh bên trái của bảng điều khiển. **Observe** (sự thật thô của những gì đã xảy ra): -- **[Luồng sự kiện](/vi/agenteye/event-stream)**: dòng sự kiện trực tiếp, từng bước của mỗi lần chạy (lệnh gọi công cụ, lệnh gọi mô hình, hook, lỗi). -- **[Phiên](/vi/agenteye/sessions)**: những sự kiện đó được tổng hợp thành một hàng trên mỗi lần chạy, mỗi cái sẵn sàng được chấm điểm, với một đồ thị thực thi kiểu git. -- **[Chỉ số hiệu suất](/vi/agenteye/telemetry)**: bản đồ nhiệt độ trễ trên mỗi bề mặt và chỉ số p50/p95/p99 cho mô hình, công cụ và hook, để một tăng đột biến ở phần đuôi nổi bật so với mức trung bình. -- **[Theo dõi lỗi](/vi/agenteye/error-tracking)**: một bề mặt phân loại cho mọi thứ không ổn, chỉ cách một cú nhấp chuột từ một cảnh báo kích hoạt. +- **[Luồng sự kiện](/vi/cloud/event-stream)**: dòng sự kiện trực tiếp, từng bước của mỗi lần chạy (lệnh gọi công cụ, lệnh gọi mô hình, hook, lỗi). +- **[Phiên](/vi/cloud/sessions)**: những sự kiện đó được tổng hợp thành một hàng trên mỗi lần chạy, mỗi cái sẵn sàng được chấm điểm, với một đồ thị thực thi kiểu git. +- **[Chỉ số hiệu suất](/vi/cloud/performance)**: bản đồ nhiệt độ trễ trên mỗi bề mặt và chỉ số p50/p95/p99 cho mô hình, công cụ và hook, để một tăng đột biến ở phần đuôi nổi bật so với mức trung bình. +- **[Theo dõi lỗi](/vi/cloud/errors)**: một bề mặt phân loại cho mọi thứ không ổn, chỉ cách một cú nhấp chuột từ một cảnh báo kích hoạt. -![Trang Tools observe: một bản đồ nhiệt độ trễ, một dải phần trăm và một thanh phân phối công cụ trên 24 thùng thời gian](/agenteye/images/tools.png) +![Trang Tools observe: một bản đồ nhiệt độ trễ, một dải phần trăm và một thanh phân phối công cụ trên 24 thùng thời gian](/cloud/images/tools.png) *Mỗi bề mặt observe kết hợp một sparkline và chỉ số p50/p95/p99 với một bản đồ nhiệt độ trễ và một dải phần trăm. Hiển thị ở đây: Công cụ.* **Analyze** (chuyển hoạt động thành câu trả lời): -- **[Truy vấn](/vi/agenteye/queries)** và **[bảng điều khiển](/vi/agenteye/dashboards)**: SQL đã lưu trên sự kiện và đánh giá của bạn, biểu đồ thành các bảng điều khiển được chia sẻ, phạm vi tổ chức. -- **[Đánh giá](/vi/agenteye/evaluations)**: điểm chất lượng do dịch vụ đánh giá của riêng bạn tạo ra, với lý do cho mỗi điểm. -- **[Kiểm toán](/vi/agenteye/audits)**: các cuộc điều tra định kỳ phát hiện các mô hình lỗi qua các phiên. -- **[Cảnh báo](/vi/agenteye/alerts)** và **[sự cố](/vi/agenteye/incidents)**: các quy tắc ngưỡng thông báo cho bạn, cộng với quy trình xử lý sự cố để phân loại chúng. +- **[Truy vấn](/vi/cloud/queries)** và **[bảng điều khiển](/vi/cloud/dashboards)**: SQL đã lưu trên sự kiện và đánh giá của bạn, biểu đồ thành các bảng điều khiển được chia sẻ, phạm vi tổ chức. +- **[Đánh giá](/vi/cloud/evaluations)**: điểm chất lượng do dịch vụ đánh giá của riêng bạn tạo ra, với lý do cho mỗi điểm. +- **[Kiểm toán](/vi/cloud/audits)**: các cuộc điều tra định kỳ phát hiện các mô hình lỗi qua các phiên. +- **[Cảnh báo](/vi/cloud/alerts)** và **[sự cố](/vi/cloud/incidents)**: các quy tắc ngưỡng thông báo cho bạn, cộng với quy trình xử lý sự cố để phân loại chúng. **Giao diện** (truy cập dữ liệu của bạn cách bạn muốn): -- **[CLI](/vi/agenteye/cli-and-agents)**: điều khiển toàn bộ triển khai của bạn từ terminal hoặc script, và để một agent lập mã làm điều đó cho bạn bằng tiếng Anh thuần túy. -- **[Trợ lý AI](/vi/agenteye/assistant)**: đặt câu hỏi về các agent của bạn bằng tiếng Anh thuần túy, ngay bên trong bảng điều khiển. -- **REST API**: mọi thứ mà bảng điều khiển và CLI làm được hỗ trợ bởi một REST API bạn có thể gọi trực tiếp với một [khóa API](/vi/agenteye/api-keys) được phân phối — nhập sự kiện, truy vấn phiên và đánh giá, và quản lý bảng điều khiển, cảnh báo, kiểm toán, người dùng và khóa, để bạn có thể tích hợp Failproof AI Observability vào công cụ của riêng bạn. +- **[CLI](/vi/cloud/cli)**: điều khiển toàn bộ triển khai của bạn từ terminal hoặc script, và để một agent lập mã làm điều đó cho bạn bằng tiếng Anh thuần túy. +- **[Trợ lý AI](/vi/cloud/assistant)**: đặt câu hỏi về các agent của bạn bằng tiếng Anh thuần túy, ngay bên trong bảng điều khiển. +- **REST API**: mọi thứ mà bảng điều khiển và CLI làm được hỗ trợ bởi một REST API bạn có thể gọi trực tiếp với một [khóa API](/vi/cloud/access) được phân phối — nhập sự kiện, truy vấn phiên và đánh giá, và quản lý bảng điều khiển, cảnh báo, kiểm toán, người dùng và khóa, để bạn có thể tích hợp FailproofAI Cloud vào công cụ của riêng bạn. **Admin** (chạy nó cho nhóm của bạn): -- **[Khóa API](/vi/agenteye/api-keys)**: token được phân phối cho bộ sưu tập, bảng điều khiển và trợ lý. +- **[Khóa API](/vi/cloud/access)**: token được phân phối cho bộ sưu tập, bảng điều khiển và trợ lý. - **Người dùng**: đăng nhập không mật khẩu, dựa trên email với danh sách cho phép. - **Cài đặt**: cấu hình trên mỗi tổ chức, bao gồm ghi đè cửa sổ ngữ cảnh mô hình. @@ -92,18 +92,18 @@ Dữ liệu chảy theo một hướng, từ mã agent của bạn tới bảng - **Bảng điều khiển**: nơi bạn khám phá mọi thứ. - **Dịch vụ tùy chọn**: một dịch vụ chấm điểm (đánh giá) và một dịch vụ trợ lý AI (chat trong bảng điều khiển). -Đối với từ vựng được sử dụng trong toàn bộ tài liệu (*event, session, evaluation, audit, finding, incident*), xem [Khái niệm](/vi/agenteye/concepts). +Đối với từ vựng được sử dụng trong toàn bộ tài liệu (*event, session, evaluation, audit, finding, incident*), xem [Khái niệm](/vi/concepts). --- -## Nhận Failproof AI Observability +## Nhận FailproofAI Cloud -Failproof AI Observability là một sản phẩm doanh nghiệp từ Failproof AI, và nó hoạt động cùng với Failproof AI Enforcement — sản phẩm chính sách và guardrail — dưới thương hiệu Failproof AI. Nó chạy hoàn toàn trong môi trường của riêng bạn. Nếu bạn chưa có quyền truy cập vào các gói, hãy yêu cầu một bản demo và chúng tôi sẽ thiết lập cho bạn: gửi email tới [nikita@befailproof.ai](mailto:nikita@befailproof.ai). +FailproofAI Cloud là một sản phẩm doanh nghiệp từ Failproof AI, và nó hoạt động cùng với FailproofAI guardrails — sản phẩm chính sách và guardrail — dưới thương hiệu Failproof AI. Nó chạy hoàn toàn trong môi trường của riêng bạn. Nếu bạn chưa có quyền truy cập vào các gói, hãy yêu cầu một bản demo và chúng tôi sẽ thiết lập cho bạn: gửi email tới [nikita@befailproof.ai](mailto:nikita@befailproof.ai). --- ## Bước tiếp theo -- [Khái niệm](/vi/agenteye/concepts): từ vựng Failproof AI Observability trong một nơi. -- [Quan sát](/vi/agenteye/observability): theo dõi những gì các agent của bạn làm, lần chạy sau lần chạy. -- [Bảo mật](/vi/agenteye/security): cách Failproof AI Observability giữ dữ liệu của bạn được cô lập và dưới sự kiểm soát của bạn. \ No newline at end of file +- [Khái niệm](/vi/concepts): từ vựng FailproofAI Cloud trong một nơi. +- [Quan sát](/vi/cloud/overview): theo dõi những gì các agent của bạn làm, lần chạy sau lần chạy. +- [Bảo mật](/vi/cloud/security): cách FailproofAI Cloud giữ dữ liệu của bạn được cô lập và dưới sự kiểm soát của bạn. \ No newline at end of file diff --git a/docs/vi/agenteye/telemetry.mdx b/docs/vi/cloud/performance.mdx similarity index 82% rename from docs/vi/agenteye/telemetry.mdx rename to docs/vi/cloud/performance.mdx index deefd7f1..399be5f9 100644 --- a/docs/vi/agenteye/telemetry.mdx +++ b/docs/vi/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "Xem ngay lập tức khi các mô hình, công cụ hoặc hook c Xem ngay lập tức khi các mô hình, công cụ hoặc hook của bạn chậm lại hoặc phát sinh chi phí, và phát hiện sự tăng độ trễ ở phía đuôi trước khi người dùng của bạn cảm nhận được. Ba trang chuyên dụng biến các thời gian thô thành p50, p95 và p99 mà bạn có thể đọc ngay tại một cái nhìn. -![Trang Models hiển thị sơ đồ nhiệt độ trễ, một dải phần trăm và các số liệu về token, chi phí và cửa sổ ngữ cảnh cho từng mô hình](/agenteye/images/models.png) +![Trang Models hiển thị sơ đồ nhiệt độ trễ, một dải phần trăm và các số liệu về token, chi phí và cửa sổ ngữ cảnh cho từng mô hình](/cloud/images/models.png) *Trang Models: sơ đồ nhiệt độ trễ, dải phần trăm và số liệu token cho mỗi mô hình, chi phí ước tính và phần trăm đầy cửa sổ ngữ cảnh.* ## Hãy dừng để trung bình ẩn các lần chạy tồi tệ nhất của bạn @@ -24,13 +24,13 @@ Một chữ thập di chuột được chia sẻ liên kết sơ đồ nhiệt v Trang Models (hiển thị ở trên) trả lời hai câu hỏi mà một hóa đơn luôn đưa ra: mô hình nào và bao nhiêu tiền. Trên cơ sở khung nhìn độ trễ được chia sẻ, nó thêm **tiêu thụ token cho mỗi mô hình**, **chi phí ước tính** và **phần trăm đầy cửa sổ ngữ cảnh**, vì vậy sự tăng trưởng của prompt bất thường và một sự nén sắp xảy ra là hiển thị trước khi chúng gây bất ngờ cho bạn. -Failproof AI Observability nhận ra các ID mô hình phổ biến một cách tự động. Nếu một cửa sổ trông không đúng, hoặc bạn chạy một mô hình riêng của riêng bạn, hãy sửa nó hoặc thêm nó trong **Settings**, trong **model context windows**, và các số liệu phần trăm đầy theo sau. +FailproofAI Cloud nhận ra các ID mô hình phổ biến một cách tự động. Nếu một cửa sổ trông không đúng, hoặc bạn chạy một mô hình riêng của riêng bạn, hãy sửa nó hoặc thêm nó trong **Settings**, trong **model context windows**, và các số liệu phần trăm đầy theo sau. ## Tools: phân biệt cái chậm với cái bị hỏng Một lệnh gọi công cụ có thể chậm, hoặc nó có thể đang im lặng thất bại, và bạn muốn biết cái nào trong vòng vài giây, chứ không phải sau khi đào xung quanh các bản ghi. -![Trang Tools hiển thị sơ đồ nhiệt độ trễ và dải phần trăm được chia sẻ bên cạnh một sự phân tích bước đầu và thất bại và một thanh phân phối công cụ](/agenteye/images/tools.png) +![Trang Tools hiển thị sơ đồ nhiệt độ trễ và dải phần trăm được chia sẻ bên cạnh một sự phân tích bước đầu và thất bại và một thanh phân phối công cụ](/cloud/images/tools.png) *Trang Tools: sơ đồ nhiệt và dải phần trăm giống nhau, cộng với sự phân tích bước đầu và thất bại và thanh phân phối công cụ.* Bên cạnh khung nhìn độ trễ được chia sẻ, trang Tools thêm một **sự phân tích bước đầu và thất bại** và một **thanh phân phối công cụ**, vì vậy bạn thấy ngay tại một cái nhìn những công cụ nào bạn dựa vào nhiều nhất và những công cụ nào đang tiêu thụ ngân sách lỗi của bạn. @@ -39,14 +39,14 @@ Bên cạnh khung nhìn độ trễ được chia sẻ, trang Tools thêm một Khi một hook vòng đời kéo một lần chạy, "hook rất chậm" không phải là điều gì bạn có thể hành động. Trang Hooks giúp bạn đến cái hook quan trọng. -![Trang Hooks hiển thị độ trễ được phân tích theo tên hook và sự kiện kích hoạt trên sơ đồ nhiệt và dải phần trăm được chia sẻ](/agenteye/images/hooks.png) +![Trang Hooks hiển thị độ trễ được phân tích theo tên hook và sự kiện kích hoạt trên sơ đồ nhiệt và dải phần trăm được chia sẻ](/cloud/images/hooks.png) *Trang Hooks: độ trễ được phân tích theo tên hook và sự kiện kích hoạt.* Trên cùng sơ đồ nhiệt độ trễ và dải phần trăm, trang Hooks chia hoạt động thành **tên hook** và **sự kiện kích hoạt**, vì vậy bạn hạ cánh trên hook đơn lẻ và sự kiện đơn lẻ cần được chú ý. ## Liên Quan -- [Event stream](/vi/agenteye/event-stream): dấu vết của từng sự kiện được mã hóa bằng màu trực tiếp. -- [Sessions](/vi/agenteye/sessions): tổng hợp các sự kiện thành một hàng cho mỗi lần chạy và mở biểu đồ thực thi của nó. -- [Error tracking](/vi/agenteye/error-tracking): một bề mặt phân loại duy nhất cho tất cả những gì bảng điều khiển vẽ màu đỏ. -- [Dashboards](/vi/agenteye/dashboards): xem tổng hợp trên toàn bộ đội của bạn. \ No newline at end of file +- [Event stream](/vi/cloud/event-stream): dấu vết của từng sự kiện được mã hóa bằng màu trực tiếp. +- [Sessions](/vi/cloud/sessions): tổng hợp các sự kiện thành một hàng cho mỗi lần chạy và mở biểu đồ thực thi của nó. +- [Error tracking](/vi/cloud/errors): một bề mặt phân loại duy nhất cho tất cả những gì bảng điều khiển vẽ màu đỏ. +- [Dashboards](/vi/cloud/dashboards): xem tổng hợp trên toàn bộ đội của bạn. \ No newline at end of file diff --git a/docs/vi/agenteye/queries.mdx b/docs/vi/cloud/queries.mdx similarity index 78% rename from docs/vi/agenteye/queries.mdx rename to docs/vi/cloud/queries.mdx index ca822294..77801132 100644 --- a/docs/vi/agenteye/queries.mdx +++ b/docs/vi/cloud/queries.mdx @@ -3,9 +3,9 @@ title: "Truy vấn" description: "Đặt bất kỳ câu hỏi nào về dữ liệu agent của bạn và nhận câu trả lời trong vài giây." --- -Đặt bất kỳ câu hỏi nào về dữ liệu agent của bạn và nhận câu trả lời trong vài giây. Failproof AI Observability cung cấp cho bạn một thư viện các truy vấn đã lưu, sẵn sàng chạy trên các sự kiện và đánh giá của bạn, để bạn bắt đầu từ một ví dụ hoạt động thay vì một trình soạn thảo SQL trống. +Đặt bất kỳ câu hỏi nào về dữ liệu agent của bạn và nhận câu trả lời trong vài giây. FailproofAI Cloud cung cấp cho bạn một thư viện các truy vấn đã lưu, sẵn sàng chạy trên các sự kiện và đánh giá của bạn, để bạn bắt đầu từ một ví dụ hoạt động thay vì một trình soạn thảo SQL trống. -![Thư viện truy vấn đã lưu: một lưới các truy vấn có thể tái sử dụng, bao gồm các cài đặt sẵn tích hợp và các truy vấn tùy chỉnh](/agenteye/images/queries.png) +![Thư viện truy vấn đã lưu: một lưới các truy vấn có thể tái sử dụng, bao gồm các cài đặt sẵn tích hợp và các truy vấn tùy chỉnh](/cloud/images/queries.png) *Thư viện truy vấn đã lưu của bạn tại `//queries`: các cài đặt sẵn tích hợp nằm cạnh các truy vấn mà nhóm bạn đã lưu.* @@ -21,7 +21,7 @@ Tìm nó tại `//queries`. Mở bất kỳ truy vấn nào và nó sẽ xuất hiện trong trình soạn thảo SQL, nơi bạn có thể điều chỉnh nó và xem câu trả lời ngay lập tức: không xuất, không vòng quay lại, không chờ đợi người khác. -![Trình soạn thảo truy vấn SQL chạy một truy vấn đã lưu, với thanh bên lược đồ và lưới kết quả trực tiếp](/agenteye/images/query-lab.png) +![Trình soạn thảo truy vấn SQL chạy một truy vấn đã lưu, với thanh bên lược đồ và lưới kết quả trực tiếp](/cloud/images/query-lab.png) *Trình soạn thảo SQL: truy vấn của bạn bên trái, thanh bên lược đồ để bạn không bao giờ phải đoán tên cột, và lưới kết quả trực tiếp bên dưới.* @@ -42,14 +42,14 @@ agenteye query list # the same saved queries, from your terminal agenteye query run errs --arg prod # run one and print the rows (add --json to pipe it) ``` - Xem [CLI và agents](/vi/agenteye/cli-and-agents) để biết bộ lệnh đầy đủ. + Xem [CLI và agents](/vi/cloud/cli) để biết bộ lệnh đầy đủ. -- **Từ trợ lý AI.** Không chắc cách diễn đạt SQL? Hỏi [trợ lý AI](/vi/agenteye/assistant) trong bảng điều khiển bằng tiếng Anh đơn giản và nó sẽ soạn thảo truy vấn và lưu nó vào thư viện của bạn. +- **Từ trợ lý AI.** Không chắc cách diễn đạt SQL? Hỏi [trợ lý AI](/vi/cloud/assistant) trong bảng điều khiển bằng tiếng Anh đơn giản và nó sẽ soạn thảo truy vấn và lưu nó vào thư viện của bạn. Chạy một truy vấn đã lưu được kiểm soát bởi quyền `queries:run`, được giữ riêng biệt với các quyền để tạo hoặc xóa truy vấn, vì vậy bạn có thể cấp quyền truy cập đọc mà không để tất cả mọi người viết lại thư viện. ## Liên quan -- [Bảng điều khiển](/vi/agenteye/dashboards): ghim kết quả truy vấn vào các biểu đồ chia sẻ, toàn bộ org. -- [Trợ lý AI](/vi/agenteye/assistant): đặt câu hỏi bằng tiếng Anh đơn giản và nhận lại một truy vấn. -- [CLI và agents](/vi/agenteye/cli-and-agents): chạy và lưu các truy vấn tương tự từ terminal của bạn. \ No newline at end of file +- [Bảng điều khiển](/vi/cloud/dashboards): ghim kết quả truy vấn vào các biểu đồ chia sẻ, toàn bộ org. +- [Trợ lý AI](/vi/cloud/assistant): đặt câu hỏi bằng tiếng Anh đơn giản và nhận lại một truy vấn. +- [CLI và agents](/vi/cloud/cli): chạy và lưu các truy vấn tương tự từ terminal của bạn. \ No newline at end of file diff --git a/docs/vi/agenteye/python-sdk.mdx b/docs/vi/cloud/sdk.mdx similarity index 92% rename from docs/vi/agenteye/python-sdk.mdx rename to docs/vi/cloud/sdk.mdx index 9cb85735..00925276 100644 --- a/docs/vi/agenteye/python-sdk.mdx +++ b/docs/vi/cloud/sdk.mdx @@ -4,11 +4,11 @@ description: "Xem chính xác những gì các AI agents của bạn đã làm t --- -Xem chính xác những gì các AI agents của bạn đã làm trong production: mọi agent run, tool call, model request, hook, và human intervention. Failproof AI Observability Python SDK ghi lại toàn bộ trail này từ bên trong code của agent để bạn có thể debug, audit, và đánh giá những gì đã xảy ra. Sử dụng nó bất cứ khi nào bạn muốn Failproof AI Observability theo dõi các agents của mình. +Xem chính xác những gì các AI agents của bạn đã làm trong production: mọi agent run, tool call, model request, hook, và human intervention. FailproofAI Cloud Python SDK ghi lại toàn bộ trail này từ bên trong code của agent để bạn có thể debug, audit, và đánh giá những gì đã xảy ra. Sử dụng nó bất cứ khi nào bạn muốn FailproofAI Cloud theo dõi các agents của mình. Bên dưới, SDK ghi các sự kiện có cấu trúc vào các file JSONL cục bộ, và daemon collector sẽ lấy chúng và gửi đến platform một cách tự động. Bạn không cần quản lý các file này. -> **Tip:** Mới bắt đầu với Failproof AI Observability? Trang này là tài liệu tham khảo SDK event hoàn chỉnh. +> **Tip:** Mới bắt đầu với FailproofAI Cloud? Trang này là tài liệu tham khảo SDK event hoàn chỉnh.
    @@ -26,7 +26,7 @@ Sau khi cài đặt, hãy xác nhận bạn có nó: python -c "import agenteye; print(agenteye.__version__)" ``` -Thích để cho một coding agent thực hiện toàn bộ tích hợp? [Python SDK Agent Skill](/vi/agenteye/python-sdk-skill) biết đường dẫn cài đặt, lên kế hoạch các điểm instrumentation, viết chúng và xác minh các events đến. +Thích để cho một coding agent thực hiện toàn bộ tích hợp? [Python SDK Agent Skill](/vi/cloud/agent-skills) biết đường dẫn cài đặt, lên kế hoạch các điểm instrumentation, viết chúng và xác minh các events đến. --- @@ -60,7 +60,7 @@ agenteye.event.agent_end(session_id="run-001", agent_id="planner", outcome="succ ### Instrumenting một cuộc gọi thực tế -Trong thực tế, bạn sẽ bao quanh code agent hiện có của mình. Đặt một model call giữa `model_request` trước và `model_response` sau, để hai event này bao phủ yêu cầu thực tế và Failproof AI Observability có thể ghép chúng lại: +Trong thực tế, bạn sẽ bao quanh code agent hiện có của mình. Đặt một model call giữa `model_request` trước và `model_response` sau, để hai event này bao phủ yêu cầu thực tế và FailproofAI Cloud có thể ghép chúng lại: ```python import anthropic @@ -99,7 +99,7 @@ Bao quanh tool calls một cách tương tự với `tool_use` và `tool_result` Đây là hình ảnh những events này khi chúng đến dashboard, được mã hóa màu theo loại và có thể lọc theo environment, agent, và session: -![The live Events stream, colour-coded by event type and filterable by environment, agent, and session](/agenteye/images/events-stream.png) +![The live Events stream, colour-coded by event type and filterable by environment, agent, and session](/cloud/images/events-stream.png) --- @@ -153,9 +153,9 @@ SDK chỉ ghi lại các trường bạn truyền một cách rõ ràng. Các pr ## Event Reference -Hầu hết các events đến theo cặp start/end chia sẻ một correlation ID: `tool_use` và `tool_result` chia sẻ một `tool_call_id`, `hook_triggered` và `hook_completed` chia sẻ một `hook_id`, và `human_wait` và `human_input` chia sẻ một `input_id`. Phát event bắt đầu, thực hiện công việc, sau đó phát event kết thúc với cùng một ID. Failproof AI Observability sẽ khớp cặp này và tính `duration_ms` cho bạn, vì vậy bạn không bao giờ truyền `duration_ms` chính mình. +Hầu hết các events đến theo cặp start/end chia sẻ một correlation ID: `tool_use` và `tool_result` chia sẻ một `tool_call_id`, `hook_triggered` và `hook_completed` chia sẻ một `hook_id`, và `human_wait` và `human_input` chia sẻ một `input_id`. Phát event bắt đầu, thực hiện công việc, sau đó phát event kết thúc với cùng một ID. FailproofAI Cloud sẽ khớp cặp này và tính `duration_ms` cho bạn, vì vậy bạn không bao giờ truyền `duration_ms` chính mình. -![A session's git-style execution graph beside its event timeline, reconstructed from the paired events, with the tool/model/hook breakdown panel](/agenteye/images/session-detail.png) +![A session's git-style execution graph beside its event timeline, reconstructed from the paired events, with the tool/model/hook breakdown panel](/cloud/images/session-detail.png) Tất cả các phương thức event đều yêu cầu hai trường này: @@ -429,5 +429,5 @@ Mỗi file được ghi atomically: SDK ghi vào một temporary file và sau đ ## Bước tiếp theo -- [Event stream](/vi/agenteye/event-stream): xem các events này đến live, được mã hóa màu và có thể lọc theo environment, agent, và session. -- [Sessions](/vi/agenteye/sessions): xem cách các paired events tái cấu trúc mỗi agent run dưới dạng một execution graph và timeline. \ No newline at end of file +- [Event stream](/vi/cloud/event-stream): xem các events này đến live, được mã hóa màu và có thể lọc theo environment, agent, và session. +- [Sessions](/vi/cloud/sessions): xem cách các paired events tái cấu trúc mỗi agent run dưới dạng một execution graph và timeline. \ No newline at end of file diff --git a/docs/vi/cloud/security.mdx b/docs/vi/cloud/security.mdx new file mode 100644 index 00000000..f329bf3c --- /dev/null +++ b/docs/vi/cloud/security.mdx @@ -0,0 +1,68 @@ +--- +title: "Bảo mật" +description: "FailproofAI Cloud được xây dựng để hoạt động gần với các agent production của bạn, có nghĩa là nó thấy được prompts, đầu vào công cụ và kết quả đầu ra của bạn." +--- + + +FailproofAI Cloud được xây dựng để hoạt động gần với các agent production của bạn, có nghĩa là nó thấy được prompts, đầu vào công cụ và kết quả đầu ra của bạn. Trang này giải thích cách nó giữ dữ liệu đó được cách ly, được kiểm soát và nằm dưới quyền của bạn. Nếu bạn đang đánh giá FailproofAI Cloud cho một bài kiểm tra bảo mật, hãy bắt đầu từ đây. + +--- + +## Dữ liệu của bạn ở lại trong môi trường của bạn + +FailproofAI Cloud được tự lưu trữ. Các sự kiện, prompts, phản hồi của mô hình và phân tích được lưu trữ trong các cơ sở dữ liệu của riêng bạn, trong môi trường của riêng bạn. Không có dữ liệu nào được gửi đến bên thứ ba SaaS để lưu trữ, và dữ liệu của bạn ở lại trong tài khoản cloud của riêng bạn. + +--- + +## Cách ly đa tổ chức + +Một instance FailproofAI Cloud có thể lưu trữ nhiều tổ chức, và mỗi tổ chức được cách ly ở lớp lưu trữ — được thực thi bởi cơ sở dữ liệu, không chỉ giao diện người dùng: + +- Dữ liệu hoạt động của một tổ chức (người dùng, khóa, bảng điều khiển, truy vấn đã lưu) được giới hạn trong tổ chức đó, và các lần đọc liên tổ chức bị chặn bởi chính cơ sở dữ liệu. +- Mỗi sự kiện được nhập đều được đánh dấu với tổ chức sở hữu, vì vậy các sự kiện của một tổ chức không bao giờ có thể được đọc bởi tổ chức khác. + +Mỗi tuyến đường bảng điều khiển được giới hạn trong một slug org (`//…`). + +--- + +## Đăng nhập + +FailproofAI Cloud sử dụng đăng nhập không mật khẩu, dựa trên email. Không có mật khẩu để lừa phishing hoặc rò rỉ. Người dùng yêu cầu một mã dùng một lần (hoặc liên kết magic một bước), được gửi email cho họ và hết hạn nhanh chóng. Đăng nhập được kiểm soát bởi một **danh sách cho phép**: chỉ những địa chỉ email (hoặc miền) mà bạn cho phép mới có thể xác thực. + +![Màn hình đăng nhập FailproofAI Cloud, gửi một mã dùng một lần đến email của bạn](/cloud/images/login.png) + +--- + +## Truy cập được giới hạn với khóa API + +Mỗi máy khách xác thực bằng khóa API có quyền granular, ít nhất. Một bộ sưu tập chỉ cần `events:add`; một khóa bảng điều khiển hoặc trợ lý có thể chỉ đọc; các hành động phá hủy (xóa, tạo lại) là các cấp riêng biệt mà bạn chọn để đưa vào. + +![Trang khóa API: các cấp quyền của mỗi khóa, được mã hóa màu theo phạm vi đọc, viết và hủy diệt](/cloud/images/api-keys.png) + +Giữ khóa bootstrap quản trị viên cho thiết lập và phát hành các khóa hẹp cho mọi thứ khác. Xem [API keys](/vi/cloud/access). + +--- + +## Trợ lý chỉ đọc, được phê duyệt + +[Trợ lý AI](/vi/cloud/assistant) trong bảng điều khiển trả lời các câu hỏi về dữ liệu của bạn, nhưng nó bị hạn chế bởi thiết kế: + +- Nó **chỉ đọc theo mặc định**: SQL của nó chạy qua một lệnh bảo vệ chỉ cho phép các truy vấn `SELECT`/`WITH`, một câu lệnh duy nhất, với một giới hạn hàng. +- Bất cứ điều gì nó tạo (một truy vấn đã lưu, một bảng điều khiển) đều **được phê duyệt**: bạn xem xét và phê duyệt mỗi lần ghi trước khi nó xảy ra. +- Nó **không bao giờ có thể xóa**. + +Vì vậy, một đồng nghiệp có thể hỏi "agents nào bị lỗi nhất tuần này?" và hành động dựa trên câu trả lời, mà không cần trợ lý có khả năng thay đổi hoặc xóa dữ liệu của bạn riêng lẻ. + +--- + +## Trong quá trình chuyển động + +Tất cả lưu lượng chạy qua HTTPS. Bạn kết thúc TLS bằng chứng chỉ của riêng bạn, vì vậy lưu lượng từ bộ sưu tập đến máy chủ và từ trình duyệt đến máy chủ được mã hóa trong quá trình chuyển động. + +--- + +## Bước tiếp theo + +- [Overview](/vi/cloud/overview): cách FailproofAI Cloud kết hợp với nhau. +- [API keys](/vi/cloud/access): giới hạn truy cập cho bộ sưu tập, bảng điều khiển và trợ lý. +- [FailproofAI Cloud](/vi/cloud/overview): những gì FailproofAI Cloud capture từ các agent của bạn. \ No newline at end of file diff --git a/docs/vi/agenteye/sessions.mdx b/docs/vi/cloud/sessions.mdx similarity index 76% rename from docs/vi/agenteye/sessions.mdx rename to docs/vi/cloud/sessions.mdx index 4cd469d7..a8a027e7 100644 --- a/docs/vi/agenteye/sessions.mdx +++ b/docs/vi/cloud/sessions.mdx @@ -4,9 +4,9 @@ description: "Mỗi sự kiện từ một lần chạy được gộp thành m --- -Hãy dừng đoán tại sao một lần chạy bị lỗi. Failproof AI Observability gộp mỗi sự kiện từ một lần chạy thành một hàng dễ đọc, sau đó vẽ toàn bộ lần chạy dưới dạng hình ảnh kiểu git mà bạn có thể hiểu trong vài giây, vì vậy bạn thấy chính xác agent của mình đã làm gì, từng bước một. +Hãy dừng đoán tại sao một lần chạy bị lỗi. FailproofAI Cloud gộp mỗi sự kiện từ một lần chạy thành một hàng dễ đọc, sau đó vẽ toàn bộ lần chạy dưới dạng hình ảnh kiểu git mà bạn có thể hiểu trong vài giây, vì vậy bạn thấy chính xác agent của mình đã làm gì, từng bước một. -![Danh sách Sessions: một hàng mỗi lần chạy, trên các môi trường và agent, với các badge trạng thái và điểm đánh giá](/agenteye/images/sessions-list.png) +![Danh sách Sessions: một hàng mỗi lần chạy, trên các môi trường và agent, với các badge trạng thái và điểm đánh giá](/cloud/images/sessions-list.png) *Một hàng mỗi lần chạy: badge trạng thái cho bạn biết cách kết thúc lần chạy ngay lập tức, và một badge điểm xuất hiện khi một evaluator được kết nối.* @@ -30,7 +30,7 @@ Sau khi bạn kết nối một evaluator, mỗi lần chạy hoàn tất sẽ ## Đọc toàn bộ lần chạy dưới dạng hình ảnh -![Biểu đồ thực thi kiểu git của một session bên cạnh dòng thời gian sự kiện của nó, với bảng phân tích tool, model, và hook](/agenteye/images/session-detail.png) +![Biểu đồ thực thi kiểu git của một session bên cạnh dòng thời gian sự kiện của nó, với bảng phân tích tool, model, và hook](/cloud/images/session-detail.png) *Biểu đồ thực thi (trái) nằm bên cạnh dòng thời gian sự kiện; thanh bên phải chia nhỏ các tool, model, hook, và chi phí token cho lần chạy.* @@ -38,7 +38,7 @@ Nhấp vào bất kỳ session nào để mở biểu đồ thực thi của nó Thanh bên phải cho bạn biết chi tiết từng lần chạy: những tool và model nào đã chạy, những hook nào được kích hoạt, và lần chạy đã chi phí bao nhiêu token. Đó là câu trả lời cho "tại sao lần chạy này lại tốn nhiều tiền như vậy?" hoặc "tool nào là cái chậm?" nằm ngay bên cạnh biểu đồ đã gây ra nó. -Các sự kiện riêng lẻ có thể được định địa chỉ, vì vậy bạn có thể trao cho ai đó một liên kết đến một thời điểm thay vì "session, khoảng hai phần ba xuống". Sao chép liên kết từ bất kỳ sự kiện nào, hoặc theo một liên kết từ kết quả [audit](/vi/agenteye/audits) hoặc lỗi, và session sẽ mở với sự kiện đó được chọn và cuộn đến. Điều này cũng áp dụng cho các lần chạy rất dài: dòng thời gian tải một cửa sổ giới hạn vì lợi ích của trình duyệt của bạn, và một liên kết trỏ vào quá cửa sổ đó vẫn tìm thấy sự kiện của nó thay vì thả bạn ở đầu. Nếu sự kiện đã lỗi thời ngoài cửa sổ retention của bạn, trang sẽ cho bạn biết điều đó thay vì yên lặng không chọn gì. +Các sự kiện riêng lẻ có thể được định địa chỉ, vì vậy bạn có thể trao cho ai đó một liên kết đến một thời điểm thay vì "session, khoảng hai phần ba xuống". Sao chép liên kết từ bất kỳ sự kiện nào, hoặc theo một liên kết từ kết quả [audit](/vi/cloud/audits) hoặc lỗi, và session sẽ mở với sự kiện đó được chọn và cuộn đến. Điều này cũng áp dụng cho các lần chạy rất dài: dòng thời gian tải một cửa sổ giới hạn vì lợi ích của trình duyệt của bạn, và một liên kết trỏ vào quá cửa sổ đó vẫn tìm thấy sự kiện của nó thay vì thả bạn ở đầu. Nếu sự kiện đã lỗi thời ngoài cửa sổ retention của bạn, trang sẽ cho bạn biết điều đó thay vì yên lặng không chọn gì. --- @@ -46,12 +46,12 @@ Các sự kiện riêng lẻ có thể được định địa chỉ, vì vậy Mỗi trang bảng điều khiển được phạm vi vào tổ chức của bạn (`//…`). Sessions nằm dưới **Observe** ở thanh bên trái, bên cạnh Events, với các bộ lọc phạm vi ngày, môi trường, agent, và session trên đầu danh sách. Mỗi hàng là một cú nhấp chuột từ biểu đồ thực thi đầy đủ của nó. -Để bật các badge điểm và lọc phạm vi điểm, hãy kết nối một evaluator: xem [Evaluations](/vi/agenteye/evaluations). +Để bật các badge điểm và lọc phạm vi điểm, hãy kết nối một evaluator: xem [Evaluations](/vi/cloud/evaluations). --- ## Liên quan -- [Event stream](/vi/agenteye/event-stream): dòng thô từng bước mà mỗi session được gộp lại từ đó. -- [Evaluations](/vi/agenteye/evaluations): kết nối một evaluator để mỗi lần chạy nhận được một badge điểm mà bạn có thể lọc. -- [Telemetry](/vi/agenteye/telemetry): cách các lần chạy đi từ agent của bạn vào các session này. \ No newline at end of file +- [Event stream](/vi/cloud/event-stream): dòng thô từng bước mà mỗi session được gộp lại từ đó. +- [Evaluations](/vi/cloud/evaluations): kết nối một evaluator để mỗi lần chạy nhận được một badge điểm mà bạn có thể lọc. +- [Telemetry](/vi/cloud/performance): cách các lần chạy đi từ agent của bạn vào các session này. \ No newline at end of file diff --git a/docs/vi/concepts.mdx b/docs/vi/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/vi/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/vi/daemon.mdx b/docs/vi/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/vi/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/vi/dashboard.mdx b/docs/vi/dashboard.mdx index 4e9f5525..e2f45f3a 100644 --- a/docs/vi/dashboard.mdx +++ b/docs/vi/dashboard.mdx @@ -69,7 +69,7 @@ Một báo cáo được hướng dẫn bởi tính cách về cách agent của 4. **Cách để cải thiện** — danh sách hàng tĩnh, một cho mỗi chính sách được quy định: tên chính sách bằng trắng, mô tả một dòng, lệnh cài đặt + nút sao chép ở bên phải. Tiêu đề phần đọc `enable all N → projected · ` (điểm số bạn sẽ đạt được với mọi bản sửa chữa được áp dụng), và nút `[install all]` của nó sao chép lệnh `failproofai policy add a b c …` kết hợp cho mỗi chính sách được quy định. 5. **Quay lại tốt hơn** — hai thẻ cạnh nhau. Bên trái: đặt nhắc nhở (`3d` / `7d` / `14d` / `30d` bộ chọn nhịp độ; tồn tại thông qua `/api/auth/reminder` khi xác thực). Bên phải: mở khóa các quyền lợi failproof — `invite a friend` mở một modal nhận một danh sách email bạn bè được phân tách bằng dấu phẩy/khoảng trắng/dòng mới (tối đa 10 trên mỗi lần gửi), POSTs chúng đến `/api/audit/invite`, được chuyển tiếp đến `POST /v0/invite` của máy chủ api. Máy chủ api gửi một email cho mỗi người nhận từ `invite@failproof.ai` với người gửi được Cc và `Reply-To` được đặt, vì vậy người nhận sẽ thấy ai đã mời họ và người gửi nhận được một bản sao trong hộp thư đến của họ. Người dùng ẩn danh được chuyển hướng qua `AuthDialog` trước tiên để email của người gửi được biết trước khi lời mời được gửi đi. Hoàn thành quyền lợi / quyền lợi là một công việc tiếp theo. -Được điều khiển bởi quá trình chạy `failproofai audit` — xem [Audit CLI](/vi/cli/audit) để biết công cụ quét cơ bản, các cờ được hỗ trợ và bất biến bộ nhớ cache cho mỗi bản ghi. Bảng điều khiển lưu cache kết quả mới nhất tại `~/.failproofai/audit-dashboard.json` (chế độ `0600`, khe duy nhất, các bộ chạy mới ghi đè) vì vậy các lần truy cập lại là tức thì; **cả bộ nhớ cache cho mỗi bản ghi và toàn bộ kết quả đều bị từ chối khi đọc sau khi chúng cũ hơn 7 ngày** vì vậy bảng điều khiển không bao giờ im lặng cung cấp kết quả cách đây một tuần — quá TTL `/audit` rơi vào trạng thái trống của nó và nhắc nhở một lần chạy tươi. Nhấp vào `[ re-audit now ]` gần dưới cùng của báo cáo POSTs `/api/audit/run` với `noCache: true` — re-audit bỏ qua bộ nhớ cache cho mỗi bản ghi và quét lại mọi bản ghi từ đầu thay vì im lặng trả lại kết quả được lưu cache — và bảng điều khiển thăm dò `/api/audit/status` ở 1Hz cho đến khi lần chạy hoàn thành; một dải tiến trình hồng dính ghép đỉnh viewport trong lần chạy có bộ đếm thời gian trôi qua, và kết quả tươi tho hoán đổi tại chỗ khi thành công (không có tải lại trang đầy đủ; kiểm toán lại không thành công để lại báo cáo trước đó nguyên vẹn). Khi thất bại, dải quay sang màu đỏ với bản sao được tính khóa từ `RerunError.kind` (`timeout` / `network` / `post_failed`). Trạng thái trống (không có bộ nhớ cache hoặc hết hạn) và trạng thái không có phiên (bộ nhớ cache tồn tại nhưng quét không tìm thấy bản ghi nào) được hiển thị riêng biệt. +Được điều khiển bởi quá trình chạy `failproofai audit` — xem [Audit CLI](/vi/audit) để biết công cụ quét cơ bản, các cờ được hỗ trợ và bất biến bộ nhớ cache cho mỗi bản ghi. Bảng điều khiển lưu cache kết quả mới nhất tại `~/.failproofai/audit-dashboard.json` (chế độ `0600`, khe duy nhất, các bộ chạy mới ghi đè) vì vậy các lần truy cập lại là tức thì; **cả bộ nhớ cache cho mỗi bản ghi và toàn bộ kết quả đều bị từ chối khi đọc sau khi chúng cũ hơn 7 ngày** vì vậy bảng điều khiển không bao giờ im lặng cung cấp kết quả cách đây một tuần — quá TTL `/audit` rơi vào trạng thái trống của nó và nhắc nhở một lần chạy tươi. Nhấp vào `[ re-audit now ]` gần dưới cùng của báo cáo POSTs `/api/audit/run` với `noCache: true` — re-audit bỏ qua bộ nhớ cache cho mỗi bản ghi và quét lại mọi bản ghi từ đầu thay vì im lặng trả lại kết quả được lưu cache — và bảng điều khiển thăm dò `/api/audit/status` ở 1Hz cho đến khi lần chạy hoàn thành; một dải tiến trình hồng dính ghép đỉnh viewport trong lần chạy có bộ đếm thời gian trôi qua, và kết quả tươi tho hoán đổi tại chỗ khi thành công (không có tải lại trang đầy đủ; kiểm toán lại không thành công để lại báo cáo trước đó nguyên vẹn). Khi thất bại, dải quay sang màu đỏ với bản sao được tính khóa từ `RerunError.kind` (`timeout` / `network` / `post_failed`). Trạng thái trống (không có bộ nhớ cache hoặc hết hạn) và trạng thái không có phiên (bộ nhớ cache tồn tại nhưng quét không tìm thấy bản ghi nào) được hiển thị riêng biệt. ### Chính sách diff --git a/docs/vi/architecture.mdx b/docs/vi/how-it-works.mdx similarity index 100% rename from docs/vi/architecture.mdx rename to docs/vi/how-it-works.mdx diff --git a/docs/vi/introduction.mdx b/docs/vi/introduction.mdx index 66971796..e0f7b998 100644 --- a/docs/vi/introduction.mdx +++ b/docs/vi/introduction.mdx @@ -54,4 +54,4 @@ failproofai policies --install # enable policies (or skip — `failproofai` wi failproofai # launch the dashboard ``` -Xem hướng dẫn [Bắt đầu](/vi/getting-started) để có hướng dẫn đầy đủ. \ No newline at end of file +Xem hướng dẫn [Bắt đầu](/vi/quickstart) để có hướng dẫn đầy đủ. \ No newline at end of file diff --git a/docs/vi/policies.mdx b/docs/vi/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/vi/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/vi/getting-started.mdx b/docs/vi/quickstart.mdx similarity index 100% rename from docs/vi/getting-started.mdx rename to docs/vi/quickstart.mdx diff --git a/docs/vi/reference/files.mdx b/docs/vi/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/vi/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/docs/zh/agent-support.mdx b/docs/zh/agent-support.mdx new file mode 100644 index 00000000..7627921c --- /dev/null +++ b/docs/zh/agent-support.mdx @@ -0,0 +1,204 @@ +--- +title: Supported agents +description: "All 12 agent CLIs FailproofAI protects — where it installs, what it can actually block on each, and where a rule would be silently inert." +icon: table +--- + +FailproofAI installs into the agent CLIs you already run, and one policy set covers all of +them. Event names, tool names, and tool-input keys are normalized before any policy +executes, so a rule you write once fires identically everywhere. + +But the CLIs are not equally capable, and pretending otherwise is how a guardrail becomes +theatre. A `deny` only means something if the CLI *reads* it at a point where the action +can still be stopped. This page states, per CLI, exactly where that is true. + +--- + +## Install command + +```bash +failproofai config # detects what's installed, sets it all up +failproofai policies --install --cli --scope project # or target one explicitly +``` + +| CLI | `--cli` name | Binary | Scopes | Status | +|---|---|---|---|---| +| Claude Code | `claude` | `claude` | user · project · local | Stable | +| OpenAI Codex | `codex` | `codex` | user · project | Stable | +| GitHub Copilot CLI | `copilot` | `copilot` | user · project | Beta | +| Cursor Agent | `cursor` | `cursor-agent` | user · project | Beta | +| OpenCode | `opencode` | `opencode` | user · project | Beta | +| Pi | `pi` | `pi` | user · project | Beta | +| Hermes | `hermes` | `hermes` | user only | Stable | +| OpenClaw | `openclaw` | `openclaw` | user only | Stable | +| Factory Droid | `factory` | `droid` | user · project | Stable | +| Devin CLI | `devin` | `devin` | user · project | Stable | +| Antigravity CLI | `antigravity` | `agy` | user · project | Stable | +| Goose | `goose` | `goose` | user · project | Stable | + + + **VS Code Copilot Chat agent mode** is covered for free. It reads hook configs from the + same paths the `copilot` and `claude` integrations already write, using the same + contract — so `failproofai policies --install --cli copilot` (or `--cli claude`) already + enforces inside VS Code agent-mode sessions. There is no separate `vscode` target. + + +--- + +## What can actually be blocked, per CLI + +Read this as: *if a policy denies here, does the agent stop?* + +- **Blocks** — the action is prevented, or the agent is forced to continue and fix it. +- **Records only** — the verdict is logged and visible, but the action proceeds. Either + the CLI discards the answer, or the action had already happened. +- **n/a** — the CLI does not fire that event at all. + +| CLI | Before a tool call | On a submitted prompt | After a tool call | At turn end | Sub-agent end | +|---|---|---|---|---|---| +| **Claude Code** | Blocks | Blocks | Records only | **Blocks** | **Blocks** | +| **OpenAI Codex** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **GitHub Copilot CLI** | Blocks | Blocks | Blocks (rewrites the result the model reads) | **Blocks** | **Blocks** | +| **Cursor Agent** | Blocks | Blocks | Records only | **Blocks** | not verified | +| **OpenCode** | Blocks | Records only | Records only | not verified | — | +| **Pi** | Blocks | Blocks | Records only | Instructs the *next* turn | — | +| **Hermes** | Blocks | — | Records only | **n/a** | Records only | +| **OpenClaw** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Factory Droid** | Blocks | Blocks | Records only | **Blocks** | Records only | +| **Devin CLI** | Blocks | Blocks | Records only | **Blocks** | — | +| **Antigravity CLI** | Blocks | Records only (instructions still work) | Records only | **Blocks** | — | +| **Goose** | Blocks | Records only | Records only | **n/a** | — | + + + **The turn-end column is the one to read before you rely on it.** The five + `require-*-before-stop` policies — commit, push, PR, no-conflicts, CI-green — work by + refusing to let the agent finish. On Hermes and Goose there is no turn-end gate for + FailproofAI to attach to, so those policies never fire there. That is a platform + limit, stated here rather than left for you to discover from a rule that quietly did + nothing. + + +Every entry in this table is derived from the same machine-readable source the product +itself uses, and a test asserts they agree. Rows that have not been verified against a +real, shipping version of a CLI say "not verified" rather than guessing — an unverified +claim about a guardrail is worse than no claim. + +--- + +## Where the hooks get written + +Each CLI has its own settings file, and setup writes into it in that CLI's own schema, +preserving whatever else is in the file. + +| CLI | User scope | Project scope | +|---|---|---| +| Claude Code | `~/.claude/settings.json` | `.claude/settings.json` (+ `.claude/settings.local.json`) | +| OpenAI Codex | `~/.codex/hooks.json` | `.codex/hooks.json` | +| GitHub Copilot CLI | `~/.copilot/hooks/failproofai.json` | `.github/hooks/failproofai.json` | +| Cursor Agent | `~/.cursor/hooks.json` | `.cursor/hooks.json` | +| OpenCode | `~/.config/opencode/opencode.json` + a generated plugin | `.opencode/opencode.json` + a generated plugin | +| Pi | `~/.pi/agent/settings.json` | `.pi/settings.json` | +| Hermes | `~/.hermes/config.yaml` | — | +| OpenClaw | `~/.openclaw/openclaw.json` | — | +| Factory Droid | `~/.factory/hooks.json` | `.factory/hooks.json` | +| Devin CLI | `~/.config/devin/config.json` | `.devin/config.json` | +| Antigravity CLI | `~/.gemini/config/hooks.json` | `.agents/hooks.json` | +| Goose | `~/.agents/plugins/failproofai/` | `.agents/plugins/failproofai/` | + +Three CLIs need something other than a shell hook, because they have no external-command +hook system at all: + +- **OpenCode** and **OpenClaw** load in-process plugins. Setup writes a small generated + shim that calls the FailproofAI binary and translates the answer into the plugin's own + return shape. +- **Pi** loads extension packages. Setup registers the extension that ships inside the + FailproofAI package. +- **Goose** auto-discovers plugin directories. Setup simply drops the directory; Goose + registers it itself at startup. + +--- + +## Gateways behave differently from coding CLIs + +**Hermes** and **OpenClaw** are self-hosted assistants your team talks to from Slack, +Telegram, a terminal, or a schedule. Two consequences worth knowing: + +- **One install covers every channel.** Hooks fire on the *tool event*, not on the source, + so a single user-scope install intercepts Slack, Telegram, CLI, and scheduled runs + uniformly — and internal sub-agents too. No per-channel configuration. +- **There is no project scope**, because there is no project. Both are user-scope only. + +Because a gateway runs headless with no TTY, installing for Hermes also enables its +automatic hook consent so the gateway can run hooks without a prompt nobody is there to +answer. + + + **Blind spot worth naming:** a gateway that spawns a separate process (for example, via + a terminal tool) does not fire its hooks for the tool calls *inside* that process. Gate + the spawn at the tool event instead. + + +--- + +## Sessions from every CLI, in one place + +Enforcement is only half of it. FailproofAI also **reads** each CLI's session transcripts — +never modifying, moving, or deleting them — which is what powers the [local +dashboard](/dashboard), the [audit](/audit), and, on a connected machine, [everything the +cloud shows you](/cloud/sessions). + +All 12 CLIs are supported as session sources. Formats vary — some write JSONL transcripts, +some keep sessions in SQLite — and FailproofAI reads each one natively. Sessions from +CLIs with a working directory group by project; gateway sessions with no working directory +group by profile and channel instead. + +Keeping transcripts somewhere non-standard — a container mount, a second checkout, a +shared volume? Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path, so two +copies of the same project stay distinct instead of merging into one confusing timeline. +[Full command reference →](/cli/harness) + +--- + +## Adding a CLI later + +Nothing about setup is one-shot. Install a new agent CLI next month and: + +```bash +failproofai config +``` + +Re-running setup detects what is now on the machine and wires it up, keeping every policy +choice you already made. You can also install ahead of time — the hook entries are written +even for a CLI you have not installed yet, and activate the moment you do. + +--- + +## Related + + + + + What travels between the agent and the policy engine, and in which direction. + + + + All 39, including which events each one listens to. + + + + Scopes, merge rules, and per-policy parameters. + + + + Every flag on the install command. + + + diff --git a/docs/zh/agenteye/cli-and-agents.mdx b/docs/zh/agenteye/cli-and-agents.mdx deleted file mode 100644 index cec9188c..00000000 --- a/docs/zh/agenteye/cli-and-agents.mdx +++ /dev/null @@ -1,79 +0,0 @@ ---- -title: "CLI" -description: "您的整个 Failproof AI 可观测性部署，一条命令即可搞定。" ---- - -您的整个 Failproof AI 可观测性部署，一条命令即可搞定。无需离开终端，即可检查生产环境、创建 API 密钥或确认事件，还可以将任意操作编写成 CI 脚本，或者用自然语言让编程智能体替您完成。 - -```bash -pipx install agenteye -agenteye login --email you@example.com # 一个 6 位验证码将发送到您的邮箱 -agenteye --json sessions --since 24h # 过去一天的所有智能体运行记录，按最新排序 -``` - -*`agenteye` CLI 与您的仪表板通信，是一个独立工具，与负责将事件发送到服务器的采集器不同。* - -## 您的整个部署，一条命令即可搞定 - -不必再为一个简单问题而反复切换标签页。`agenteye` CLI 通过单一二进制文件读取您的数据并管理您的组织，原本需要在仪表板中点来点去才能完成的检查，现在只需一行命令即可复用、设置别名或粘贴到操作手册中。它提供四个操作入口： - -- **读取数据：** `sessions`、`events`、`evals` 和 `errors`，支持按时间、智能体和环境过滤。 -- **管理组织：** `keys`、`users`、`settings`、`alerts` 和 `incidents`。 -- **运行分析：** 支持保存的 SQL 查询，以及针对事件数据的即席 `query` 执行。 -- **询问助手：** `agent ask` 可访问与仪表板中相同的只读分析助手。 - -使用 `pipx` 一次性安装，通过邮件发送的 6 位验证码登录，即可开始使用。会话有效期约为一天，过期后重新运行 `agenteye login` 即可。无需打开浏览器，直接用它检查生产环境、创建密钥或快速处理正在触发的事件： - -```bash -agenteye errors --since 24h --aggregate # 查看故障情况，按错误类型分组 -agenteye incidents list --state firing # 查看当前正在触发的事件 -agenteye keys create ci --add events:add # 创建一个仅能推送事件的密钥，密钥值仅显示一次 -``` - -有一个使用习惯需要注意：`--json` 等全局选项必须放在命令之前。`agenteye --json sessions` 是正确的，`agenteye sessions --json` 则不行。 - -## 编写脚本，集成到 CI - -每条命令都支持 `--json`，这带来了质的变化。干净的 JSON 输出到 stdout，而人类可读的状态信息和警告则输出到 stderr，因此使用 `--json` 捕获的内容可以直接通过管道传给 `jq`，无需过滤多余的行。这也是 CLI 既适合您在终端直接使用，也适合编程智能体解析输出的原因： - -```bash -agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id' -``` - -它专为无人值守运行而设计。当没有终端连接时，确认提示会自动跳过，因此在管道中不会卡住，而且每条命令都会返回有意义的退出码：`0` 表示成功，`4` 表示未登录，`5` 表示缺少权限（消息中会指明具体权限，例如 `alerts:write`），`3` 表示仪表板无法访问。脚本可以根据 `4` 进行重新认证，或根据 `5` 精确告知需要向管理员申请哪些权限，而不是盲目失败。 - -## 用自然语言让编程智能体来驱动 - -更好的是，您根本不需要记住这些参数。**CLI 技能**是一个名为 `agenteye-cli` 的小型 Agent Skill 文件夹，它可以教会 Claude Code 或 Codex 等编程智能体通过自然语言来驱动 CLI。只需问"今天有什么问题吗？"，智能体就会选择合适的命令，以您的身份执行，并以文字形式给出答复。 - -对于 Claude Code，将 `agenteye-cli` 文件夹放入 `~/.claude/skills/` 即可自动发现。Failproof AI 可观测性提供该文件夹；无需额外安装任何内容，因为它只是驱动您已经安装的 CLI。请先自行登录：技能无法替您完成邮件验证码登录流程。 - -由于智能体以您的身份运行 CLI，它可以执行您的登录权限所允许的一切操作，包括读写：创建密钥、修改设置、解决事件。CLI 的"您确定吗？"提示对智能体不会触发，因此技能的设计是：在执行任何变更操作前，先明确说明将要执行的命令，并等待您确认。您就是那个确认步骤。 - -```text -您 为什么会话 run-001 失败了？ - -智能体 正在运行：agenteye --json events --session-id run-001 --all - checkout-agent 在第三次工具调用时遇到了 TimeoutError。 -``` - -读取操作立即完成，所有写入操作都会先等待您确认： - -```text -您 给 CI 创建一个只能推送事件的密钥。 - -智能体 这将创建一个 API 密钥，我将执行： - agenteye keys create ci --add events:add - 确认继续吗？ - -您 是的 - -智能体 完成。已创建密钥 "ci"，仅具有 events:add 权限。密钥值仅显示一次，请立即保存。 -``` - -## 相关链接 - -- [CLI 参考文档](/zh/agenteye/cli)：所有命令、参数和 JSON 格式说明。 -- [面向智能体的 CLI 使用示例](/zh/agenteye/cli-recipes)：可直接复制的 `jq` 模式和退出码处理方法。 -- [CLI 智能体技能](/zh/agenteye/cli-skill)：安装和运行 `agenteye-cli` 技能。 -- [AI 助手](/zh/agenteye/assistant)：仪表板内置分析助手，也是 `agent ask` 的访问目标。 \ No newline at end of file diff --git a/docs/zh/agenteye/cli-skill.mdx b/docs/zh/agenteye/cli-skill.mdx deleted file mode 100644 index 75f6023c..00000000 --- a/docs/zh/agenteye/cli-skill.mdx +++ /dev/null @@ -1,159 +0,0 @@ ---- -title: "Failproof AI 可观测性 CLI Agent Skill" -description: "向你的编程 Agent 询问「今天有什么问题吗？」，让它直接从你的实时 Failproof AI 可观测性数据中给出答案，无需记忆任何命令。" ---- - - -向你的编程 Agent 询问*「今天有什么问题吗？」*，让它直接从你的实时 Failproof AI 可观测性数据中给出答案，无需记忆任何命令。**Failproof AI 可观测性 CLI skill**（`agenteye-cli`）是一种 *Agent Skill*：一个包含说明文件的小型文件夹，供 Claude Code 或 Codex 等编程 Agent 按需加载。它使 Agent 能够通过 [`agenteye` CLI](/zh/agenteye/cli)，以自然语言请求（如*「给 CI 创建一个只能推送事件的密钥」*或*「确认正在触发的告警并将其分配给我」*）来操作你的可观测性部署。 - -它**不是**服务或独立的二进制文件，无需任何部署。它构建在你已安装的 CLI 之上：Agent 调用 `agenteye --json …`，解析干净的 JSON，然后用自然语言回答你。它能做的一切，你都可以自己输入相同的命令来完成。 - ---- - -## 与 Failproof AI 可观测性其他接口的关系 - -Failproof AI 可观测性提供四种方式访问相同的数据和控制功能，它们相互补充： - -| 接口 | 说明 | 运行环境 | 适用场景 | -|---|---|---|---| -| **[CLI](/zh/agenteye/cli)** | `agenteye` 的命令与参数参考文档 | 你的终端 | 需要运行或脚本化某个具体命令时 | -| **[CLI 使用示例](/zh/agenteye/cli-recipes)** | 可直接复制的 `jq`/管道模式 | 你的终端 / 脚本 | 将 CLI 集成到自动化流程时 | -| **CLI skill**（本文档） | 基于 CLI 的自然语言入口 | 你工作站上的编程 Agent | 想要直接提问、让 Agent 选择命令时 | -| **[Evaluator skill](/zh/agenteye/evaluator-skill)** | 用于设计和构建评分服务的同类 skill | 你工作站上的编程 Agent | 想要*生成*评估分数而非读取时 | -| **[Python SDK skill](/zh/agenteye/python-sdk-skill)** | 为你的 Agent 添加遥测数据发送能力的同类 skill | 你工作站上的编程 Agent | 想让你的 Agent *生成*本 skill 所读取的事件时 | -| **[仪表盘内置 AI 助手](/zh/agenteye/assistant)** | 内嵌于仪表盘的聊天功能 | 服务端（仪表盘内） | 需要在仪表盘中对数据进行问答时 | - -Skill 本身没有任何特权，它只是将你的语言转化为以你身份运行的 CLI 调用： - -```mermaid -flowchart TD - YOU["你：「确认正在触发的告警」"] --> AGENT["编程 Agent（Claude Code / Codex）
    加载 agenteye-cli skill"] - AGENT --> CLI["agenteye --json incidents ack ..."] - CLI -->|你已认证的 CLI 会话| API["可观测性仪表盘 API"] -``` - -### 与仪表盘内置 AI 助手的重要区别 - -这是两个截然不同的工具，影响范围差异显著： - -- **仪表盘内置 AI 助手**（[AI 助手](/zh/agenteye/assistant)）是嵌入仪表盘的聊天功能，由 Agent 服务提供支持。它**只读，且写入操作需要明确审批**：可以起草已保存的查询和仪表盘，但每次写入都会暂停并等待你的明确点击确认，且不会执行删除操作。它受 `agent:use` 权限限制，只能查看你当前所在组织的数据。 -- **CLI skill** 在*你的*工作站上运行，在*你的*编程 Agent 内部以**你的身份**驱动 `agenteye` CLI。它可以执行 CLI 的**全部功能，包括变更操作**（创建/轮换/禁用 API 密钥、修改组织设置、解决告警、删除已保存的查询），仅受你的 CLI 登录权限约束。请像对待手动输入这些命令一样谨慎对待它。 - ---- - -## 前置条件 - -1. 已安装 **`agenteye` CLI** 并添加到 `PATH`（参见 [CLI](/zh/agenteye/cli) 参考文档：`pipx install agenteye`）。 -2. 已设置**仪表盘 URL**（`AGENTEYE_DASHBOARD_URL`，或由 Agent 传入 `--base-url`）。 -3. 已**登录会话**：需先自行运行 `agenteye login`。Skill **无法**代你完成邮件一次性验证码登录；若会话缺失或过期（CLI 退出码 `4`），它会提示你运行 `agenteye login`。 - ---- - -## 获取方式 - -Skill 发布于 Failproof AI 的公开 skill 集合中： - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli) - -无任何访问限制——该仓库完全公开，skill 本身不需要任何凭证，因为它只是使用*你*登录的会话，通过**公开的** `agenteye` CLI 访问*你的*仪表盘。你无需向任何人申请。 - -请注意，它作为独立文件夹发布，**不包含**在 `pipx install agenteye` 包中，请勿在该包中查找。 - -## 安装 Skill - -最快捷的方式是使用 [`skills`](https://skills.sh) CLI，它会自动获取文件夹并放置到 Agent 的查找路径中： - -```bash -# Claude Code，仅限当前项目 -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code - -# 所有项目（安装至 ~/.claude/skills/） -npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy - -# 使用 Codex -npx skills add FailproofAI/skills --skill agenteye-cli -a codex -``` - -管理方式与其他 skill 相同： - -```bash -npx skills list -a claude-code # 查看已安装的 skill -npx skills update agenteye-cli # 拉取最新版本 -npx skills remove agenteye-cli # 移除 skill -``` - -prefer 手动安装？Agent Skill 本质上只是一个包含 `SKILL.md`（以及可选参考文件）的文件夹，直接复制即可： - -- **Claude Code**：将 `agenteye-cli/` 文件夹放入 `~/.claude/skills/`（所有项目）或 `<你的仓库>/.claude/skills/`（仅该仓库）。Claude Code 会自动发现它——通过 `/skills` 列表验证，或直接提问一个与其描述匹配的问题。 -- **Codex（OpenAI）**：Codex 读取相同的 `SKILL.md`。内置的 `agents/openai.yaml` 设置了 `allow_implicit_invocation: true`，因此当任务匹配时 Codex 会自动选择该 skill；否则可通过 `$agenteye-cli` 显式调用。 - ---- - -## 安全注意事项：Agent 运行 CLI 时变更操作不会出现确认提示 - -> **警告：** 在让 Agent 执行变更操作前，请先阅读本节。 - -`agenteye` CLI 通常会在执行破坏性操作前询问*「确定吗？」*。**当它未连接到终端时（这正是编程 Agent 的运行方式），该确认会被自动跳过；`--json` 参数也会跳过确认。** 因此，安全确认提示对 Agent **不会**触发。 - -Skill 的设计已对此进行补偿：它会在执行任何状态变更前，说明将要运行的确切命令，并等待你明确的**确认**。请保持这一规范。当你通过 Agent 操作 Failproof AI 可观测性时，*你*就是确认步骤。需要特别注意的变更类命令： - -- `keys create` / `update` / `disable` / `regenerate` -- `users create` / `update` / `disable` / `enable` -- `settings set` -- `alerts create` / `update` / `delete` / `test` -- 写入类 `incidents` 子命令：`ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe` -- `query create` / `update` / `delete` -- `agent rename` / `delete` -- `orgs switch` - -**观测**类命令（`events`、`sessions`、`evals`、`errors`、`list`、`whoami`、`orgs list/current/perms`）均为只读，不会产生任何变更。 - -由于 Agent 以**你的身份**操作，它只能执行你的登录所允许的操作；权限按**组织**级别解析（参见 [API 密钥](/zh/agenteye/api-keys)）。若某个命令超出你的权限，会返回退出码 `5` 并精确说明所缺权限，Agent 可以告诉你需要向管理员申请什么权限，而不是模糊地报错。 - ---- - -## 你可以问它什么 - -以下是真实的交互示例——先是一次只读查询，然后是一次等待你确认的变更操作： - -```text -you ▸ 最近一天有什么问题吗？ - -agent ▸ 正在运行：agenteye --json errors --since 24h --aggregate - 过去 24 小时内有 3 种错误类型。最多的是 TimeoutError - （18 次，全部发生在 checkout-agent），其次是 RateLimited（5 次）。 - 需要查看 TimeoutError 的失败会话吗？ - -you ▸ 给 CI 创建一个只能推送事件的密钥。 - -agent ▸ 这将创建一个 API 密钥。我将运行： - agenteye keys create ci --add events:add - 确认继续吗？ - -you ▸ 是的 - -agent ▸ 完成。密钥「ci」已创建，仅具有 events:add 权限。 - 密钥只显示一次，请立即保存。我无法再次显示它。 -``` - -Skill 将每种自然语言意图映射到正确的 `agenteye` 命令，会先查询有效值（`list `、`whoami`）而不是猜测，并在任何变更前说明确切命令。更多示例： - -- *「最近 24 小时有什么问题/故障吗？」* → `errors --since 24h --aggregate`，然后给出明细。 -- *「为什么会话 `run-001` 失败了？」* → `events --session-id run-001 --all` + `evals --session-id run-001`。 -- *「本周质量趋势如何？」* → `evals --aggregate --since 7d`，然后深入查看低分运行。 -- *「给 CI 创建一个只能推送事件的密钥。」* → `keys create ci --add events:add`（说明命令后创建，并捕获一次性密钥）。 -- *「谁有访问权限？将 Dana 设为只读。」* → `users list` → `users update dana@… --permission-set read-only`（向你确认后执行）。 -- *「确认正在触发的告警并分配给我。」* → `incidents list --state firing` → `incidents ack ` / `incidents assign you@…`。 - -有关这些操作背后的确切命令、参数和 JSON 格式，请参见 [CLI](/zh/agenteye/cli) 参考文档和 [Agent 的 CLI 使用示例](/zh/agenteye/cli-recipes)。 - ---- - -## 下一步 - -- **[CLI](/zh/agenteye/cli)**：`agenteye` 完整命令与参数参考文档。 -- **[Agent 的 CLI 使用示例](/zh/agenteye/cli-recipes)**：可直接复制的 `jq` 模式和退出码处理方法。 -- **[Evaluator agent skill](/zh/agenteye/evaluator-skill)**：同类 skill，用于构建 `agenteye evals` 读取其分数的评估器。 -- **[Python SDK agent skill](/zh/agenteye/python-sdk-skill)**：同类 skill，用于为 Agent 添加遥测数据发送能力，使 `agenteye` 能够读取相应数据。 -- **[AI 助手](/zh/agenteye/assistant)**：仪表盘内置助手（与本终端 skill 不同）。 -- **[API 密钥](/zh/agenteye/api-keys)**：限定 skill 可执行操作范围的按组织权限模型。 \ No newline at end of file diff --git a/docs/zh/agenteye/codex-capture.mdx b/docs/zh/agenteye/codex-capture.mdx deleted file mode 100644 index 480e4bdb..00000000 --- a/docs/zh/agenteye/codex-capture.mdx +++ /dev/null @@ -1,55 +0,0 @@ ---- -title: "Codex 会话捕获" -description: "将团队本地 OpenAI Codex 会话以普通会话和事件的形式导入 AgentEye，无需改变现有的 Codex 使用方式。" ---- - -您的工程师每天都在使用 OpenAI Codex。Codex 会话捕获功能将这些编码会话以普通会话和事件的形式引入 AgentEye，让您可以对其进行搜索、回放，并与其他所有观测数据放在一起进行评估。它与 [Python SDK](/zh/agenteye/python-sdk) 相辅相成：SDK 用于对您自己编写的 Agent 进行埋点，而此功能则捕获团队日常使用 Codex 产生的会话数据——无需任何改动。 - -一个轻量级的后台采集器会在 Codex 本地会话记录写入时实时读取，并将其上报至 AgentEye。每台机器只需部署一个采集器，即可同时捕获所有本地 Codex 界面的数据，无需逐一配置。 - -同一采集器还支持其他 Agent 的捕获——详见 [OpenClaw](/zh/agenteye/openclaw-capture) 和 [Hermes](/zh/agenteye/hermes-capture)。您可以按需启用，单个采集器能够同时捕获多个来源。 - ---- - -## 捕获内容 - -所有**本地**运行的 Codex 界面都会生成相同的磁盘会话记录，采集器会统一采集以下来源： - -- Codex **CLI** 及 `codex exec` -- **VS Code / IDE 扩展** -- **桌面应用**（仅限本地执行的会话） - -每个 Codex 会话都会成为 AgentEye 中的一个[会话](/zh/agenteye/sessions)；其中的用户消息、助手消息、推理过程、工具调用、工具结果以及 Token 用量将转化为对应的[事件](/zh/agenteye/event-stream)。每个会话的来源界面（CLI、IDE 或桌面应用）均会被记录，便于您加以区分。 - -> **云端会话不在捕获范围内。** 桌面应用越来越多地将会话运行在 Codex 云端，本地仅保留元数据，没有可供读取的本地记录。只有本地执行的会话才会被捕获。 - ---- - -## 启用捕获 - -捕获功能默认关闭，需手动启用。请使用具有 `events:add` 权限的 API 密钥（参见 [API 密钥](/zh/agenteye/api-keys)）安装采集器，并开启 Codex 捕获： - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --codex-enabled -``` - -该命令将安装采集器、将其注册为后台服务并立即开始捕获。运行以下命令确认服务正常运行： - -```bash -agenteye-collector health -``` - -首次运行时，现有的 Codex 会话将被一次性回填，此后的新活动将在数秒内实时上报。采集器对 Codex 的本地文件仅执行读取操作，不会对其进行修改、移动或删除，且每个会话只会上报一次，重启后亦然。 - ---- - -## 数据展示 - -捕获的会话将出现在 **Sessions** 中，其事件将出现在 **Events** 流中，与您观测的其他 Agent 数据完全一致——因此[会话回放](/zh/agenteye/sessions)、[搜索](/zh/agenteye/queries)、[评估](/zh/agenteye/evaluations)和[告警](/zh/agenteye/alerts)均可正常使用。通过筛选 Codex Agent，可单独查看其数据。 - ---- - -## 隐私说明 - -Codex 会话记录包含完整的会话内容——包括命令输出、文件内容以及 Codex 读写的所有信息——可能涉及敏感数据。捕获的会话将原样上报，因此请仅在适合将相关内容集中存储至 AgentEye 的机器和团队中启用此功能，并为采集器配置仅限 `events:add` 权限的密钥。数据隔离机制详见[安全说明](/zh/agenteye/security)。 \ No newline at end of file diff --git a/docs/zh/agenteye/concepts.mdx b/docs/zh/agenteye/concepts.mdx deleted file mode 100644 index 1ec4cb14..00000000 --- a/docs/zh/agenteye/concepts.mdx +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: "概念" -description: "Failproof AI 可观测性的术语词汇表——事件、会话、评估、审计、发现与事件单——集中定义于此。" ---- - - -本页定义了 Failproof AI 可观测性所使用的术语词汇。如果您在其他指南中遇到不熟悉的术语，可在此查找其定义。无需从头到尾通读：快速浏览即可，或在遇到需要明确含义的词汇时随时跳回查阅。 - ---- - -## 数据模型 - -**事件（Event）** -最小数据单元。一条事件记录了您的 Agent 执行的单个步骤：一次 `tool_use`、一次 `model_request`、一次 `hook_completed`、一次 `error` 等等。您的 Agent 通过 [Python SDK](/zh/agenteye/python-sdk) 发出事件，这些事件会实时显示在**事件（Events）**页面上。 - -**会话（Session）** -一次 Agent 运行，以 `session_id` 标识。会话是共享同一 ID 的所有事件的集合，在**会话（Sessions）**页面上汇总为单行，并在详情页上以执行图的形式呈现。会话通常以 `agent_start` 开始，以 `agent_end` 结束。 - -**Agent** -一次运行中具名的参与者，以 `agent_id` 标识。一次运行可以涉及多个 Agent：例如一个规划器（planner）派生出一个摘要子 Agent。子 Agent 携带 `parent_id`，正是这个字段使 Failproof AI 可观测性能够在执行图中将它们绘制在各自的泳道上。 - -**环境（Environment）** -标识运行发生位置的标签：`production`、`staging`、`dev`。您在配置 SDK 时设置一次，几乎每个仪表板页面都可以按环境进行筛选。 - -**上下文窗口占用率（Context-window fill）** -模型响应所消耗的上下文窗口百分比。Failproof AI 可观测性会在其识别的模型的 `model_response` 事件上标记该数值，使提示词增长趋势和即将到来的压缩行为直接在事件流中可见。 - ---- - -## 质量 - -**评估（Evaluation）** -由您运行的评分服务为已完成会话生成的质量分数。评估为选装功能：在接入评估器之前，会话只会被记录，不会被评分。每次评估可以包含多个命名维度的分数（例如 `helpfulness`、`factuality`、`tool_efficiency`），每个维度附带简短的推理说明。请参阅[评估套件（Evaluation suite）](/zh/agenteye/evaluation-suite)。 - -**分数键（Score key）** -评估器报告的某一维度名称，例如 `helpfulness`。告警规则和审计可以随时间追踪特定的分数键。 - -**评估器（Evaluator）** -您的评分服务。Failproof AI 可观测性会将已完成运行的对话记录以 POST 方式发送给它，并存储其返回的分数。平台不内置默认评估器，评分逻辑由您自行实现。 - ---- - -## 发现与修复故障 - -**Hook** -您的 Agent 框架在某个步骤前后运行的守卫或副作用逻辑：内容安全检查、PII 脱敏、预算限制等。Hook 会发出 `hook_triggered` / `hook_completed` 事件，携带 `outcome`（allow、deny、modify），并拥有独立的观测页面。 - -**告警规则（Alert rule）** -当指标超过您设定的阈值时触发的规则：错误率、p95 延迟、Token 成本或评估分数。规则触发时，会创建一个事件单并通过您选择的渠道（邮件、Slack、Webhook、仪表板内）发送通知。请参阅[告警（Alerts）](/zh/agenteye/alerts)。 - -**事件单（Incident）** -告警规则触发时创建的待处理问题。事件单具有生命周期（确认、分配、解决）以及记录每次操作的活动时间线。您也可以手动创建事件单。 - -**审计（Audit）** -定期（每小时至每周）运行的调查任务，在*跨会话*的日志中挖掘您尚未编写规则的故障模式：错误聚类、低分、延迟异常值、工具调用循环以及未正常结束的运行。告警监控的是您已知的指标，而审计则告诉您下一步应该关注什么。请参阅[审计（Audits）](/zh/agenteye/audits)。 - -**发现（Finding）** -一次审计运行产出的带有排名和证据支撑的结果。发现会描述一种模式，关联到其背后的具体会话，并具有分级处理生命周期（确认、解决、静默、忽略）。Failproof AI 可观测性会对多次运行间的发现进行去重，使已知模式得到更新而不是不断堆积。 - -**AI 助手（The AI assistant）** -仪表板内的对话工具，能够用自然语言回答关于您的 Agent 的问题，基于您自己的数据。默认为只读模式；其创建的任何内容（已保存的查询、仪表板）均需审批，且无法执行删除操作。请参阅 [AI 助手（AI assistant）](/zh/agenteye/assistant)。 - ---- - -## 运行方式 - -**组织（Organization / 租户）** -隔离的工作空间。一个 Failproof AI 可观测性实例可以托管多个组织，每个组织拥有独立的用户、密钥和数据。所有仪表板 URL 均限定在您的组织标识符下（`//…`）。 - -**采集器（Collector）** -`agenteye-collector`，运行在每台 Agent 机器上的轻量级守护进程，负责批量处理 SDK 写入磁盘的事件并将其发送到服务器。 - -**API 密钥（API key）** -用于向服务器验证客户端身份的作用域令牌。密钥携带细粒度权限（例如，采集器使用 `events:add`，仪表板密钥使用只读作用域）。请参阅 [API 密钥（API keys）](/zh/agenteye/api-keys)。 - -**服务器（Server）** -数据采集和 API 服务。负责采集事件、将运行状态存储到您的数据库中，并提供仪表板和 CLI 服务。 - -**仪表板（Dashboard）** -Web 界面。每个页面均限定在某个组织范围内，通过服务器 API 读取数据。 - ---- - -## 后续步骤 - -- [概览（Overview）](/zh/agenteye/overview)：了解各组件如何协同工作。 -- [可观测性（Observability）](/zh/agenteye/observability)：各观测页面（Events、Sessions、Models、Tools、Hooks、Errors）的详细介绍。 \ No newline at end of file diff --git a/docs/zh/agenteye/evaluator-skill.mdx b/docs/zh/agenteye/evaluator-skill.mdx deleted file mode 100644 index eb24fb2d..00000000 --- a/docs/zh/agenteye/evaluator-skill.mdx +++ /dev/null @@ -1,167 +0,0 @@ ---- -title: "Failproof AI 可观测性评估器 Agent 技能" -description: "让您的编程 Agent 既负责决策又负责构建，从「我觉得我们的 Agent 有时表现很差」直接走向部署完毕的评分服务。" ---- - - -让您的编程 Agent 既负责决策又负责构建，从*「我觉得我们的 Agent 有时表现很差」*直接走向部署完毕的评分服务。**Failproof AI 可观测性评估器技能**（`agenteye-evaluator`）是一种 *Agent Skill*：一个小型指令文件夹，供 Claude Code 或 Codex 等编程 Agent 按需加载。它能引导 Agent 确定哪些质量维度值得为*您的* Agent 跟踪，然后编写、测试并部署对这些维度进行评分的[评估器服务](/zh/agenteye/evaluation-suite)。 - -它**不是**一个托管评分器、一个您上传到的注册表，也不是插件系统。您的评估器始终是运行在您自己基础设施上的 HTTP 服务，与[评估套件](/zh/agenteye/evaluation-suite)指南中所描述的完全一致。该技能只是教您的 Agent 如何把它构建好——它所做的一切，您完全可以自己动手写同样的代码来实现。 - ---- - -## 难点在于决定评分什么 - -SDK 接口很简洁——一个装饰器和两个模型——Agent 仅凭[契约](/zh/agenteye/evaluation-suite#http-contract)就能把代码写出来。评估器真正的失败之处不在这里。它们失败是因为评错了东西，而评错对象的评估器比没有还糟：它产出的仪表盘会让所有人习惯性地无视。 - -因此，该技能的大部分工作发生在任何代码存在之前。它让 Agent 对您进行访谈（*「描述一次进展顺利的运行；再描述一次进展糟糕的」*），然后通过 [`agenteye` CLI](/zh/agenteye/cli) 提取您的真实会话并从头到尾阅读。这两部分通常会出现分歧，而这个差距正是关键所在：您打算衡量什么，与您的对话记录实际上能支撑什么，往往并不一致。一个维度只有在**可从事件中计算**且**具有区分度**时才能保留——如果它在您的好运行和差运行上都打出 0.9 分，那什么也说明不了，直接剔除。 - -最终返回的是一份包含 2-4 个维度的提案，附带推理说明，供您在写下任何一行代码之前确认。 - -```mermaid -flowchart TD - YOU["您：「我想为我的支持机器人做评估」"] --> AGENT["编程 Agent（Claude Code / Codex）
    加载 agenteye-evaluator 技能"] - AGENT -->|"访谈：好的表现和差的表现分别是什么样的？"| YOU - AGENT -->|"agenteye --json sessions / events"| DATA["您的真实会话
    实际发生的情况"] - DATA --> DIMS["2-4 个维度，由您确认"] - DIMS --> SVC["您的评估器服务
    agenteye-evaluator SDK"] - SVC --> SCORES["评分出现在仪表盘
    和 agenteye evals 中"] -``` - ---- - -## 与其他评估组件的关系 - -共有四份文档涵盖评分相关内容，它们按顺序相互衔接： - -| 页面 | 内容 | 适用场景 | -|---|---|---| -| **[评估（Evaluations）](/zh/agenteye/evaluations)** | 该功能：会话网格上的评分、仪表盘、重新评估 | 您想了解自动评分能带来什么 | -| **[评估套件（Evaluation suite）](/zh/agenteye/evaluation-suite)** | HTTP 契约、SDK、服务器环境变量 | 您正在自行实现或调试评估器 | -| **评估器技能**（本文档） | 设计*并*构建评分器的自然语言入口 | 您想从「我想要评估」走到一个正在运行的服务 | -| **[CLI 技能](/zh/agenteye/cli-skill)** | `agenteye` CLI 的自然语言入口 | 您想*读取*已有的评分结果 | -| **[Python SDK 技能](/zh/agenteye/python-sdk-skill)** | 为您的 Agent 添加埋点的自然语言入口 | 您的 Agent 尚未输出会话——还没有东西可以评分 | - -### 与 CLI 技能的区别：构建 vs. 读取 - -这两个技能在职责上刻意不重叠，同时安装两者是常规配置——Agent 会根据您的提问在二者之间切换： - -- **`agenteye-evaluator`**（本文档）构建*产生*评分的东西。它的任务在评分首次出现时结束。 -- **[`agenteye-cli`](/zh/agenteye/cli-skill)** 读取已存在的评分（`agenteye evals`）。「本周质量下降了吗？」是它回答的问题，不是本技能的职责。 - ---- - -## 前提条件 - -1. **已安装并登录 `agenteye` CLI**（`pipx install agenteye`，然后 `agenteye login`）。该技能会用到它两次：拉取真实会话用于设计，以及在最后确认评分是否落地。您的登录账户需要 `events:read` 权限，以及用于最终检查的 `evaluations:read` 权限。与 CLI 技能一样，它**无法**替您完成邮件一次性验证码登录。 -2. **一个放置评估器的地方。** 评估器会被构建成镜像并作为长期运行的服务运行，因此它需要一个真实的代码仓库，而不是临时文件。评估器通常独立存在于自己的仓库中，与被评分的 Agent 分开——该技能会寻找现有仓库，并在搭建新仓库之前征询您的意见。 -3. **`agenteye-evaluator` SDK wheel**——在让您的 Agent 开始输入 `pip` 命令之前，请先阅读下一节。 - ---- - -## 获取方式 - -该技能发布于 Failproof AI 的公共技能集合中： - -**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-evaluator/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-evaluator) - -该仓库是公开的，技能本身不需要任何凭据——它只是用*您*登录时的会话驱动 `agenteye` CLI，并在*您的*仓库中写代码。请注意，它以独立文件夹的形式发布，**不在** `pipx install agenteye` 包内，请勿在那里寻找它。 - -## 安装技能 - -最快的方式是使用 [`skills`](https://skills.sh) CLI，它会拉取文件夹并放到您的 Agent 查找的位置： - -```bash -# Claude Code，仅限当前项目 -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code - -# 所有项目（安装到 ~/.claude/skills/） -npx skills add FailproofAI/skills --skill agenteye-evaluator -a claude-code -g --copy - -# 改用 Codex -npx skills add FailproofAI/skills --skill agenteye-evaluator -a codex -``` - -然后像管理其他技能一样管理它： - -```bash -npx skills list -a claude-code # 查看已安装的技能 -npx skills update agenteye-evaluator # 拉取最新版本 -npx skills remove agenteye-evaluator # 移除它 -``` - -喜欢手动安装？Agent Skill 只是一个包含 `SKILL.md`（以及可选引用文件）的文件夹，直接复制也可以： - -- **Claude Code**：将 `agenteye-evaluator/` 文件夹放入 `~/.claude/skills/`（所有项目）或 `/.claude/skills/`（仅该仓库）。Claude Code 会自动发现它——通过 `/skills` 列表验证，或者直接询问评估相关问题即可。 -- **Codex（OpenAI）**：Codex 读取同一个 `SKILL.md`。捆绑的 `agents/openai.yaml` 设置了 `allow_implicit_invocation: true`，因此当任务匹配时 Codex 会自动选择该技能；否则可以通过 `$agenteye-evaluator` 显式调用它。 - ---- - -## SDK 不在公共 PyPI 上 - -> **警告：** 在让 Agent 安装 SDK 之前，请先阅读本节。 - -该技能是公开的；它所驱动的 SDK 则不是。`agenteye-evaluator` 仅作为私有发布产物发布，且与 `agenteye` 不同，该名称在**公共 PyPI 上尚未被注册**——因此直接执行 `pip install agenteye-evaluator` 可能会将陌生人的包安装到读取您生产对话记录的服务中。这是一个供应链问题，而不是笔误。 - -该技能了解这一点，因此会按照安装梯队依次尝试，在第一个适用的环节停下：如果您在 AgentEye 仓库内，则使用 monorepo 源码；否则使用 GitHub Releases 上的私有发布 wheel（需要访问权限）；如果两者都无法访问，它会**停止并告诉您联系 Failproof AI 联系人获取 wheel**，而不是自行发挥。 - -因此，如果您的 Agent 提议从公共 PyPI 直接执行 `pip install agenteye-evaluator`，这就说明该技能根本没有加载。请立即停止并检查技能是否已安装。 - ---- - -## 可以问它什么 - -一次真实的完整流程从模糊的需求开始，以一个经过确认的设计方案结束，而不是直接以代码开始： - -```text -您 ▸ 我想为我们的支持机器人做评估。我觉得它有时表现很差。 - -Agent ▸ 在我写任何东西之前：描述一次进展顺利的运行， - 以及一次进展糟糕的。 - -您 ▸ 好的情况：它从文档中给出答案。差的情况：它胡编乱造， - 或者放弃并说「请联系支持」。 - -Agent ▸ 正在运行：agenteye --json sessions --status error --since 7d - 读取 5 个会话，包括您认为表现差的两个。 - - 您的描述说的是事实性和推诿。对话记录在推诿方面 - 与描述吻合（5 个中有 2 个以转接支持结束， - 在 model_response.content 中清晰可见），但事实性 - 需要您的事件中没有的基准真值。 - - 因此我会评分三件事： - - deflection 基于规则，成本低，能区分您的两次运行 - - tool_efficiency 基于规则，能捕捉 run-004 中的重试循环 - - groundedness LLM 评判：每个声明都在它检索的文档中吗？ - - 「事实性」需要我们没有的基准真值；groundedness - 是您的对话记录实际上能支撑的最接近的替代。 - 这三个名称可以吗？之后重命名会导致历史记录断裂。 - -您 ▸ 好的 - -Agent ▸ 这个仓库中没有评估器。要在这里搭建一个， - 还是您在别处已经有了？ -``` - -之后，它会先编写基于规则的维度（免费、即时、确定性），针对真实捕获的会话（包括那些会让朴素评估器崩溃的空会话和未完成会话）进行测试，只在主观维度上才会使用 LLM 评判器。它了解[调度器的限制](/zh/agenteye/evaluation-suite#configuring-the-server)——30 秒请求超时和全局 8 个并发调用——因此如果评判器不能可靠地在时限内完成，它会使用 `JobPending` 异步处理，而不是让您的评判器被取消并以五倍成本重试五次。 - -然后它进行部署，设置两个服务器环境变量，并通过 `agenteye --json evals --session-id ` 确认评分确实落地。评分落地是唯一的证明。 - ---- - -## 需要注意的事项 - -- **维度名称几乎是永久性的。** 评分键是任意字符串，平台会对您发送的任何内容进行趋势分析，这意味着下游没有任何东西能纠正一个错误的选择。之后重命名会导致历史记录断裂：旧会话保留旧键，趋势就此中断。这就是为什么该技能在写代码之前要明确征得您的同意——请认真对待那个提示。 -- **测试夹具是真实的生产对话记录。** 针对真实会话进行设计意味着要将它们拉取到磁盘上，而它们可能包含客户数据。该技能会在将其提交到 git 之前征询您的意见；如有疑虑，请将 `fixtures/` 排除在仓库之外，让每位开发者自行拉取。 -- **Agent 会编写并部署一个读取所有对话记录的服务。** 它以您的身份行事，受您的 CLI 登录权限约束，但请像审查其他接触生产数据的代码一样审查评估器。 - ---- - -## 后续步骤 - -- **[评估套件（Evaluation suite）](/zh/agenteye/evaluation-suite)**：HTTP 契约、SDK 以及该技能所配置的服务器环境变量。 -- **[评估（Evaluations）](/zh/agenteye/evaluations)**：评分落地后出现的位置。 -- **[CLI 技能](/zh/agenteye/cli-skill)**：与本技能配套的技能，用于读取结果而非构建评分器。 -- **[CLI](/zh/agenteye/cli)**：该技能所依赖的会话数据背后的命令参考。 \ No newline at end of file diff --git a/docs/zh/agenteye/hermes-capture.mdx b/docs/zh/agenteye/hermes-capture.mdx deleted file mode 100644 index 606645f9..00000000 --- a/docs/zh/agenteye/hermes-capture.mdx +++ /dev/null @@ -1,53 +0,0 @@ ---- -title: "Hermes 会话捕获" -description: "将团队的 Hermes 网关会话（包括 Slack、Telegram、CLI 和定时运行）作为普通会话和事件引入 AgentEye。" ---- - -[Hermes](https://hermes-agent.nousresearch.com) 可以在团队常用的任意平台上为他们提供解答——Slack、Telegram、CLI 或定时运行。Hermes 会话捕获功能将所有这些内容作为普通会话和事件引入 AgentEye，让团队每天交互的助手与你自己编写的 Agent 一样具备可观测性。 - -一个小型后台采集器会在 Hermes 本地会话存储写入时读取其内容，并将会话发送至 AgentEye。其工作方式与 [Codex](/zh/agenteye/codex-capture) 和 [OpenClaw](/zh/agenteye/openclaw-capture) 捕获相同，且单个采集器可以同时捕获多个来源。 - ---- - -## 捕获内容 - -机器上的每一个 Hermes 会话都会被捕获，无论来自哪个渠道。每个会话都会成为 AgentEye 中的一个 [session（会话）](/zh/agenteye/sessions)；其中的用户消息、助手消息、工具调用和工具结果将成为对应的 [events（事件）](/zh/agenteye/event-stream)。 - -会话的来源渠道——Slack、Telegram、CLI 或定时运行——会被记录在会话上，便于区分和按渠道筛选。同时还会记录会话运行所用的模型、发起会话的聊天窗口和用户，以及当某个会话由另一个会话派生时，其指向父会话的关联链接。 - -无论是否已有消息，只要 Hermes 启动会话，该会话即可立即呈现；每轮对话的回复及其工具调用均按实际发生顺序排列。会话结束时，你还可以获取会话结束的原因、消耗的费用以及使用的 token 数量。 - ---- - -## 启用方法 - -捕获功能默认关闭，需手动启用。使用具有 `events:add` 权限的 API 密钥（参见 [API keys](/zh/agenteye/api-keys)）安装采集器，并开启 Hermes 捕获： - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --hermes-enabled -``` - -该命令会安装采集器、将其注册为后台服务并开始捕获。确认其运行状态： - -```bash -agenteye-collector health -``` - -需要在同一台机器上捕获多个 Agent？在同一命令中添加各自的标志即可，例如 `--hermes-enabled --codex-enabled`。 - -首次运行时，已有的 Hermes 会话会被一次性回填，此后新活动将在数秒内实时流入。Hermes 自身的数据仅会被读取，不会被修改或删除；即使采集器重启，每条消息也只会发送一次。 - -`health` 命令还会显示采集器捕获的内容是否全部成功到达 AgentEye。若某批数据无法投递，会被保留并重试，而非丢弃；只要有任何数据仍在等待中，检查结果就会显示为不健康——因此"健康"意味着数据已成功送达，而非仅仅进程在运行。 - ---- - -## 数据呈现位置 - -捕获的会话出现在 **Sessions** 中，其事件出现在 **Events** 流中，与其他被观测的 Agent 完全一致——因此 [session replay（会话回放）](/zh/agenteye/sessions)、[search（搜索）](/zh/agenteye/queries)、[evaluations（评估）](/zh/agenteye/evaluations) 和 [alerts（告警）](/zh/agenteye/alerts) 均可对其使用。通过筛选 Hermes Agent 可单独查看这些会话。 - ---- - -## 隐私说明 - -Hermes 会话包含完整的对话记录——包括命令输出、文件内容以及 Agent 读取或写入的任何内容——并可能包含敏感信息。捕获的会话将原样发送，因此请仅在将该内容集中存储至 AgentEye 合适的场景下启用捕获功能，并为采集器提供仅限 `events:add` 权限范围的密钥。数据隔离保护的详细说明请参见 [Security（安全性）](/zh/agenteye/security)。 \ No newline at end of file diff --git a/docs/zh/agenteye/observability.mdx b/docs/zh/agenteye/observability.mdx deleted file mode 100644 index d428e307..00000000 --- a/docs/zh/agenteye/observability.mdx +++ /dev/null @@ -1,23 +0,0 @@ ---- -title: "观察" -description: "观察界面让你实时查看 Agent 的运行情况，并深入分析任意单次运行记录。" ---- - - -观察界面让你实时查看 Agent 的运行情况，并深入分析任意单次运行记录。这里的所有内容都是实时的，以组织为范围，可按日期范围、环境、Agent 和会话进行筛选，让你能在几秒内从"感觉有点不对"定位到具体的运行记录。 - -![实时事件流，按类型进行颜色标注，可按环境、Agent 和会话筛选](/agenteye/images/events-stream.png) - -四个界面，各有独立页面： - -- **[事件流](/zh/agenteye/event-stream)**：所有 Agent 每次运行的实时逐步追踪记录，最新在前。是你组织的首页，也是分诊排查的第一站。 -- **[会话与执行图](/zh/agenteye/sessions)**：将事件汇总为每次运行一行，并以类似 Git 的图形展示每次运行的展开过程。 -- **[性能指标](/zh/agenteye/telemetry)**：模型、工具和 Hook 的延迟热图及 p50/p95/p99 关键指标，让尾部毛刺从中位数中一眼凸显。 -- **[错误追踪](/zh/agenteye/error-tracking)**：统一呈现所有异常情况的分诊界面，一键从触发的告警跳转到出问题的运行记录。 - -## 相关内容 - -- [评估](/zh/agenteye/evaluations)：对每次运行进行质量评分。 -- [告警](/zh/agenteye/alerts)：将任意阈值转化为告警规则。 -- [审计](/zh/agenteye/audits)：让 Failproof AI Observability 自动为你发现跨会话的故障模式。 -- [CLI 与 Agent](/zh/agenteye/cli-and-agents)：在终端中获得同等的可观测能力。 \ No newline at end of file diff --git a/docs/zh/agenteye/openclaw-capture.mdx b/docs/zh/agenteye/openclaw-capture.mdx deleted file mode 100644 index 0e5b540e..00000000 --- a/docs/zh/agenteye/openclaw-capture.mdx +++ /dev/null @@ -1,49 +0,0 @@ ---- -title: "OpenClaw 会话捕获" -description: "将团队本地的 OpenClaw 会话作为普通会话和事件接入 AgentEye，无需更改 OpenClaw 的运行方式。" ---- - -如果你的团队使用 [OpenClaw](https://docs.openclaw.ai)，OpenClaw 会话捕获功能可将这些会话作为普通会话和事件引入 AgentEye，方便你与其他观测数据一起进行搜索、回放和评估。该功能与 [Python SDK](/zh/agenteye/python-sdk) 互为补充：SDK 用于对你自行编写的 Agent 进行插桩，而本功能则捕获团队日常使用 OpenClaw 所产生的工作内容——无需改变任何使用习惯。 - -一个轻量级后台采集器会在 OpenClaw 本地会话记录写入时实时读取，并将其传输至 AgentEye。其工作方式与 [Codex 捕获](/zh/agenteye/codex-capture) 相同，且同一采集器可同时捕获两者。 - ---- - -## 捕获内容 - -机器上 OpenClaw 配置中的每个 Agent 都会被该机器的采集器捕获，无需针对单个 Agent 进行额外配置。 - -每个 OpenClaw 会话对应 AgentEye 中的一个[会话](/zh/agenteye/sessions)；其用户消息、助手消息、工具调用及工具结果将成为对应的[事件](/zh/agenteye/event-stream)。 - ---- - -## 开启方式 - -捕获功能默认关闭，需手动启用。使用具有 `events:add` 权限的 API 密钥（参见 [API 密钥](/zh/agenteye/api-keys)）安装采集器，并开启 OpenClaw 捕获： - -```bash -curl -fsSL https://raw.githubusercontent.com/FailproofAI/agenteye-collector/main/install.sh \ - | sh -s -- --key --openclaw-enabled -``` - -该命令会安装采集器、将其注册为后台服务并开始捕获。确认服务正在运行： - -```bash -agenteye-collector health -``` - -同一台机器上需捕获多个 Agent？在同一命令中添加各自的标志即可，例如 `--openclaw-enabled --codex-enabled`。 - -首次运行时，已有的 OpenClaw 会话会被一次性回填，此后的新活动将在数秒内实时传输。OpenClaw 的本地文件仅供读取，不会被修改、移动或删除；每个会话即使跨越重启也只会被传输一次。 - ---- - -## 数据呈现位置 - -捕获的会话显示在 **Sessions** 中，其事件显示在 **Events** 流中，与其他被观测的 Agent 完全一致——因此[会话回放](/zh/agenteye/sessions)、[搜索](/zh/agenteye/queries)、[评估](/zh/agenteye/evaluations)和[告警](/zh/agenteye/alerts)均适用。按 OpenClaw Agent 进行筛选可单独查看其数据。 - ---- - -## 隐私说明 - -OpenClaw 记录包含完整的会话内容——包括命令输出、文件内容以及 Agent 读写的所有信息——可能涉及敏感数据。捕获的会话将原样传输，因此请仅在适合将相关内容集中存储至 AgentEye 的机器和团队中启用捕获功能，并将采集器的密钥权限限定为仅 `events:add`。有关数据隔离保护措施，请参阅[安全性](/zh/agenteye/security)。 \ No newline at end of file diff --git a/docs/zh/agenteye/python-sdk-skill.mdx b/docs/zh/agenteye/python-sdk-skill.mdx deleted file mode 100644 index 7c99880c..00000000 --- a/docs/zh/agenteye/python-sdk-skill.mdx +++ /dev/null @@ -1,131 +0,0 @@ ---- -title: "Failproof AI Observability Python SDK Agent Skill" -description: "从未插桩的 Agent 到可观测的事件——让你的编码 Agent 找到插桩点、完成编写并验证落地。" ---- - -告诉你的编码 Agent *"为这个 Agent 添加 Failproof AI Observability"*，让它读取你的循环逻辑，找出插桩位置，完成编写，并在宣告任务完成之前验证事件是否正常产生。 - -**Python SDK skill**（`agenteye-python-sdk`）是一个 *Agent Skill*：一个包含指令的文件夹，当任务与之匹配时，Claude Code 或 Codex 等编码 Agent 会按需加载它。它教会 Agent 使用 [Python SDK](/zh/agenteye/python-sdk)——它本身不是一个库，也不会改变 SDK 的任何工作方式。 - -## 插桩容易写，也容易悄无声息地出错 - -SDK 很小巧：十三个事件方法，全部仅支持关键字参数。编码 Agent 读完 [Python SDK](/zh/agenteye/python-sdk) 参考文档，一分钟内就能写出看似合理的插桩代码。 - -问题在于，这个 SDK 在你出错时不会抛出异常，而错误的插桩和正确的插桩看起来一模一样——直到有人打开仪表板，发现什么都没有。真正浪费时间的错误都是「沉默型」的： - -| 错误类型 | 你看到的现象 | -|---|---| -| 缺少 `agent_start` | 每个事件都落地，零个 session。 | -| 环境变量从未设置 | 一切正常运行，但都归档在 `dev` 下。 | -| `outcome="failure"` | 运行显示绿色——只有 `failed`、`error`、`timeout`、`rejected` 才会被计入。 | -| 字段名拼写错误 | 被接受并存储为新字段。 | -| 从线程池中发送事件 | 被静默丢弃。 | - -这些错误都不会抛出异常，也不会在测试中暴露。每一种都已在 skill 中说明，并以合约形式附上对应的检测方法。 - -## 它的执行步骤 - -该 skill 会执行经验丰富的工程师会做的三个步骤： - -1. **规划。** 读取你的 Agent 循环，并提出只有你能回答的两个问题：什么算作一次运行（你的 `session_id`），以及哪些是可区分的执行者（你的 `agent_id`）。它会在开始写代码之前就这些问题达成共识，因为事后修改会导致历史数据分裂，趋势图也会随之断裂。 -2. **编写。** 在每次运行时绑定一次身份，而不是在每个调用点都传递一遍；并选择并发安全的实现方式——这一点很重要，因为看似简便的做法会悄悄地将两个并发运行混入同一个 session，而且毫无提示。 -3. **验证。** 运行你的 Agent，读取生成的事件文件，检查 `agent_start` 是否存在、环境是否正确、一次运行是否对应一个 session。 - -第三步是人们最常跳过的。SDK 将事件写入本地文件，因此完整的集成可以在笔记本电脑上、无需服务器、无需 API 密钥、无需网络的情况下得到验证——这正是该 skill 坚持执行这一步的原因。 - -## 它与其他 skill 的关系 - -三个 skill，职责清晰划分： - -| Skill | 适用场景 | 操作范围 | -|---|---|---| -| **Python SDK skill**（本页） | 你想让 Agent *发出*遥测数据——"添加可观测性"、"为什么我的 Agent 没有出现？" | 在你 Agent 的代码仓库中写代码，不读取任何内容。 | -| **[Evaluator skill](/zh/agenteye/evaluator-skill)** | 你想对运行结果*评分*——"我们到底该衡量什么？" | 在你的代码仓库中写代码；读取遥测数据。 | -| **[CLI skill](/zh/agenteye/cli-skill)** | 你想*读取*发生了什么，或者操作你的部署 | 以你的身份驱动 CLI，包括变更操作。 | - -它们按顺序衔接：本 skill 让事件开始流动，evaluator 对其评分，CLI 读取结果。在你的 Agent 发出 session 之前，没有任何内容可评估，也没有任何内容可读取——所以如果你从零开始，就从这里开始。 - -## 前置条件 - -1. **Python 3.10+** 以及你想要插桩的 Agent 代码库。 -2. **SDK。** 它以私有 wheel 包的形式分发给客户，而非通过公共索引——你的入门指南会介绍如何获取和安装它。该 skill 知道安装路径，如果找不到，会向你询问，而不是自行猜测。 -3. **无需其他任何东西。** 不需要登录仪表板、不需要 API 密钥、不需要网络。该 skill 通过 SDK 写入的事件文件进行验证，因此可以在离线状态下完成工作并证明其有效性。 - -## 获取方式 - -该 skill 位于公开的 [`FailproofAI/skills`](https://github.com/FailproofAI/skills) 集合中： - -```bash -npx skills add FailproofAI/skills --skill agenteye-python-sdk -a claude-code -``` - -添加 `-g` 可以为所有项目安装，而不仅限于当前项目；如果你的环境不支持符号链接，请添加 `--copy`。对于 Codex，请传入 `-a codex`。 - -## 手动安装 - -Agent Skill 是包含 `SKILL.md` 及相关引用文件的文件夹。如果你不想使用安装程序： - -- **Claude Code**：将 `agenteye-python-sdk/` 文件夹复制到 `~/.claude/skills/`（适用于所有项目）或 `/.claude/skills/`（仅适用于该仓库）。Claude Code 会自动发现它——查看 `/skills` 列表，或者直接提问一个与之匹配的问题。 -- **Codex**：Codex 读取相同的 `SKILL.md`。捆绑的 `agents/openai.yaml` 设置了 `allow_implicit_invocation: true`，因此当任务匹配时会自动选中；否则可以通过 `$agenteye-python-sdk` 显式调用。 - -**在包含你想要插桩的代码的仓库中**运行你的 Agent——该 skill 在提出任何建议之前会先读取你的 Agent 循环。 - -## 一次对话示例 - -```text -you ▸ Add Failproof AI Observability to this agent. - -agent ▸ Read your loop — one tool dispatcher in tools.py, one Anthropic - wrapper in llm.py. Two questions before I write anything. - - What counts as one run? You handle requests concurrently, so I'd - use the request id you already have in ctx.request_id — then a - session here and a request in your logs are the same string. - - And who are the actors? I see a planner path and a researcher - path. Label them separately, or is that one agent to you? - -you ▸ Request id is right. They're one agent, call it "assistant". - -agent ▸ Done. Wrapped the dispatcher and the LLM client; agent_start and - agent_end bracket handle_request, including the error path. Used - a context-local for identity — your dispatcher runs tools in a - thread pool, and the simpler approach would have mixed concurrent - runs into one session without complaining. - - Verified against a real run: 11 events across 4 files — - agent_start, model_request/response, tool_use/tool_result x3, - agent_end (outcome=success). environment=production, one session - per request, durations paired. - - One thing I did NOT do: your worker pool in batch.py submits - straight to the executor, so events from there would be dropped. - Want me to fix those too? -``` - -值得关注的模式：它在提出建议之前先读取了代码，只问了你才能回答的问题，复用了你已有的 ID，*因为*看到了线程池而选择了并发安全的实现方式，并且通过**读取实际事件**来验证，而非直接宣告成功——然后还指出了那个已知会悄悄失败的地方。 - -## 你可以问它什么 - -- *"为什么我的 Agent 没有出现在仪表板上？"* → 逐层排查：事件是否在写入，`agent_start` 是否存在，环境是否正确，采集器是否在读取同一个位置。 -- *"所有数据都落在 dev 下。"* → 环境变量从未设置，或被后续调用重置了。 -- *"添加 token 追踪。"* → 找到你的 LLM wrapper，记录模型、停止原因和用量。 -- *"也为子 Agent 插桩。"* → 同一个 session，不同的 Agent 标签，嵌套在各自的父级下。 -- *"为插桩代码编写测试。"* → 将 SDK 指向一个临时目录，并对写入的事件进行断言。 - -## 注意事项 - -**让它执行验证。** 让这个 skill 物有所值的正是最后一步——运行你的 Agent 并读取事件。一个写完插桩就停下来的 Agent 只做了容易的那一半，而悄悄失败的恰恰是另一半。 - -**在写代码之前先确定命名。** `session_id` 和 `agent_id` 是所有视图分组的轴。事后重命名会导致历史数据分裂：旧的运行保留旧标签，趋势图随之断裂。该 skill 会主动询问；这个问题值得花一分钟认真思考。 - -**如果你的 Agent 提议从公共索引安装 SDK，说明 skill 没有加载。** SDK 是私有分发的。这个提议是一个可靠的信号，表明你的编码 Agent 在凭空猜测而非遵循 skill——在那里停下来，检查 skill 是否已正确安装。 - -除此之外，它的影响范围很小：它在你的工作目录中写代码，在你指定的位置写事件文件。它不读取你的部署内容，也不对其做任何修改。 - -## 下一步 - -- **[Python SDK](/zh/agenteye/python-sdk)**：完整的事件参考——本 skill 所自动化的每种事件类型和字段。 -- **[Sessions](/zh/agenteye/sessions)**：事件落地后，你的插桩所产生的内容。 -- **[Evaluator Agent Skill](/zh/agenteye/evaluator-skill)**：运行数据开始积累后的下一步——对其评分。 -- **[CLI Agent Skill](/zh/agenteye/cli-skill)**：读取你的遥测数据。 \ No newline at end of file diff --git a/docs/zh/agenteye/security.mdx b/docs/zh/agenteye/security.mdx deleted file mode 100644 index 91196f3e..00000000 --- a/docs/zh/agenteye/security.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: "安全性" -description: "Failproof AI Observability 被设计为紧邻您的生产环境 Agent 运行，这意味着它能看到您的提示词、工具输入和输出内容。" ---- - - -Failproof AI Observability 被设计为紧邻您的生产环境 Agent 运行，这意味着它能看到您的提示词、工具输入和输出内容。本页说明它如何确保数据隔离、受控，并始终掌握在您手中。如果您正在对 Failproof AI Observability 进行安全审查评估，请从这里开始。 - ---- - -## 您的数据保留在您的环境中 - -Failproof AI Observability 采用自托管模式。事件、提示词、模型响应和分析数据均存储在您自己的数据库和环境中。数据不会被发送至任何第三方 SaaS 平台存储，始终保留在您自己的云账户内。 - ---- - -## 租户隔离 - -一个 Failproof AI Observability 实例可以托管多个组织，每个组织在存储层面相互隔离——这由数据库强制执行，而不仅仅依赖 UI 层面的限制： - -- 组织的运营数据（用户、密钥、仪表盘、已保存查询）仅限于该组织访问，跨组织读取由数据库本身拦截阻止。 -- 每个采集的事件都标记了所属组织，因此一个组织的事件永远无法被另一个组织读取。 - -每个仪表盘路由都以组织 slug 为前缀（`//…`）。 - ---- - -## 登录方式 - -Failproof AI Observability 采用无密码、基于邮件的登录方式，不存在可被钓鱼或泄露的密码。用户申请一次性验证码（或一键魔法链接），系统将其发送至用户邮箱，且会在短时间内过期。登录受**白名单**限制：只有您允许的邮箱地址（或域名）才能完成认证。 - -![Failproof AI Observability 登录界面，将一次性验证码发送至您的邮箱](/agenteye/images/login.png) - ---- - -## 通过 API 密钥实现精细化访问控制 - -每个客户端均使用携带精细化最小权限的 API 密钥进行认证。数据采集器只需 `events:add` 权限；仪表盘或助手密钥可设为只读；破坏性操作（删除、重新生成）作为独立权限授予，由您自行决定是否开放。 - -![API 密钥页面：每个密钥的权限授予情况，按读取、写入和破坏性范围用颜色区分](/agenteye/images/api-keys.png) - -保留管理员引导密钥用于初始配置，其余场景均应颁发权限受限的密钥。详见 [API 密钥](/zh/agenteye/api-keys)。 - ---- - -## 只读、需审批的 AI 助手 - -仪表盘内的 [AI 助手](/zh/agenteye/assistant) 可基于您的数据回答问题，但在设计上受到严格约束： - -- **默认只读**：其执行的 SQL 经过守卫过滤，仅允许 `SELECT`/`WITH` 查询，单条语句执行，并设有行数上限。 -- 它创建的任何内容（已保存查询、仪表盘）均需**审批才能生效**：每一次写入操作发生前，您都需要审查并确认。 -- **它永远无法执行删除操作**。 - -因此，团队成员可以询问"本周哪些 Agent 报错最多？"并基于答案采取行动，而无需担心助手会自行修改或删除您的数据。 - ---- - -## 传输安全 - -所有流量均通过 HTTPS 传输。您使用自己的证书终止 TLS，确保采集器到服务器以及浏览器到服务器的流量在传输过程中全程加密。 - ---- - -## 后续步骤 - -- [概览](/zh/agenteye/overview)：了解 Failproof AI Observability 的整体架构。 -- [API 密钥](/zh/agenteye/api-keys)：为采集器、仪表盘和助手配置访问权限。 -- [可观测性](/zh/agenteye/observability)：了解 Failproof AI Observability 从您的 Agent 中采集的数据内容。 \ No newline at end of file diff --git a/docs/zh/cli/audit.mdx b/docs/zh/audit.mdx similarity index 100% rename from docs/zh/cli/audit.mdx rename to docs/zh/audit.mdx diff --git a/docs/zh/cli/backfill.mdx b/docs/zh/cli/backfill.mdx new file mode 100644 index 00000000..5611ddd2 --- /dev/null +++ b/docs/zh/cli/backfill.mdx @@ -0,0 +1,75 @@ +--- +title: failproofai backfill +description: "Re-send history the collector already read past — after connecting late, clearing a dashboard, or re-enrolling a machine." +icon: clock-rotate-left +--- + +```bash +failproofai backfill +failproofai backfill --since 6m +failproofai backfill --dry-run +``` + +A connected machine ships new agent activity as it happens and remembers how far it has +read. `backfill` rewinds that mark so history is sent again. + +Reach for it when: + +- you **connected a machine after** the work you want to see happened +- you **cleared a dashboard** and want the sessions back +- you **re-enrolled** a machine and its history did not follow +- you **added a [capture path](/cli/harness)** that already contained sessions + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--since ` | How far back: `30d`, `6m`, `2y`, or an explicit `YYYY-MM-DD`. Default: 30 days. | +| `--dry-run` | Report what would be re-read. Changes nothing. | + +```bash +failproofai backfill --since 30d +failproofai backfill --since 2026-01-01 +failproofai backfill --since 6m --dry-run +``` + +--- + +## What it does and doesn't do + +- **It re-reads, it does not duplicate.** Sessions are shipped once, so running backfill + twice does not double anything up. +- **It only covers what is still on disk.** Agent CLIs prune their own transcripts; anything + they have deleted is gone before FailproofAI ever sees it. +- **It respects your transcript setting.** On a machine connected with `--no-transcripts`, + backfill re-sends decisions and not transcripts, exactly like live capture. +- **It needs a connection.** On an unconnected machine there is nowhere to send anything. + +Start with `--dry-run` on a long window. A year of transcripts across a busy machine is a +lot of data, and it is better to see the size before you send it. + +--- + +## Related + + + + + Deliver what is already spooled, right now. + + + + What is captured, from which CLIs. + + + + Capture from non-standard locations. + + + + Getting a machine reporting in the first place. + + + diff --git a/docs/zh/cli/config.mdx b/docs/zh/cli/config.mdx new file mode 100644 index 00000000..5d05627c --- /dev/null +++ b/docs/zh/cli/config.mdx @@ -0,0 +1,145 @@ +--- +title: failproofai config +description: "Setup, status, cloud connection, and time-boxed pauses — one command." +icon: gear +--- + +```bash +failproofai config # guided setup +failproofai configure # alias +failproofai setup # alias +``` + +`config` is the front door. With no flags it runs the setup wizard; with flags it becomes +the non-interactive surface for everything about this machine's state. + +--- + +## Guided setup + +Two questions, then it writes everything: + + + + **Recommended** applies 16 policies globally to every agent CLI detected on this + machine. **Customize** lets you pick the scope, combine [presets](/policies#presets), + and choose the CLIs yourself. + + + Paste an API key to connect, or stay local and connect later. Nothing is lost either + way — re-running `config` picks up where you left off. + + + +It then confirms the exact files it will change before changing them, installs the +[`failproofaid` service](/daemon), and reports what it did. + +Re-run it any time — after installing a new agent CLI, after an upgrade, or to change your +mind. It shows your current state rather than resetting it. + + + Setup needs root to install the service, and uses `sudo -n` rather than prompting. If it + cannot elevate it writes **nothing** and prints the commands for you to run. On an + unsupported platform it refuses outright rather than leaving a half-configured machine. + + +--- + +## Cloud connection + +```bash +failproofai config --connect --token +failproofai config --connect --token --no-transcripts +failproofai config --machine-label "build-runner-3" +failproofai config --disconnect +failproofai config --status +``` + +| Flag | Meaning | +|---|---| +| `--connect ` | Cloud base URL — your dashboard origin. | +| `--token ` | An API key for your organization. | +| `--machine-id ` | Stable id for this machine. Defaults to the one already here, or a fresh random one. | +| `--machine-label ` | Display name in the dashboard. **Used alone, it renames an already-connected machine.** | +| `--no-transcripts` | Send policy decisions only, never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Connection, service, and pause state. | + +One connection configures **two capabilities**: this machine pulls centrally-managed +policy (`policies:pull`) and reports what its hooks decided (`events:add`). Both are +checked against the server *before* anything is written, and reported separately — a key +carrying one and not the other connects for what it can and says exactly why the other +half is missing. + + + Connecting sends **both** policy decisions and full session transcripts. A transcript + carries prompts, file contents, and whatever was pasted into a terminal. That is the + point of connecting, and it is stated here rather than buried behind a flag. Use + `--no-transcripts` for decisions only; `--status` always says which is in effect. + + +Tokens are stored owner-only in `~/.failproofai/`, never in the service definition — that +file is world-readable. Connecting, rotating, and disconnecting all need no `sudo`. + +[Full guide, including fleet provisioning →](/cloud/connect) + +--- + +## Pausing enforcement + +```bash +failproofai config --pause # this directory's newest session, 30m +failproofai config --pause 10m # 10 minutes (s / m / h; a bare number means minutes) +failproofai config --pause --session +failproofai config --resume +failproofai config --resume --all # end every active pause +failproofai config --status # what is paused, and when it lifts +``` + +A pause suspends **built-in, custom, and convention** policies for **one session**, and +always expires on its own. Maximum 8 hours; renewing extends the same stretch rather than +restarting the ceiling, so enforcement cannot be kept off indefinitely one legal command at +a time. + +Two things a pause does **not** do: + +- It does not touch [cloud-managed policies](/cloud/managed-policies) — those keep + enforcing. +- It is not configuration. Pause state is machine-local, so it can never be committed and + travel to everyone who checks out the branch. + +With `block-self-pause` enabled (it is, under Recommended), an agent cannot pause on its own +behalf. + +--- + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Success — including a user who cancelled the wizard. Cancelling is not a failure. | +| `1` | Setup could not complete — for example the required service could not be installed. A fleet script can branch on this to tell "the user pressed Esc" from "this machine is unconfigured". | + +--- + +## Related + + + + + The whole setup path, start to finish. + + + + Permissions, machine identity, and troubleshooting. + + + + What gets installed, and why it needs root. + + + + What Recommended turns on, and the presets behind Customize. + + + diff --git a/docs/zh/cli/flush.mdx b/docs/zh/cli/flush.mdx new file mode 100644 index 00000000..b0604240 --- /dev/null +++ b/docs/zh/cli/flush.mdx @@ -0,0 +1,64 @@ +--- +title: failproofai flush +description: "Deliver everything already spooled, now, instead of waiting for the next sweep." +icon: paper-plane +--- + +```bash +failproofai flush +failproofai flush --wait +failproofai flush --wait --timeout 120 +``` + +A connected machine batches what it collects and uploads on its own schedule. `flush` +delivers everything waiting immediately. + +Use it when you are standing in front of the dashboard wondering whether something arrived +— which is exactly the moment a background sweep interval feels longest. + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--wait` | Block until the spool drains, or the timeout expires. | +| `--timeout ` | How long to wait with `--wait`. Default: 60. | + +Without `--wait` the command asks for a delivery and returns immediately. With `--wait` it +returns only once there is nothing left outstanding — which makes it useful at the end of a +CI job, or as the last line of a provisioning script. + +--- + +## Why the spool exists + +Delivery failures do not discard data. A batch that cannot be delivered is **kept and +retried**, and the machine reports as unhealthy while anything is still outstanding. + +That is what makes "healthy" mean *your data arrived*, rather than merely *the process is +alive*. `failproofai config --status` reports it. + +--- + +## Related + + + + + Re-send history the collector already passed. + + + + Connection, service, and delivery state. + + + + What gets collected in the first place. + + + + What does the collecting and uploading. + + + diff --git a/docs/zh/cli/harness.mdx b/docs/zh/cli/harness.mdx new file mode 100644 index 00000000..817075bf --- /dev/null +++ b/docs/zh/cli/harness.mdx @@ -0,0 +1,126 @@ +--- +title: failproofai harness +description: "Capture agent sessions from paths outside a CLI's default location — containers, mounted volumes, second checkouts." +icon: folder-tree +--- + +```bash +failproofai harness list +failproofai harness add-path +failproofai harness remove-path +``` + +FailproofAI knows where each supported agent CLI keeps its sessions. `harness` is for when +yours are somewhere else: a container mount, a second checkout, a shared volume, a VM disk +you attached to inspect. + +--- + +## Harness names + +One of the [12 supported CLIs](/agent-support): + +```text +claude codex copilot openclaw pi factory +antigravity cursor goose opencode devin hermes +``` + +A name that isn't in that list is rejected. That check exists because it is the one failure +with no other detector — a typo'd harness produces a perfectly valid configuration file +that captures absolutely nothing, silently. + +--- + +## Adding a path + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +``` + +`~` is expanded. From then on, sessions under that path are captured alongside the default +location. + +### Labels + +```bash +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness add-path codex "vm-b=/mnt/vm-b/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without a +label, two copies of the same project collapse into one timeline that makes no sense; with +one, `vm-a` and `vm-b` stay distinct everywhere you look. + +Omit the label and the folder name is used. + +### Two rejections, and why + +| Rejected | Because | +|---|---| +| A path that overlaps a default location | It would be collected **twice**, under two different agent ids — the same work appearing as two agents. | +| Two entries sharing a label | They would share progress state, so **both** would re-read from the beginning after every restart. | + +Both failures are silent if allowed, which is exactly why they are refused up front. + +--- + +## Listing and removing + +```bash +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +`list` shows every configured extra path, grouped by harness. + +--- + +## Containers + +Environment variables override the file, per source — useful when the config file is baked +into an image but the mount points differ per run: + +```bash +FAILPROOFAI_CLAUDE_EXTRA_PATHS=/mnt/a/.claude/projects,/mnt/b/.claude/projects +FAILPROOFAI_CODEX_EXTRA_PATHS=vm-a=/mnt/vm-a/.codex/sessions +``` + +Comma-separated, same `label=path` grammar. + +--- + +## What happens next + +Each accepted path becomes its own capture task with its own progress tracking, so one +slow or unreadable path never stalls the others. + +New paths are read from the beginning on their first pass. To pull in older history from a +path you added late: + +```bash +failproofai backfill --since 6m +``` + +--- + +## Related + + + + + What gets captured, and how to narrow it. + + + + Re-read history the collector already passed. + + + + Every harness name and where its sessions normally live. + + + + Every variable, including the per-harness overrides. + + + diff --git a/docs/zh/cli/migrate.mdx b/docs/zh/cli/migrate.mdx new file mode 100644 index 00000000..fbf6435f --- /dev/null +++ b/docs/zh/cli/migrate.mdx @@ -0,0 +1,117 @@ +--- +title: Migrate the home directory +description: "Bring ~/.failproofai up to the layout this version speaks, and see what would happen first" +--- + +```bash +failproofai migrate --dry-run # print the plan, change nothing +failproofai migrate # run it +``` + +Most people never type this. It runs by itself on the first command after an +upgrade, and [`failproofai update`](/cli/update) includes it. Reach for it +directly when you want to see the plan before it happens, or to run the migration +on its own. + +## Keyed on the layout, not the version + +`~/.failproofai/VERSION` records a **layout** number — the shape of the directory, +not the release that wrote it. Migrations are keyed on that number, which is what +makes a long gap cheap: + +- npm versions change on every release, dozens of them between two layouts. +- So a machine that skips thirty releases with **no layout change** runs **zero** + migrations, not thirty no-ops. +- And a machine that skips several layouts at once runs each step in order, each + step knowing only its own two ends. + +That matters because npm cannot update an installed package on its own. A machine +sitting on one version for months and then jumping several layouts is the normal +case, not the exotic one. + +## The dry run + +`--dry-run` prints the exact chain and the files that would be saved first, and +changes nothing at all — no migration, no backup, no ledger entry: + +``` +Layout 2 on disk; this build speaks 3. +1 step(s) would run: + 2 → 3 layout 2 → 3: carry config.toml and credentials.toml into JSON, move + custom-policies/ back up into policies/, nest the policy config at the root + +These would be copied to ~/.failproofai/migrations/backup-layout2 first: + VERSION + config.toml + credentials.toml +``` + +## What is carried, and what is rebuilt + +Every path in the home declares what kind of data it holds, and that decides +whether a migration may throw it away. The rule: **derived and re-fetchable may be +dropped; anything you typed, anything not yet delivered, and anything that +identifies the machine is carried.** + +| Carried | Rebuilt or re-fetched | +|---|---| +| `config.json` — settings, `daemon.configured`, extra capture paths | The audit cache | +| `credentials.json` — your cloud enrolment | Cloud-managed deployments (re-fetched and digest-verified on the next poll) | +| `policies-config.json` — your policy selection and params | Daemon scratch state | +| `policies/` — your own policy files and the helpers they import | | +| `hook-activity/` — the decision log the dashboard reads | | +| Undelivered events still queued for upload | | +| `cursors/` — collector watermarks | | +| The daemon binary in `bin/` | | + + + Undelivered events are carried rather than dropped because the loss would be + permanent, not slow: the collector's watermark has already advanced past + anything sitting in the spool, so nothing would ever read that range of a + transcript again. The migration also asks the daemon to deliver what is spooled + as soon as it finishes, so the usual outcome is that there is nothing left to + carry. + + +Keys a *newer* version wrote into `config.json`, `credentials.json` or +`policies-config.json` are preserved too, rather than dropped by an older reader. + +## The record it leaves + +``` +~/.failproofai/migrations/ + applied.json one entry per step: layout, CLI, timestamp, duration, result + backup-layout/ copies of the irreplaceable files, taken before the first step +``` + +`applied.json` is what answers "what has this machine actually been through" — the +first question worth asking when something looks wrong after an upgrade. Attach it +to a bug report. + +The backup is deliberately small rather than a copy of the whole directory: the +migration no longer deletes anything irreplaceable by design, so what is worth +insuring against is a *defect in a step*, and these few files are where such a +defect would hurt. + +## If a step fails + +The chain stops there. `VERSION` is stamped only by a step that completed, so the +home stays marked with its old layout and the next command retries it — a home is +never marked current on the strength of a partial migration. The step is recorded +in `applied.json` with `"ok": false`, and the backup is where it was taken. + +## A newer home is refused, not migrated + +If `~/.failproofai/` was written by a **newer** failproofai than the one you are +running, the command stops and tells you to upgrade instead. That data is fine and +a newer CLI reads it; migrating "forward" from it is not a thing that exists, and +resetting it would destroy something recoverable. + +``` +This machine's failproofai directory was written by a newer version (layout 4; +this build speaks 3). Upgrade rather than migrate: + npm install -g failproofai@latest +``` + +The daemon applies the same rule: `failproofaid` refuses to start against a layout +it does not speak, rather than reading and writing paths that have moved. diff --git a/docs/zh/cli/uninstall.mdx b/docs/zh/cli/uninstall.mdx new file mode 100644 index 00000000..b0031865 --- /dev/null +++ b/docs/zh/cli/uninstall.mdx @@ -0,0 +1,95 @@ +--- +title: failproofai uninstall +description: "Remove FailproofAI from a machine completely — hook entries from every agent CLI, and the background service." +icon: trash +--- + +```bash +failproofai uninstall +failproofai uninstall --dry-run +failproofai uninstall --purge --yes +``` + +Removes the hook entries FailproofAI wrote into every agent CLI, and the +[`failproofaid` service](/daemon). + + + **Run this before `npm rm -g failproofai`.** npm runs no uninstall script, so removing + the package on its own leaves both the hook entries and the background service behind — + hooks pointing at a binary that no longer exists, and a service nobody remembers + installing. + + +--- + +## Options + +| Flag | Meaning | +|---|---| +| `--purge` | Also delete `~/.failproofai` — settings, credentials, audit history, and the service binary. | +| `--dry-run` | Show what would be removed. Changes nothing. | +| `--yes`, `-y` | Skip the confirmation prompt. | + +Without `--purge`, your configuration survives. Reinstalling and running `failproofai +config` puts you back exactly where you were. + +--- + +## What it does, in order + + + + Unconditionally, and before anything else. Leaving that flag set with no service to + reach would **deny every hook event** on the machine, across all 12 CLIs — recoverable + only by hand-editing a config file. + + + Each CLI's own settings file is edited in place, keeping everything else in it. + + + Including any older user-scope service left behind by a previous version. + + + Only with `--purge`. + + + +Run `--dry-run` first if you want the list before the action. + +--- + +## Leaving your organization + +If the machine is [connected to the cloud](/cloud/connect) and you only want to stop that — +not remove the guardrails — disconnect instead: + +```bash +failproofai config --disconnect +``` + +That clears the credentials **and** stops enforcing the cloud-managed deployment, while +local policies keep working exactly as before. + +--- + +## Related + + + + + Setup, status, connect, disconnect. + + + + What gets installed, and how it is supervised. + + + + Disable individual policies without uninstalling. + + + + Upgrading rather than removing. + + + diff --git a/docs/zh/cli/update.mdx b/docs/zh/cli/update.mdx new file mode 100644 index 00000000..8d28ab47 --- /dev/null +++ b/docs/zh/cli/update.mdx @@ -0,0 +1,94 @@ +--- +title: Update after an upgrade +description: "Finish the half of an upgrade npm cannot do: migrate the home and match the daemon" +--- + +```bash +npm install -g failproofai@latest && failproofai update +``` + +That is the whole upgrade. `npm` replaces the CLI; `failproofai update` does the +rest. + +## Why a second command exists + +`npm install -g` replaces one thing — the CLI. Two other pieces of a failproofai +install live outside the package on purpose, and neither moves when npm runs: + +- **`~/.failproofai/`**, your settings, cloud enrolment, policy selection and + history. A new version may organise it differently, and the reorganisation has + to be done by code that knows both shapes. +- **The `failproofaid` daemon binary**, at + `~/.failproofai/bin/failproofaid-`. It is deliberately *not* inside + `node_modules`: an upgrade that swapped the file under a running service would + repoint a live daemon at a binary built from different source, and removing the + package would delete it out from under a service that then crash-loops at every + boot. + +So after `npm install -g` alone, the CLI is new and the daemon is not. +`failproofaid` refuses to start against a home layout it does not speak — the loud +version of that mismatch rather than the silent one — so the two halves need +bringing together. `failproofai update` is that step. + +## What it does + + + + Reads the layout recorded in `~/.failproofai/VERSION` and runs the steps that + bring it to the one this version speaks. Usually none — see + [`failproofai migrate`](/cli/migrate). + + + From the platform package npm already downloaded where possible (no network), + otherwise from the release asset for this exact version, SHA-256 verified + before it is used. + + + Probed rather than assumed — a service manager reports a process active the + moment it forks, which is not the same as it working. + + + +## Options + +| Flag | Effect | +|------|--------| +| `--no-daemon` | Migrate the home only, leaving the daemon at its current version. | + + + `--no-daemon` leaves a version-skewed daemon in place. On a machine configured + to require the daemon, every hook event **fails closed** if the daemon cannot + answer — and a daemon that refuses to start against a migrated home cannot + answer. Prefer letting the daemon half run. + + +## If something goes wrong + +The command exits non-zero and says which half failed. Two cases worth knowing: + +- **A migration step did not finish.** The home is left marked with its *old* + layout, so the next command retries it — no home is ever marked current on the + strength of a partial migration. Copies of your settings and enrolment were + saved before anything ran, in `~/.failproofai/migrations/backup-layout/`. +- **The daemon could not be restarted without a password.** `sudo -n` is used + deliberately, so nothing ever prompts from under a progress display. The + command prints the exact line to run yourself. + + + Nothing here needs the interactive setup wizard. Your settings, cloud + enrolment and policy selection survive an upgrade, so a migrated machine + enforces exactly as it did before — which matters most on the machines with + nobody sitting at them: a CI runner, a fleet box, a headless gateway. + + +## Automating it + +`failproofai update` is non-interactive and safe to run when there is nothing to +do — it reports "no migration was needed" and exits 0. Putting it after every +upgrade in a provisioning script or Dockerfile is the intended use: + +```dockerfile +RUN npm install -g failproofai@latest && failproofai update --no-daemon +``` + +(`--no-daemon` in an image build, where there is no service to restart yet.) diff --git a/docs/zh/agenteye/api-keys.mdx b/docs/zh/cloud/access.mdx similarity index 95% rename from docs/zh/agenteye/api-keys.mdx rename to docs/zh/cloud/access.mdx index 05e82748..446ad973 100644 --- a/docs/zh/agenteye/api-keys.mdx +++ b/docs/zh/cloud/access.mdx @@ -1,10 +1,10 @@ --- title: "API 密钥" -description: "API 密钥控制谁以及什么可以访问您的 Failproof AI Observability 服务器，使采集器可以发送事件而无需获得读取或管理员权限。" +description: "API 密钥控制谁以及什么可以访问您的 FailproofAI Cloud 服务器，使采集器可以发送事件而无需获得读取或管理员权限。" --- -API 密钥控制谁以及什么可以访问您的 Failproof AI Observability 服务器，使采集器可以发送事件而无需获得读取或管理员权限。每个密钥携带一个或多个权限，每个权限控制特定的服务器路由；您只需授予某项工作所需的少量权限。大多数部署只需创建三种类型的密钥。 +API 密钥控制谁以及什么可以访问您的 FailproofAI Cloud 服务器，使采集器可以发送事件而无需获得读取或管理员权限。每个密钥携带一个或多个权限，每个权限控制特定的服务器路由；您只需授予某项工作所需的少量权限。大多数部署只需创建三种类型的密钥。 ## 大多数部署所需的 3 种密钥 @@ -82,7 +82,7 @@ API 密钥控制谁以及什么可以访问您的 Failproof AI Observability 服 这些权限支撑仪表板的**用户**页面，每个成员授予的作用域以标签形式显示： -![用户页面：每个仪表板用户一张卡片，显示其电子邮件、已授予的权限以及编辑/禁用控件](/agenteye/images/users.png) +![用户页面：每个仪表板用户一张卡片，显示其电子邮件、已授予的权限以及编辑/禁用控件](/cloud/images/users.png) ### 操作设置 @@ -91,7 +91,7 @@ API 密钥控制谁以及什么可以访问您的 Failproof AI Observability 服 | `settings:read` | `GET /settings`、`GET /settings/schema`、`GET /settings/model-context-windows`、`GET /settings/model-context-windows/resolve` | 查看仪表板管理的操作设置及其元数据；列出每个模型的上下文窗口覆盖值；以及解析模型的有效窗口。 | | `settings:write` | `PUT /settings/:key`、`PUT /settings/model-context-windows`、`DELETE /settings/model-context-windows` | 编辑操作设置，以及添加、更改或删除每个模型的上下文窗口覆盖值。更改会影响新事件，无需重启服务器。 | -![设置页面：仪表板管理的操作设置，例如允许的登录方式和会话/OTP 有效期，可在不重启的情况下编辑](/agenteye/images/settings.png) +![设置页面：仪表板管理的操作设置，例如允许的登录方式和会话/OTP 有效期，可在不重启的情况下编辑](/cloud/images/settings.png) ### 告警与事件 @@ -247,7 +247,7 @@ curl -s -X POST http://your-server/keys//regenerate \ 从仪表板创建密钥时，您无需提供密文；仪表板会为您生成强密文，并在创建时**仅显示一次**。请立即复制并安全存储；与重新生成密钥一样，它不会再次显示。您仍然可以直接选择密钥的权限，或从权限集初始化（见下文）。 -![API 密钥页面：每个密钥一张卡片，显示其名称、已授予的权限和创建时间，以及重新生成和禁用操作；受保护的密钥（如 `admin`）会被标记](/agenteye/images/api-keys.png) +![API 密钥页面：每个密钥一张卡片，显示其名称、已授予的权限和创建时间，以及重新生成和禁用操作；受保护的密钥（如 `admin`）会被标记](/cloud/images/api-keys.png) --- @@ -276,5 +276,5 @@ curl -s -X POST http://your-server/keys//regenerate \ ## 后续步骤 -- [Python SDK](/zh/agenteye/python-sdk)：您的 Agent 代码在发送事件时如何进行身份验证。 -- [安全性](/zh/agenteye/security)：登录、访问控制和每个组织的数据隔离如何工作。 \ No newline at end of file +- [Python SDK](/zh/cloud/sdk)：您的 Agent 代码在发送事件时如何进行身份验证。 +- [安全性](/zh/cloud/security)：登录、访问控制和每个组织的数据隔离如何工作。 \ No newline at end of file diff --git a/docs/zh/cloud/agent-skills.mdx b/docs/zh/cloud/agent-skills.mdx new file mode 100644 index 00000000..9c06c739 --- /dev/null +++ b/docs/zh/cloud/agent-skills.mdx @@ -0,0 +1,219 @@ +--- +title: Agent skills +description: "Three installable skills that let your coding agent operate FailproofAI Cloud, instrument your own agents, and build your evaluator — from plain-English requests." +icon: wand-magic-sparkles +--- + +You should not have to memorize a flag to ask *"is anything broken today?"* + +FailproofAI publishes three **Agent Skills** — small folders of instructions that a coding +agent like Claude Code or Codex loads on demand when a task matches. They are not services, +libraries, or plugins. Each one teaches your agent to drive something you already have, +using credentials you already hold. + +| Skill | Ask it to | What it touches | +|---|---|---| +| **`agenteye-cli`** | Read your data and run your organization — *"which sessions errored today?"*, *"give CI a key that can only push events"* | Drives the [CLI](/cloud/cli) as you | +| **`agenteye-python-sdk`** | Instrument your own agent so it reports at all — *"add observability to this agent"* | Writes code in your agent's repo | +| **`agenteye-evaluator`** | Decide what quality means for you, then build the scorer | Writes code in your repo; reads your sessions | + +They hand off in that order: the SDK skill gets events flowing, the evaluator skill scores +them, the CLI skill reads them back. Starting from scratch? Start at the top of that list. + +--- + +## Getting them + +All three live in FailproofAI's **public** skills collection: +**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** + +Nothing is gated. None of them carries a credential of its own — each only drives a tool +you have already installed, with the session you logged in with. + +```bash +# Claude Code, this project only +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code + +# every project (installs to ~/.claude/skills/) +npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy + +# Codex instead +npx skills add FailproofAI/skills --skill agenteye-cli -a codex + +# then manage them like any other skill +npx skills list -a claude-code +``` + +Swap the `--skill` value for `agenteye-python-sdk` or `agenteye-evaluator`. Installing +several is the normal setup — the agent picks between them from what you ask. + + + The skills ship as their own folders and are **not** inside the `pipx install agenteye` + package, so don't look for them there. + + +--- + +## `agenteye-cli` — operate your deployment by asking + +```text +you Why did session run-001 fail? + +agent Running: agenteye --json events --session-id run-001 --all + The checkout-agent hit a TimeoutError on its third tool call. +``` + +Reads are instant. Every write pauses for you: + +```text +you Give CI a key that can only push events. + +agent This creates an API key. I'll run: + agenteye keys create ci --add events:add + OK to proceed? + +you yes + +agent Done. Key "ci" created with events:add only. The secret is shown once — store it now. +``` + +**Prerequisites:** the [`agenteye` CLI](/cloud/cli) installed and on `PATH`, your dashboard +URL set, and a logged-in session (`agenteye login`). The skill **cannot** complete the +emailed one-time-code login for you — it will tell you to run `agenteye login` when the +session is missing or expired. + + + **This skill has your full permissions, including writes.** It runs the CLI *as you*, so + it can do anything your login can: create and rotate keys, change settings, resolve + incidents, delete saved queries. The CLI's "are you sure?" prompt does not fire for a + non-interactive caller, so the skill is written to state the exact command and wait for + your OK before any change. **You are the confirmation step.** + + This is a different blast radius from the [in-dashboard assistant](/cloud/assistant), + which is read-only with approval-gated authoring and can never delete. + + +--- + +## `agenteye-python-sdk` — instrument an agent, correctly + +The [SDK](/cloud/sdk) is small — thirteen event methods, all keyword-only — and a coding +agent can produce plausible instrumentation from the reference in a minute. + +The catch is that wrong instrumentation looks exactly like right instrumentation until +someone opens a dashboard and finds it empty. The expensive mistakes are all **silences**: + +| The mistake | What you see | +|---|---| +| No `agent_start` | Every event lands. Zero sessions. | +| Environment never set | Everything works, filed under `dev`. | +| `outcome="failure"` | The run shows green — only `failed`, `error`, `timeout`, `rejected` count. | +| A typo'd field name | Accepted, and stored as a brand new field. | +| Events emitted from a thread pool | Silently dropped. | + +None of these raise. None show up in tests. Every one is in the skill, stated as a contract +with the check that catches it. + +The skill works in three steps, in the order a careful engineer would: + + + + It reads your agent loop and asks the two questions only you can answer: what counts as + one run (your `session_id`), and who the distinguishable actors are (your `agent_id`). + Both get agreed *before* code is written — changing them later splits your history and + breaks every trend built on it. + + + It binds identity once per run instead of threading it through every call site, and + picks a concurrency-safe shape. That detail matters: the obvious shortcut silently + merges two overlapping runs into one session. + + + It runs your agent and reads the resulting event files, checking that `agent_start` is + present, the environment is right, and one run produced exactly one session. + + + +That third step is the one people skip, and the SDK writes events to local files — so a +complete integration can be proven on a laptop with **no server, no API key, and no +network**. Which is exactly why the skill insists on doing it. + +**Prerequisites:** Python 3.10+, the agent codebase, and the SDK. Nothing else — no +dashboard login, no key. + +--- + +## `agenteye-evaluator` — decide what to score, then build the scorer + +The hard part of evaluation is not the code. The [HTTP contract](/cloud/evaluators) is +small enough that an agent can implement it from the spec alone. Evaluators fail because +they **score the wrong thing** — and an evaluator that scores the wrong thing is worse than +none, because it produces a dashboard everyone learns to ignore. + +So most of this skill is the part before any code exists: + +```mermaid +flowchart TD + YOU["you: 'I want evals for my support bot'"] --> AGENT["coding agent
    loads the agenteye-evaluator skill"] + AGENT -->|"interview: what does good vs bad look like?"| YOU + AGENT -->|"reads your real sessions"| DATA["what actually happens"] + DATA --> DIMS["2-4 dimensions, you sign off"] + DIMS --> SVC["your evaluator service"] + SVC --> SCORES["scores land in the dashboard"] +``` + +It interviews you (*"describe a run that went well; now one that went badly"*), then pulls +your real sessions and reads them end to end. Those two halves usually disagree, and the +gap is the point: what you *intend* to measure versus what your transcripts can actually +support. + +A dimension only survives two tests. It must be **computable** from the events, and it must +be **discriminating** — if it scores 0.9 on both your good run and your bad one, it teaches +nothing and gets cut. What comes back is a proposal of 2–4 dimensions with the reasoning +attached, for you to approve before a line is written. + +**Prerequisites:** the CLI installed and logged in (with `events:read`, plus +`evaluations:read` for the final check), and somewhere real for the evaluator to live — it +becomes a long-running service, so it needs a repo, not a scratch file. Evaluators often +live in their own repo, separate from the agent being scored; the skill looks for one and +asks before scaffolding. + +--- + +## How these compare to the in-dashboard assistant + +Two natural-language front doors, very different blast radii: + +| | Agent skills | [In-dashboard assistant](/cloud/assistant) | +|---|---|---| +| Runs | On your workstation, in your coding agent | Server-side, in the dashboard | +| Authenticates as | You, via your CLI session | Your dashboard session, scoped to your read permissions | +| Can mutate | **Yes** — the CLI's full surface | Only saved queries and dashboards, each approval-gated | +| Can delete | **Yes** | **Never** | +| Best for | Doing things: provisioning, triage, building | Asking things: "how is quality trending this week?" | + +Both are useful, and most teams run both. Just know which one you are talking to. + +--- + +## Related + + + + + Every command, flag, and JSON shape the CLI skill drives. + + + + `jq` patterns and exit-code handling for scripts and agents. + + + + The event reference the SDK skill writes against. + + + + The scoring contract the evaluator skill implements. + + + diff --git a/docs/zh/agenteye/alerts.mdx b/docs/zh/cloud/alerts.mdx similarity index 74% rename from docs/zh/agenteye/alerts.mdx rename to docs/zh/cloud/alerts.mdx index cdd3e3ad..fdfab401 100644 --- a/docs/zh/agenteye/alerts.mdx +++ b/docs/zh/cloud/alerts.mdx @@ -4,9 +4,9 @@ description: "在问题越过你的底线时立即获知，通过团队已在使 --- -在问题越过你的底线时立即获知，通过团队已在使用的渠道，而不是等到客户反映才知道。规则只需设置一次，Failproof AI Observability 便会按计划检查，并通过邮件、Slack、webhook 或直接在仪表盘中通知你。 +在问题越过你的底线时立即获知，通过团队已在使用的渠道，而不是等到客户反映才知道。规则只需设置一次，FailproofAI Cloud 便会按计划检查，并通过邮件、Slack、webhook 或直接在仪表盘中通知你。 -![告警页面：告警规则卡片网格，每张卡片显示触发条件、评估窗口、通知渠道，以及信息、警告或严重等级标识](/agenteye/images/alerts.png) +![告警页面：告警规则卡片网格，每张卡片显示触发条件、评估窗口、通知渠道，以及信息、警告或严重等级标识](/cloud/images/alerts.png) *一览所有告警规则：监控内容、检查频率、通知渠道及紧急程度。* ## 在用户发现之前，先行了解问题 @@ -22,9 +22,9 @@ description: "在问题越过你的底线时立即获知，通过团队已在使 ## 用表单配置规则，而非 JSON -你只需在表单中描述什么叫"出了问题"，Failproof AI Observability 会自动生成底层规则。JSON 格式不过是表单背后生成的产物，你可以通过读它来理解规则，但几乎不需要手写。 +你只需在表单中描述什么叫"出了问题"，FailproofAI Cloud 会自动生成底层规则。JSON 格式不过是表单背后生成的产物，你可以通过读它来理解规则，但几乎不需要手写。 -![新建告警表单：名称与描述、启用开关，以及包含指标阈值、自定义 SQL、评估分数、复合评估、单事件条件的触发器选项](/agenteye/images/alert-new.png) +![新建告警表单：名称与描述、启用开关，以及包含指标阈值、自定义 SQL、评估分数、复合评估、单事件条件的触发器选项](/cloud/images/alert-new.png) *选择触发器后，表单会自动切换为对应字段；点击保存即写入规则。* 常规流程很快：填写名称、选择**触发器**（监控什么）、设置**阈值和窗口**（偏差多大、持续多久）、绑定至少一个**通知渠道**，然后**保存**，再点击**测试**发送一条模拟通知，确认每个目标渠道都已正确配置。在底层，这会生成一个简洁的规则描述，例如： @@ -43,7 +43,7 @@ description: "在问题越过你的底线时立即获知，通过团队已在使 | **复合评估** | 多项分数检查通过 any、all 或至少 N 项逻辑组合，用于捕捉仅在多项指标上共同体现的退化 | | **单事件** | 某个匹配的事件出现：特定 Agent、特定错误类型，或特定消息子字符串 | -已经在[错误页面](/zh/agenteye/error-tracking)盯着某个故障看了？每一行都有一个 **+ alert** 按钮，点击即可打开预填好的表单，专门用于捕获该故障的再次发生——你刚刚排查过的事件，下次出现时就会主动通知你。 +已经在[错误页面](/zh/cloud/errors)盯着某个故障看了？每一行都有一个 **+ alert** 按钮，点击即可打开预填好的表单，专门用于捕获该故障的再次发生——你刚刚排查过的事件，下次出现时就会主动通知你。 **在哪里找到它：** 告警位于 `//alerts`。创建、编辑、删除和测试规则需要 **`alerts:write`** 权限；仅查看只需 `alerts:read`。接收人选择器会按姓名列出你组织的成员，无需离开表单即可指定通知对象。 @@ -53,11 +53,11 @@ description: "在问题越过你的底线时立即获知，通过团队已在使 ## 告警触发后会发生什么 -一旦触发，系统会创建一个**事件**并通知你的渠道一次。之后由团队确认、分配负责人、跟进处理并最终解决——全程有清晰归属的记录可查。这套分诊流程有专属页面，详见[事件管理](/zh/agenteye/incidents)。 +一旦触发，系统会创建一个**事件**并通知你的渠道一次。之后由团队确认、分配负责人、跟进处理并最终解决——全程有清晰归属的记录可查。这套分诊流程有专属页面，详见[事件管理](/zh/cloud/incidents)。 ## 相关内容 -- [事件管理](/zh/agenteye/incidents)：追踪告警从触发到确认再到解决的全过程。 -- [错误追踪](/zh/agenteye/error-tracking)：对 Agent 故障进行分组，一键将其转化为告警规则。 -- [仪表盘](/zh/agenteye/dashboards)：查看共享看板，告警所依据的阈值均来源于此。 -- [CLI 与 Agents](/zh/agenteye/cli-and-agents)：从终端创建告警、确认事件，或将其脚本化集成到 CI 中。 \ No newline at end of file +- [事件管理](/zh/cloud/incidents)：追踪告警从触发到确认再到解决的全过程。 +- [错误追踪](/zh/cloud/errors)：对 Agent 故障进行分组，一键将其转化为告警规则。 +- [仪表盘](/zh/cloud/dashboards)：查看共享看板，告警所依据的阈值均来源于此。 +- [CLI 与 Agents](/zh/cloud/cli)：从终端创建告警、确认事件，或将其脚本化集成到 CI 中。 \ No newline at end of file diff --git a/docs/zh/agenteye/assistant.mdx b/docs/zh/cloud/assistant.mdx similarity index 84% rename from docs/zh/agenteye/assistant.mdx rename to docs/zh/cloud/assistant.mdx index 44ec814e..eeef89a7 100644 --- a/docs/zh/agenteye/assistant.mdx +++ b/docs/zh/cloud/assistant.mdx @@ -4,9 +4,9 @@ description: "用自然语言向 Agent 数据提问，并直接获取链接到 --- -用自然语言向 Agent 数据提问，并直接获取链接到具体证据的答案。无需编写 SQL，无需翻查仪表盘——**Failproof AI Observability** 助手是团队中任何人获取 Agent 相关答案的最快方式。 +用自然语言向 Agent 数据提问，并直接获取链接到具体证据的答案。无需编写 SQL，无需翻查仪表盘——**FailproofAI Cloud** 助手是团队中任何人获取 Agent 相关答案的最快方式。 -![Failproof AI Observability 助手在仪表盘中回答自然语言问题的界面，展示了实时 Agent 活动表、按 Agent 划分的模型使用情况，以及文字摘要，所执行的查询也内联显示](/agenteye/images/assistant.png) +![FailproofAI Cloud 助手在仪表盘中回答自然语言问题的界面，展示了实时 Agent 活动表、按 Agent 划分的模型使用情况，以及文字摘要，所执行的查询也内联显示](/cloud/images/assistant.png) *用自然语言提问，答案直接来自你自己的数据。这里它分解了哪些 Agent 最繁忙、它们使用了哪些模型，并展示了执行的查询，方便你核实每一个数字。* 无需任何学习成本。打开对话框，输入你想了解的内容，然后点击它返回的链接： @@ -36,7 +36,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a 在 **Queries** 页面，助手更进一步，化身 SQL 编写者：描述你想要的查询（"显示过去 7 天内各 Agent 的错误率"），它会将 SQL 直接流式输入编辑器，并打开差异视图，让你在内容落定前选择 **Accept** 或 **Reject**。 -![Observability Queries 页面及其 SQL 编辑器](/agenteye/images/query-lab.png) +![FailproofAI Cloud Queries 页面及其 SQL 编辑器](/cloud/images/query-lab.png) *Queries 页面：编辑器是助手流式生成草稿查询的地方，查询为只读状态，供你接受或拒绝。* 在此通过提问来编写 SQL 使用的是 `queries:run` 权限，与编辑器中 **Run** 按钮背后的权限相同。其他地方的对话则需要 `agent:use` 权限。 @@ -49,7 +49,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a - **每次写入操作都需要你确认。** 已保存查询和仪表盘只有在你明确点击 Approve 后才会创建，且没有任何设置可以关闭这道审批门。 - **它永远无法删除任何内容。** 没有删除工具暴露给助手，它也不持有删除权限。删除操作始终由你在仪表盘中亲自完成。 - **它仅限于你的组织内部。** 助手只能查看你当前所在的组织。 -- **你的问题属于你自己。** 提示词和答案存储在你自己的 Observability 数据库中；产品分析功能只记录使用元数据，从不记录你的提示词文本。 +- **你的问题属于你自己。** 提示词和答案存储在你自己的 FailproofAI Cloud 数据库中；产品分析功能只记录使用元数据，从不记录你的提示词文本。 ## 在哪里找到它 @@ -57,7 +57,7 @@ AI: This run took 12 steps across 3 tools and failed near the end when a ## 相关内容 -- [CLI 与 Agents](/zh/agenteye/cli-and-agents) -- [查询](/zh/agenteye/queries) -- [仪表盘](/zh/agenteye/dashboards) -- [评估套件](/zh/agenteye/evaluation-suite) \ No newline at end of file +- [CLI 与 Agents](/zh/cloud/cli) +- [查询](/zh/cloud/queries) +- [仪表盘](/zh/cloud/dashboards) +- [评估套件](/zh/cloud/evaluators) \ No newline at end of file diff --git a/docs/zh/agenteye/audits.mdx b/docs/zh/cloud/audits.mdx similarity index 80% rename from docs/zh/agenteye/audits.mdx rename to docs/zh/cloud/audits.mdx index b1400d83..9f21f053 100644 --- a/docs/zh/agenteye/audits.mdx +++ b/docs/zh/cloud/audits.mdx @@ -1,10 +1,10 @@ --- title: "审计：您的自动可靠性分析师" -description: "Failproof AI Observability 会主动发现那些您从未为之编写规则的故障，并为您提供一份按优先级排列、有证据支撑的待办清单，告诉您究竟需要修复什么。" +description: "FailproofAI Cloud 会主动发现那些您从未为之编写规则的故障，并为您提供一份按优先级排列、有证据支撑的待办清单，告诉您究竟需要修复什么。" --- -Failproof AI Observability 会主动发现那些您从未为之编写规则的故障，并为您提供一份按优先级排列、有证据支撑的待办清单，告诉您究竟需要修复什么。这就像每晚都有一位分析师梳理您的日志，然后在清晨将简短的清单放在您的桌上。 +FailproofAI Cloud 会主动发现那些您从未为之编写规则的故障，并为您提供一份按优先级排列、有证据支撑的待办清单，告诉您究竟需要修复什么。这就像每晚都有一位分析师梳理您的日志，然后在清晨将简短的清单放在您的桌上。
    @@ -12,7 +12,7 @@ Failproof AI Observability 会主动发现那些您从未为之编写规则的 *两分钟概览：从计划运行到可付诸行动的修复方案。* -![审计页面：定期扫描会话以发现故障模式的周期性任务，每项任务都有计划和灵敏度设置](/agenteye/images/audits.png) +![审计页面：定期扫描会话以发现故障模式的周期性任务，每项任务都有计划和灵敏度设置](/cloud/images/audits.png) *每个审计都是一个周期性任务，负责挖掘您的会话数据并输出按优先级排列、有证据支撑的改进建议。* ## 不再猜测下一步修复什么 @@ -48,7 +48,7 @@ Failproof AI Observability 会主动发现那些您从未为之编写规则的 ## 相关内容 -- [告警](/zh/agenteye/alerts)：在您已知的阈值被触发的瞬间收到通知。 -- [评估](/zh/agenteye/evaluations)：对每次运行进行评分，让质量回归问题自动浮现。 -- [错误追踪](/zh/agenteye/error-tracking)：对 Agent 抛出的错误进行分组和跟踪。 -- [事件](/zh/agenteye/incidents)：将审计发现的问题追踪至最终修复完成。 \ No newline at end of file +- [告警](/zh/cloud/alerts)：在您已知的阈值被触发的瞬间收到通知。 +- [评估](/zh/cloud/evaluations)：对每次运行进行评分，让质量回归问题自动浮现。 +- [错误追踪](/zh/cloud/errors)：对 Agent 抛出的错误进行分组和跟踪。 +- [事件](/zh/cloud/incidents)：将审计发现的问题追踪至最终修复完成。 \ No newline at end of file diff --git a/docs/zh/cloud/capture.mdx b/docs/zh/cloud/capture.mdx new file mode 100644 index 00000000..071dd028 --- /dev/null +++ b/docs/zh/cloud/capture.mdx @@ -0,0 +1,177 @@ +--- +title: Session capture +description: "Bring the agent work your team already does — across all 12 supported CLIs — into the cloud as ordinary sessions, with no change to how anyone works." +icon: satellite-dish +--- + +Your engineers already run coding agents every day. Session capture brings that work into +FailproofAI Cloud as ordinary sessions and events, so you can search, replay, score, and +alert on it next to everything else you observe. + +It complements the [Python SDK](/cloud/sdk): the SDK instruments agents *you write*, while +capture covers the agent CLIs your team *already uses* — with no change to how they run +them. + +--- + +## Turning it on + +There is nothing extra to install. Capture is part of connecting a machine: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +That is it. The [background service](/daemon) already on the machine reads each agent CLI's +own session files as they are written and ships them, alongside the policy decisions it is +already reporting. + +```bash +failproofai config --status # is this machine connected, and what is it sending? +failproofai flush --wait # deliver everything spooled right now +``` + +On first run, the sessions already on the machine are backfilled once; new activity then +streams within seconds. + +--- + +## What gets captured + +Every one of the [12 supported agent CLIs](/agent-support) is a capture source: + +| | | | +|---|---|---| +| Claude Code | OpenAI Codex | GitHub Copilot CLI | +| Cursor Agent | OpenCode | Pi | +| Hermes | OpenClaw | Factory Droid | +| Devin CLI | Antigravity CLI | Goose | + +One machine, one connection, every CLI on it. There is no per-CLI setup and no per-project +step. + +Each session becomes a cloud [session](/cloud/sessions); its user and assistant messages, +reasoning, tool calls, tool results, and token usage become the matching +[events](/cloud/event-stream). Everything downstream then works on them — +[replay](/cloud/sessions), [search](/cloud/queries), [evaluations](/cloud/evaluations), +[audits](/cloud/audits), and [alerts](/cloud/alerts). + +Where a CLI records it, the **surface** a session came from is preserved too: whether a +Codex session ran in the CLI, the IDE extension, or the desktop app; which channel a +Hermes or OpenClaw session came in on (Slack, Telegram, terminal, or a scheduled run); and +when a session spawned another, the link back to its parent. + +**Your files are only ever read.** Never modified, never moved, never deleted. Each session +is shipped once, even across restarts. + + + **Cloud-executed sessions are not captured.** Some agent CLIs increasingly run sessions + on their vendor's own infrastructure and keep only metadata on the machine — there is no + local transcript to read. Only locally-executed sessions are captured. + + +--- + +## Transcripts in a non-standard place + +Containers, second checkouts, shared volumes, mounted VM disks — a transcript directory is +not always where the CLI puts it by default. Point FailproofAI at it: + +```bash +failproofai harness add-path claude ~/work/mirror/.claude/projects +failproofai harness add-path codex "vm-a=/mnt/vm-a/.codex/sessions" +failproofai harness list +failproofai harness remove-path codex "vm-a=/mnt/vm-a/.codex/sessions" +``` + +The optional `label=` prefix namespaces the agent ids that come out of that path. Without +it, two copies of the same project collapse into one confusing timeline; with it, they stay +distinct. + +Two rejections that exist to prevent silent failures: + +- **A path overlapping a default location is refused.** It would be collected twice, under + two different agent ids. +- **Two entries sharing a label are refused.** They would share progress state, and both + would re-read from the beginning after every restart. + +For containers, `FAILPROOFAI__EXTRA_PATHS` (comma-separated) overrides the file +per source. [Full command reference →](/cli/harness) + +--- + +## Catching up on history + +Connected a machine after the work happened? Cleared a dashboard? Re-enrolled a host? + +```bash +failproofai backfill --since 6m # re-read the last six months +failproofai backfill --since 30d # or a shorter window +failproofai backfill --dry-run # report what would be re-read, change nothing +``` + +Backfill re-sends history the collector has already read past. Sessions are shipped once, +so re-running it does not duplicate anything. + +--- + +## Delivery you can trust + +`failproofai config --status` tells you whether what was captured actually **arrived** — +not merely that a process is alive. + +If a batch cannot be delivered it is **kept and retried**, not discarded, and the machine +reports as unhealthy while anything is still outstanding. "Healthy" means your data landed. + +--- + +## Privacy + + + Agent transcripts contain the **whole session** — prompts, model responses, file contents + the agent read or wrote, and command output. They can contain secrets. Captured sessions + are shipped as they are. + + Enable capture only on machines and for teams where centralizing that content is + appropriate, and give each machine a key scoped to what it actually needs. + + +Want the fleet view without the transcripts? + +```bash +failproofai config --connect --token --no-transcripts +``` + +Policy decisions still flow — which policy fired, on which tool, in which session, with +what verdict — so you keep enforcement visibility across the fleet without centralizing +file contents. `--status` always reports which mode is in effect. + +Note that the local [sanitize policies](/built-in-policies#secrets-sanitizers) redact +secrets from tool output *before the model reads them*, which reduces (but does not +eliminate) what a transcript can contain. Treat transcripts as sensitive regardless. + +[How your data is isolated →](/cloud/security) + +--- + +## Related + + + + + The command, the permissions, and what leaves the machine. + + + + Where captured sessions land, and how to read them. + + + + Instrument agents you write yourself. + + + + Every CLI, and what enforcement each supports. + + + diff --git a/docs/zh/agenteye/cli-recipes.mdx b/docs/zh/cloud/cli-recipes.mdx similarity index 91% rename from docs/zh/agenteye/cli-recipes.mdx rename to docs/zh/cloud/cli-recipes.mdx index f8a55923..c3389200 100644 --- a/docs/zh/agenteye/cli-recipes.mdx +++ b/docs/zh/cloud/cli-recipes.mdx @@ -4,9 +4,9 @@ description: "可直接复制粘贴的查询模式和 jq 脚本，将会话、 --- -直接通过脚本或 AI Coding Agent 拉取会话、事件和评估数据（并触发重新评估），输出干净的 JSON 到 stdout，可直接通过管道传入 `jq`。这些脚本将 Failproof AI Observability 的数据转化为终端用户或 AI Coding Agent（Claude Code、Cursor）可以查询和自动化处理的格式，无需点击仪表盘。 +直接通过脚本或 AI Coding Agent 拉取会话、事件和评估数据（并触发重新评估），输出干净的 JSON 到 stdout，可直接通过管道传入 `jq`。这些脚本将 FailproofAI Cloud 的数据转化为终端用户或 AI Coding Agent（Claude Code、Cursor）可以查询和自动化处理的格式，无需点击仪表盘。 -以下模式均可直接复制粘贴，适用于 Failproof AI Observability CLI（`agenteye`）。安装、认证及完整选项列表请参阅 [CLI](/zh/agenteye/cli)；运行 `agenteye -h` 或 `agenteye -h` 查看内置帮助。 +以下模式均可直接复制粘贴，适用于 FailproofAI Cloud CLI（`agenteye`）。安装、认证及完整选项列表请参阅 [CLI](/zh/cloud/cli)；运行 `agenteye -h` 或 `agenteye -h` 查看内置帮助。 ## 基本规则 @@ -173,7 +173,7 @@ esac ## 下一步 -- [CLI](/zh/agenteye/cli)：安装、认证及每个命令的完整选项参考。 -- [CLI agent skill](/zh/agenteye/cli-skill)：将这些脚本打包为 AI Coding Agent 可加载的技能。 -- [API keys](/zh/agenteye/api-keys)：创建并限定 CLI、SDK 和 collector 认证所用密钥的权限范围。 -- [Python SDK](/zh/agenteye/python-sdk)：向 Failproof AI Observability 发送事件，为上述脚本提供可查询的数据。 \ No newline at end of file +- [CLI](/zh/cloud/cli)：安装、认证及每个命令的完整选项参考。 +- [CLI agent skill](/zh/cloud/agent-skills)：将这些脚本打包为 AI Coding Agent 可加载的技能。 +- [API keys](/zh/cloud/access)：创建并限定 CLI、SDK 和 collector 认证所用密钥的权限范围。 +- [Python SDK](/zh/cloud/sdk)：向 FailproofAI Cloud 发送事件，为上述脚本提供可查询的数据。 \ No newline at end of file diff --git a/docs/zh/agenteye/cli.mdx b/docs/zh/cloud/cli.mdx similarity index 91% rename from docs/zh/agenteye/cli.mdx rename to docs/zh/cloud/cli.mdx index f141ed47..5c9b679d 100644 --- a/docs/zh/agenteye/cli.mdx +++ b/docs/zh/cloud/cli.mdx @@ -1,10 +1,10 @@ --- title: "CLI" -description: "通过终端或脚本驱动所有 Failproof AI Observability 功能，无需往返控制台。" +description: "通过终端或脚本驱动所有 FailproofAI Cloud 功能，无需往返控制台。" --- -通过终端或脚本驱动所有 Failproof AI Observability 功能，无需往返控制台。`agenteye` CLI 可查询您的数据（会话、事件日志、评估结果）并管理您的组织（API 密钥、用户、设置、告警、事件、已保存查询），非常适合自动化检查、将 Observability 集成到 CI 流程，或让编码智能体检查生产环境。每个命令均支持 `--json` 标志，因此无论是您在终端交互使用，还是编码智能体（Claude Code、Cursor）调用并解析结果，都同样适用。 +通过终端或脚本驱动所有 FailproofAI Cloud 功能，无需往返控制台。`agenteye` CLI 可查询您的数据（会话、事件日志、评估结果）并管理您的组织（API 密钥、用户、设置、告警、事件、已保存查询），非常适合自动化检查、将 FailproofAI Cloud 集成到 CI 流程，或让编码智能体检查生产环境。每个命令均支持 `--json` 标志，因此无论是您在终端交互使用，还是编码智能体（Claude Code、Cursor）调用并解析结果，都同样适用。 使用这一个二进制文件，您可以： @@ -69,7 +69,7 @@ agenteye --version agenteye --help ``` -> **注意：** Failproof AI Observability Python SDK 也使用 `agenteye` 这个发行包名称。使用 `pipx` 或 `uv tool` 安装 CLI（而非 `pip install` 到共享虚拟环境中）可以避免两者冲突。只有在同一环境中未安装 SDK 的情况下，才可以直接使用 `pip install agenteye`。 +> **注意：** FailproofAI Cloud Python SDK 也使用 `agenteye` 这个发行包名称。使用 `pipx` 或 `uv tool` 安装 CLI（而非 `pip install` 到共享虚拟环境中）可以避免两者冲突。只有在同一环境中未安装 SDK 的情况下，才可以直接使用 `pip install agenteye`。 --- @@ -91,7 +91,7 @@ agenteye logout # 吊销会话并清除存储的令牌 `whoami` 在会话缺失或过期时不会报错，而是返回 `logged_in: false`，因此脚本或智能体可以安全地探测认证状态（如果未设置 base URL 或控制台不可达，仍可能以非零状态退出）。 -**要求：** 您的邮箱必须获准登录控制台（请联系您的 Failproof AI Observability 管理员），且控制台必须可通过其 base URL 访问（参见[配置](#configuration)）。如果申请了验证码但未收到，您的邮箱可能尚未开通控制台访问权限。 +**要求：** 您的邮箱必须获准登录控制台（请联系您的 FailproofAI Cloud 管理员），且控制台必须可通过其 base URL 访问（参见[配置](#configuration)）。如果申请了验证码但未收到，您的邮箱可能尚未开通控制台访问权限。 --- @@ -338,13 +338,13 @@ agenteye agent rename --title "error triage" ; agenteye agent delete | 5 | 已认证，但账户缺少所需权限（消息中会指明具体权限） | | 6 | 请求的资源未找到（例如未知的会话或事件 id） | -这些退出码使 CLI 适合脚本化使用：编码智能体可以根据 `4` 提示您重新认证，或根据 `5` 提示缺少的权限。请参阅 [CLI 智能体使用食谱](/zh/agenteye/cli-recipes)，了解退出码处理模式和 JSON 输出结构。 +这些退出码使 CLI 适合脚本化使用：编码智能体可以根据 `4` 提示您重新认证，或根据 `5` 提示缺少的权限。请参阅 [CLI 智能体使用食谱](/zh/cloud/cli-recipes)，了解退出码处理模式和 JSON 输出结构。 --- ## 下一步 -- **[CLI 智能体使用食谱](/zh/agenteye/cli-recipes)**：可直接复用的查询模式、`jq` 单行命令、`--fields` 投影、退出码处理以及 JSON 输出结构，专为驱动 CLI 的编码智能体编写。 -- **[CLI 智能体技能](/zh/agenteye/cli-skill)**：将此 CLI 打包为可安装的 Claude Code / Codex *技能*，让编码智能体通过自然语言请求驱动 Failproof AI Observability。 -- **[API 密钥](/zh/agenteye/api-keys)**：`keys create --add …` 背后的权限模型。 -- **[AI 助手](/zh/agenteye/assistant)**：启用 `agent ask` 所使用的助手。 \ No newline at end of file +- **[CLI 智能体使用食谱](/zh/cloud/cli-recipes)**：可直接复用的查询模式、`jq` 单行命令、`--fields` 投影、退出码处理以及 JSON 输出结构，专为驱动 CLI 的编码智能体编写。 +- **[CLI 智能体技能](/zh/cloud/agent-skills)**：将此 CLI 打包为可安装的 Claude Code / Codex *技能*，让编码智能体通过自然语言请求驱动 FailproofAI Cloud。 +- **[API 密钥](/zh/cloud/access)**：`keys create --add …` 背后的权限模型。 +- **[AI 助手](/zh/cloud/assistant)**：启用 `agent ask` 所使用的助手。 \ No newline at end of file diff --git a/docs/zh/cloud/connect.mdx b/docs/zh/cloud/connect.mdx new file mode 100644 index 00000000..5495f6a8 --- /dev/null +++ b/docs/zh/cloud/connect.mdx @@ -0,0 +1,289 @@ +--- +title: Connect a machine +description: "One command, one key, two capabilities — and a plain statement of exactly what leaves the machine." +icon: plug +--- + +Connecting a machine to FailproofAI Cloud opens two streams in opposite directions: + +```mermaid +flowchart LR + subgraph M["Your machine"] + D["failproofaid"] + end + subgraph C["FailproofAI Cloud"] + S["your organization"] + end + S -->|"policy down · policies:pull"| D + D -->|"activity + sessions up · events:add"| S +``` + +You give it one URL and one key, and both are configured from that. Asking twice is what +made this feel like two products — connect for policy, see an empty dashboard, and +reasonably conclude the thing is broken. + +--- + +## The command + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +Or run `failproofai config` and choose **Paste an API key** when it asks. Both paths write +byte-identical state, so a machine set up interactively and one set up by a script end up +the same. + +Don't have a key? Create one at +[befailproof.ai/get-started](https://befailproof.ai/get-started/). + +| Flag | What it does | +|---|---| +| `--connect ` | The cloud base URL. Your dashboard origin is the right value. | +| `--token ` | An API key for your organization. See [which permissions it needs](#what-the-key-needs). | +| `--machine-id ` | A stable id for this machine. Defaults to the one already recorded here, or a fresh random one. | +| `--machine-label ` | The human-readable name shown in the dashboard. Defaults to the hostname. | +| `--no-transcripts` | Send policy decisions only — never session transcripts. | +| `--disconnect` | Stop pulling policy and stop sending activity. | +| `--status` | Show connection, service, and pause state. | + + + Connecting needs **no root**. It writes a credential file the service reads rather than + baking a token into the service definition — that file is world-readable, so a token + there would hand an organization-scoped key to every local user. Re-connecting, rotating + a token, and disconnecting are all unprivileged, and an already-running service can be + connected without reinstalling anything. + + +--- + +## What leaves this machine + +Read this section before you connect a machine that touches anything sensitive. + +Connecting turns on **both** streams by default: + +| Stream | Contents | +|---|---| +| **Policy decisions** | Which policy fired, on which tool, in which session, with what verdict and reason. Tool *names*, never file contents. | +| **Session transcripts** | The full agent session — prompts, model responses, file contents the agent read or wrote, and command output. | + +Transcripts are the point. A dashboard that shows only decisions is the empty-dashboard +problem in a different costume: you can see that something was blocked, but not what your +agents actually did. That is also exactly why it is stated here in plain words rather than +buried behind a flag nobody finds. + +**If that is more than you want to centralize:** + +```bash +failproofai config --connect --token --no-transcripts +``` + +Decisions still flow, transcripts never do. `failproofai config --status` always reports +which mode is in effect, so nobody has to guess. + +Whichever you choose, the machine keeps enforcing locally either way — connecting adds +visibility and central policy, it never removes protection. + +--- + +## What the key needs + +One key, two independent permissions: + +| Permission | Enables | +|---|---| +| `policies:pull` | Receiving centrally-managed policy | +| `events:add` | Reporting decisions and sessions | + +Both are verified **before anything is written**, and reported **separately** — because a +key carrying one and not the other is a real, supported state, not a broken setup. + +| Key carries | What happens | +|---|---| +| Both | Fully connected. Policy arrives, activity flows, the dashboard fills. | +| `policies:pull` only | Connected for policy. Enforcement works; the CLI tells you the dashboard will stay empty and exactly why. | +| `events:add` only | Connected for reporting. The machine keeps enforcing its **local** policies and reports what they decide, but receives no central ones. | +| Neither | Nothing is written. A credential file that does not work is worse than none, because `--status` would then report a connection the machine does not have. | + +The organization the key belongs to is named on every outcome, including the partial ones. +A key pasted from the wrong organization authenticates perfectly and reports somewhere +nobody is looking — naming the org on screen is what makes that visible immediately. + +[Creating scoped keys →](/cloud/access) + +--- + +## Machine identity + +Two separate things, and the distinction matters: + +- **Machine id** — the stable identity your fleet history, deployments, and enrolment are + keyed on. Reconnecting reuses the id already on the machine, so `--connect` is idempotent + and never "moves" a host. +- **Machine label** — the human-readable name in the dashboard. Defaults to the hostname, + and is display-only. + +A machine that has never carried an id gets a **random** one — deliberately not the +hostname. Two hosts sharing a hostname (fresh cloud VMs, cloned images) would otherwise +silently merge into one machine on the server, stranding one host's history and making the +fleet page lie about your coverage. + +Renaming later needs no re-enrolment: + +```bash +failproofai config --machine-label "build-runner-3" +``` + +--- + +## Environments + +Label what a machine belongs to — `production`, `staging`, `dev` — and almost every +dashboard surface can filter by it. It is set on the machine's collector settings and +stamped on everything it reports. + + + An environment name must not contain a comma. Dashboard filters pass environments as a + comma-separated list, so `prod,blue` would be read as two values. Events carrying one are + rejected at ingest. + + +--- + +## Checking it worked + +```bash +failproofai config --status +``` + +Reports the connection (including which organization and which mode), whether the service +is running, and whether enforcement is paused on any session. + +Two commands for when you want to stop waiting: + +```bash +failproofai flush --wait # deliver everything spooled right now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +`backfill` is the one to reach for after clearing a dashboard, re-enrolling a machine, or +connecting later than the work you want to see. `--dry-run` reports what would be re-read +without changing anything. + +--- + +## Connecting a fleet without a human at each keyboard + +`--connect` is non-interactive by design, so it drops straight into whatever you already +use to configure machines: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +A few things that make this safe to run unattended: + +- **Idempotent.** Re-running it on a connected machine reuses the existing id and re-verifies + the key rather than creating a second machine. +- **Verified before written.** A typo'd or revoked key fails at connect time with a precise + reason, instead of becoming a silent pile of rejected uploads discovered a week later. +- **Refuses plaintext.** A token is never sent to a non-`https` host — except `localhost`, + where there is no network to intercept. +- **Exit codes mean something.** A failed connect exits non-zero with the reason on stderr. + + + Bake the guardrails into your machine image and connect at boot. A machine that has + FailproofAI but is not connected still enforces locally — it just does not appear in your + fleet view, which is the one gap the [fleet page](/cloud/fleet) is built to make obvious. + + +--- + +## Disconnecting + +```bash +failproofai config --disconnect +``` + +This does both halves properly: it clears the credentials **and** stops enforcing the +cloud-managed deployment. Clearing credentials alone would stop the machine *refreshing* +policy while every artifact already on disk kept being enforced on every tool call — so a +machine that deliberately left an organization would go on being governed by whatever +deployment happened to be current when it left, indefinitely, while `--status` reported it +as unconnected. + +Local policies are untouched. The machine keeps enforcing exactly what it enforced before +it was ever connected. + +--- + +## Troubleshooting + + + + + The key was not accepted at all. Check it was copied whole — keys are long, and a + truncated paste looks like a valid string. + + + + The key is valid but too narrow. Create one with the permission you need, or add it to + the existing key. See [Access](/cloud/access). + + + + You pointed at the dashboard's web front end rather than its API path. Pass the plain + origin (`https://app.befailproof.ai`) and let the CLI derive the rest — it accepts either + form, but a redirect that lands on a login page would otherwise look like success while + every upload was silently lost. + + + + Almost always a key with `policies:pull` and not `events:add`. `failproofai config + --status` names the missing permission. If both are present, run `failproofai flush + --wait` to force a delivery and see the result immediately. + + + + Something changed the machine id between connections — usually an explicit `--machine-id` + on one run and not the other. Reconnect with the id you want to keep; the id, not the + label, is what history is keyed on. + + + + That is the [fail-closed guarantee](/daemon#fail-closed) doing its job: on a configured + machine, a guardrail that cannot answer denies. Check the service is running with + `failproofai config --status`. If it reports a protocol-version mismatch, run + `failproofai config` to bring both halves back into step. + + + + +--- + +## Related + + + + + What comes down the policy stream, and how to roll it out safely. + + + + Every machine, its deployment, and its coverage. + + + + Creating a key with exactly the two permissions this needs. + + + + What actually moves the data, and what happens when it can't. + + + diff --git a/docs/zh/agenteye/dashboards.mdx b/docs/zh/cloud/dashboards.mdx similarity index 66% rename from docs/zh/agenteye/dashboards.mdx rename to docs/zh/cloud/dashboards.mdx index 2ba487d6..bd17b461 100644 --- a/docs/zh/agenteye/dashboards.mdx +++ b/docs/zh/cloud/dashboards.mdx @@ -6,7 +6,7 @@ description: "将实时智能体数据转化为团队共享的统一视图。" 将实时智能体数据转化为团队共享的统一视图。将重要查询固定为图表，团队所有人一眼即可看到相同的数据，无需重复执行任何查询。 -![基于已保存查询构建的仪表板：每小时事件折线图、按类型划分的错误柱状图、延迟面积图和按模型划分的 token 用量](/agenteye/images/dashboard-fleet.png) +![基于已保存查询构建的仪表板：每小时事件折线图、按类型划分的错误柱状图、延迟面积图和按模型划分的 token 用量](/cloud/images/dashboard-fleet.png) *一块看板，四个已保存查询：每小时事件数、按类型划分的错误、延迟和按模型划分的 token 用量。* @@ -25,15 +25,15 @@ description: "将实时智能体数据转化为团队共享的统一视图。" ## 固定已保存的查询 -每个图块都从已保存的查询开始。在[查询](/zh/agenteye/queries)库（包含内置预设以及您自定义的查询，覆盖事件和评估数据）中构建并保存您关心的查询，然后将其固定到仪表板，选择最适合数据的图表类型：**折线图**用于展示随时间变化的趋势，**柱状图**用于对比各分类，**面积图**用于展示数据量，**饼图**用于展示占比分布。 +每个图块都从已保存的查询开始。在[查询](/zh/cloud/queries)库（包含内置预设以及您自定义的查询，覆盖事件和评估数据）中构建并保存您关心的查询，然后将其固定到仪表板，选择最适合数据的图表类型：**折线图**用于展示随时间变化的趋势，**柱状图**用于对比各分类，**面积图**用于展示数据量，**饼图**用于展示占比分布。 由于图块本质上就是将已保存的查询渲染为图表，因此无需手动同步任何内容。只需更新一次查询，所有使用该查询的仪表板都会自动更新。 ## 关注质量，而不仅仅是数量 -数量告诉您智能体正在忙碌运行，质量才能告诉您它们是否真正完成了工作。将仪表板指向您的[评估分数](/zh/agenteye/evaluations)，即可获得一块追踪运行质量随时间变化的看板，让质量下降以图表曲线低谷的形式呈现，而不是来自用户的意外投诉。 +数量告诉您智能体正在忙碌运行，质量才能告诉您它们是否真正完成了工作。将仪表板指向您的[评估分数](/zh/cloud/evaluations)，即可获得一块追踪运行质量随时间变化的看板，让质量下降以图表曲线低谷的形式呈现，而不是来自用户的意外投诉。 -![基于已保存评估查询构建的质量仪表板](/agenteye/images/dashboard-quality.png) +![基于已保存评估查询构建的质量仪表板](/cloud/images/dashboard-quality.png) *质量看板将评估分数置于核心位置，与运营数据并排展示。* @@ -41,6 +41,6 @@ description: "将实时智能体数据转化为团队共享的统一视图。" ## 相关内容 -- [查询](/zh/agenteye/queries)：构建并保存成为图块的查询。 -- [评估](/zh/agenteye/evaluations)：对运行结果评分，以便随时间追踪质量变化。 -- [告警](/zh/agenteye/alerts)：对任意指标设置阈值并触发通知。 \ No newline at end of file +- [查询](/zh/cloud/queries)：构建并保存成为图块的查询。 +- [评估](/zh/cloud/evaluations)：对运行结果评分，以便随时间追踪质量变化。 +- [告警](/zh/cloud/alerts)：对任意指标设置阈值并触发通知。 \ No newline at end of file diff --git a/docs/zh/agenteye/error-tracking.mdx b/docs/zh/cloud/errors.mdx similarity index 66% rename from docs/zh/agenteye/error-tracking.mdx rename to docs/zh/cloud/errors.mdx index 7802ac30..a77bd9bb 100644 --- a/docs/zh/agenteye/error-tracking.mdx +++ b/docs/zh/cloud/errors.mdx @@ -5,14 +5,14 @@ description: "在一处查看所有 Agent 产生的失败，并自动归组， 在一处查看所有 Agent 产生的失败，并自动归组，让密集的错误爆发呈现为单一问题。你只需一键，便能从"某处出现红色报错"直接跳转到确切的出问题运行记录，无需滚动实时日志去寻找。 -![错误页面：顶部是错误随时间分布的直方图，下方是分组的红色错误行，每行都有一键式"+ alert"按钮](/agenteye/images/errors.png) +![错误页面：顶部是错误随时间分布的直方图，下方是分组的红色错误行，每行都有一键式"+ alert"按钮](/cloud/images/errors.png) *错误页面：顶部是错误随时间分布的直方图，重复失败会折叠为每个事件一行。* ## 所有失败，自动为你汇总 Agent 出错时，你不应该还要滚动实时事件流，焦急地等待红色行出现，又担心它们随即消失。**错误**页面替你完成收集工作。它将仪表板中所有标红的内容汇聚到一个统一的分诊界面，让你第一眼看到的是哪里出了问题，而不是去哪里找问题。 -它捕获的不只是显而易见的错误。除了显式的 `error` 事件，Failproof AI Observability 还会把那些悄无声息的失败浮出水面：任何携带失败信息的 `tool_result`、`hook_completed` 或 `agent_end` 都会出现在这里。工具返回了错误，或者 hook 异常退出，即使没有抛出明显的异常，它们也不会再悄悄溜走。 +它捕获的不只是显而易见的错误。除了显式的 `error` 事件，FailproofAI Cloud 还会把那些悄无声息的失败浮出水面：任何携带失败信息的 `tool_result`、`hook_completed` 或 `agent_end` 都会出现在这里。工具返回了错误，或者 hook 异常退出，即使没有抛出明显的异常，它们也不会再悄悄溜走。 页面顶部的直方图展示了错误随时间的分布情况。一眼即可判断这是持续的背景噪音，还是几分钟前突然出现的峰值——让你立刻决定是否需要放下手头的工作去处理。 @@ -22,19 +22,19 @@ Agent 出错时，你不应该还要滚动实时事件流，焦急地等待红 一个依赖损坏可能每分钟触发数百次相同的错误。如果原始展示，那就是一大堵几乎相同的日志行，把你真正需要看的信息完全淹没。 -Failproof AI Observability 会将同一会话中相同错误类型的重复失败折叠为一行。一次爆发呈现为一个事件。你数的是问题数，而不是日志行数，关键信号始终置于顶端，不会被自身的数量所淹没。 +FailproofAI Cloud 会将同一会话中相同错误类型的重复失败折叠为一行。一次爆发呈现为一个事件。你数的是问题数，而不是日志行数，关键信号始终置于顶端，不会被自身的数量所淹没。 ## 从"某处出现红色"直达确切事件 点击任意一行，即可直接进入该运行的会话，并定位到出错的确切事件。无需复制会话 ID，无需滚动寻找出问题的时刻：你直接就站在那里，完整的执行图一目了然，让你能清楚看到 Agent 在出错前都做了什么。 -如果你拥有 `alerts:write` 权限，每一行还带有一个 **+ alert** 按钮。点击后，Observability 会打开一条新的告警规则，并预填好内容以捕获相同的失败。你刚刚处理过的事件，下次发生时会主动通知你，而不是再次让你措手不及。 +如果你拥有 `alerts:write` 权限，每一行还带有一个 **+ alert** 按钮。点击后，FailproofAI Cloud 会打开一条新的告警规则，并预填好内容以捕获相同的失败。你刚刚处理过的事件，下次发生时会主动通知你，而不是再次让你措手不及。 **访问路径：** **错误**页面位于仪表板的观测区域，路径为 `//errors`。 ## 相关内容 -- [告警](/zh/agenteye/alerts)：将任何失败转化为通知规则。 -- [事件](/zh/agenteye/incidents)：追踪从触发到解决的完整告警过程。 -- [会话](/zh/agenteye/sessions)：打开任意错误背后的完整运行记录。 -- [审计](/zh/agenteye/audits)：让 Observability 自动为你发现运行中的失败模式。 \ No newline at end of file +- [告警](/zh/cloud/alerts)：将任何失败转化为通知规则。 +- [事件](/zh/cloud/incidents)：追踪从触发到解决的完整告警过程。 +- [会话](/zh/cloud/sessions)：打开任意错误背后的完整运行记录。 +- [审计](/zh/cloud/audits)：让 FailproofAI Cloud 自动为你发现运行中的失败模式。 \ No newline at end of file diff --git a/docs/zh/agenteye/evaluations.mdx b/docs/zh/cloud/evaluations.mdx similarity index 67% rename from docs/zh/agenteye/evaluations.mdx rename to docs/zh/cloud/evaluations.mdx index 72e9c0bd..d6356162 100644 --- a/docs/zh/agenteye/evaluations.mdx +++ b/docs/zh/cloud/evaluations.mdx @@ -4,15 +4,15 @@ description: "质量问题主动找上门，而不是等到用户投诉时你才 --- -质量问题主动找上门，而不是等到用户投诉时你才得知。只需接入一次你自己的评分服务，Failproof AI Observability 就会自动对每一次完成的运行打分——帮助性下降或幻觉激增等问题会在用户察觉之前自动浮现。 +质量问题主动找上门，而不是等到用户投诉时你才得知。只需接入一次你自己的评分服务，FailproofAI Cloud 就会自动对每一次完成的运行打分——帮助性下降或幻觉激增等问题会在用户察觉之前自动浮现。 -![Sessions 网格中的分数列：每次运行都带有评估状态标记，以及颜色编码的帮助性、真实性和工具效率徽章](/agenteye/images/sessions-list.png) +![Sessions 网格中的分数列：每次运行都带有评估状态标记，以及颜色编码的帮助性、真实性和工具效率徽章](/cloud/images/sessions-list.png) *Sessions 网格中的每次运行都携带其评分；红色、琥珀色和绿色徽章让问题运行一眼可见，无需打开任何一条记录。* ## 告别手动抽样检查 -过去你只能抽查少数几次运行，然后祈祷其余的没有问题。现在，每一次已完成的会话在结束的那一刻就会按你关心的维度自动评分：帮助性、工具效率、真实性、安全性，以及任何你设定的质量标准。你来定义评分键；Failproof AI Observability 负责存储、追踪并展示评估器返回的所有内容。没有任何运行会漏掉评分，你也不必再从支持工单里得知回归问题。 +过去你只能抽查少数几次运行，然后祈祷其余的没有问题。现在，每一次已完成的会话在结束的那一刻就会按你关心的维度自动评分：帮助性、工具效率、真实性、安全性，以及任何你设定的质量标准。你来定义评分键；FailproofAI Cloud 负责存储、追踪并展示评估器返回的所有内容。没有任何运行会漏掉评分，你也不必再从支持工单里得知回归问题。 评分会随着会话展示在 **`//sessions`** 的 Sessions 网格上（侧边栏 → *observe* → *sessions*），每行一组徽章簇。只想查看表现不达标的运行？按分数范围筛选，比如帮助性低于 0.5，精准定位值得深入阅读的运行。查看评分需要 `evaluations:read` 权限。 @@ -20,7 +20,7 @@ description: "质量问题主动找上门，而不是等到用户投诉时你才 数字告诉你某次运行表现不佳；会话页面则告诉你原因。打开任意一次运行，右侧面板首先显示总体摘要，随后按维度展示评分条，每条下方附有评估器自身的推理说明——让你在几秒内从"真实性评分 0.4"定位到具体出错的那个论断。 -![会话的右侧面板：顶部是评估摘要，下方是各维度评分条及各条推理说明，旁边是完整的事件时间线](/agenteye/images/session-detail.png) +![会话的右侧面板：顶部是评估摘要，下方是各维度评分条及各条推理说明，旁边是完整的事件时间线](/cloud/images/session-detail.png) *会话详情视图：摘要、各维度评分条，以及每项评分背后的推理说明，与运行事件时间线并排显示。* @@ -30,7 +30,7 @@ description: "质量问题主动找上门，而不是等到用户投诉时你才 单次运行低分是噪声；整个批次下滑才是信号。已保存的仪表板将你的评分转化为可一目了然的趋势：本周与上周的平均帮助性对比，按 Agent、按环境分别呈现。 -![质量仪表板：各评估维度的平均分柱状图，以及时间趋势折线](/agenteye/images/dashboard-quality.png) +![质量仪表板：各评估维度的平均分柱状图，以及时间趋势折线](/cloud/images/dashboard-quality.png) *已保存的质量仪表板展示你关注的评分键趋势，让缓慢的下滑在演变为事故之前早早显现。* @@ -38,14 +38,14 @@ description: "质量问题主动找上门，而不是等到用户投诉时你才 ## 一次接入评估器 -评分功能为可选项，在你将 Failproof AI Observability 指向一个评分服务之前，始终保持关闭状态。你只需搭建一个小型 HTTP 服务（Observability 提供了一个可直接复制的参考实现），在服务器上设置两个值，此后每次运行都会自动获得评分。完整操作指南、评分契约和 SDK 详见深度指南。 +评分功能为可选项，在你将 FailproofAI Cloud 指向一个评分服务之前，始终保持关闭状态。你只需搭建一个小型 HTTP 服务（FailproofAI Cloud 提供了一个可直接复制的参考实现），在服务器上设置两个值，此后每次运行都会自动获得评分。完整操作指南、评分契约和 SDK 详见深度指南。 -不确定该从哪些维度开始评分？[评估器 Agent 技能](/zh/agenteye/evaluator-skill)可以让你的编码 Agent 结合你自己的会话数据找出答案，然后构建并部署该服务。 +不确定该从哪些维度开始评分？[评估器 Agent 技能](/zh/cloud/agent-skills)可以让你的编码 Agent 结合你自己的会话数据找出答案，然后构建并部署该服务。 ## 相关内容 -- [评估套件](/zh/agenteye/evaluation-suite)：接入评估器、评分契约与 SDK。 -- [评估器 Agent 技能](/zh/agenteye/evaluator-skill)：让编码 Agent 选定评分维度并构建评估器。 -- [Sessions](/zh/agenteye/sessions)：展示评分的逐次运行网格。 -- [仪表板](/zh/agenteye/dashboards)：在组织内保存并共享质量趋势。 -- [审计](/zh/agenteye/audits)：Observability 的另一项自动质量功能，用于跨会话调查。 \ No newline at end of file +- [评估套件](/zh/cloud/evaluators)：接入评估器、评分契约与 SDK。 +- [评估器 Agent 技能](/zh/cloud/agent-skills)：让编码 Agent 选定评分维度并构建评估器。 +- [Sessions](/zh/cloud/sessions)：展示评分的逐次运行网格。 +- [仪表板](/zh/cloud/dashboards)：在组织内保存并共享质量趋势。 +- [审计](/zh/cloud/audits)：FailproofAI Cloud 的另一项自动质量功能，用于跨会话调查。 \ No newline at end of file diff --git a/docs/zh/agenteye/evaluation-suite.mdx b/docs/zh/cloud/evaluators.mdx similarity index 82% rename from docs/zh/agenteye/evaluation-suite.mdx rename to docs/zh/cloud/evaluators.mdx index 8eb6f0cb..33f99a7e 100644 --- a/docs/zh/agenteye/evaluation-suite.mdx +++ b/docs/zh/cloud/evaluators.mdx @@ -1,20 +1,20 @@ --- title: "评估套件" -description: "Failproof AI Observability 可以自动对每次已完成的 Agent 运行进行质量评分：您提供一个小型评分服务，Observability 负责其余一切。" +description: "FailproofAI Cloud 可以自动对每次已完成的 Agent 运行进行质量评分：您提供一个小型评分服务，FailproofAI Cloud 负责其余一切。" --- -Failproof AI Observability 可以自动对每次已完成的 Agent 运行进行质量评分：您提供一个小型评分服务，Observability 负责其余一切。使用它来追踪您关心的维度（有用性、工具效率、事实准确性、安全性；由您决定），及早发现回归问题，并一眼比较不同 Agent 或环境的表现。评分功能为可选项：在服务器上设置 `EVALUATOR_ENDPOINT` 之前，该流水线不会执行任何操作。 +FailproofAI Cloud 可以自动对每次已完成的 Agent 运行进行质量评分：您提供一个小型评分服务，FailproofAI Cloud 负责其余一切。使用它来追踪您关心的维度（有用性、工具效率、事实准确性、安全性；由您决定），及早发现回归问题，并一眼比较不同 Agent 或环境的表现。评分功能为可选项：在服务器上设置 `EVALUATOR_ENDPOINT` 之前，该流水线不会执行任何操作。 -> **注意：** 评分维度由您自行定义。您的评估器可以返回任意数值键；Observability 会存储、趋势分析并展示您返回的所有内容。 +> **注意：** 评分维度由您自行定义。您的评估器可以返回任意数值键；FailproofAI Cloud 会存储、趋势分析并展示您返回的所有内容。 ## 概览 -1. **编写评分器。** 搭建一个小型 HTTP 服务，读取会话转录并返回评分。Observability 附带一个可直接复制使用的参考实现。请参阅[使用 SDK 编写评估器](#writing-an-evaluator-with-the-sdk)。 -2. **将 Observability 指向该服务。** 在服务器进程上设置 `EVALUATOR_ENDPOINT`（以及共享的 `EVALUATOR_TOKEN`）。 +1. **编写评分器。** 搭建一个小型 HTTP 服务，读取会话转录并返回评分。FailproofAI Cloud 附带一个可直接复制使用的参考实现。请参阅[使用 SDK 编写评估器](#writing-an-evaluator-with-the-sdk)。 +2. **将 FailproofAI Cloud 指向该服务。** 在服务器进程上设置 `EVALUATOR_ENDPOINT`（以及共享的 `EVALUATOR_TOKEN`）。 3. **查看评分结果。** 每个已完成的会话都会被自动评分；结果显示在会话详情页、会话列表和已保存的仪表盘上。 -![会话详情视图，右侧边栏显示评估摘要、各维度评分条及推理文本](/agenteye/images/session-detail.png) +![会话详情视图，右侧边栏显示评估摘要、各维度评分条及推理文本](/cloud/images/session-detail.png) *配置评估器后，每次已完成的运行都会被评分，结果出现在会话的右侧边栏：顶部为摘要，其下为带推理说明的各维度评分条。* @@ -24,7 +24,7 @@ Failproof AI Observability 可以自动对每次已完成的 Agent 运行进行 ```mermaid flowchart LR - ING["ingest /events
    agent_end"] --> SRV["Observability server"] + ING["ingest /events
    agent_end"] --> SRV["FailproofAI Cloud server"] SRV -->|"POST /evaluate"| EV["Evaluator service"] EV -->|"done or pending"| SRV SRV -->|"poll GET /evaluate/{job_id}"| EV @@ -32,14 +32,14 @@ flowchart LR SRV --> RES["evaluations
    terminal results"] ``` -当 Observability SDK 为某个会话发出 `agent_end` 事件时，服务器会调度一次评估。随后它将完整的事件转录以 POST 方式发送到您的评估器服务，评估器可以： +当 FailproofAI Cloud SDK 为某个会话发出 `agent_end` 事件时，服务器会调度一次评估。随后它将完整的事件转录以 POST 方式发送到您的评估器服务，评估器可以： - **内联返回结果**，格式为 `{"status":"done", "scores":{...}, "reasoning":{...}, "summary":"..."}`。结果将追加到该会话的评估时间线中。`reasoning` 和 `summary` 为可选字段。 -- **延迟处理**，返回 `{"status":"pending", "job_id":"abc-123"}`。Observability 随后会轮询 `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123`，直到评估器返回 `{"status":"done", ...}` 或 `{"status":"error", "error":"..."}`。 +- **延迟处理**，返回 `{"status":"pending", "job_id":"abc-123"}`。FailproofAI Cloud 随后会轮询 `GET {EVALUATOR_ENDPOINT}/evaluate/abc-123`，直到评估器返回 `{"status":"done", ...}` 或 `{"status":"error", "error":"..."}`。 - 轮询频率按任务设置：`pending` 响应中可包含 `next_poll_secs` 来覆盖默认值；否则 Observability 使用 `GET /config` 返回的 `default_poll_interval_secs`；若未设置则回退到服务器的 `EVALUATOR_POLLING_INTERVAL_SECS`（默认 10 秒）。所有值均被限制在 [1s, 1h] 范围内。 + 轮询频率按任务设置：`pending` 响应中可包含 `next_poll_secs` 来覆盖默认值；否则 FailproofAI Cloud 使用 `GET /config` 返回的 `default_poll_interval_secs`；若未设置则回退到服务器的 `EVALUATOR_POLLING_INTERVAL_SECS`（默认 10 秒）。所有值均被限制在 [1s, 1h] 范围内。 -从未发出 `agent_end` 的会话（例如 Agent 进程崩溃）也可以被处理：评估器的 `GET /config` 可返回 `{"inactivity_timeout_secs": 1800}`，Observability 将对闲置超过该时长的会话进行评估。将该字段设为 `null` 或省略可禁用此回退机制。 +从未发出 `agent_end` 的会话（例如 Agent 进程崩溃）也可以被处理：评估器的 `GET /config` 可返回 `{"inactivity_timeout_secs": 1800}`，FailproofAI Cloud 将对闲置超过该时长的会话进行评估。将该字段设为 `null` 或省略可禁用此回退机制。 当 `EVALUATOR_ENDPOINT` 未设置时，该流水线完全为空操作。 @@ -47,7 +47,7 @@ flowchart LR 闲置回退机制在已恢复的会话中同样生效：如果在上一次终态评估之后有新事件到达，且会话随后再次闲置超过 `inactivity_timeout_secs`，则会入队一次新的评估。 -暂时性失败（5xx、429、超时、网络错误）将以指数退避方式重试，最多重试 `EVALUATOR_MAX_ATTEMPTS` 次；4xx 响应为终态错误。Observability 支持多实例水平扩展运行，工作会被分区处理，确保同一会话不会被同时分发两次。 +暂时性失败（5xx、429、超时、网络错误）将以指数退避方式重试，最多重试 `EVALUATOR_MAX_ATTEMPTS` 次；4xx 响应为终态错误。FailproofAI Cloud 支持多实例水平扩展运行，工作会被分区处理，确保同一会话不会被同时分发两次。 --- @@ -55,7 +55,7 @@ flowchart LR 所有需要认证的路由均使用**Bearer Token 认证**。两端必须配置相同的值： -- Observability 服务器：环境变量 `EVALUATOR_TOKEN` +- FailproofAI Cloud 服务器：环境变量 `EVALUATOR_TOKEN` - 评估器服务：以相同方式配置（`agenteye-evaluator` SDK 按惯例读取 `EVALUATOR_TOKEN`） 如果 `EVALUATOR_TOKEN` 未设置，服务器将不发送 `Authorization` 请求头；评估器可以接受匿名请求，这在纯内部网络中是可以接受的，但不建议在公共互联网上使用。 @@ -126,7 +126,7 @@ flowchart LR 您不必手动实现 HTTP 协议规范。`agenteye-evaluator` Python 包提供了一个带类型的 FastAPI 封装，帮您处理认证、路由以及请求/响应格式。 -Failproof AI Observability 还附带了一个**可直接使用的参考评估器**，它根据转录的结构为 `helpfulness`、`tool_efficiency` 和 `factuality` 进行评分。您可以将其作为起点，替换为自己的逻辑：LLM 裁判、规则引擎，或任何适合您质量标准的方法。 +FailproofAI Cloud 还附带了一个**可直接使用的参考评估器**，它根据转录的结构为 `helpfulness`、`tool_efficiency` 和 `factuality` 进行评分。您可以将其作为起点，替换为自己的逻辑：LLM 裁判、规则引擎，或任何适合您质量标准的方法。 最小可用评估器示例： @@ -149,7 +149,7 @@ def run(req: EvalRequest) -> EvalResponse: `app` 实例可在任何 ASGI 服务器下运行，使用 `uvicorn module:app` 即可启动。 -对于需要延迟执行高开销任务的评估器，可返回 `JobPending` 并注册一个 `@app.job_lookup` 处理器；Observability 服务器会轮询 `GET /evaluate/{job_id}`，直到您返回终态状态或达到 `EVALUATOR_MAX_POLL_DURATION_SECS` 上限（默认 1 小时）。 +对于需要延迟执行高开销任务的评估器，可返回 `JobPending` 并注册一个 `@app.job_lookup` 处理器；FailproofAI Cloud 服务器会轮询 `GET /evaluate/{job_id}`，直到您返回终态状态或达到 `EVALUATOR_MAX_POLL_DURATION_SECS` 上限（默认 1 小时）。 完整的 API 参考、异步模式和事件模式请参阅 `agenteye-evaluator` SDK 的 README。 @@ -157,7 +157,7 @@ def run(req: EvalRequest) -> EvalResponse: ## 运行您的评估器 -评估器是**您自己的服务** —— Failproof AI Observability 不提供默认评估器，因此您需要在自己的服务基础设施中构建并运行它。它可在任何 ASGI 服务器下运行（例如 `uvicorn my_evaluator:app`）；按照 [HTTP 协议规范](#http-contract) 提供 `/health`、`/config` 和 `/evaluate` 路由，然后将服务器指向该地址（参见[配置服务器](#configuring-the-server)）。 +评估器是**您自己的服务** —— FailproofAI Cloud 不提供默认评估器，因此您需要在自己的服务基础设施中构建并运行它。它可在任何 ASGI 服务器下运行（例如 `uvicorn my_evaluator:app`）；按照 [HTTP 协议规范](#http-contract) 提供 `/health`、`/config` 和 `/evaluate` 路由，然后将服务器指向该地址（参见[配置服务器](#configuring-the-server)）。 评估器可访问后，`GET /health` 将返回 `{"status":"ok"}`。Agent 完整运行结束后，在服务器上执行 `GET /evaluations` 将返回一条 `status: "done"` 的记录及您的评估器产生的评分。 @@ -255,7 +255,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. - **`/sessions`**：可过滤的会话列表；评分列一眼显示每个会话的评估状态和评分。 - **`/dashboards`**：已保存的评估健康视图（参见下方[仪表盘](#dashboards)）。 -![会话列表，显示每个会话的评估状态标签和颜色编码的评分徽章（helpfulness、factuality、tool_efficiency、safety、coherence）](/agenteye/images/sessions-list.png) +![会话列表，显示每个会话的评估状态标签和颜色编码的评分徽章（helpfulness、factuality、tool_efficiency、safety、coherence）](/cloud/images/sessions-list.png) *会话列表一眼显示每次运行的评估状态和评分；红/黄/绿徽章让低评分一目了然。* @@ -272,7 +272,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. 每张卡片显示匹配会话数量、done/error/timeout 分类统计、每个重点评分的平均值，以及小型趋势迷你图。打开仪表盘可查看全尺寸面板；**"在会话中打开"**可跳转至预先过滤到该数据片段的会话页面。指标通过 `GET /evaluations/aggregate` 在服务端对整个匹配集进行精确计算，结果为精确值而非采样值。 -![评估健康仪表盘，显示每个评估维度的平均评分条、工具正常/错误分类统计、热门工具及每小时事件趋势](/agenteye/images/dashboard-quality.png) +![评估健康仪表盘，显示每个评估维度的平均评分条、工具正常/错误分类统计、热门工具及每小时事件趋势](/cloud/images/dashboard-quality.png) **权限：** 查看需要同时具备 `dashboards:read` 和 `evaluations:read`；创建和编辑需要 `dashboards:write`；删除需要 `dashboards:delete`。引导管理员会自动获得所有这些权限。 @@ -294,7 +294,7 @@ GET /evaluations?score_filters=helpfulness:0.5..,factuality:0.9.. ## 后续步骤 -- [评估器 Agent 技能](/zh/agenteye/evaluator-skill)：让编码 Agent 针对真实会话设计您的评估维度并为您构建该服务。 -- [Python SDK](/zh/agenteye/python-sdk)：发出触发评分的 `agent_end` 事件。 -- [API 密钥](/zh/agenteye/api-keys)：`evaluations:read` 和 `evaluations:trigger` 权限。 -- [审计](/zh/agenteye/audits)：Observability 的另一个自动化质量功能，用于基于策略的审查。 \ No newline at end of file +- [评估器 Agent 技能](/zh/cloud/agent-skills)：让编码 Agent 针对真实会话设计您的评估维度并为您构建该服务。 +- [Python SDK](/zh/cloud/sdk)：发出触发评分的 `agent_end` 事件。 +- [API 密钥](/zh/cloud/access)：`evaluations:read` 和 `evaluations:trigger` 权限。 +- [审计](/zh/cloud/audits)：FailproofAI Cloud 的另一个自动化质量功能，用于基于策略的审查。 \ No newline at end of file diff --git a/docs/zh/agenteye/event-stream.mdx b/docs/zh/cloud/event-stream.mdx similarity index 85% rename from docs/zh/agenteye/event-stream.mdx rename to docs/zh/cloud/event-stream.mdx index b852b4f2..ff748e80 100644 --- a/docs/zh/agenteye/event-stream.mdx +++ b/docs/zh/cloud/event-stream.mdx @@ -6,7 +6,7 @@ description: "智能体一有动作，你立刻知晓。" 智能体一有动作，你立刻知晓。事件流是你对生产环境中每个智能体的实时脉搏：无需等待，无需翻查日志，无需猜测刚刚发生了什么。 -![实时事件流：颜色编码的事件行实时滚动，可按环境、智能体、会话、事件类型和自由文本过滤](/agenteye/images/events-stream.png) +![实时事件流：颜色编码的事件行实时滚动，可按环境、智能体、会话、事件类型和自由文本过滤](/cloud/images/events-stream.png) *来自你组织中每个智能体的所有事件，最新的排在最前，实时更新。* @@ -43,8 +43,8 @@ description: "智能体一有动作，你立刻知晓。" ## 相关内容 -- [Sessions](/zh/agenteye/sessions)：相同的事件按每次运行汇总为一行，并附有 git 风格的执行图。 -- [Telemetry](/zh/agenteye/telemetry)：你的智能体发送什么内容，以及事件如何到达流。 -- [Error tracking](/zh/agenteye/error-tracking)：统一的错误排查界面，涵盖所有出错情况。 -- [Alerts](/zh/agenteye/alerts)：将任意阈值转化为告警规则。 -- [CLI and agents](/zh/agenteye/cli-and-agents)：从终端获取相同的实时追踪。 \ No newline at end of file +- [Sessions](/zh/cloud/sessions)：相同的事件按每次运行汇总为一行，并附有 git 风格的执行图。 +- [Telemetry](/zh/cloud/performance)：你的智能体发送什么内容，以及事件如何到达流。 +- [Error tracking](/zh/cloud/errors)：统一的错误排查界面，涵盖所有出错情况。 +- [Alerts](/zh/cloud/alerts)：将任意阈值转化为告警规则。 +- [CLI and agents](/zh/cloud/cli)：从终端获取相同的实时追踪。 \ No newline at end of file diff --git a/docs/zh/cloud/fleet.mdx b/docs/zh/cloud/fleet.mdx new file mode 100644 index 00000000..71ced5d6 --- /dev/null +++ b/docs/zh/cloud/fleet.mdx @@ -0,0 +1,120 @@ +--- +title: Fleet +description: "Every machine running agents in your organization, which deployment it is actually on, and which ones have no guardrails at all." +icon: server +--- + +The question a fleet view exists to answer is not "how many machines do we have?" It is +**"is the rule I wrote last Tuesday actually running everywhere it needs to?"** + +Every other way of answering that is a guess. Asking in a channel gets you replies from +the people who read channels. Checking a config in git tells you what *should* be true on +machines that pulled. The fleet page tells you what is true right now, on each host, from +the host itself. + +--- + +## What a machine reports + +Each connected machine appears with: + +| | | +|---|---| +| **Label** | The human-readable name — the hostname by default, renameable at any time. | +| **Machine id** | The stable identity everything is keyed on. Two hosts that share a hostname stay distinct. | +| **Deployment** | The numbered [policy deployment](/cloud/managed-policies) this machine has actually fetched and verified — not the one you assigned, the one it is running. | +| **Environment** | `production`, `staging`, `dev` — whatever you labelled it. | +| **Last seen** | When it last reported in. | +| **What it sends** | Decisions only, or decisions and transcripts. | + +The distinction between *assigned* and *actually running* is the whole point of the +column. A machine that has been offline since Thursday shows Thursday's deployment number, +which is exactly the fact you want in front of you before you assume a rollout landed. + +--- + +## Unguarded machines + +The most valuable row on this page is the one you did not expect to be there. + +A machine can be reporting activity without receiving policy — a key scoped to +`events:add` and not `policies:pull`, an install that was never connected for policy, a +host somebody set up before the organization had managed policy at all. Those machines are +running agents. They show up in your sessions. And they are enforcing nothing you +assigned. + +The fleet view surfaces them as unguarded rather than letting them blend into a count of +"machines reporting." That is the false reading this page exists to prevent: a healthy +looking dashboard, full of activity, from hosts your policy never reached. + +The fix is one command on the machine, with a key that carries both permissions: + +```bash +failproofai config --connect https://app.befailproof.ai --token +``` + +[Which permissions a key needs →](/cloud/connect#what-the-key-needs) + +--- + +## Machines vs. agents vs. sessions + +Three levels, easy to conflate: + +| Level | What it is | +|---|---| +| **Machine** | One host. Guardrails are installed and enforced here. | +| **Agent** | A named actor inside a run — a coding CLI, a planner, a sub-agent. Several per machine is normal. | +| **Session** | One run, from start to finish. Many per agent. | + +Grouping by machine is what makes a fleet legible: it answers coverage questions. Grouping +by agent or session is what makes an incident legible: it answers *what happened* +questions. The dashboard lets you move between them in a click — a machine's row leads to +its sessions, a session leads back to the machine that ran it. + +--- + +## Adding machines as your team grows + +Connecting is a single non-interactive command, so it belongs in whatever already +provisions your machines — an onboarding script, a Dockerfile, a configuration-management +run, a golden image: + +```bash +npm install -g failproofai +failproofai config --connect "$FAILPROOFAI_URL" \ + --token "$FAILPROOFAI_KEY" \ + --machine-id "$(cat /etc/machine-id)" \ + --machine-label "$(hostname)" +``` + +Re-running it is safe: the machine keeps its existing id rather than appearing twice. + + + Give each provisioning path its own key. Revoking one then cuts off exactly one class of + machine, instead of forcing you to re-key the whole fleet because one image leaked. + + +--- + +## Related + + + + + What a deployment is, and how to roll one out safely. + + + + The command, the permissions, and what gets sent. + + + + What those machines' agents actually did. + + + + Scoped keys, per provisioning path. + + + diff --git a/docs/zh/agenteye/incidents.mdx b/docs/zh/cloud/incidents.mdx similarity index 84% rename from docs/zh/agenteye/incidents.mdx rename to docs/zh/cloud/incidents.mdx index 7937776f..4acfc32d 100644 --- a/docs/zh/agenteye/incidents.mdx +++ b/docs/zh/cloud/incidents.mdx @@ -6,7 +6,7 @@ description: "当告警触发时，所有人都能看到事故已开启、负责 当告警触发时，第一个问题永远是"谁在处理？"事故功能给出了答案：一旦发生违规，所有人都能立即看到事故已开启、负责人是谁，以及目前已发生的一切——形成一份干净、有归属的记录，可以直接用于事后复盘。 -![事故收件箱：与告警关联的事故卡片和手动创建的事故卡片，按状态分组，每张卡片带有严重程度徽章和负责人信息](/agenteye/images/incidents.png) +![事故收件箱：与告警关联的事故卡片和手动创建的事故卡片，按状态分组，每张卡片带有严重程度徽章和负责人信息](/cloud/images/incidents.png) *收件箱按状态将未处理的事故分组，并支持按严重程度和负责人筛选，让你快速看到当前需要人工介入的内容。* ## 一眼知道谁在负责 @@ -17,10 +17,10 @@ description: "当告警触发时，所有人都能看到事故已开启、负责 事故结束时，你已经有了现成的复盘素材。打开任意事故，你会看到违规证据、负责人和订阅者、用于协调沟通的评论线程，以及一条只能追加的活动时间线。 -![事故详情视图：父告警与违规摘要、负责人和订阅者、有归属标注的活动时间线，以及评论线程](/agenteye/images/incident-detail.png) +![事故详情视图：父告警与违规摘要、负责人和订阅者、有归属标注的活动时间线，以及评论线程](/cloud/images/incident-detail.png) *所有发生过的事，按时间顺序排列，每一行都标注了操作人。* -每一个操作（开启、确认、解决等）都会写入时间线，且永远不会被编辑删除。每条记录都有归属：操作员按邮箱标注，由 Failproof AI Observability 自动执行的操作（例如在违规时自动开启事故）则标注为 **automated**。没有匿名记录，没有信息丢失，事后复盘几乎可以自动生成。 +每一个操作（开启、确认、解决等）都会写入时间线，且永远不会被编辑删除。每条记录都有归属：操作员按邮箱标注，由 FailproofAI Cloud 自动执行的操作（例如在违规时自动开启事故）则标注为 **automated**。没有匿名记录，没有信息丢失，事后复盘几乎可以自动生成。 ## 事故的流转方式 @@ -45,6 +45,6 @@ stateDiagram-v2 ## 相关内容 -- [告警](/zh/agenteye/alerts)：当阈值被突破时触发事故的规则。 -- [错误追踪](/zh/agenteye/error-tracking)：在一处查看所有故障，并将其中一个提升为告警。 -- [审计](/zh/agenteye/audits)：定期运行的分析器，用于发现没有规则在监控的故障。 \ No newline at end of file +- [告警](/zh/cloud/alerts)：当阈值被突破时触发事故的规则。 +- [错误追踪](/zh/cloud/errors)：在一处查看所有故障，并将其中一个提升为告警。 +- [审计](/zh/cloud/audits)：定期运行的分析器，用于发现没有规则在监控的故障。 \ No newline at end of file diff --git a/docs/zh/cloud/managed-policies.mdx b/docs/zh/cloud/managed-policies.mdx new file mode 100644 index 00000000..76344e75 --- /dev/null +++ b/docs/zh/cloud/managed-policies.mdx @@ -0,0 +1,182 @@ +--- +title: Managed policies +description: "Write a guardrail once, assign it, and every connected machine enforces it — with an observe-only rollout so you can see what it would block before it blocks anything." +icon: cloud-arrow-down +--- + +Committing a policy to `.failproofai/policies/` is the right answer for one repository and +a team that all works in it. It stops being the answer the moment you have twelve machines, +four repositories, and a contractor whose laptop you have never touched. + +Managed policies close that gap. You assign a policy in the dashboard; every connected +machine fetches it, verifies it, and enforces it — with no git pull, no re-install, and no +message in a channel asking everyone to please update. + +--- + +## How a deployment reaches a machine + + + + The set of policies assigned to a machine (or a group of machines) is its **desired + state**. Changing that set produces a new, numbered **deployment**. + + + Each connected machine asks what it should be running. The answer names the deployment + and every policy artifact in it, with a digest for each. + + + Artifacts are content-addressed, so a deployment that changes one policy re-downloads + one policy. A machine that has been offline catches up in a single pass. + + + Every artifact's SHA-256 is checked before the deployment goes live, **and again + immediately before each policy is loaded on the hook path**. A file that does not match + its digest is refused rather than executed — the machine keeps enforcing its previous + deployment rather than half-applying a new one. + + + +The result: a machine is always enforcing exactly one complete, verified deployment. There +is no state where half a rollout is live. + +--- + +## Roll out in observe mode first + +The risk with fleet-wide policy is not that a rule is wrong in theory. It is that a rule +that looks obviously correct turns out to block something forty engineers do all day. + +Every assignment carries an **effect**: + +| Effect | What happens on the machine | +|---|---| +| `enforce` | The verdict is acted on. A deny blocks the action. | +| `observe` | The policy is evaluated exactly as normal, then its verdict is **discarded**. Nothing is blocked; everything is recorded. | + +So the safe rollout is: + + + + Assign the policy with `observe` and let it run against real traffic. + + + The decisions land in your dashboard like any other. Filter to that policy and look at + what it would have blocked — on real work, from real people, not from a test you wrote + to confirm your own assumption. + + + Add the allowlist entry you now know you need, then switch the effect. The machines + pick up the change on their next poll. + + + + + `enforce` is the default when an assignment does not say. That is deliberate: a manifest + written before observe mode existed must not silently downgrade a machine to observation. + The default has to be the one that keeps enforcing. + + +--- + +## What a machine does when the cloud is unreachable + +It keeps enforcing the last deployment it successfully fetched. + +That is the behaviour you want in both directions. A network blip does not quietly disarm a +fleet, and a machine that has been on a plane for six hours is not stuck on a policy set +from last quarter — it catches up on its next successful poll. + +Two related guarantees worth knowing: + +- **A local [pause](/policies#pausing-enforcement) does not suspend managed policies.** + Someone can pause their own local rules for twenty minutes; they cannot pause what the + organization deployed. +- **Disconnecting actually disconnects.** `failproofai config --disconnect` clears the + active deployment as well as the credentials, so a machine that leaves your organization + stops being governed by it. Artifacts already on disk are inert and left in place, which + makes reconnecting cheap. + +--- + +## Where managed policies sit in evaluation + +They run **after** the built-ins and **before** anything local: + +1. Built-in policies +2. **Cloud-managed policies** +3. Explicit custom files +4. Convention files (project, then user) + +The first `deny` wins and short-circuits the rest, so a managed policy that denies is final +regardless of what a local file would have said. Instructions from every layer accumulate +and are delivered together. + +[Full evaluation order →](/how-it-works#step-3-policies-run-in-order) + +--- + +## What you can deploy + +Managed policies use the **same authoring API** as the ones you write locally — the same +`allow` / `deny` / `instruct` helpers, the same context object, the same event matching. A +policy that works in `.failproofai/policies/` works as a managed policy without changes. + +```js +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-prod-database-writes", + description: "Nobody's agent touches the production database, from any machine", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Bash") return allow(); + const cmd = ctx.toolInput?.command ?? ""; + if (/psql.*prod|mysql.*prod/.test(cmd)) { + return deny("Production database access is blocked. Use the read replica."); + } + return allow(); + }, +}); +``` + +[Authoring reference →](/custom-policies) + +--- + +## Local policies still work + +Managed policies add a layer; they do not take one away. Teams keep using +`.failproofai/policies/` for rules that belong to one repository, and reserve managed +policies for rules that belong to the organization. + +A useful split: + +| Rule belongs in | When | +|---|---| +| **The repo** (`.failproofai/policies/`) | It is about this codebase — its conventions, its build, its deploy process. It should travel with a branch and be reviewed in a PR. | +| **The cloud** (managed) | It is about the organization — credentials, production access, compliance. It must apply to machines whose repositories you do not control, and it must not be removable by editing a file locally. | + +--- + +## Related + + + + + Which machines are on which deployment, and which have no guardrails at all. + + + + The `policies:pull` half of a connection. + + + + The authoring API shared by local and managed policies. + + + + The 39 rules you can enable without writing anything. + + + diff --git a/docs/zh/agenteye/overview.mdx b/docs/zh/cloud/overview.mdx similarity index 50% rename from docs/zh/agenteye/overview.mdx rename to docs/zh/cloud/overview.mdx index 14357637..0ef3cad0 100644 --- a/docs/zh/agenteye/overview.mdx +++ b/docs/zh/cloud/overview.mdx @@ -1,16 +1,16 @@ --- title: "Failproof AI：监控 Agent 故障" -description: "Failproof AI Observability 是一个自托管平台，用于在生产环境中观测、评估和改进您的 AI agent。" +description: "FailproofAI Cloud 是一个自托管平台，用于在生产环境中观测、评估和改进您的 AI agent。" --- -Failproof AI Observability 是一个自托管平台，用于在生产环境中观测、评估和改进您的 AI agent。它记录 agent 的所有行为（每次工具调用、模型请求、hook 和错误），对每次运行的质量进行评分，并在您自己基础设施内运行的仪表板中呈现您未曾预料到的故障。 +FailproofAI Cloud 是一个自托管平台，用于在生产环境中观测、评估和改进您的 AI agent。它记录 agent 的所有行为（每次工具调用、模型请求、hook 和错误），对每次运行的质量进行评分，并在您自己基础设施内运行的仪表板中呈现您未曾预料到的故障。 -如果您正在部署 AI agent，并且厌倦了猜测某次运行出错的原因，这就是您的起点。本文将介绍 Failproof AI Observability 能为您提供什么，以及各模块如何协同工作——无需先安装任何东西。 +如果您正在部署 AI agent，并且厌倦了猜测某次运行出错的原因，这就是您的起点。本文将介绍 FailproofAI Cloud 能为您提供什么，以及各模块如何协同工作——无需先安装任何东西。 -> **Failproof AI Observability 是 Failproof AI 的企业级产品。** 想亲眼看看它的效果？申请演示，请发邮件至 [nikita@befailproof.ai](mailto:nikita@befailproof.ai)。 +> **FailproofAI Cloud 是 Failproof AI 的企业级产品。** 想亲眼看看它的效果？申请演示，请发邮件至 [nikita@befailproof.ai](mailto:nikita@befailproof.ai)。 -![Failproof AI Observability 会话以 git 风格的执行图呈现，旁边是事件时间线，右侧栏按运行维度展示工具、模型和 hook 的详细信息](/agenteye/images/session-detail.png) +![FailproofAI Cloud 会话以 git 风格的执行图呈现，旁边是事件时间线，右侧栏按运行维度展示工具、模型和 hook 的详细信息](/cloud/images/session-detail.png) *每次 agent 运行均以 git 风格的执行图（左）呈现，旁边配有事件时间线。并行子 agent 各占独立泳道；右侧栏按运行维度列出工具、模型、hook 及 token 消耗明细。* @@ -30,52 +30,52 @@ Failproof AI Observability 是一个自托管平台，用于在生产环境中
    -*Failproof Audit：让 Failproof AI Observability 跨会话挖掘您的日志，并告诉您需要修复的问题。* +*Failproof Audit：让 FailproofAI Cloud 跨会话挖掘您的日志，并告诉您需要修复的问题。* --- ## 团队使用它的理由 - **看清 agent 实际做了什么。** 每次运行都会生成一个可读的 git 风格执行图：哪些工具并行运行、哪些子 agent 分支启动、在哪里卡住，以及消耗了多少资源。 -- **自动捕获质量回归。** 接入一个小型评分服务后，Failproof AI Observability 会对每次完成的运行进行评分，帮助性下降或幻觉激增时会自动浮现。 +- **自动捕获质量回归。** 接入一个小型评分服务后，FailproofAI Cloud 会对每次完成的运行进行评分，帮助性下降或幻觉激增时会自动浮现。 - **发现您未曾定义规则的故障。** 周期性审计跨会话挖掘日志，查找错误聚类、延迟异常值、低分运行和卡死任务，并将按优先级排序、附有证据支撑的发现呈现给您。 - **在关键时刻收到告警。** 基于错误率、延迟、成本或评估分数的阈值规则会触发告警，生成可确认、分配和解决的事件。 - **用自然语言提问。** 仪表板内置 AI 助手，可以用中文直接询问「本周生产环境的质量趋势如何？」，基于您自己的数据作答。任何变更均需审批方可生效。 -- **数据完全归您所有。** Failproof AI Observability 采用自托管方式：事件、提示词和分析数据始终保存在您掌控的基础设施中。 +- **数据完全归您所有。** FailproofAI Cloud 采用自托管方式：事件、提示词和分析数据始终保存在您掌控的基础设施中。 --- ## 功能概览 -Failproof AI Observability 围绕三个核心理念组织：**观测（observe）**、**分析（analyze）** 和 **管理（admin）**，并在仪表板左侧边栏中一一对应。 +FailproofAI Cloud 围绕三个核心理念组织：**观测（observe）**、**分析（analyze）** 和 **管理（admin）**，并在仪表板左侧边栏中一一对应。 **观测**（运行的原始真相）： -- **[事件流](/zh/agenteye/event-stream)**：每次运行的实时逐步记录（工具调用、模型调用、hook、错误）。 -- **[会话](/zh/agenteye/sessions)**：将这些事件汇总为每次运行一行，每行均可评分，并附有 git 风格的执行图。 -- **[性能指标](/zh/agenteye/telemetry)**：按维度划分的延迟热力图，以及模型、工具和 hook 的 p50/p95/p99 关键指标，让尾部延迟从中位数中一眼凸显。 -- **[错误追踪](/zh/agenteye/error-tracking)**：所有异常的统一分类界面，一键直达触发中的告警。 +- **[事件流](/zh/cloud/event-stream)**：每次运行的实时逐步记录（工具调用、模型调用、hook、错误）。 +- **[会话](/zh/cloud/sessions)**：将这些事件汇总为每次运行一行，每行均可评分，并附有 git 风格的执行图。 +- **[性能指标](/zh/cloud/performance)**：按维度划分的延迟热力图，以及模型、工具和 hook 的 p50/p95/p99 关键指标，让尾部延迟从中位数中一眼凸显。 +- **[错误追踪](/zh/cloud/errors)**：所有异常的统一分类界面，一键直达触发中的告警。 -![工具观测页：24 个时间段内的延迟热力图、百分位带和工具分布条形图](/agenteye/images/tools.png) +![工具观测页：24 个时间段内的延迟热力图、百分位带和工具分布条形图](/cloud/images/tools.png) *每个观测界面均将 p50/p95/p99 关键指标与延迟热力图及百分位带配对展示。图中所示：工具页。* **分析**（将活动转化为洞察）： -- **[查询](/zh/agenteye/queries)** 和 **[仪表板](/zh/agenteye/dashboards)**：基于事件和评估数据的已保存 SQL 查询，以图表形式呈现在团队共享的组织级仪表板中。 -- **[评估](/zh/agenteye/evaluations)**：由您自己的评估服务产出的质量分数，附带每项评分的推理过程。 -- **[审计](/zh/agenteye/audits)**：周期性调查，跨会话发现故障模式。 -- **[告警](/zh/agenteye/alerts)** 和 **[事件](/zh/agenteye/incidents)**：触发通知的阈值规则，以及用于分类处理的事件工作流。 +- **[查询](/zh/cloud/queries)** 和 **[仪表板](/zh/cloud/dashboards)**：基于事件和评估数据的已保存 SQL 查询，以图表形式呈现在团队共享的组织级仪表板中。 +- **[评估](/zh/cloud/evaluations)**：由您自己的评估服务产出的质量分数，附带每项评分的推理过程。 +- **[审计](/zh/cloud/audits)**：周期性调查，跨会话发现故障模式。 +- **[告警](/zh/cloud/alerts)** 和 **[事件](/zh/cloud/incidents)**：触发通知的阈值规则，以及用于分类处理的事件工作流。 **接口**（以您喜欢的方式访问数据）： -- **[CLI](/zh/agenteye/cli-and-agents)**：通过终端或脚本驱动整个部署，也可以让编码 agent 用自然语言替您完成操作。 -- **[AI 助手](/zh/agenteye/assistant)**：直接在仪表板内用自然语言询问关于您 agent 的问题。 -- **REST API**：仪表板和 CLI 的所有功能均由 REST API 提供支持，您可以使用有权限范围限制的 [API 密钥](/zh/agenteye/api-keys) 直接调用——摄取事件、查询会话和评估数据、管理仪表板、告警、审计、用户和密钥，将 Failproof AI Observability 接入您自己的工具链。 +- **[CLI](/zh/cloud/cli)**：通过终端或脚本驱动整个部署，也可以让编码 agent 用自然语言替您完成操作。 +- **[AI 助手](/zh/cloud/assistant)**：直接在仪表板内用自然语言询问关于您 agent 的问题。 +- **REST API**：仪表板和 CLI 的所有功能均由 REST API 提供支持，您可以使用有权限范围限制的 [API 密钥](/zh/cloud/access) 直接调用——摄取事件、查询会话和评估数据、管理仪表板、告警、审计、用户和密钥，将 FailproofAI Cloud 接入您自己的工具链。 **管理**（为您的团队运维）： -- **[API 密钥](/zh/agenteye/api-keys)**：适用于采集器、仪表板和助手的范围化令牌。 +- **[API 密钥](/zh/cloud/access)**：适用于采集器、仪表板和助手的范围化令牌。 - **用户**：基于邮件的无密码登录，支持白名单管理。 - **设置**：组织级配置，包括模型上下文窗口覆盖项。 @@ -91,18 +91,18 @@ Failproof AI Observability 围绕三个核心理念组织：**观测（observe - **仪表板**：您浏览一切数据的地方。 - **可选服务**：评分服务（评估）和 AI 助手服务（仪表板内聊天）。 -有关文档中使用的术语（*事件、会话、评估、审计、发现、事件*），请参阅[概念](/zh/agenteye/concepts)。 +有关文档中使用的术语（*事件、会话、评估、审计、发现、事件*），请参阅[概念](/zh/concepts)。 --- -## 获取 Failproof AI Observability +## 获取 FailproofAI Cloud -Failproof AI Observability 是 Failproof AI 的企业级产品，与 Failproof AI Enforcement（策略与护栏产品）同属 Failproof AI 品牌，并可协同使用。它完全运行在您自己的环境中。如果您尚未获得软件包访问权限，请申请演示，我们将为您完成配置：发送邮件至 [nikita@befailproof.ai](mailto:nikita@befailproof.ai)。 +FailproofAI Cloud 是 Failproof AI 的企业级产品，与 FailproofAI guardrails（策略与护栏产品）同属 Failproof AI 品牌，并可协同使用。它完全运行在您自己的环境中。如果您尚未获得软件包访问权限，请申请演示，我们将为您完成配置：发送邮件至 [nikita@befailproof.ai](mailto:nikita@befailproof.ai)。 --- ## 后续步骤 -- [概念](/zh/agenteye/concepts)：Failproof AI Observability 术语的集中说明。 -- [可观测性](/zh/agenteye/observability)：逐次追踪您 agent 的行为。 -- [安全性](/zh/agenteye/security)：Failproof AI Observability 如何确保您的数据隔离并保持在您的掌控之下。 \ No newline at end of file +- [概念](/zh/concepts)：FailproofAI Cloud 术语的集中说明。 +- [可观测性](/zh/cloud/overview)：逐次追踪您 agent 的行为。 +- [安全性](/zh/cloud/security)：FailproofAI Cloud 如何确保您的数据隔离并保持在您的掌控之下。 \ No newline at end of file diff --git a/docs/zh/agenteye/telemetry.mdx b/docs/zh/cloud/performance.mdx similarity index 81% rename from docs/zh/agenteye/telemetry.mdx rename to docs/zh/cloud/performance.mdx index 11994458..119df2aa 100644 --- a/docs/zh/agenteye/telemetry.mdx +++ b/docs/zh/cloud/performance.mdx @@ -6,7 +6,7 @@ description: "即时发现模型、工具或 hook 的性能下降或费用飙升 即时发现模型、工具或 hook 的性能下降或费用飙升，在用户察觉之前捕捉尾延迟峰值。三个专属页面将原始计时数据转化为一目了然的 p50、p95 和 p99 指标。 -![模型页面展示了延迟热力图、百分位数区间，以及每个模型的 token 数量、成本和上下文窗口占用情况](/agenteye/images/models.png) +![模型页面展示了延迟热力图、百分位数区间，以及每个模型的 token 数量、成本和上下文窗口占用情况](/cloud/images/models.png) *模型页面：延迟热力图、百分位数区间，以及每个模型的 token 数量、预估成本和上下文窗口填充情况。* ## 别让平均值掩盖最糟糕的情况 @@ -24,13 +24,13 @@ description: "即时发现模型、工具或 hook 的性能下降或费用飙升 模型页面（如上图所示）直接回答账单上的两个问题：哪个模型，花了多少钱。在共享延迟视图之上，它还增加了**每模型 token 消耗量**、**预估成本**和**上下文窗口填充情况**，让提示词无节制增长和即将触发的压缩操作在发生之前就能被发现。 -Failproof AI Observability 能自动识别常见的模型 ID。如果某个窗口显示有误，或者您使用的是自有私有模型，可在 **Settings** 的 **model context windows** 中进行修正或添加，填充率读数将随之更新。 +FailproofAI Cloud 能自动识别常见的模型 ID。如果某个窗口显示有误，或者您使用的是自有私有模型，可在 **Settings** 的 **model context windows** 中进行修正或添加，填充率读数将随之更新。 ## 工具：区分慢速与故障 一次工具调用可能只是速度慢，也可能是在悄悄失败，您希望在几秒内知道是哪种情况，而不是翻遍日志之后才发现。 -![工具页面展示了共享的延迟热力图和百分位数区间，以及成功/失败分类统计和工具分布条形图](/agenteye/images/tools.png) +![工具页面展示了共享的延迟热力图和百分位数区间，以及成功/失败分类统计和工具分布条形图](/cloud/images/tools.png) *工具页面：相同的热力图和百分位数区间，加上成功/失败分类统计和工具分布条形图。* 在共享延迟视图的基础上，工具页面额外提供**成功/失败分类统计**和**工具分布条形图**，让您一眼看出哪些工具最常被调用，哪些正在侵蚀您的错误预算。 @@ -39,14 +39,14 @@ Failproof AI Observability 能自动识别常见的模型 ID。如果某个窗 当某个生命周期 hook 拖慢了运行速度，"hook 太慢了"这样的结论根本无从下手。Hook 页面能帮您直接定位到问题所在。 -![Hook 页面在共享热力图和百分位数区间之上，按 hook 名称和触发事件细分延迟数据](/agenteye/images/hooks.png) +![Hook 页面在共享热力图和百分位数区间之上，按 hook 名称和触发事件细分延迟数据](/cloud/images/hooks.png) *Hook 页面：按 hook 名称和触发事件细分的延迟数据。* 在相同的延迟热力图和百分位数区间之上，Hook 页面将活动按 **hook 名称**和**触发事件**进行细分，让您精准锁定需要关注的单个 hook 和单个事件。 ## 相关内容 -- [事件流](/zh/agenteye/event-stream)：每个事件的实时彩色追踪记录。 -- [会话](/zh/agenteye/sessions)：将事件汇总为每次运行一行，并打开其执行图。 -- [错误追踪](/zh/agenteye/error-tracking)：统一处理仪表板标红的所有问题。 -- [仪表板](/zh/agenteye/dashboards)：跨全局的汇总视图。 \ No newline at end of file +- [事件流](/zh/cloud/event-stream)：每个事件的实时彩色追踪记录。 +- [会话](/zh/cloud/sessions)：将事件汇总为每次运行一行，并打开其执行图。 +- [错误追踪](/zh/cloud/errors)：统一处理仪表板标红的所有问题。 +- [仪表板](/zh/cloud/dashboards)：跨全局的汇总视图。 \ No newline at end of file diff --git a/docs/zh/agenteye/queries.mdx b/docs/zh/cloud/queries.mdx similarity index 82% rename from docs/zh/agenteye/queries.mdx rename to docs/zh/cloud/queries.mdx index b99058fd..e0853406 100644 --- a/docs/zh/agenteye/queries.mdx +++ b/docs/zh/cloud/queries.mdx @@ -6,7 +6,7 @@ description: "向 Agent 数据提问，秒级获取答案。" 向 Agent 数据提问，秒级获取答案。Failproof AI 可观测性为您提供一个已保存、可直接运行的查询库，覆盖您的事件与评估数据，让您从现成示例出发，而无需面对空白的 SQL 编辑器。 -![已保存查询库：一个包含可复用查询的网格视图，涵盖内置预设和自定义查询](/agenteye/images/queries.png) +![已保存查询库：一个包含可复用查询的网格视图，涵盖内置预设和自定义查询](/cloud/images/queries.png) *您的已保存查询库，位于 `//queries`：内置预设与团队保存的查询并排展示。* @@ -22,7 +22,7 @@ description: "向 Agent 数据提问，秒级获取答案。" 打开任意查询，它会直接加载到 SQL 编辑器中，您可以即时调整并查看结果——无需导出，无需往返传递，无需等待他人。 -![SQL 查询编辑器正在运行一个已保存查询，左侧有 Schema 侧边栏，下方有实时结果网格](/agenteye/images/query-lab.png) +![SQL 查询编辑器正在运行一个已保存查询，左侧有 Schema 侧边栏，下方有实时结果网格](/cloud/images/query-lab.png) *SQL 编辑器：左侧是您的查询，Schema 侧边栏让您无需猜测列名，下方是实时结果网格。* @@ -43,14 +43,14 @@ agenteye query list # 在终端查看相同的已保存查询 agenteye query run errs --arg prod # 运行并打印数据行（添加 --json 可进行管道传输） ``` - 完整命令集请参阅 [CLI 与 Agents](/zh/agenteye/cli-and-agents)。 + 完整命令集请参阅 [CLI 与 Agents](/zh/cloud/cli)。 -- **从 AI 助手获取。** 不确定如何编写 SQL？用自然语言向仪表板内置的 [AI 助手](/zh/agenteye/assistant) 提问，它会为您起草查询并自动保存到查询库。 +- **从 AI 助手获取。** 不确定如何编写 SQL？用自然语言向仪表板内置的 [AI 助手](/zh/cloud/assistant) 提问，它会为您起草查询并自动保存到查询库。 运行已保存查询需要 `queries:run` 权限，该权限与创建或删除查询的权限相互独立，因此您可以授予只读访问权限，而无需允许所有人改写查询库。 ## 相关内容 -- [仪表板](/zh/agenteye/dashboards)：将查询结果固定为组织共享的图表。 -- [AI 助手](/zh/agenteye/assistant)：用自然语言提问，获取对应查询。 -- [CLI 与 Agents](/zh/agenteye/cli-and-agents)：从终端运行和保存相同的查询。 \ No newline at end of file +- [仪表板](/zh/cloud/dashboards)：将查询结果固定为组织共享的图表。 +- [AI 助手](/zh/cloud/assistant)：用自然语言提问，获取对应查询。 +- [CLI 与 Agents](/zh/cloud/cli)：从终端运行和保存相同的查询。 \ No newline at end of file diff --git a/docs/zh/agenteye/python-sdk.mdx b/docs/zh/cloud/sdk.mdx similarity index 96% rename from docs/zh/agenteye/python-sdk.mdx rename to docs/zh/cloud/sdk.mdx index 754fb43a..8e25cfe1 100644 --- a/docs/zh/agenteye/python-sdk.mdx +++ b/docs/zh/cloud/sdk.mdx @@ -26,7 +26,7 @@ SDK 以私有 wheel 包的形式分发给客户，而非通过公共包索引。 python -c "import agenteye; print(agenteye.__version__)" ``` -希望让编码智能体完成整个集成工作？[Python SDK Agent Skill](/zh/agenteye/python-sdk-skill) 了解安装路径，能够规划插桩点、编写代码并验证事件是否正确落地。 +希望让编码智能体完成整个集成工作？[Python SDK Agent Skill](/zh/cloud/agent-skills) 了解安装路径，能够规划插桩点、编写代码并验证事件是否正确落地。 --- @@ -99,7 +99,7 @@ agenteye.event.model_response( 以下是这些事件到达仪表板后的样式，按类型用颜色区分，并支持按环境、智能体和会话进行筛选： -![实时事件流，按事件类型颜色编码，可按环境、智能体和会话筛选](/agenteye/images/events-stream.png) +![实时事件流，按事件类型颜色编码，可按环境、智能体和会话筛选](/cloud/images/events-stream.png) --- @@ -155,7 +155,7 @@ SDK 仅记录您显式传递的字段。提示词、消息、工具输入输出 大多数事件以共享关联 ID 的开始/结束对形式出现：`tool_use` 和 `tool_result` 共享一个 `tool_call_id`，`hook_triggered` 和 `hook_completed` 共享一个 `hook_id`，`human_wait` 和 `human_input` 共享一个 `input_id`。发送开始事件，执行工作，然后使用相同 ID 发送结束事件。Failproof AI 可观测性会自动匹配这一对事件并为您计算 `duration_ms`，因此您无需自行传递 `duration_ms`。 -![会话的 git 风格执行图及其事件时间线（由配对事件重建），以及工具/模型/钩子分解面板](/agenteye/images/session-detail.png) +![会话的 git 风格执行图及其事件时间线（由配对事件重建），以及工具/模型/钩子分解面板](/cloud/images/session-detail.png) 所有事件方法均需要以下两个字段： @@ -429,5 +429,5 @@ agenteye.event.tool_use( ## 后续步骤 -- [事件流](/zh/agenteye/event-stream)：实时查看这些事件，按事件类型颜色编码，可按环境、智能体和会话进行筛选。 -- [会话](/zh/agenteye/sessions)：了解配对事件如何将每次智能体运行重建为执行图和时间线。 \ No newline at end of file +- [事件流](/zh/cloud/event-stream)：实时查看这些事件，按事件类型颜色编码，可按环境、智能体和会话进行筛选。 +- [会话](/zh/cloud/sessions)：了解配对事件如何将每次智能体运行重建为执行图和时间线。 \ No newline at end of file diff --git a/docs/zh/cloud/security.mdx b/docs/zh/cloud/security.mdx new file mode 100644 index 00000000..4f29635b --- /dev/null +++ b/docs/zh/cloud/security.mdx @@ -0,0 +1,68 @@ +--- +title: "安全性" +description: "FailproofAI Cloud 被设计为紧邻您的生产环境 Agent 运行，这意味着它能看到您的提示词、工具输入和输出内容。" +--- + + +FailproofAI Cloud 被设计为紧邻您的生产环境 Agent 运行，这意味着它能看到您的提示词、工具输入和输出内容。本页说明它如何确保数据隔离、受控，并始终掌握在您手中。如果您正在对 FailproofAI Cloud 进行安全审查评估，请从这里开始。 + +--- + +## 您的数据保留在您的环境中 + +FailproofAI Cloud 采用自托管模式。事件、提示词、模型响应和分析数据均存储在您自己的数据库和环境中。数据不会被发送至任何第三方 SaaS 平台存储，始终保留在您自己的云账户内。 + +--- + +## 租户隔离 + +一个 FailproofAI Cloud 实例可以托管多个组织，每个组织在存储层面相互隔离——这由数据库强制执行，而不仅仅依赖 UI 层面的限制： + +- 组织的运营数据（用户、密钥、仪表盘、已保存查询）仅限于该组织访问，跨组织读取由数据库本身拦截阻止。 +- 每个采集的事件都标记了所属组织，因此一个组织的事件永远无法被另一个组织读取。 + +每个仪表盘路由都以组织 slug 为前缀（`//…`）。 + +--- + +## 登录方式 + +FailproofAI Cloud 采用无密码、基于邮件的登录方式，不存在可被钓鱼或泄露的密码。用户申请一次性验证码（或一键魔法链接），系统将其发送至用户邮箱，且会在短时间内过期。登录受**白名单**限制：只有您允许的邮箱地址（或域名）才能完成认证。 + +![FailproofAI Cloud 登录界面，将一次性验证码发送至您的邮箱](/cloud/images/login.png) + +--- + +## 通过 API 密钥实现精细化访问控制 + +每个客户端均使用携带精细化最小权限的 API 密钥进行认证。数据采集器只需 `events:add` 权限；仪表盘或助手密钥可设为只读；破坏性操作（删除、重新生成）作为独立权限授予，由您自行决定是否开放。 + +![API 密钥页面：每个密钥的权限授予情况，按读取、写入和破坏性范围用颜色区分](/cloud/images/api-keys.png) + +保留管理员引导密钥用于初始配置，其余场景均应颁发权限受限的密钥。详见 [API 密钥](/zh/cloud/access)。 + +--- + +## 只读、需审批的 AI 助手 + +仪表盘内的 [AI 助手](/zh/cloud/assistant) 可基于您的数据回答问题，但在设计上受到严格约束： + +- **默认只读**：其执行的 SQL 经过守卫过滤，仅允许 `SELECT`/`WITH` 查询，单条语句执行，并设有行数上限。 +- 它创建的任何内容（已保存查询、仪表盘）均需**审批才能生效**：每一次写入操作发生前，您都需要审查并确认。 +- **它永远无法执行删除操作**。 + +因此，团队成员可以询问"本周哪些 Agent 报错最多？"并基于答案采取行动，而无需担心助手会自行修改或删除您的数据。 + +--- + +## 传输安全 + +所有流量均通过 HTTPS 传输。您使用自己的证书终止 TLS，确保采集器到服务器以及浏览器到服务器的流量在传输过程中全程加密。 + +--- + +## 后续步骤 + +- [概览](/zh/cloud/overview)：了解 FailproofAI Cloud 的整体架构。 +- [API 密钥](/zh/cloud/access)：为采集器、仪表盘和助手配置访问权限。 +- [可观测性](/zh/cloud/overview)：了解 FailproofAI Cloud 从您的 Agent 中采集的数据内容。 \ No newline at end of file diff --git a/docs/zh/agenteye/sessions.mdx b/docs/zh/cloud/sessions.mdx similarity index 79% rename from docs/zh/agenteye/sessions.mdx rename to docs/zh/cloud/sessions.mdx index 28d838e0..15e55727 100644 --- a/docs/zh/agenteye/sessions.mdx +++ b/docs/zh/cloud/sessions.mdx @@ -5,7 +5,7 @@ description: "将一次运行的所有事件汇总为一行可读记录，并以 不再猜测运行失败的原因。Failproof AI 可观测性将一次运行的所有事件汇总为一行可读记录，再将整个运行过程绘制成 git 风格的图示，让你几秒内看清全貌，逐步了解智能体究竟做了什么。 -![会话列表：每次运行占一行，跨越多个环境和智能体，附带状态标签和评估分数徽章](/agenteye/images/sessions-list.png) +![会话列表：每次运行占一行，跨越多个环境和智能体，附带状态标签和评估分数徽章](/cloud/images/sessions-list.png) *每次运行占一行：状态标签让你一眼看出运行结果，连接评估器后还会显示分数徽章。* @@ -29,7 +29,7 @@ description: "将一次运行的所有事件汇总为一行可读记录，并以 ## 以图示读懂整个运行过程 -![会话的 git 风格执行图与事件时间线并排显示，右侧面板展示工具、模型和 hook 的详细拆解](/agenteye/images/session-detail.png) +![会话的 git 风格执行图与事件时间线并排显示，右侧面板展示工具、模型和 hook 的详细拆解](/cloud/images/session-detail.png) *执行图（左侧）与事件时间线并排显示；右侧栏对本次运行使用的工具、模型、hook 以及 token 消耗进行详细拆解。* @@ -37,7 +37,7 @@ description: "将一次运行的所有事件汇总为一行可读记录，并以 右侧栏提供逐次运行的详细拆解：哪些工具和模型参与了运行、哪些 hook 触发了、以及本次运行消耗了多少 token。「这次运行为什么这么贵？」或「哪个工具最慢？」的答案就在执行图旁边。 -每个单独事件都有固定链接，因此你可以把某一时刻的链接直接分享给他人，而不是说「在那个会话里，大概三分之二的位置」。从任意事件复制链接，或从[审计](/zh/agenteye/audits)发现或错误中跳转，会话将打开并定位到该事件。对于非常长的运行同样适用：时间线出于浏览器性能考虑只加载有限的时间窗口，但指向窗口之外的链接仍然能定位到对应事件，而不是把你扔到最开始。如果该事件已超出你的数据保留窗口，页面会明确提示，而不是静默地选中空白内容。 +每个单独事件都有固定链接，因此你可以把某一时刻的链接直接分享给他人，而不是说「在那个会话里，大概三分之二的位置」。从任意事件复制链接，或从[审计](/zh/cloud/audits)发现或错误中跳转，会话将打开并定位到该事件。对于非常长的运行同样适用：时间线出于浏览器性能考虑只加载有限的时间窗口，但指向窗口之外的链接仍然能定位到对应事件，而不是把你扔到最开始。如果该事件已超出你的数据保留窗口，页面会明确提示，而不是静默地选中空白内容。 --- @@ -45,12 +45,12 @@ description: "将一次运行的所有事件汇总为一行可读记录，并以 每个控制台页面都限定在你的组织范围内（`//…`）。会话功能位于左侧边栏的 **Observe** 下，紧邻 Events，列表顶部提供日期范围、环境、智能体和会话等筛选条件。每行点击一次即可进入完整执行图。 -要开启分数徽章和按分数范围筛选的功能，请连接评估器，详见[评估](/zh/agenteye/evaluations)。 +要开启分数徽章和按分数范围筛选的功能，请连接评估器，详见[评估](/zh/cloud/evaluations)。 --- ## 相关内容 -- [事件流](/zh/agenteye/event-stream)：每个会话汇总自原始的逐步事件记录。 -- [评估](/zh/agenteye/evaluations)：连接评估器，让每次运行都获得可供筛选的分数徽章。 -- [遥测](/zh/agenteye/telemetry)：了解运行数据如何从你的智能体传入这些会话。 \ No newline at end of file +- [事件流](/zh/cloud/event-stream)：每个会话汇总自原始的逐步事件记录。 +- [评估](/zh/cloud/evaluations)：连接评估器，让每次运行都获得可供筛选的分数徽章。 +- [遥测](/zh/cloud/performance)：了解运行数据如何从你的智能体传入这些会话。 \ No newline at end of file diff --git a/docs/zh/concepts.mdx b/docs/zh/concepts.mdx new file mode 100644 index 00000000..24d965b3 --- /dev/null +++ b/docs/zh/concepts.mdx @@ -0,0 +1,196 @@ +--- +title: Concepts +description: "Every term these docs use — policy, decision, session, machine, deployment, finding, incident — defined once, in one place." +icon: book +--- + +You don't need to read this page end to end. Skim it once, then come back when a word in +another guide isn't pinned down. + +--- + +## Guardrails + +**Policy** +One rule, evaluated against one agent action. A policy has a name, the events it listens +to, and a function that returns a decision. Policies come from four places — [built +in](/built-in-policies), [written by you](/custom-policies), dropped into a +`.failproofai/policies/` directory by convention, or [deployed from the +cloud](/cloud/managed-policies). + +**Decision** +What a policy returns: **allow** (proceed), **deny** (block the action and tell the agent +why), or **instruct** (let it proceed, and add context to keep it on track). `allow` can +carry a message too — useful for confirming a check passed rather than staying silent. + +**Hook event** +The moment a policy runs. `PreToolUse` (before a tool call), `PostToolUse` (after it), +`UserPromptSubmit`, `Stop` (the agent is about to finish its turn), `SubagentStop`, +`SessionStart`, `SessionEnd`, `Notification`, `PreCompact`. Not every agent CLI fires +every event — see [the support matrix](/agent-support). + +**Agent CLI (harness)** +One of the 12 coding agents FailproofAI hooks into: Claude Code, OpenAI Codex, GitHub +Copilot CLI, Cursor Agent, OpenCode, Pi, Hermes, OpenClaw, Factory Droid, Devin CLI, +Antigravity CLI, and Goose. "Harness" is the word used where the distinction matters — +for example [`failproofai harness add-path`](/cli/harness). + +**Scope** +Where a piece of configuration lives: **project** (`.failproofai/`, committed), **local** +(`.failproofai/*.local.json`, gitignored), or **global** (`~/.failproofai/`). Policies +merge across all three; see [Configuration](/configuration#merge-rules). + +**Preset** +A themed bundle of built-in policies the setup wizard offers — *Secrets & data*, *Git +safety*, *Ship discipline*, *Cloud & infra*. Presets are additive: tick several and you +get the union. + +**Convention policy** +A policy file discovered automatically because of where it sits, with no configuration at +all. Any file matching `*policies.{js,mjs,ts}` in `.failproofai/policies/` (project) or +`~/.failproofai/policies/` (user) is loaded on the next hook event. + +**Pause** +A time-boxed suspension of local enforcement for **one session**. Always expires on its +own — 30 minutes by default, 8 hours maximum, never unbounded. Cloud-managed policies keep +enforcing through a pause, and agents cannot pause on their own behalf while +`block-self-pause` is on. See [`failproofai config --pause`](/cli/config#pausing-enforcement). + +**Fail closed** +The property that a guardrail which cannot answer denies rather than allows. On a +configured machine, that is what makes stopping the service a way to stop working, not a +way to work unguarded. See [the daemon](/daemon#fail-closed). + +--- + +## What runs on a machine + +**`failproofai`** +The CLI. Runs setup, installs and lists policies, launches the local dashboard, runs the +audit, and connects the machine to the cloud. + +**`failproofaid`** +The background service that evaluates policy on a configured machine, collects what your +agents did, and exchanges it with the cloud. Installed by setup as a system service that +starts at boot and survives logout. See [the daemon](/daemon). + +**Machine** +One host, identified to the cloud by a stable **machine id** and shown under a +human-readable **machine label** (the hostname, by default). The id is what your fleet +history is keyed on; the label is only for reading. Two hosts that happen to share a +hostname stay distinct. + +**Environment** +A label for what a machine or run belongs to: `production`, `staging`, `dev`, `local`. +Set once, attached to everything, and available as a filter almost everywhere in the cloud +dashboard. + +**Deployment** +A numbered, immutable snapshot of the policy set assigned to a machine. The daemon fetches +a deployment, verifies each artifact's digest, and switches to it atomically. `--status` +and the cloud dashboard both report which deployment a machine is actually on — which is +how you tell "rolled out" from "rolled out everywhere." + +**Effect (`enforce` / `observe`)** +Whether a cloud-managed policy's verdict is acted on or recorded and discarded. `observe` +lets you measure a new rule against real traffic before it can block anyone. + +--- + +## What gets recorded + +**Hook activity** +The local decision log: one entry per non-allow decision, with the policy, the tool, the +session, the reason, and how long it took. Read by the local dashboard, and shipped to the +cloud on a connected machine. + +**Transcript** +The agent CLI's own record of a session, in its own format, in its own location. +FailproofAI reads transcripts; it never writes to them. They contain prompts, file +contents, and command output — which is why sending them to the cloud is an explicit, +disclosed choice. + +**Session** +One agent run, identified by a `session_id`. In the cloud, a session is every event +sharing that id, rolled into one row and drawn as an execution graph. + +**Event** +The smallest unit of recorded data: one step an agent took. `tool_use`, `tool_result`, +`model_request`, `model_response`, `hook_triggered`, `hook_completed`, `error`, +`agent_start`, `agent_end`, and the human-in-the-loop events. + +**Agent** +A named actor inside a run, identified by an `agent_id`. One run can involve several — a +planner that spawns a summarizer, for example. Sub-agents carry a `parent_id`, which is +what puts them on their own lane in the execution graph. + +**Context-window fill** +How much of a model's context window a response consumed, stamped on `model_response` +events for recognized models. Makes prompt growth and an approaching compaction visible +before they bite. + +--- + +## Quality and operations, in the cloud + +**Evaluation** +A quality score for a finished run, produced by a scoring service **you** run. Opt-in: +until you connect one, runs are recorded but not scored. Each evaluation can carry several +named scores, each with a line of reasoning. + +**Score key** +The name of one dimension your evaluator reports — `helpfulness`, `factuality`, +`tool_efficiency`, whatever your quality bar is. You define them; the cloud stores, trends, +and displays whatever you send. + +**Evaluator** +Your scoring service. The cloud POSTs a finished run's transcript to it and stores what +comes back. FailproofAI ships no default evaluator — the scoring logic is yours. See +[Evaluators](/cloud/evaluators). + +**Saved query** +A named, shared SQL query over your events and evaluations. Read-only by construction — +only `SELECT` and `WITH`, with a statement timeout and a row cap. + +**Dashboard (cloud)** +A shared, org-wide board built from saved queries rendered as charts. Not to be confused +with the [local dashboard](/dashboard), which runs on your own machine. + +**Alert rule** +A rule that fires when something crosses a threshold you set — error rate, p95 latency, +token spend, an evaluator score, a custom SQL result, or a single matching event. When it +fires it opens an incident and notifies your channels. + +**Incident** +An open issue created when an alert fires, with a lifecycle (acknowledge → assign → +resolve) and an append-only, attributed activity timeline. One alert holds at most one open +incident at a time, so a flapping rule cannot bury you. + +**Audit (cloud)** +A recurring investigation that mines your sessions *across* runs for failure patterns +nobody wrote a rule for: error clusters, drift, goal failures, tool misuse, coverage gaps. +Where an alert watches something you already know about, an audit tells you what to look at +next. + +**Finding** +One ranked, evidence-backed result from an audit run. Names a pattern, links the exact +sessions and events behind it, and carries its own triage lifecycle. + +**Organization** +Your isolated workspace in the cloud. Users, keys, machines, policies, and data all belong +to exactly one. Every dashboard URL is scoped under its slug (`//…`). + +**API key** +A scoped token that authenticates a client. Keys carry granular permissions — `events:add` +for a machine that only reports, `policies:pull` for one that only receives policy, +read-only scopes for a dashboard integration. See [Access and permissions](/cloud/access). + +--- + + + Two things share the word **audit**, and they are different features. The [local + audit](/audit) replays the transcripts already on your machine through the policy engine + and scores your agent's habits. The [cloud audit](/cloud/audits) is a scheduled + investigation across your organization's sessions that produces ranked findings. The + local one needs no account; the cloud one needs a connected fleet. + diff --git a/docs/zh/daemon.mdx b/docs/zh/daemon.mdx new file mode 100644 index 00000000..3f36b954 --- /dev/null +++ b/docs/zh/daemon.mdx @@ -0,0 +1,267 @@ +--- +title: The failproofaid service +description: "The background service that makes enforcement fail closed, keeps evaluation fast, and connects a machine to your fleet." +icon: server +--- + +`failproofaid` is the background service FailproofAI installs during setup. It does three +jobs, and each one is the answer to a way guardrails fail quietly in the real world. + + + + + Every hook event on a configured machine is answered by the service — from a process + that is already warm, so nobody pays a cold start on a tool call. + + + + If the service cannot answer, the tool call is **denied**. Stopping it is a way to stop + working, not a way to work unguarded. + + + + Pulls your organization's policy down, ships what your agents did up, and keeps both + working across restarts and outages. + + + + +--- + +## Fail closed + +This is the property everything else on this page exists to protect. + +On a machine that completed setup, **`failproofaid` is the only evaluator**. Every way of +not getting an answer denies: + +| Situation | Result | +|---|---| +| The service is not running | Tool call denied | +| The socket is unreachable | Tool call denied | +| The service and the CLI disagree on the protocol version | Tool call denied, with a message naming the version and pointing at `failproofai config` | + +There is deliberately **no in-process fallback** on this path. A second policy engine you +can reach by stopping the first is not a guarantee, and a machine where killing one service +silently disables every guardrail is not a guarded machine. + +The version-mismatch case gets its own message because the remedy is different from "the +service is down," and telling those two apart is the whole value of distinguishing them. +The cost is real and worth stating: the first time the protocol changes, a machine whose +CLI updated before its service did will deny until `failproofai config` runs. Both halves +ship from the same release and every CLI command warns when it detects the skew, so the +window is short and announces itself. + +### The two situations that do *not* use the service + +In-process evaluation still exists, and is reachable only when a machine was never +configured for the daemon: + +1. **A machine that has not been set up.** No hooks are installed either, so nothing is + evaluating anything. +2. **The FailproofAI repository's own development configs.** Contributors run the engine + in-process against the package they are editing — a flaky in-development service must + not block the tool calls of the people developing it. + +Neither is a configured user machine. + +--- + +## Platform support + +`failproofaid` runs on **Linux and macOS**. + +On anything else — Windows, today — `failproofai config` **refuses to run**. It prints +why and exits before drawing a single prompt: no hooks installed, no partial state, no +machine that reads as configured while enforcing something weaker than every other +configured machine. + +That is a deliberate change from earlier behaviour, which skipped the service requirement +and let setup complete anyway. Refusing is the more honest failure: it says plainly that +the platform is not supported yet, instead of shipping a quieter guarantee under the same +name. + +--- + +## How it is supervised + +The service is **system-scope, user-run**: + +| Platform | What is installed | +|---|---| +| Linux | `/etc/systemd/system/failproofaid@.service`, with `User=` and `WantedBy=multi-user.target` | +| macOS | A `LaunchDaemon` plist in `/Library/LaunchDaemons` with `UserName` set | + +It starts at boot, needs no login, and survives logout. + +That last property is why it is a system service rather than a per-user one. A user-level +service does not start at boot without extra configuration and stops with the last login +session — so the daemon died on logout, and because a configured machine **fails closed**, +anything running without a login session (a detached tmux, a cron job, a CI runner) then +hit denials. + +Three consequences follow, each handled explicitly: + +- **Installing needs root.** Setup checks `sudo -n` *before* writing anything. If it + cannot elevate, it writes nothing and hands you the exact commands to run. Never an + interactive password prompt — one fired from underneath a full-screen wizard is + unreadable. +- **A system service has no login environment.** The service is pointed at the exact Node + binary that ran setup, not a bare `node`. The most common Node install puts its binary + on no system PATH at all, which would resolve fine while you watch and then fail + silently inside the service. +- **Any older user-scope service is removed first**, on every install and uninstall. It + holds the same lock the new one needs, so leaving one behind means the new service + starts, loses the race, and the machine sits failing closed against a daemon that never + came up. + +Checking on it needs no privileges: + +```bash +systemctl status failproofaid@$USER # Linux +failproofai config --status # either platform — connection, service, pause state +``` + +Install waits for the service to reach **and hold** a running state before reporting +success. A service that reports "active" the instant it forks would otherwise pass a check +even if it died at startup. + +--- + +## How the binary reaches your machine + +The npm package carries no binary — one package serves every platform — so the binary +arrives through one of two channels, tried in this order: + + + + Platform-specific packages are published alongside the CLI, so `npm install failproofai` + already downloaded the one matching your machine and skipped the others. Installing + from it involves **no network at all**, which makes it the channel that works + air-gapped or behind a proxy that blocks GitHub. + + + A compressed binary plus a checksum manifest, fetched for this CLI's exact version and + **SHA-256 verified before it is decompressed**. This covers installs that skipped + optional dependencies, packages installed from disk, and standalone service installs. + + The URL is *constructed* from the installed version, never discovered. No API call, no + "latest" redirect, no rate limit — and no way to end up running a service built from + different source than the CLI talking to it. + + + +Both land the file in `~/.failproofai/bin/`, under a versioned filename. The service is +never pointed into `node_modules`: a global package upgrade would otherwise swap the file +under a running service, and uninstalling the package would delete it out from under a +service that then crash-loops at every boot. + +Two escape hatches: + +| Variable | Effect | +|---|---| +| `FAILPROOFAI_NO_DOWNLOAD=1` | Never reach out to fetch a binary; fail with a reason instead. An already-installed binary keeps working, and the npm channel is unaffected — this gates *fetching*, not copying. | +| `FAILPROOFAI_DAEMON_BASE_URL` | Point the download at an internal mirror. | + +Only the install path does any of this. The hook path is a pure disk check, so it can +never block on the network. + +--- + +## Upgrading + +```bash +npm install -g failproofai@latest +failproofai update +``` + +`failproofai update` finishes what npm cannot: it migrates `~/.failproofai` to the new +layout if the layout changed, puts the matching service binary in place, and restarts the +service. + +**Your configuration is carried across, not reset:** + +| Kept | Rebuilt | +|---|---| +| Your policy selection and parameters | The audit cache | +| Your machine settings, including extra capture paths | Cloud-managed deployments — re-fetched and digest-verified on the next poll | +| Your cloud connection | Service scratch state | +| Your own policy files, and the helpers they import | | +| The decision log, and anything not yet delivered to the cloud | | + +Settings written by a *newer* version are preserved rather than dropped by an older +reader, so moving between versions does not silently discard anything in either direction. +Every migration is recorded, and the irreplaceable files are copied to a backup directory +before anything runs. + +You do **not** need to re-run setup after an upgrade. A migrated machine enforces exactly +as it did before — which is what makes upgrading safe on machines with nobody sitting at +them. + +See [`failproofai update`](/cli/update) and [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## What it does for a connected machine + +On a machine [connected to FailproofAI Cloud](/cloud/connect), the same service handles +both directions of traffic: + +- **Policy down.** Polls for this machine's desired state, downloads any policy artifact it + does not already have, verifies each one's digest, and switches deployments atomically. A + machine that loses its network keeps enforcing the last deployment it successfully + fetched. +- **Activity up.** Reads the local decision log and — unless you connected with + `--no-transcripts` — your agent CLIs' session transcripts, spools them to disk, and + uploads in batches. If delivery fails, the spool is retained and retried; nothing is + dropped because the network blinked. + +```bash +failproofai flush --wait # deliver everything spooled, now +failproofai backfill --since 6m # re-read history the collector already passed +``` + +--- + +## Uninstalling + +```bash +failproofai uninstall +``` + +Removes the hook entries from every agent CLI **and** the service. Add `--purge` to also +delete `~/.failproofai` (settings, credentials, audit history, and the service binary). + +Uninstall clears the daemon-configured flag **first and unconditionally**. Leaving that +flag set with no service to reach would deny every hook event on the machine, across all 12 +CLIs, recoverable only by hand-editing a config file. + + + Run `failproofai uninstall` **before** `npm rm -g failproofai`. npm runs no uninstall + script, so removing the package on its own leaves both the hook entries and the service + behind. + + +--- + +## Related + + + + + The full path from a tool call to a decision. + + + + What the service sends, and what it receives. + + + + Setup, status, connect, disconnect, pause. + + + + Every variable, including the download escape hatches. + + + diff --git a/docs/zh/dashboard.mdx b/docs/zh/dashboard.mdx index 54438c6c..697ff7dc 100644 --- a/docs/zh/dashboard.mdx +++ b/docs/zh/dashboard.mdx @@ -69,7 +69,7 @@ Hermes 和 OpenClaw 是用户范围的，没有可用于分组的工作目录， 4. **改进建议** — 平和的列表行，每个推荐策略一行：左侧为白色策略名称和一行描述，右侧为安装命令和复制按钮。区块标题显示 `enable all N → projected · `（应用所有修复后可达到的评分），其 `[install all]` 按钮会复制针对所有推荐策略的组合 `failproofai policy add a b c …` 命令。 5. **下次更好** — 两张并排卡片。左侧：设置提醒（`3d` / `7d` / `14d` / `30d` 周期选择器；认证后通过 `/api/auth/reminder` 持久化）。右侧：解锁 failproof 特权——`invite a friend` 打开一个模态框，接受逗号/空格/换行分隔的好友邮件列表（每次最多 10 个），通过 `/api/audit/invite` 发送 POST 请求，转发至 api-server 的 `POST /v0/invite`。api-server 从 `invite@failproof.ai` 向每位收件人发送邮件，同时将发件人抄送，并设置 `Reply-To`，以便收件人知道是谁邀请了他们，发件人也会在收件箱中收到一份副本。匿名用户会先通过 `AuthDialog` 进行路由，以便在发送邀请前获知发件人的邮件地址。权限/特权兑现为后续功能。 -由 `failproofai audit` 运行时驱动——请参阅 [审计 CLI](/zh/cli/audit) 了解底层扫描引擎、支持的标志和每个记录的缓存不变量。控制台将最新结果缓存在 `~/.failproofai/audit-dashboard.json`（模式 `0600`，单槽，新运行覆盖），以便再次访问时立即加载；**每个记录的缓存和整体结果缓存在读取时若超过 7 天则被拒绝**，因此控制台不会静默返回一周前的结果——超过 TTL 后 `/audit` 会回落到空状态并提示重新运行。点击报告底部附近的 `[ re-audit now ]` 会以 `noCache: true` 向 `/api/audit/run` 发送 POST 请求——重新审计会绕过每个记录的缓存，从头重新扫描每份记录，而不是静默返回缓存结果——控制台以 1Hz 轮询 `/api/audit/status` 直到运行完成；运行期间，一条粉色进度条固定在视口顶部并显示已用时间，完成后新结果会原地替换（无需整页刷新；重新审计失败时保留之前的报告）。失败时进度条变为红色，并根据 `RerunError.kind`（`timeout` / `network` / `post_failed`）显示对应文案。空状态（无缓存或已过期）和零会话状态（缓存存在但扫描未发现任何记录）会分别显示。 +由 `failproofai audit` 运行时驱动——请参阅 [审计 CLI](/zh/audit) 了解底层扫描引擎、支持的标志和每个记录的缓存不变量。控制台将最新结果缓存在 `~/.failproofai/audit-dashboard.json`（模式 `0600`，单槽，新运行覆盖），以便再次访问时立即加载；**每个记录的缓存和整体结果缓存在读取时若超过 7 天则被拒绝**，因此控制台不会静默返回一周前的结果——超过 TTL 后 `/audit` 会回落到空状态并提示重新运行。点击报告底部附近的 `[ re-audit now ]` 会以 `noCache: true` 向 `/api/audit/run` 发送 POST 请求——重新审计会绕过每个记录的缓存，从头重新扫描每份记录，而不是静默返回缓存结果——控制台以 1Hz 轮询 `/api/audit/status` 直到运行完成；运行期间，一条粉色进度条固定在视口顶部并显示已用时间，完成后新结果会原地替换（无需整页刷新；重新审计失败时保留之前的报告）。失败时进度条变为红色，并根据 `RerunError.kind`（`timeout` / `network` / `post_failed`）显示对应文案。空状态（无缓存或已过期）和零会话状态（缓存存在但扫描未发现任何记录）会分别显示。 ### 策略 diff --git a/docs/zh/architecture.mdx b/docs/zh/how-it-works.mdx similarity index 100% rename from docs/zh/architecture.mdx rename to docs/zh/how-it-works.mdx diff --git a/docs/zh/introduction.mdx b/docs/zh/introduction.mdx index 4cc7b60a..fc4c505c 100644 --- a/docs/zh/introduction.mdx +++ b/docs/zh/introduction.mdx @@ -54,4 +54,4 @@ failproofai policies --install # enable policies (or skip — `failproofai` wi failproofai # launch the dashboard ``` -完整流程请参阅[快速入门](/zh/getting-started)指南。 \ No newline at end of file +完整流程请参阅[快速入门](/zh/quickstart)指南。 \ No newline at end of file diff --git a/docs/zh/policies.mdx b/docs/zh/policies.mdx new file mode 100644 index 00000000..41c03bf4 --- /dev/null +++ b/docs/zh/policies.mdx @@ -0,0 +1,267 @@ +--- +title: Policies +description: "What a policy is, where policies come from, the order they run in, and how to turn them on, tune them, and switch them off." +icon: shield-halved +--- + +A policy is one rule, evaluated against one thing an agent is about to do. It is the unit +of everything FailproofAI enforces — the 39 built-in rules, the ones you write, and the +ones your organization deploys from the cloud all use the same shape and the same three +answers. + +--- + +## The three decisions + +```js +allow() // proceed, silently +allow("CI is green.") // proceed, and tell the model something useful +deny("sudo is blocked here") // stop the action, and say why +instruct("Run tests first.") // proceed, with extra context to stay on track +``` + +| Decision | What the agent experiences | +|---|---| +| **allow** | Nothing. The tool call runs as normal. With a message, the model also receives that line as context. | +| **deny** | The call never runs. The model is told `Blocked by failproofai: ` and typically routes around it on its own. | +| **instruct** | The call runs. The model receives your message alongside the result. | + +The reason text matters more than it looks. A denial is not an error the agent hits and +gives up on — it is a sentence the model reads and acts on. `deny("Don't do that")` gets +you a retry loop; `deny("Pushes to main are blocked — open a PR from a feature branch +instead")` gets you a pull request. + + + Reach for **instruct** more than you expect. Most agent failures are not a dangerous + command — they are drift, redundancy, and stopping early. Those are steering problems, + and steering costs nothing. + + +--- + +## Where policies come from + +Four sources, all evaluated together, each with a different reason to exist. + + + + + 39 rules covering the failure modes every team hits. Enable by name, tune by parameter, + no code. + + + + JavaScript, with the same `allow` / `deny` / `instruct` API. For failure modes specific + to your codebase. + + + + Any `*policies.mjs` file in `.failproofai/policies/`, discovered automatically. Commit + it and the whole team has it. + + + + Policy your organization assigns centrally. Digest-verified on this machine, and + deployable in observe-only mode first. + + + + +--- + +## The order they run in + + + + In definition order, each with its parameters resolved from your config merged over + the policy's own defaults. + + + Whatever your organization deployed here. Each artifact's SHA-256 is verified + immediately before it loads. Anything deployed in `observe` mode is evaluated and then + has its verdict discarded. + + + Files you named with `--custom`, in configured order. + + + Project `.failproofai/policies/` first, then user `~/.failproofai/policies/`. + Alphabetical within each — prefix with `01-`, `02-` if order matters to you. + + + +Then: + +- **The first `deny` wins and stops everything after it.** Its reason is the answer. +- **All `instruct` messages accumulate** and are delivered together. +- **All `allow` messages accumulate** the same way. + +--- + +## Turning policies on + +The fastest path is setup, which offers **Recommended** — 16 policies, globally, for every +agent CLI on the machine: + +```bash +failproofai config +``` + + +| Group | Policies | Why | +|---|---|---| +| Secrets never reach the model or disk | `sanitize-jwt`, `sanitize-api-keys`, `sanitize-connection-strings`, `sanitize-private-key-content`, `sanitize-bearer-tokens`, `protect-env-vars`, `block-env-files`, `block-secrets-write` | A leaked credential is the one failure you cannot undo by reverting a commit. | +| The agent cannot disable its own guardrails | `block-self-pause`, `block-failproofai-commands` | An agent that can turn off enforcement has no enforcement. | +| Commands that are unrecoverable when wrong | `block-sudo`, `block-curl-pipe-sh`, `block-rm-rf` | Everything here destroys state that no undo brings back. | +| Git history stays recoverable | `block-push-master`, `block-force-push` | `--force-with-lease` still works; blind clobbering does not. | + +Recommended is a deliberate, separate list — not "everything that happens to default on". +A test asserts no default-on policy is missing from it, so a machine set up by pressing +Enter is never guarded *less* than one configured by hand. + + +### Presets + +Choosing **Customize** gives you themed bundles instead. They are additive — tick several +and you get the union. + +| Preset | What it covers | +|---|---| +| **Secrets & data** | Redact secrets in tool output, block `.env` and secret-file writes, keep reads inside the repo | +| **Git safety** | Block force-push and pushes to main, warn on history-rewriting git operations | +| **Ship discipline** | Don't let the agent finish until changes are committed, pushed, PR'd, and CI is green | +| **Cloud & infra** | Block `kubectl` / `terraform` / `aws` / `gcloud` / `az` / `helm` / `gh` pipeline commands | + +### One at a time + +```bash +failproofai policy add block-rm-rf +failproofai policy remove warn-git-amend +failproofai policies # list everything, with status and parameters +``` + +Or toggle any policy from the [local dashboard's](/dashboard) Policies page. + +--- + +## Tuning a policy without writing code + +Most built-in policies take parameters. Set them in +`policies-config.json` under `policyParams`: + +```json +{ + "policyParams": { + "block-sudo": { + "allowPatterns": ["sudo systemctl status", "sudo journalctl"] + }, + "block-push-master": { + "protectedBranches": ["main", "release", "prod"] + }, + "warn-large-file-write": { "thresholdKb": 512 } + } +} +``` + +Allowlist patterns are matched **token by token against the parsed command**, not against +the raw string. An entry for `sudo systemctl status *` cannot be bypassed by appending +`; rm -rf /`. + +### `hint` — extra guidance on any policy + +Every policy accepts a `hint`, appended to whatever reason it gives: + +```json +{ + "policyParams": { + "block-force-push": { "hint": "Branch off and open a PR instead." } + } +} +``` + +The agent then sees: *"Force-pushing is blocked. Branch off and open a PR instead."* Works +on built-in, custom, and convention policies alike — no code change. + +[Full configuration reference →](/configuration) + +--- + +## Pausing enforcement + +Sometimes you genuinely need a policy out of the way for ten minutes. Pausing is +deliberately **not** configuration: + +```bash +failproofai config --pause # this directory's newest session, 30 minutes +failproofai config --pause 10m # a specific duration (max 8h) +failproofai config --resume # end it early +failproofai config --status # what is paused, and when it lifts +``` + +The rules that make this safe to have at all: + +- **One session, not the machine.** It applies to the agent session you are actually + sitting in front of. +- **Always time-boxed.** 30 minutes by default, 8 hours maximum, never unbounded. Renewing + extends the same stretch rather than restarting the ceiling, so you cannot pause forever + one legal command at a time. +- **Never committed.** Pause state lives in machine-local state, not in a config file that + would travel to everyone who checks out the branch. +- **Cloud-managed policies keep enforcing.** A local pause does not suspend what your + organization deployed. +- **Agents cannot pause themselves.** `block-self-pause` is on by default and blocks an + agent from running the pause command on its own behalf. + +--- + +## Writing your own + +When the failure mode is specific to your codebase, write the rule: + +```js +// .failproofai/policies/team-policies.mjs +import { customPolicies, allow, deny } from "failproofai"; + +customPolicies.add({ + name: "no-production-writes", + description: "Block writes to paths containing 'production'", + match: { events: ["PreToolUse"] }, + fn: async (ctx) => { + if (ctx.toolName !== "Write" && ctx.toolName !== "Edit") return allow(); + const path = ctx.toolInput?.file_path ?? ""; + return path.includes("production") + ? deny("Writes to production paths are blocked") + : allow(); + }, +}); +``` + +Custom policies are **fail-open**: a syntax error, a thrown exception, or a function that +runs longer than 10 seconds is logged and treated as allow. Your own broken rule never +takes the built-ins down with it. + +[Full authoring guide →](/custom-policies) · [Testing your policies →](/testing) + +--- + +## Related + + + + + Every rule, what it catches, and its parameters. + + + + Which decisions actually block, per CLI. + + + + Scopes, merge rules, and the config file format. + + + + One deployment, every machine, with an observe-only rollout. + + + diff --git a/docs/zh/getting-started.mdx b/docs/zh/quickstart.mdx similarity index 100% rename from docs/zh/getting-started.mdx rename to docs/zh/quickstart.mdx diff --git a/docs/zh/reference/files.mdx b/docs/zh/reference/files.mdx new file mode 100644 index 00000000..fd1ba55d --- /dev/null +++ b/docs/zh/reference/files.mdx @@ -0,0 +1,117 @@ +--- +title: Files and paths +description: "Everything FailproofAI writes on a machine, what each file holds, and which ones are safe to delete." +icon: folder +--- + +FailproofAI writes to exactly two places: `~/.failproofai/` and a `.failproofai/` directory +in any project you configure. The only exception is the hook entry it adds to each agent +CLI's own settings file, so that CLI knows to call it. + +--- + +## `~/.failproofai/` — the machine + +| Path | Holds | Safe to delete? | +|---|---|---| +| `policies-config.json` | Your global policy selection and parameters | Only if you want to lose your setup | +| `policies/` | **Your own policy files.** Drop `*policies.mjs` in; no config needed | No — this is your code | +| `policies/cloud-policies/` | Policies your organization deployed here | Yes — re-fetched and verified on the next poll | +| `config.json` | Machine settings: daemon, collector, capture paths, audit schedule | Only if you want to re-run setup | +| `credentials.toml` | Cloud tokens. **Owner-only (`0600`)** | Yes — you will need to reconnect | +| `hook-activity/` | The decision log the dashboard reads | Yes — you lose local history | +| `bin/` | The downloaded service binary, versioned | Yes — reinstalled by `failproofai config` | +| `run/` | The service's runtime socket and lock | Yes — recreated at start | +| `state/` | Pause state and scheduler progress | Yes — pauses end, schedules restart | +| `cache/` | The audit's per-transcript cache | Yes — the next audit is just slower | +| `logs/`, `hook.log` | Debug output from custom policy errors | Yes | +| `migrations/` | Applied-migration records and pre-migration backups | Keep until you are sure an upgrade went well | + + + Put your own policy files **directly** in `policies/`. The `cloud-policies/` folder + beside them is managed for you, and discovery does not descend into subdirectories — so + the two can never collide. + + +--- + +## `.failproofai/` — the project + +| Path | Holds | Commit it? | +|---|---|---| +| `policies-config.json` | Project policy selection and parameters | **Yes** — this is your team's standard | +| `policies-config.local.json` | Your personal overrides for this repo | **No** — gitignore it | +| `policies/` | Convention policy files for this repo | **Yes** | + +A project's config layers over your global one. [Merge rules →](/configuration#merge-rules) + +--- + +## Agent CLI settings files + +FailproofAI adds a hook entry to each agent CLI's own configuration, in that CLI's own +schema, preserving everything else in the file. [The full list of paths, per +CLI →](/agent-support#where-the-hooks-get-written) + +These are the only files outside `~/.failproofai/` and `.failproofai/` that FailproofAI +writes to, and `failproofai uninstall` removes exactly what it added. + +--- + +## Agent transcripts — read, never written + +Each agent CLI writes its own session records, in its own format and location. FailproofAI +**reads** them to render session replay, to run the [audit](/audit), and — on a connected +machine — to give the cloud a picture of the run. + +They are never modified, moved, or deleted. If your transcripts live somewhere +non-standard, [`failproofai harness add-path`](/cli/harness) points at them. + +--- + +## Permissions + +- `credentials.toml` is written `0600`, and the directory around it is tightened to match. A + `0600` file inside a world-readable directory is still reachable by every local user. +- Cloud tokens are deliberately **not** placed in the service definition file, which is + installed world-readable. That is also why connecting, rotating a token, and disconnecting + all work without `sudo`. + +--- + +## What an upgrade does to all of this + +A new version may reorganize `~/.failproofai/`. When it does, the first command after the +upgrade migrates it and **carries your configuration across** — policy selection, machine +settings, cloud connection, your own policy files and the helpers they import, the decision +log, and anything not yet delivered. + +Rebuilt rather than migrated: the audit cache, cloud deployments (re-fetched and verified), +and service scratch state. + +Irreplaceable files are copied to a backup directory before anything runs, and every +migration is recorded. See [`failproofai migrate --dry-run`](/cli/migrate). + +--- + +## Related + + + + + What goes in each config file, and how scopes merge. + + + + Overrides for nearly every path on this page. + + + + What the service reads and writes. + + + + Removing all of it cleanly. + + + diff --git a/scripts/translate-docs/mintlify-nav.ts b/scripts/translate-docs/mintlify-nav.ts index 24b6f5f3..1df69500 100644 --- a/scripts/translate-docs/mintlify-nav.ts +++ b/scripts/translate-docs/mintlify-nav.ts @@ -85,8 +85,13 @@ export function buildLanguageNav( const groupNameMap: Record = { "Getting Started": t.gettingStarted, + // The unified navigation renamed several groups. Only genuine synonyms are + // mapped onto an existing translation; the rest stay English until + // NAV_TRANSLATIONS gains a key for them, which is better than a wrong word. + "Start here": t.gettingStarted, "Core Concepts": t.coreConcepts, CLI: t.cli, + "CLI reference": t.cli, Tools: t.tools, Advanced: t.advanced, Examples: t.examples, From 541b256501c085f36c270d02e2a201f2fd342f2a Mon Sep 17 00:00:00 2001 From: NiveditJain Date: Wed, 12 Aug 2026 23:55:01 +0530 Subject: [PATCH 2/2] docs: correct the changelog PR references to #687 Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_014ZHcJRck9qdLpGEZeL51js --- CHANGELOG.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index af66cef4..84ad4681 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,10 +4,10 @@ ### Docs -- Rebuilt the documentation as one product instead of two. The site had two top-level tabs — "Enforcement" and "Observability" — which asked every reader to work out, before reading anything, which half of a single product they were in. There is now one navigation, ordered as a journey: install and guard a machine, see what happened on it, then govern the fleet from the cloud. The observability section is repositioned as **FailproofAI Cloud** and lives at `/cloud/*` (was `/agenteye/*`), with redirects from every old URL. (#684) -- Documented the integration surface that had no docs at all. `failproofai config --connect` was described only in `--help`: new pages cover [connecting a machine](https://docs.befailproof.ai/cloud/connect) (both capabilities, what leaves the machine, fleet provisioning, troubleshooting), [managed policies](https://docs.befailproof.ai/cloud/managed-policies) (deployments, digest verification, observe-mode rollout), [the fleet view](https://docs.befailproof.ai/cloud/fleet), and [session capture](https://docs.befailproof.ai/cloud/capture) across all 12 CLIs — replacing three per-CLI capture pages written for a separate collector. (#684) -- Added the pages a reader kept needing and could not find: [How it works](https://docs.befailproof.ai/how-it-works) (tool call → decision → dashboard, end to end, with the failure-mode table), [the failproofaid service](https://docs.befailproof.ai/daemon) (fail-closed, supervision, how the binary arrives, upgrades), [Supported agents](https://docs.befailproof.ai/agent-support) (per-CLI matrix of what a deny can actually block — the one fact that decides whether a policy does anything), [Policies](https://docs.befailproof.ai/policies) as a hub, one merged [Concepts](https://docs.befailproof.ai/concepts) glossary, and a [files and paths](https://docs.befailproof.ai/reference/files) reference. New CLI pages for `config`, `harness`, `backfill`, `flush`, and `uninstall`. (#684) -- Moved per-CLI hook-schema internals out of the Configuration page and into the support matrix, so configuration is about configuring again rather than a wall of vendor contracts. (#684) +- Rebuilt the documentation as one product instead of two. The site had two top-level tabs — "Enforcement" and "Observability" — which asked every reader to work out, before reading anything, which half of a single product they were in. There is now one navigation, ordered as a journey: install and guard a machine, see what happened on it, then govern the fleet from the cloud. The observability section is repositioned as **FailproofAI Cloud** and lives at `/cloud/*` (was `/agenteye/*`), with redirects from every old URL. (#687) +- Documented the integration surface that had no docs at all. `failproofai config --connect` was described only in `--help`: new pages cover [connecting a machine](https://docs.befailproof.ai/cloud/connect) (both capabilities, what leaves the machine, fleet provisioning, troubleshooting), [managed policies](https://docs.befailproof.ai/cloud/managed-policies) (deployments, digest verification, observe-mode rollout), [the fleet view](https://docs.befailproof.ai/cloud/fleet), and [session capture](https://docs.befailproof.ai/cloud/capture) across all 12 CLIs — replacing three per-CLI capture pages written for a separate collector. (#687) +- Added the pages a reader kept needing and could not find: [How it works](https://docs.befailproof.ai/how-it-works) (tool call → decision → dashboard, end to end, with the failure-mode table), [the failproofaid service](https://docs.befailproof.ai/daemon) (fail-closed, supervision, how the binary arrives, upgrades), [Supported agents](https://docs.befailproof.ai/agent-support) (per-CLI matrix of what a deny can actually block — the one fact that decides whether a policy does anything), [Policies](https://docs.befailproof.ai/policies) as a hub, one merged [Concepts](https://docs.befailproof.ai/concepts) glossary, and a [files and paths](https://docs.befailproof.ai/reference/files) reference. New CLI pages for `config`, `harness`, `backfill`, `flush`, and `uninstall`. (#687) +- Moved per-CLI hook-schema internals out of the Configuration page and into the support matrix, so configuration is about configuring again rather than a wall of vendor contracts. (#687) ## 1.0.0 — 2026-08-12