Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 37 additions & 2 deletions .github/workflows/release-please.yml
Original file line number Diff line number Diff line change
Expand Up @@ -211,8 +211,9 @@ jobs:
gh release upload "${TAG}" dist/* SHA256SUMS
env-pack-assets/* ENV_PACK_SHA256SUMS

# Prove the published artifact installs and imports from the public index in a
# clean environment (no checkout, no local links).
# Prove the published artifact installs, imports, and executes the README
# quickstart from the public index in a clean environment (no checkout or
# local distribution links).
smoke:
needs: [release-please, publish]
if: >-
Expand Down Expand Up @@ -250,6 +251,40 @@ jobs:
assert raes_adapters.__version__ == version("raes-adapters")
PY
test "$(/tmp/smoke/bin/python -c 'import raes_adapters; print(raes_adapters.__version__)')" = "${VERSION}"
- name: Run the published README quickstart
run: |
set -euo pipefail
quickstart_root="$(mktemp -d)"
cd "${quickstart_root}"
PYTHONPATH='' PYTHONSAFEPATH=1 /tmp/smoke/bin/raes-adapters run --mode conformance --suite pr --output cage2-quickstart > quickstart.json
/tmp/smoke/bin/python - <<'PY'
import json
from pathlib import Path

expected = {
"disposition": "succeeded",
"evidence_basis": "hermetic-live",
"inventory": "inventory.json",
"mode": "conformance",
"run_count": 1,
}
assert json.loads(Path("quickstart.json").read_text()) == expected
artifacts = sorted(
path.as_posix()
for path in Path("cage2-quickstart").rglob("*")
if path.is_file()
)
assert artifacts == [
"cage2-quickstart/index.json",
"cage2-quickstart/inventory.json",
"cage2-quickstart/runs/cyborg-pr-seed-3/conformance/backend-conformance.json",
]
inventory = json.loads(Path("cage2-quickstart/inventory.json").read_text())
assert [item["path"] for item in inventory["artifacts"]] == [
"index.json",
"runs/cyborg-pr-seed-3/conformance/backend-conformance.json",
]
PY

# After a release, the version bump + CHANGELOG land on `main`, so `dev` falls
# one release commit behind. Open a back-merge PR so `dev` is resynced. This job
Expand Down
3 changes: 3 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ ADR-069 §8). CAGE-2 replication is one backend served by the CybORG adapter
rather than the scope of the repository. Contributions land through the Ground
Control `/implement` workflow.

The [developer index](docs/maintainers/index.md) collects repository layout,
verification, packaging/release, governance, and adapter workflow references.

## Branching

- `main` — production. PRs only (from `dev`). No direct pushes, no force-push.
Expand Down
652 changes: 162 additions & 490 deletions README.md

Large diffs are not rendered by default.

21 changes: 7 additions & 14 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,22 @@
# RAES adapters

This repository contains a single distribution, `raes-adapters`, that connects
concrete simulator backends to published RAES contracts. It ships shared base
plumbing plus backend modules whose dependencies can live behind separate
extras in one lock. Maintainer-selected backends are admitted; backend-local
qualification evidence records attainable claim strength and limitations. The
`cyborg` and `cyberbattlesim` extras are dependency-light because their selected
native sources have no governed publishable artifact, so users acquire those
simulators separately and the base install remains independent. The admitted
CybORG backend is supported through its documented source installation despite
the absence of an automatically installed native simulator; its extra installs
the published environment-pack validator used by the researcher command.

The shared `raes_adapters.base` module provides plumbing only. RAES remains the
semantic and protocol authority.
`raes-adapters` connects concrete simulator backends to published RAES
contracts. Start with the installed conformance quickstart, then select the
researcher command or evidence recipe that matches the claim you need to make.
Scenario validity, adapter conformance, native run completion, and scientific
evidence are separate boundaries.

## Start here

- [Repository overview](https://github.com/OpenRAE/adapters#readme)
- [Researcher guide](researcher-guide.md)
- [Installed researcher command](researcher-command.md)
- [Offline bundle verifier](bundle-verifier.md)
- [NASim researcher command](nasim-researcher-command.md)
- [CyberBattleSim researcher command](cyberbattlesim-researcher-command.md)
- [CyberBattleSim baseline reproduction](cyberbattlesim-baseline-reproduction.md)
- [Contribution guide](https://github.com/OpenRAE/adapters/blob/dev/CONTRIBUTING.md)
- [Developer index](maintainers/index.md)
- [Architecture decisions](decisions/adrs/README.md)
- [CybORG/CAGE-2 backend qualification guardrails](decisions/cyborg-cage2-runtime-qualification-guardrails.md)
- [CybORG/CAGE-2 source-ledger guardrails](decisions/cyborg-cage2-source-ledger-guardrails.md)
Expand Down
91 changes: 91 additions & 0 deletions docs/maintainers/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
# Developer index

This index is for adapter implementers, contributors, release maintainers, and
repository operators. Researchers should start with the
[README](https://github.com/OpenRAE/adapters#readme) and
[researcher guide](../researcher-guide.md).

## Architecture boundary

RAES owns the published semantic contracts. This repository adapts concrete
simulators to those contracts:

- backend concepts stay under `src/raes_adapters/<simulator>/`;
- `raes_adapters.base` contains composition plumbing, not schemas, profiles,
backend protocols, diagnostic envelopes, stores, or policy authority;
- portable artifacts exclude native simulator objects, raw logs, hidden truth,
arguments/environment dumps, tokens, and full tracebacks; and
- accepted repository decisions are recorded under `docs/decisions/adrs/` and
content-pinned in `adr-index.yaml`.

Start with [ADR-002](../decisions/adrs/adr-002-raes-authority-and-adapter-boundaries.md)
and [ADR-003](../decisions/adrs/adr-003-single-distribution-and-trusted-publishing.md).
Backend design notes under `docs/decisions/` record source-specific evidence
and guardrails; they are authority records, not researcher tutorials.

## Repository layout

| Path | Purpose |
| --- | --- |
| `src/raes_adapters/base/` | Shared adapter plumbing over published RAES APIs. |
| `src/raes_adapters/<simulator>/` | Backend implementation, qualification, mappings, profiles, and packaged examples. |
| `tests/` | Distribution and backend behavior tests. |
| `tools/` and `tools/tests/` | Repository policy and project-service checks. |
| `docs/decisions/` | Accepted ADRs plus backend-scoped design/evidence guardrails. |
| `noxfile.py` | Canonical local and CI verification graph. |
| `.github/workflows/` | Parallel PR gates and Trusted Publishing release automation. |

## Local development

Follow [CONTRIBUTING](https://github.com/OpenRAE/adapters/blob/dev/CONTRIBUTING.md)
for environment setup, hooks, change conventions, and backend additions. The
canonical completion command is:

```shell
uv tool run --from 'nox[uv]==2026.4.10' nox -f noxfile.py -s verify -- --skip-requirement
```

Use `--skip-requirement` only for genuine requirement-free maintenance. Normal
issue work supplies the governing requirement UID through Ground Control.

The graph runs hygiene, policy, lint, strict typing, tests with coverage,
clean-built distribution probes, and strict documentation. CI runs the same
sessions as independent jobs joined by the `PR Gate`; see
[continuous integration](ci.md) for targeted reproduction commands.

## Packaging and releases

`raes-adapters` is one distribution with optional per-simulator extras and one
`uv.lock`. A mutually incompatible simulator stack is isolated through uv
extras conflicts rather than a second distribution, workspace, or lockfile.

Release Please owns the version and `CHANGELOG.md` from Conventional Commit
history. Trusted Publishing builds and verifies the tagged commit, publishes
through PyPI OIDC, attaches immutable distributions/checksums, and runs the
exact-version public-index smoke. Do not hand-edit the changelog, create a
changelog fragment, store a PyPI token, or silently replace an existing release
artifact. Service identities and workflow boundaries are listed under
[project services](project-services.md).

## Add or change an adapter

1. Keep native imports behind the backend module boundary so the base install
remains usable.
2. Reuse published RAES contracts and existing repository helpers; do not add a
local semantic model or shared backend registry.
3. Bind qualification to immutable source evidence and disclose unsupported
facts or losses rather than promoting them through static claims.
4. Add behavioral tests at the narrowest boundary and extend the clean-wheel
proof for installed behavior.
5. Update researcher documentation when a command, output, support statement,
limitation, or evidence interpretation changes.
6. Add or amend an ADR only for a durable repository decision; regenerate the
ADR pin in the same change.

## Governance and services

- [Contribution guide](https://github.com/OpenRAE/adapters/blob/dev/CONTRIBUTING.md)
- [Continuous integration](ci.md)
- [Project services](project-services.md)
- [ADR overview](../decisions/adrs/README.md)
- [Repository issue tracker](https://github.com/OpenRAE/adapters/issues)
146 changes: 146 additions & 0 deletions docs/researcher-guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Researcher guide

Use this guide after completing the README conformance quickstart. The
quickstart proves that an installed adapter can emit bounded RAES conformance
evidence; it does not execute a simulator study.

## Choose the evidence you need

Before selecting a command, decide which statement the work must support:

| Intended statement | Minimum evidence boundary |
| --- | --- |
| The authored scenario and experiment are well-formed. | Validate the environment pack, SDL, task, experiment, participants, controls, and all content digests. |
| The adapter exercises selected RAES backend contracts. | Retain a conformance report with its execution basis, cases, diagnostics, capability gaps, and declared non-claims. |
| A native run completed. | Admit the pinned simulator source, complete every scheduled run, verify cleanup, validate the portable records, and seal the inventory last. |
| A result reproduces or supports a research claim. | Predeclare the method and comparison rule, retain the complete evidence joins, account for losses and stochastic controls, and report only the strongest tier the evidence supports. |

Do not infer a stronger row from a successful weaker row. In particular,
scenario validity is not conformance, conformance is not native completion,
and native completion is not scientific validity.

## Work with the CAGE-2 pack

The installed `cage2-research` pack carries the portable Scenario 2 SDL, task,
experiment, Blue participant manifest/selection/configuration, compatibility
record, content manifest, and provenance ledger. The
[checked-in pack](https://github.com/OpenRAE/adapters/tree/dev/src/raes_adapters/cyborg/examples/cage2-research)
is the review surface; installed package resources are the execution surface.

The current pack has status `built`, not `golden`. Its native task remains
fail-closed because the pinned RAES contract cannot validate the task's
semantic reward-component witness. Pack validity and source qualification do
not override that gate.

## Change a study deliberately

### Participant implementation

A Red policy can be selected only when the pack declares that variant. A new
Blue implementation needs a published participant manifest, selection, and
configuration whose identities and digests join the task and pack. Record the
implementation source or model bytes; do not represent an unavailable
submitted artifact through a similarly behaving substitute.

### Seed allocation

Seeds belong to the experiment design. Record which random streams each seed
actually controls and disclose the unbound streams. A shared numeric seed does
not imply deterministic replay across simulator, policy, Python, NumPy, or Gym
state.

### Trial length

Trial length is part of the declared condition, not an arbitrary cutoff. A new
length changes termination and comparison semantics, so update the experiment,
task joins, pack content manifest, and digests before execution.

### Environment pack

Changing the scenario, task, participant, or experiment creates a new pack
identity/version. Validate and reseal it through the published environment-pack
tools. The researcher CLI intentionally has no ambient profile root, arbitrary
driver import, hidden default, or runtime download seam.

## CAGE-2 reproduction boundary

The [installed researcher command](researcher-command.md) documents both the
two-seed authored example and the frozen public protocol reproduction. The
public protocol schedules 3 trial lengths × 3 Red variants × 1,000 episodes and
uses the declared study-scoped Python stream initialized with seed 153. It is a
substantial compute and storage workload, not a quickstart.

The public submitted Blue artifact is unavailable. The retained comparison can
therefore support only the predeclared behavioral-baseline outcome tier; it
cannot establish submitted-agent identity, state or observation equivalence,
deterministic replay, or native conformance from score similarity.

## Limitations to carry into a report

- Portable artifacts exclude native state, observations, action identifiers,
reward vectors, object representations, raw logs, hidden truth, environment
dumps, and tracebacks.
- Stochastic controls are partial wherever the pinned source exposes no binding
seam.
- Qualification applies only to its repository, commit, selected files,
patches, interpreter/platform evidence, and dependency graph.
- Capability gaps and missing evidence witnesses remain negative facts; a
successful adjacent check does not satisfy them.
- Full studies can be expensive in episodes, wall time, memory, and evidence
storage. Estimate all four before execution.
- The apparatus is not a production control, security certification, or
operational defense guarantee.

## Citation and provenance checklist

Record these identities with the retained evidence:

1. `raes-adapters` release version and artifact hash;
2. RAES and environment-pack dependency versions;
3. environment pack name, version, content digest, and compatibility record;
4. adapter qualification profile, upstream repository, source commit, and any
admitted patch digest;
5. scenario, task, experiment, and participant artifact digests;
6. run controls, seed allocation, runtime/software inventory, and declared
losses; and
7. the final `inventory.json` plus the archive or repository location that
preserves the referenced files.

Cite an immutable release and the upstream simulator project. A branch name or
working-tree path is useful context but is not an executed artifact identity.

## Troubleshooting

### The package is not available

Confirm Python 3.12, the configured public package index, and the requested
release. An editable checkout is suitable for development but must not be
reported as a published-distribution reproduction.

### The output path is rejected

Every output root is invocation-relative, new, and exclusively reserved. Use a
new directory name. Absolute paths, parent traversal, reuse, and resolved
symlink escape are rejected before evidence writes.

### Validation exits with status 3

Read the stable error code. For the current CAGE-2 native task,
`researcher.validation.evidence-unverifiable` is expected: validation stopped
before runtime planning because the evidence witness cannot be verified. Do
not remove the requirement or replace it with a local allowlist.

### Native source is unavailable

The `cyborg` extra installs pack validation, not CybORG. Native execution needs
the exact separately installed qualified source and packaging fix recorded in
the qualification. The adapter does not clone, download, or select source from
an ambient environment variable.

### A run fails

Retain the bounded command error, package/source identities, and any sealed
portable inventory. Do not attach native logs, raw observations, environment
contents, tokens, or full tracebacks. The [exit-status table](researcher-command.md#exit-status)
separates usage, validation, output, native execution, artifact, and internal
failures.
2 changes: 2 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,14 @@ markdown_extensions:

nav:
- Home: index.md
- Researcher guide: researcher-guide.md
- Researcher command: researcher-command.md
- Offline bundle verifier: bundle-verifier.md
- NASim researcher command: nasim-researcher-command.md
- CyberBattleSim researcher command: cyberbattlesim-researcher-command.md
- CyberBattleSim baseline reproduction: cyberbattlesim-baseline-reproduction.md
- Maintainers:
- Developer index: maintainers/index.md
- Continuous integration: maintainers/ci.md
- Project services: maintainers/project-services.md
- Decisions:
Expand Down
Loading
Loading