diff --git a/README.md b/README.md index 80de679b..c8d8f413 100644 --- a/README.md +++ b/README.md @@ -184,3 +184,8 @@ Two things sit alongside the licence: ## Support Open an issue. For commercial support, get in touch. + +**Found a security vulnerability?** Do not open an issue — +[report it privately](https://github.com/teemops/tops/security/advisories/new) or email +security@teemops.com. See [SECURITY.md](SECURITY.md) for scope, what to expect, and the +issues already known. diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 00000000..5161741a --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,119 @@ +# Security policy + +TOPS is a security tool that reads customers' AWS accounts. A vulnerability here can expose +another organisation's cloud posture, so we would rather hear about a problem early and +awkwardly than late and politely. + +## Reporting a vulnerability + +**Do not open a public issue.** Use one of these instead: + +1. **[Report a vulnerability privately on GitHub](https://github.com/teemops/tops/security/advisories/new)** + — preferred. It keeps the report private, and lets us develop and review the fix in a + private fork before anything is disclosed. +2. **security@teemops.com** — if you would rather not use GitHub, or the form is unavailable. + +You do not need to have a fix, a CVE, or a polished write-up. A rough description of +something that looks wrong is worth sending. + +### What to include + +Whatever you have. The more of this you can give us, the faster we can confirm it: + +- What the issue is, and what an attacker gets out of it +- The steps to reproduce, ideally against a local install +- The version — `git rev-parse HEAD`, or the image tag from `docker compose images app` +- Whether you have told anyone else, and whether you intend to publish + +## What happens next + +| | | +| --- | --- | +| **Acknowledgement** | Within 5 working days | +| **Initial assessment** | Within 10 working days — whether we can reproduce it, and our severity view | +| **Fix and release** | Depends on severity. We will tell you the plan and keep you updated | +| **Credit** | Named in the advisory and the changelog, unless you would rather not be | + +TOPS is maintained by a small team. If you have not heard from us inside those windows, +please chase — the most likely explanation is that a message went astray, not that we are +ignoring it. + +## What we are most worried about + +If you are looking for somewhere to start, these are the failures that would hurt most, in +order: + +1. **Cross-organisation data access.** Every query touching tenant data is supposed to be + scoped by `organization_id`. Anything that lets one organisation read another's accounts, + scans or findings is the highest-severity class of bug in this codebase. +2. **Authentication or authorisation bypass** on any route. +3. **Anything that lets a third party influence account linking** — the CloudFormation + custom resource, the SNS to SQS path, or the handling of `external_id` — such that an + account is linked, unlinked, or repointed without the owner's action. +4. **Privilege escalation through the CloudFormation templates**, or a template that grants + more in a customer account than the documentation says it does. +5. **Secrets handling** — anything that writes credentials to logs, images, or the repository. + +## Scope + +**In scope** + +- The application in `app/` +- The CloudFormation templates in `templates/` and `infra/` +- The installer (`install.sh`), the Docker images, and `docker-compose.yml` +- The published `teem/tops` and `teem/tops-base` images + +**Out of scope** + +- **Anything that requires shell access to the host TOPS runs on.** This is a self-hosted + application; an operator with a shell can read the database by design. See + [D-9](docs/roadmap.md) on why application-level encryption at rest was deliberately removed. +- **Misconfiguration of your own install** — running with `APP_DEBUG=true` on a public + address, exposing MySQL's port, or reusing credentials. +- **Findings the scanner reports about your AWS account.** Those are the product working. If + you think a rule is wrong, open a normal issue. +- Missing security headers, cookie flags, or scanner output with no demonstrated impact. +- Denial of service against teemops.com, social engineering, and physical attacks. +- Vulnerabilities in AWS itself — report those to AWS. + +**Never test against infrastructure you do not own.** Reproduce against a local install or +your own AWS account. A report is not worth another organisation's data. + +## Already known — please do not report these + +These are public, tracked, and being worked on. Reporting them again is not a finding. + +| | | +| --- | --- | +| [#101](https://github.com/teemops/tops/issues/101) | The parent account's `teemops-sns` topic accepts `sns:Publish` from any AWS principal. Publish-only, and acting on a message still requires guessing a 122-bit `external_id`, but it is open and it is being fixed. The issue links the full research and the options considered | +| [#100](https://github.com/teemops/tops/issues/100) | The account-linking consumer does not cross-check the account in `TopsRoleArn` against the one in `StackId`, nor require a `pending` status | +| [#102](https://github.com/teemops/tops/issues/102) | `templates/iam.role.audit.account.cfn.yaml` defaults to a vendor AWS account and is not on the onboarding path | + +## Supported versions + +Only the latest release. TOPS is pre-1.0 and moving quickly; fixes land on `develop` and go +out in the next release rather than being backported. + +| Version | Supported | +| --- | --- | +| 0.4.x | Yes | +| < 0.4 | No — upgrade | + +Upgrading is a tag change and a restart. See +[Upgrading and rolling back](https://github.com/teemops/tops#upgrading-and-rolling-back). + +## Disclosure + +We aim to publish an advisory once a fix is released, describing the issue, the affected +versions and the fix. We will agree timing with you, and we would ask that you hold off +publishing until a fixed release exists — or 90 days from your report, whichever comes first. + +If a vulnerability is already being exploited, tell us and we will move immediately rather +than to any schedule. + +## What we will not do + +We will not pursue legal action against anyone who reports a vulnerability in good faith, +follows this policy, and does not access or destroy data that is not theirs. There is no +bug bounty — this is an open-source project without a budget for one, and we would rather +say so than imply otherwise. diff --git a/app/DOCS.md b/app/DOCS.md index 17266292..4053e0e1 100644 --- a/app/DOCS.md +++ b/app/DOCS.md @@ -5,7 +5,7 @@ This file provides quick links to all project documentation. For detailed docume ## Quick Links ### Essential Setup -- [Quick Start](../docs/quick-start.md) - Get started quickly +- [Install TOPS](../README.md#install) - the canonical setup path - [Environment Setup](../docs/laravel-app/ENV_SETUP.md) - Configure environment variables - [Database Setup](../docs/laravel-app/database-setup.md) - Database configuration diff --git a/design/architecture/README.md b/design/architecture/README.md new file mode 100644 index 00000000..da0ba08a --- /dev/null +++ b/design/architecture/README.md @@ -0,0 +1,20 @@ +# Architecture explainer — design exploration + +`aws-integration.html` is the standalone page that was built first, to work out how to +explain the AWS integration to a security reviewer. It did its job twice over: it settled the +structure of the explainer archetype, and writing it is what surfaced the open SNS topic now +tracked as [N-11](../../docs/roadmap.md#n-11--lock-down-the-account-linking-sns-topic). + +**It is not the canonical version.** That is +[`user-docs/start-here/how-tops-connects-to-aws.md`](../../user-docs/start-here/how-tops-connects-to-aws.md), +which is what publishes to docs.teemops.com. If the two disagree, the docs page wins. + +The difference that matters: this page embeds its diagrams inline at a 1160px viewBox, sized +for a full-bleed layout. The docs page uses standalone 900px SVGs in +`user-docs/assets/diagrams/`, re-authored so the text survives being scaled into a docs +column. See the sample-test findings in +[`design/docs-site/information-architecture.md`](../docs-site/information-architecture.md). + +Kept because the exploration is worth having on record, not because it needs maintaining. +**Do not edit it to track product changes** — edit the docs page. This one can be deleted +whenever it stops being interesting. diff --git a/design/architecture/aws-integration.html b/design/architecture/aws-integration.html new file mode 100644 index 00000000..5ebeda02 --- /dev/null +++ b/design/architecture/aws-integration.html @@ -0,0 +1,1107 @@ +How TOPS connects to AWS + + +
+ +
+
TOPS · self-hosted AWS security scanning · architecture
+

How TOPS connects to AWS

+

+ TOPS runs on your infrastructure and reads your AWS accounts through cross-account IAM roles. + There are no stored access keys, no inbound network path to the install, and no data path to us. + This page shows exactly what gets created, who creates it, and what crosses each boundary. +

+
+ v0.4.0 + Docker Compose + Laravel 11 + Vue + CloudFormation · SNS · SQS · STS + Apache-2.0, source-available +
+
+ +
+
+
Your infrastructure
+

The server you run TOPS on. Owns the database, the findings, and the only copy of the encrypted role ARNs.

+
+
+
Your parent AWS account
+

One account you nominate. Holds the messaging resources — an SNS topic, SQS queues, one S3 bucket. Deployed once.

+
+
+
Child accounts
+

Every account you want scanned. Each gets one CloudFormation stack and one IAM role. Add as many as you have.

+
+
+ + +
+
STAGE
01
+
+

Standing up the parent account

+

Run once · ./install.sh --aws · one region

+

+ The installer runs as a throwaway container on your own machine, using your own AWS CLI credentials. + It deploys two CloudFormation stacks into an account you choose, then writes the resulting queue names and + ARNs back to disk. Nothing here is shared with other installs — these are your resources, in your account. +

+ + + +
    +
  1. Deploy. The installer checks sts:get-caller-identity, confirms the account and region with you, then runs sam deploy.
    2. +
  2. Create. CloudFormation creates the topic, the queues, the dead-letter queue and the bucket. You own all of it.
    3. +
  3. Subscribe. teemops-sns is subscribed to teemops_main, so a notification becomes a queue message.
    4. +
  4. Report back. Stack outputs are written to generated/teemops.env and loaded by the containers on restart.
    5. +
+ +

+ Skippable. TOPS runs without this step — you just cannot link an account yet. Come back later with + ./install.sh --aws-only. Everything above can be deleted by removing two CloudFormation stacks. +

+
+
+ + +
+
STAGE
02
+
+

Connecting child accounts, one link at a time

+

Repeat per account · no limit · no per-account infrastructure in the parent

+

+ TOPS never asks for credentials to a child account. It hands the account's own administrator a + CloudFormation quick-create link, pre-filled with an ExternalId generated for that account alone. + The administrator creates the stack in their own console, under their own session. The stack calls home once + to say which role ARN was created — and that is the whole handshake. +

+ + + +
    +
  1. Hand over a link. A console quick-create URL carrying four parameters: your parent account id, the region, the account's ExternalId and its UniqueId.
    2. +
  2. The admin creates the stack. In their account, with their permissions. TOPS has no session there and cannot create it for them.
    3. +
  3. The stack calls home. A custom resource publishes the new role ARN, external id and unique id to your SNS topic, which fans into teemops_main.
    4. +
  4. Your worker picks it up. Outbound long-poll from inside your network. The message is accepted only if both ids match a record you already created.
    5. +
  5. The stack completes. TOPS replies to the CloudFormation ResponseURL; the admin sees CREATE_COMPLETE, or a failure with a reason.
    6. +
+ +

+ Why there is no account limit. Linking adds one stack in the child account and one database row in yours — + no queue, no topic, no bucket, no per-account resource in the parent. The tenth account and the thousandth cost the + same to connect. Deleting the stack fires the same custom resource in reverse and unlinks it. +

+
+
+ + +
+
STAGE
03
+
+

What actually runs: six containers

+

One docker-compose.yml · one host · docker compose up -d

+

+ TOPS is a Laravel monolith and a MySQL database, split across containers by role rather than by service boundary. + The web tier never talks to AWS. The worker tier does — outbound only, assuming a role per scan and holding the + credentials for the life of one job. Scan jobs travel on the database queue, so a scan needs no AWS messaging at all. +

+ + + +
    +
  1. The UI enqueues, it does not scan. Starting a scan writes a job row. The web tier holds no AWS credentials at all.
    2. +
  2. Workers poll the database. Scan jobs use the database queue driver, with SELECT … FOR UPDATE SKIP LOCKED so five workers never collide.
    3. +
  3. Account linking polls SQS. The only pool that needs AWS messaging, and it stays switched off until TOPS_SQS_ARN is set.
    4. +
  4. Scanning assumes the role. One AssumeRole per job, temporary credentials, read calls, results written home.
    5. +
+ +

+ Air-gapped from us by construction. The parent account, the queues, the database and the findings are all yours. + There is no telemetry endpoint, no license check, and no hosted control plane in this path — the container image is + the only thing you pull from outside, and you can build it yourself with install-build.sh. +

+
+
+ + +
+

What crosses the boundary

+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
DirectionTrafficCarriesWhen
outboundHTTPS to AWS API endpoints, signed with credentials from sts:AssumeRoleDescribe / List / Get calls against the account being scannedDuring a scan
outboundSQS ReceiveMessage / DeleteMessage long-pollNothing — an empty poll until a stack is createdContinuously, if AWS messaging is set up
into your AWSOne sns:Publish from the child account's CloudFormation stackRole ARN, external id, unique idOnce per account, at link and at unlink
neverInbound connections to the TOPS hostThere is no listener. AWS is polled, not called back.—
neverLong-lived AWS access keysNone are requested, entered, or stored. Only role ARNs, encrypted.—
neverFindings, resource inventory, or account metadata leaving your installScan output is written to your MySQL and stays there—
+
+
+ + +
+

What the child-account role can actually do

+

+ Worth being precise about, because this is the first question every security reviewer asks. The role is + not read-only. Scanning uses only the read half; the write half exists for the operational features + that provision and manage AWS security services on your behalf. +

+
+
+
Readused by every scan
+
    +
  • ReadOnlyAccess (AWS managed)
    • +
  • ResourceGroupsandTagEditorReadOnlyAccess
    • +
+

This is the whole permission set the scanner exercises. If you only want scanning, this is all you need.

+
+
+
Writeonly for the ops features
+
    +
  • securityhub:* · guardduty:* · macie:* · inspector:*
    • +
  • config:Put* / Delete* — rules, recorders, delivery channels, conformance packs
    • +
  • cloudtrail: create, update, delete, start/stop logging
    • +
  • logs: create/delete log groups, streams, metric filters
    • +
  • cloudformation: create, update, delete, change sets
    • +
  • s3:CreateBucket / DeleteBucket (for Config delivery)
    • +
  • iam:PassRole → arn:aws:iam::*:role/tops*
    • +
+

+ The template is a plain CloudFormation file in your own S3 bucket. Strip the policies you do not want before + you publish the link — the scanner keeps working, the ops features stop. +

+
+
+
+ + +
+

The shape of it, in numbers

+
+
+
2
+
stacks in the parent
+

Deployed once. Delete them and the integration is gone.

+
+
+
1
+
stack per child account
+

Created by that account's own admin. Revocable by them at any time.

+
+
+
0
+
stored AWS keys
+

Cross-account roles with a per-account external id, assumed per job.

+
+
+
0
+
inbound ports to AWS
+

Every connection is opened from inside your network.

+
+
+
~153
+
region jobs per full scan
+

Fanned across five worker processes by default.

+
+
+
6
+
containers on one host
+

app, worker, mysql, backup, maildev, and a restore container that stays parked.

+
+
+
+ + + +
diff --git a/design/docs-site/archetypes/explainer.md b/design/docs-site/archetypes/explainer.md new file mode 100644 index 00000000..d656a4fc --- /dev/null +++ b/design/docs-site/archetypes/explainer.md @@ -0,0 +1,104 @@ + + +# Where your data lives + +> **Written for security reviewers.** If you have been asked to approve TOPS, this page is +> the whole answer. No prior knowledge of TOPS is assumed. + + + +Every finding TOPS produces is written to a MySQL database on a server you run. Nothing is +sent anywhere else. This page shows where each category of data is stored, who can read it, +and what happens to it when you remove an account or uninstall. + + + +![Description of what the diagram shows, written as a full sentence for screen readers and for anyone the image fails to load for.](../assets/diagrams/name.svg) + + + +## What the diagram does not say + +Prose here — the qualifications, the edge cases, the "except when". The diagram carries the +shape; this carries the caveats that would clutter it. + + + +## The specifics + +| Data | Stored in | Readable by | Removed when | +| --- | --- | --- | --- | +| Findings | Your MySQL, `findings` table | Members of the owning organisation | The account is unlinked | +| IAM role ARNs | Your MySQL, `aws_accounts` | Members of the owning organisation | The account is unlinked | +| Scan logs | Container stdout, your host | Anyone with shell on the host | Your log rotation says so | + + + +## What we do not claim + +- TOPS does not encrypt findings at the application layer. Encryption at rest is delegated + to the host — see [D-9](https://github.com/teemops/tops/blob/develop/docs/roadmap.md). +- Anyone with shell access to your host can read the database. There is no protection + against your own operators. + + + +## Related + +- [How TOPS connects to AWS](../start-here/how-tops-connects-to-aws.md) +- [Backups and restore](../running-the-service/backups-and-restore.md) + +*Source of truth for this page: `path/to/file` in the TOPS repository.* + + diff --git a/design/docs-site/archetypes/reference.md b/design/docs-site/archetypes/reference.md new file mode 100644 index 00000000..d58db5ee --- /dev/null +++ b/design/docs-site/archetypes/reference.md @@ -0,0 +1,73 @@ + + +# Configuration reference + +Every environment variable TOPS reads, what it does, and what happens if you leave it unset. +Set these in `.env` in your install directory, then restart with `docker compose up -d`. + + + +## Jump to + +[Application](#application) · [Database](#database) · [AWS](#aws) · [Workers](#workers) · [Backups](#backups) + + + +## Application + +| Variable | Default | What it does | +| --- | --- | --- | +| `APP_URL` | `http://localhost:8080` | The URL TOPS builds links with. Set it to the address people actually browse to, or emailed links break | +| `APP_KEY` | generated at install | Laravel's encryption key. Rotate with `php artisan key:generate`; everyone is logged out | +| `APP_DEBUG` | `true` | Leave `false` anywhere reachable by someone you do not trust — debug pages expose configuration | + +## AWS + +| Variable | Default | What it does | +| --- | --- | --- | +| `TOPS_DEPLOYMENT_REGION` | *unset* | The region the parent stacks were deployed into. Set by the AWS install step | +| `TOPS_SQS_ARN` | *unset* | The account-linking queue. **While unset, account linking is switched off** and the worker pool does not start | +| `TOPS_WORKER_PROCESSES` | `5` | Region workers in parallel. Each holds ~60–120 MB and one MySQL connection | + + + +## Related + +- [Upgrading and rolling back](upgrading-and-rolling-back.md) +- [Workers, queues and scan speed](workers-queues-and-scan-speed.md) — before raising `TOPS_WORKER_PROCESSES` + + diff --git a/design/docs-site/archetypes/task.md b/design/docs-site/archetypes/task.md new file mode 100644 index 00000000..880de320 --- /dev/null +++ b/design/docs-site/archetypes/task.md @@ -0,0 +1,92 @@ + + +# Connect an AWS account + + + +Links one AWS account to TOPS so it can be scanned. Takes about five minutes, and most of +that is waiting for CloudFormation. + + + +## Before you start + +- TOPS is installed and you can sign in +- You ran the AWS step during install — if you skipped it, run `./install.sh --aws-only` first +- You can create a CloudFormation stack in the account you want to scan, or you know who can + + + +## Steps + +1. **Open AWS accounts and choose Add account.** TOPS creates a pending record and + generates a link that is specific to this account. + +2. **Send the link to whoever administers the AWS account**, or open it yourself if that + is you. It opens the CloudFormation quick-create page, pre-filled. + + The link carries an `ExternalId` generated for this account alone. Treat it like a + password — anyone with it can complete this link. + +3. **Create the stack.** The defaults are correct. Acknowledge the IAM capability + checkbox — the stack creates a role, so AWS requires it. + + ``` + Stack name: tops-vendor-audit + Status: CREATE_IN_PROGRESS → CREATE_COMPLETE + ``` + +4. **Wait for TOPS to pick it up.** Usually a few seconds after the stack completes. The + account moves from *Pending* to *Active* on its own — you do not need to refresh or + paste anything back. + + + +## When it goes wrong + +**The stack sits at CREATE_IN_PROGRESS for more than a few minutes.** +TOPS never received the notification, so CloudFormation is waiting for a reply that is not +coming. Check that the AWS step ran during install (`TOPS_SQS_ARN` is set) and that the +account-linking worker is running: `docker compose logs worker | grep account-queue`. + +**The stack fails with "Account not found".** +The link was generated for a different TOPS organisation, or the pending record was +deleted. Generate a fresh link and create the stack again. + +**The account shows Active but scans return nothing.** +The role exists but cannot be assumed. Confirm the account is not in a region your +credentials cannot reach, and see [Troubleshooting](../running-the-service/troubleshooting.md). + + + +## Next + +- [Run your first scan](run-your-first-scan.md) +- [What the IAM role can do](../aws-accounts/what-the-iam-role-can-do.md) — if you were asked to justify the permissions + + diff --git a/design/docs-site/information-architecture.md b/design/docs-site/information-architecture.md new file mode 100644 index 00000000..42c5bb22 --- /dev/null +++ b/design/docs-site/information-architecture.md @@ -0,0 +1,384 @@ +# docs.teemops.com — information architecture and content map + +Design work, not built. Scope: the structure and content plan for the user documentation +site, plus the page design for **How TOPS connects to AWS**, which is the page that prompted +it. + +**Out of scope for this worktree:** `teemops.com` marketing site wireframes and design. That +continues elsewhere. Where this document touches the marketing site it is only to draw the +boundary between the two. + +## What D-10 already decided + +[D-10](../../docs/roadmap.md#d-10-in-full-where-user-documentation-lives) settled most of the +structural questions. Nothing here relitigates them: + +| Decision | Consequence for this design | +| --- | --- | +| A new top-level `user-docs/`, not `docs/` | `docs/` stays a contributor knowledge base in an engineering voice. Nothing in this plan moves it | +| One site, structured **by task**, not by audience | No "developer track" / "business track" split. Depth increases *down the page* instead | +| `docs.teemops.com`, Cloudflare Pages | Already the plan for the marketing site; this is the same pattern on a second directory | +| **Plain Markdown; renderer decided later** | This design deliberately does **not** pick a static-site generator. See [Tooling](#tooling-stays-deferred-on-purpose) | +| `README.md` stays canonical for install | The site links to it rather than copying it. Duplication is the thing D-10 was most concerned about | + +### One place this design extends D-10 + +D-10 scoped `user-docs/` to pick up **from "you're logged in"** — findings, remediation, team +management, scan profiles — with `README.md` owning everything before that. + +That holds for *operators*. It does not hold for the audience that prompted this work. +**A CISO deciding whether to permit TOPS in their AWS organisation reads before anything is +installed, and will not read a GitHub README to do it.** They want trust boundaries, IAM +permissions, and what crosses the wire. That is a third audience D-10 did not model. + +**Proposed amendment, for the decisions log:** `user-docs/` owns **evaluate** and **use**. +`README.md` stays canonical for the *commands* — the one-liner, the flags, the upgrade steps +— and the site links out to it rather than restating them. This keeps D-10's +anti-duplication rule intact while admitting the evaluator. Suggest recording as **D-13** +before the first page is written, so it isn't rediscovered. + +## Audiences + +Three, in the order they arrive. The task-based structure serves all three without splitting +the site, because the tasks themselves are ordered this way. + +| | Reads | Arrives from | Leaves when | +| --- | --- | --- | --- | +| **Evaluator** — CISO, CTO, security reviewer | Architecture, trust boundaries, IAM permissions, licence | A vendor-review request, or the marketing site | They can answer "is this safe to allow in our org?" | +| **Operator** — DevOps, platform engineer | Install, connect accounts, upgrade, back up, troubleshoot | GitHub, or the evaluator handing it over | TOPS is running and scanning | +| **User** — whoever reads the findings | Findings, remediation, insights, scan profiles, team | The app itself, usually mid-task | They know what a finding means and what to do about it | + +The evaluator is the smallest audience and the highest-stakes: they are a gate, not a user. +One unanswered question on their list stops the other two ever arriving. + +## Site map + +``` +docs.teemops.com +│ +├── Start here +│ ├── What TOPS is — and what it is not +│ ├── How TOPS connects to AWS ← evaluator entry point +│ ├── What it costs to run +│ ├── Install → links README, does not copy it +│ ├── Connect your first AWS account +│ └── Run your first scan +│ +├── Using TOPS +│ ├── The dashboard +│ ├── Running a scan +│ ├── Scan profiles and scan types +│ ├── Reading a finding +│ ├── Resolving a finding +│ └── Insights +│ +├── AWS accounts +│ ├── Adding an account +│ ├── What the IAM role can do ← the honest permissions page +│ ├── Removing an account +│ └── Many accounts at once +│ +├── Your organization +│ ├── Organizations and data isolation +│ ├── Inviting and managing members +│ ├── Roles and permissions +│ └── Your profile +│ +├── Running the service +│ ├── Configuration reference +│ ├── Upgrading and rolling back +│ ├── Backups and restore +│ ├── Workers, queues and scan speed +│ ├── Troubleshooting +│ └── Uninstalling +│ +├── Security +│ ├── The security model +│ ├── Where your data lives +│ ├── Reporting a vulnerability +│ └── Licence and trademark +│ +└── Reference + ├── Glossary + ├── What TOPS scans — services and rules + ├── Command reference + └── Release notes +``` + +Eight sections is at the top of what a sidebar carries before it needs search. If it grows, +**Security** folds into **Start here** before anything else does — its pages are read once, +by one audience, in one sitting. + +## Content inventory + +Status is about the *content*, not the file. "Exists" means the substance is written +somewhere and needs porting and a change of voice, not that a `user-docs/` page is ready. + +| Page | For | Status | Source | Priority | +| --- | --- | --- | --- | :---: | +| **Start here** | +| What TOPS is — and what it is not | All | ✍️ Write | `README.md` intro, roadmap Direction | 1 | +| How TOPS connects to AWS | Evaluator | ✅ **Built this session** | `design/architecture/aws-integration.html` | 1 | +| What it costs to run | Evaluator, Operator | ⚠️ Partial | `install.sh` `aws_intro()`, `docker-compose.README.md` | 2 | +| Install | Operator | ✅ Exists | `README.md` § Install — link, don't copy | 1 | +| Connect your first AWS account | Operator | ✅ Exists | `README.md` § Scanning a real AWS account | 1 | +| Run your first scan | Operator, User | ⚠️ Thin | `README.md`, `docs/features/scan-*.md` | 1 | +| **Using TOPS** | +| The dashboard | User | ✍️ Write | `Pages/Dashboard.vue` | 3 | +| Running a scan | User | ✍️ Write | `Pages/Scans/*`, `NewScanModal.vue` | 2 | +| Scan profiles and scan types | User | ✍️ Write | `ScanProfilesService`, `ScanTypesService`, `config/scan.php` | 2 | +| Reading a finding | User | ✅ **Written** | `Pages/Findings/*`, `rules/rulesets/*.json` | **1** | +| Resolving a finding | User | ✅ **Written** | `FindingsEngine`, `ScanResult`, D-11 durable findings | **1** | +| Insights | User | ✍️ Write | `Pages/Insights/`, `docs/features/insights-by-service.md` | 3 | +| **AWS accounts** | +| Adding an account | Operator | ✅ Exists | `docs/features/onboarding-flow.md` (rewrite — it is a spec, not a guide) | 2 | +| What the IAM role can do | Evaluator | ✅ **Written** | `templates/iam.role.child.account.cfn.yaml` | **1** | +| Removing an account | Operator | ✍️ Write | Stack deletion → `Delete` custom resource | 3 | +| Many accounts at once | Operator, Evaluator | ✍️ Write | This session's research | 3 | +| **Your organization** | +| Organizations and data isolation | Evaluator, User | ✍️ Write | `OrganizationPermission`, multi-tenancy practice | 2 | +| Inviting and managing members | User | ✍️ Write | `Pages/Organizations/*`, `organization-team-management-plan.md` | 2 | +| Roles and permissions | User | ⚠️ Unstable | `OrganizationPermission` — see [#—/X-5](../../docs/roadmap.md), reconciliation open | 3 | +| Your profile | User | ✍️ Write | `Pages/Profile/` | 4 | +| **Running the service** | +| Configuration reference | Operator | ⚠️ Scattered | `.env.docker.example`, `docker-compose.yml` comments | 2 | +| Upgrading and rolling back | Operator | ✅ Exists | `README.md` § Upgrading | 2 | +| Backups and restore | Operator | ✅ Exists, strong | `docs/backups.md` | 2 | +| Workers, queues and scan speed | Operator | ✅ Exists | `worker-supervisord.conf` comments, `parallel-region-scans.md` | 3 | +| Troubleshooting | Operator | ✅ Exists | `DEBUG.md`, `README.md` § If something goes wrong | 2 | +| Uninstalling | Operator, Evaluator | ⚠️ Partial | `docker-compose.README.md`; the two CFN stacks | 3 | +| **Security** | +| The security model | Evaluator | ✅ **Built this session** | The architecture page's boundary table | **1** | +| Where your data lives | Evaluator | ✍️ Write | Nothing leaves the install — needs stating plainly | 2 | +| Reporting a vulnerability | Evaluator | ✍️ Write | No `SECURITY.md` exists — **gap** | **1** | +| Licence and trademark | Evaluator | ✅ Exists | `LICENSE`, `TRADEMARK.md`, roadmap D-7 | 3 | +| **Reference** | +| Glossary | All | ✅ Exists | `docs/GLOSSARY.md` | 3 | +| What TOPS scans — services and rules | Evaluator, User | ✅ Exists | `docs/planning.md`, the rule JSON | 2 | +| Command reference | Operator | ✍️ Write | `artisan` commands, `backup.sh`, `install.sh` flags | 4 | +| Release notes | All | ✅ Exists | `CHANGELOG.md` | 4 | + +**Totals:** 34 pages — 12 exist in substance, 5 partial or scattered, 17 to write from +scratch. Roughly a third of the site is a porting-and-rewriting job rather than an authoring +job, which is the argument for doing the IA before the writing. + +### Two gaps this inventory exposed + +1. ~~**There is no `SECURITY.md`.**~~ **Written 2026-08-02.** A public repo with a `security` + label, three open security issues and a CISO-facing architecture page had nowhere to + report a vulnerability, and `README.md` said "open an issue" — precisely the wrong advice. + Now at the repository root, with the README pointing to it. **Two follow-ups it depends + on:** GitHub private vulnerability reporting must be enabled (it is currently off), and + `security@teemops.com` must be created and routed. Until both are done the policy names + channels that do not answer. +2. **`docs/quick-start.md` is dead.** It describes a Nuxt 3 + Serverless + Prisma monorepo — + the pre-pivot stack. It is linked from `docs/README.md` and would mislead anyone who found + it. Delete it rather than port it. + +## What not to document yet + +Per `docs/practices/product.md` — no page for a feature that does not exist. Verified against +`docs/PROGRESS.md`, which is checked against the code: + +| | Why not | +| --- | --- | +| **Firebase / OAuth sign-in** | Built but **off by default** (`FIREBASE_USER_AUTH=false`). Documenting it as a path implies the default install needs a Firebase project. It does not. One line under Configuration reference, no page | +| **MFA** | Partially built, deliberately gated, and not reachable without Firebase yet | +| **Report export, scheduled scans, multi-cloud** | Not started. No controller, no route, no UI | + +## Page archetypes + +Three, which is enough. Every page in the map is one of them. + +**1. Task page** — the default. Title, one-sentence outcome, prerequisites as a short list, +numbered steps with real command output, then a "when it goes wrong" section. Depth increases +down the page: the first screen is enough for a non-technical reader, the last third is for +someone debugging. + +**2. Explainer** — architecture, security model, permissions. Diagram first, prose second, +a table of specifics last. This is the archetype **How TOPS connects to AWS** already +implements: zoned diagrams, numbered captions, a boundary table, and an honest +read/write permissions split. + +**3. Reference** — glossary, commands, rules, release notes. Scannable, tabular, no +narrative. Assumed to be arrived at from search or a link, never read top to bottom. + +Copyable skeletons, with the guidance inline as comments that do not render: +[`archetypes/task.md`](archetypes/task.md) · +[`archetypes/explainer.md`](archetypes/explainer.md) · +[`archetypes/reference.md`](archetypes/reference.md). + +## House voice + +D-10's decision to structure by task rather than by audience only works if every page is +written the same way. Pages will be written by whoever ships the feature, months apart, so +the rules have to be few enough to remember and specific enough to check. + +**Depth increases down the page.** The first screen serves someone non-technical; the last +third serves someone debugging. This is the mechanism that lets one site serve three +audiences without splitting into tracks — if a page cannot be arranged this way, it is +probably two pages. + +**Write from the reader's side of the screen.** They manage *accounts* and *findings*, not +`aws_accounts` rows and rule evaluations. Name things the way the UI names them. Internal +vocabulary belongs in `docs/`, not here. + +**Symptom before cause.** Troubleshooting entries are reached by searching an error message, +so lead with the literal text the reader has in front of them — then the cause, then the fix. +A section organised by subsystem is unfindable by the person who needs it. + +**Show real output.** Readers compare their screen to the page character by character. +Paraphrased output silently breaks that. + +**Never claim more than the code does.** The child IAM role is not read-only, so no page says +it is. A reader who catches one overstatement stops trusting the rest of the site, and +reviewers are exactly the audience that checks. Every explainer carries a *What we do not +claim* section for this reason. + +**Link, do not restate.** Install commands live in `README.md`; the site links to them. Two +canonical copies drift, and the stale one is the one someone follows. + +**No page for a feature that does not exist.** Verified against `docs/PROGRESS.md`, not +against the roadmap — the roadmap describes intent, `PROGRESS.md` describes the code. + +## Writing order + +Priority in the inventory says what matters; this says what unblocks what. Roughly dependency +order, not importance order. + +1. **`SECURITY.md`** — repo hygiene, blocked by nothing, and the one gap with an active + reason to exist today. +2. **The two evaluator pages** — already written. *How TOPS connects to AWS* is done; *What + the IAM role can do* is a straight extraction from it and the child template. +3. **Reading a finding**, then **Resolving a finding.** In that order: resolution is + meaningless to someone who cannot yet interpret severity and evidence. These two are what + a design partner hits within minutes of a first scan, and nothing currently describes them. +4. **Deploy the site.** Cloudflare Pages, `docs.teemops.com`. Do it once there are four real + pages rather than before — a live site with one page invites filler. +5. **Trim `README.md`'s setup section** to link inward. Only after an install page exists to + link *to*; doing it earlier points at nothing. +6. **Delete `docs/quick-start.md`.** +7. **Everything else, as the feature that needs it ships.** No documentation sprint. + +The one ordering trap: **Configuration reference** looks cheap because the values already +exist in `.env.docker.example` and the compose file. It is not — it is a second copy of +something the code states, and it goes stale silently. Write it late, and only after deciding +whether it can be generated. + +## Sample test: the architecture page as a real Markdown page + +Built and measured, not theorised. `user-docs/start-here/how-tops-connects-to-aws.md` plus +three standalone SVGs in `user-docs/assets/diagrams/`, rendered through a deliberately +minimal shell (python-markdown + ~60 lines of CSS — no framework, which was the point). + +**Verdict: Markdown-first survives. The diagrams need one theme rule and some rework.** + +### What worked + +Plain CommonMark carried the whole page — headings, tables, lists, blockquote callout, +linked images. Nothing needed MDX, a component, or a build step. The three diagrams are +external `.svg` files referenced with `![alt](path.svg)`, which keeps the Markdown readable +and the diagrams independently editable. Each SVG carries its own `prefers-color-scheme` +block, so dark mode works without inheriting page tokens. + +### Two bugs that only appear once the SVG is standalone + +Both would have shipped invisibly if the diagrams had stayed inline in HTML. + +1. **HTML named entities are undefined in XML.** `—`, `×`, `→` are fine in + inline SVG inside an HTML document, because the HTML parser resolves them. A standalone + `.svg` served as `image/svg+xml` is parsed as XML, where only five entities exist — the + file fails to render **entirely**, not partially. All must be numeric references + (`—`). Worth a lint rule if more diagrams follow. +2. **`` inside a CSS comment breaks the parse.** An XML parser reads it as a tag. + Angle brackets cannot appear anywhere in a standalone SVG's CSS, even in comments. + +Validate every diagram with an XML parser before committing. A malformed SVG shows as a +broken image, which reads as a missing file rather than a syntax error. + +### The real finding: diagram width versus prose measure + +The diagrams were first authored at a 1160px viewBox, carried over from a full-bleed HTML +page. That does not survive a docs column. Measured effective body-text size after scaling: + +| Diagram viewBox | Layout | Diagram width | Scale | Body text | Readable | +| --- | --- | --- | --- | --- | --- | +| 1160px | Prose column (68ch), default | 686px | 0.59 | 6.8px | No | +| 1160px | Full content column | 757px | 0.65 | 7.5px | No | +| 1160px | Wide variant, 1680px viewport | 1202px | 1.04 | 11.9px | Yes, but only above ~1600px | +| **900px** | **Standard column, 1440px viewport** | **892px** | **0.99** | **12.4px** | **Yes** | +| **900px** | **Standard column, 1100px viewport** | **767px** | **0.85** | **10.7px** | **Yes** | +| **900px** | **Standard column, 820px viewport** | **765px** | **0.85** | **10.6px** | **Yes** | + +**The diagrams were re-authored at a 900px viewBox** with proportionally larger type (body +12.5px, mono 13px, titles 15.5px) and zones stacked vertically rather than placed side by +side, which is what pays for the larger text. They now read at every desktop width in the +*standard* layout. + +Two rules follow, and they are the whole theme requirement: + +- **Images must not inherit the prose measure.** Markdown wraps an image in a `

`, so + `p { max-width: 68ch }` silently caps every diagram at 686px. One rule fixes it + (`main p:has(> img) { max-width: none }`) and it is the single most important line of CSS + on the site. +- **The theme must collapse its side rails at the usual breakpoints** — on-page TOC below + ~1200px, sidebar below ~900px. Every candidate renderer already does this by default. + +**The earlier recommendation of a wide layout variant for explainer pages is withdrawn.** It +was compensating for over-wide diagrams. With the diagrams sized correctly, the standard +layout is enough, and the site keeps one layout instead of two. + +Below ~800px — phones — no diagram of this density is readable inline, and no amount of +layout work changes that. Every diagram is therefore wrapped in a link to itself, +`[![alt](x.svg)](x.svg)`, plain CommonMark, so one tap opens the standalone SVG and it scales +to the viewport. One line, and it is the right answer on small screens regardless. + +### Guardrails for the next diagram + +Hand-placed SVG at this density does not survive being eyeballed. Both defects found in the +re-authored set were geometric — a 4px card overlap and a text run escaping its card — and +both were caught by loading the SVGs in a browser and comparing every `getBBox()` against its +containing rectangle. Worth keeping as a check when diagrams are added: + +- no text extends past the card that contains it, or past its zone +- no two non-decorative cards overlap +- nothing sits outside the viewBox + +Numbered chips that sit on a boundary-crossing arrow will trip a naive version of that check; +exclude them by class rather than loosening the tolerance. + +### What this settles for tooling + +The page needs: Markdown with tables, static assets served alongside, per-page layout +selection, and a theme whose image CSS is one rule away from correct. That is the whole +requirement list, and every MkDocs- or Docsify-class renderer meets it. **Nothing found here +argues for a framework.** + +## Tooling stays deferred, on purpose + +D-10 said plain Markdown, renderer chosen once there is content. **This design does not +change that**, and the wireframes should not be read as a vote for a framework. + +What the wireframes commit to is only what any renderer in the MkDocs/Docsify class gives +you free: a left sidebar of sections, a content column at a readable measure, an on-page +table of contents, and search. Nothing in the IA needs MDX, React components, or a build +step. The one page with real layout demands — the architecture explainer — is already a +self-contained HTML page with inline SVG, which every candidate renderer can embed. + +**Revisit when:** the first slice is written and the section count or search need actually +bites. Not before. + +## First slice + +D-10's own first slice, sharpened by the inventory above. Ordered by what unblocks the most. + +1. `SECURITY.md` in the repo root — not a site page, and not blocked by any of this. +2. `user-docs/` with the four priority-1 evaluator/user pages: **How TOPS connects to AWS**, + **What the IAM role can do**, **Reading a finding**, **Resolving a finding**. +3. Cloudflare Pages connected, `docs.teemops.com` resolving. +4. `README.md`'s setup section trimmed to a summary that links in, so install has one + canonical copy. +5. Delete `docs/quick-start.md`. + +After that, D-10's rule holds: a page per feature as it ships, written by whoever ships it. +No documentation sprint. diff --git a/design/docs-site/wireframes.html b/design/docs-site/wireframes.html new file mode 100644 index 00000000..04da7697 --- /dev/null +++ b/design/docs-site/wireframes.html @@ -0,0 +1,565 @@ +docs.teemops.com — structure and wireframes + + +

+ +
+
TOPS · user documentation · structure and wireframes
+

docs.teemops.com

+

+ One site, structured by task, serving three audiences without splitting into tracks. + This is the shell, the three page archetypes every page fits, and the full content map — + 34 pages, of which 12 already exist in substance somewhere in the repo. +

+
+ D-10: user-docs/, not docs/ + Cloudflare Pages + Plain Markdown — no framework yet + Marketing site out of scope +
+
+ + +
+
01 · The shell
+

What every page sits inside

+

+ Sidebar of sections, content at a readable measure, on-page contents on the right. Nothing + here needs a framework — it is what any renderer in the MkDocs or Docsify class gives you + out of the box, which is what keeps D-10's "decide the tooling later" honest. + Shown with the page that prompted the site. +

+ +
+
+
+
+
docs.teemops.com/start-here/how-tops-connects-to-aws
+
+
+ + + +
+
Start here / How TOPS connects to AWS
+

How TOPS connects to AWS

+
Written for security reviewers
+

+ TOPS runs on your infrastructure and reads your AWS accounts through cross-account IAM + roles. No stored access keys, no inbound network path, no data path to us. +

+ +
+
Stage 1 — Standing up the parent account
+
zoned SVG · installer → CloudFormation → SNS / SQS / S3
+
+ +
+ +
+
Stage 2 — Connecting child accounts
+
quick-create URL → IAM role → custom resource → your worker
+
+ +
+ +
+
DirectionTrafficCarries
+
outboundAWS API, signed by AssumeRoleRead calls during a scan
+
neverInbound connectionsThere is no listener
+
neverLong-lived access keysRole ARNs only
+
+
+ + + +
+
+
+ +

+ The audience chip is the one non-obvious element. D-10 ruled out splitting the site into + audience tracks, and that is right — but a CISO landing here from a vendor review needs to know + in one glance that this page was written for them, not that they have wandered into an operator + runbook. One chip does that without forking the navigation. +

+
+ + +
+
02 · Page archetypes
+

Three shapes, and every page is one of them

+

+ Constraining to three is what keeps a docs site consistent when pages are written by whoever + shipped the feature, months apart. Depth increases down the page in all three — that is how + one site serves a non-technical reader and an engineer without a track switch. +

+ +
+ +
+
+
Archetype 1 — the default
+
Task page
+
Someone is trying to do a thing right now. Get them to done, then help them when it breaks.
+
+
+
H1 — the task, as a verb
+
One sentence: what you will have at the end
+
Before you start — short list
+
Numbered steps, with real output
+
Numbered steps, with real output
+
When it goes wrong — symptom → cause → fix
+
Next: the page they will need after this
+
+
18 of 34 pages. Install, connect an account, invite a member, restore a backup.
+
+ +
+
+
Archetype 2 — the gate
+
Explainer
+
Someone is deciding whether to trust this. Diagram first, prose second, specifics last.
+
+
+
H1 + audience chip
+
Thesis: the claim, in two sentences
+
Diagram — zoned, with numbered captions
+
Prose: what the diagram does not say
+
Table of specifics — the honest one
+
What we do not claim
+
+
6 pages. How TOPS connects to AWS, the security model, what the IAM role can do.
+
+ +
+
+
Archetype 3 — the lookup
+
Reference
+
Arrived at from search or a link, never read top to bottom. Scannable, tabular, no narrative.
+
+
+
H1 + one line of scope
+
Jump links / filter
+
Table, sorted the way it is looked up
+
Table, sorted the way it is looked up
+
Table, sorted the way it is looked up
+
+
10 pages. Glossary, what TOPS scans, command reference, configuration, release notes.
+
+ +
+
+ + +
+
03 · Content map
+

All 34 pages, and what already exists

+

+ Status is about the content, not the file — "have it" means the substance is written somewhere + in the repo and needs porting and a change of voice, not that a page is ready. Roughly a third of + this site is a rewriting job rather than an authoring job, which is the argument for mapping it + before writing any of it. +

+ +
+ ● Have it — port and rewrite for the reader + ◑ Partial or scattered across files + ○ Write from scratch +
+ +
+ +
+
Start here 6
+
○What TOPS is — and what it is not
+
●How TOPS connects to AWS
+
◑What it costs to run
+
●Install — links README, does not copy
+
●Connect your first AWS account
+
◑Run your first scan
+
+ +
+
Using TOPS 6
+
○The dashboard
+
○Running a scan
+
○Scan profiles and scan types
+
○Reading a finding
+
◑Resolving a finding
+
○Insights
+
+ +
+
AWS accounts 4
+
●Adding an account
+
●What the IAM role can do
+
○Removing an account
+
○Many accounts at once
+
+ +
+
Your organization 4
+
○Organizations and data isolation
+
○Inviting and managing members
+
◑Roles and permissions
+
○Your profile
+
+ +
+
Running the service 6
+
◑Configuration reference
+
●Upgrading and rolling back
+
●Backups and restore
+
●Workers, queues and scan speed
+
●Troubleshooting
+
◑Uninstalling
+
+ +
+
Security 4
+
●The security model
+
○Where your data lives
+
○Reporting a vulnerability
+
●Licence and trademark
+
+ +
+
Reference 4
+
●Glossary
+
●What TOPS scans — services and rules
+
○Command reference
+
●Release notes
+
+ +
+
Not yet —
+
✕Firebase / OAuth sign-in — off by default
+
✕MFA — partial, deliberately gated
+
✕Report export — not started
+
✕Scheduled scans — not started
+
✕Azure, GCP — AWS only
+
+ +
+ +

+ Bold pages are the first slice. Four of them, chosen because they unblock the most: the two + evaluator pages are already written as of this session, and the two finding pages are what a design + partner hits within minutes of their first scan. Everything else waits for the feature that needs it. +

+
+ + +
+
04 · What the map exposed
+

Two gaps that are not docs problems

+ +
+
+
34
+
pages in the map
+

Eight sections. At the point where a sidebar still works without search.

+
+
+
12
+
already written
+

Substance exists in the repo. Needs porting and a change of voice, not authoring.

+
+
+
5
+
partial or scattered
+

True but spread across compose comments, install output and feature specs.

+
+
+
17
+
to write
+

Mostly "Using TOPS" — the half of the product no document currently describes.

+
+
+ +

+ There is no SECURITY.md. A public repo with three open security issues and a CISO-facing + architecture page has nowhere to report a vulnerability. That is repo hygiene, not a docs-site + page, and it should not wait for the site to be built. +

+ +

+ docs/quick-start.md is dead. It describes a Nuxt 3 + Serverless + Prisma monorepo + — the pre-pivot stack — and it is linked from docs/README.md. Anyone who finds it is + actively misled. Delete it rather than port it. +

+
+ + + +
diff --git a/docs/README.md b/docs/README.md index 05cef017..643dceb1 100644 --- a/docs/README.md +++ b/docs/README.md @@ -5,7 +5,7 @@ This directory contains all project documentation organized by topic. ## Quick Links ### Getting Started -- [Quick Start Guide](./quick-start.md) - Get up and running quickly +- [Install TOPS](../README.md#install) - the canonical setup path, in the repository README - [Laravel App Setup](./laravel-app/setup.md) - Detailed setup instructions ### Architecture & Planning @@ -67,7 +67,6 @@ docs/ ├── README.md (this file) ├── architecture.md ├── planning.md -├── quick-start.md ├── PROGRESS.md ├── laravel-app/ # Laravel application documentation │ ├── README.md diff --git a/docs/architecture.md b/docs/architecture.md index dbc50070..77c8172f 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -231,4 +231,4 @@ Future: Integration with AWS Config and Security Hub for additional insights. See the [Roadmap](./roadmap.md) for what's planned and why, [Progress](./PROGRESS.md) for what's built, [AWS Scanner Coverage](./planning.md) for the scanner expansion plan, and -[Quick Start](./quick-start.md) for setup instructions. +the [repository README](../README.md#install) for setup instructions. diff --git a/docs/features/features-spec.md b/docs/features/features-spec.md index ac527437..f18e8370 100644 --- a/docs/features/features-spec.md +++ b/docs/features/features-spec.md @@ -1,6 +1,6 @@ # Feature Specifications -This document contains feature specifications using our [User Story Template](../docs/templates/user-story-template.md). All features follow our [Practices](../docs/practices/) and [Feature Development Process](../docs/processes/feature-development.md). +This document contains feature specifications using our [User Story Template](../templates/user-story-template.md). All features follow our [Practices](../practices/) and [Feature Development Process](../processes/feature-development.md). ## Architecture Context @@ -206,10 +206,10 @@ https://console.aws.amazon.com/cloudformation/home?#/stacks/quickcreate?template - **User Satisfaction**: Account addition process is clear and straightforward ### Related Practices -- [Product Practices](../docs/practices/product.md) - Simplicity First, User Experience -- [Security Practices](../docs/practices/security.md) - Data Protection (Encryption), Secrets Management, API Security -- [Database Practices](../docs/practices/database.md) - Schema Design, Data Integrity -- [Architecture Practices](../docs/practices/architecture.md) - Simplicity First, Standard Patterns +- [Product Practices](../practices/product.md) - Simplicity First, User Experience +- [Security Practices](../practices/security.md) - Data Protection (Encryption), Secrets Management, API Security +- [Database Practices](../practices/database.md) - Schema Design, Data Integrity +- [Architecture Practices](../practices/architecture.md) - Simplicity First, Standard Patterns ### Technical Notes - IAM Role ARN must be encrypted at rest using AES-256 @@ -724,9 +724,9 @@ Flash Message: success - "Successfully logged in via [Provider]!" ### Related Practices -- [Product Practices](../docs/practices/product.md) - User Experience, Simplicity First -- [Frontend Practices](../docs/practices/frontend.md) - Component Design, Accessibility -- [Security Practices](../docs/practices/security.md) - XSS Prevention +- [Product Practices](../practices/product.md) - User Experience, Simplicity First +- [Design Practices](../practices/design.md) - UX principles, component design, accessibility +- [Security Practices](../practices/security.md) - XSS Prevention --- @@ -798,7 +798,7 @@ $schedule->command('aws:process-sqs --once')->everyMinute(); ## Notes -- All features follow our [Practices](../docs/practices/) documents -- Features are developed incrementally following [Feature Development Process](../docs/processes/feature-development.md) -- See [PROGRESS.md](../docs/PROGRESS.md) for detailed implementation status +- All features follow our [Practices](../practices/) documents +- Features are developed incrementally following [Feature Development Process](../processes/feature-development.md) +- See [PROGRESS.md](../PROGRESS.md) for detailed implementation status diff --git a/docs/quick-start.md b/docs/quick-start.md deleted file mode 100644 index 035e1357..00000000 --- a/docs/quick-start.md +++ /dev/null @@ -1,253 +0,0 @@ -# Quick Start Guide - -## Immediate First Steps - -### 1. Prerequisites -- Node.js 18+ installed -- AWS CLI configured (for Serverless deployment) -- Firebase project created -- MySQL database (local or AWS RDS) -- Serverless Framework CLI: `npm install -g serverless` - -### 2. Initialize Monorepo Structure - -```bash -# Create project root -mkdir -p tops/{frontend,backend,shared} -cd tops - -# Initialize frontend (Nuxt 3) -npx nuxi@latest init frontend -cd frontend -npm install - -# Initialize backend (TypeScript Lambda Functions) -cd ../backend -npm init -y -npm install --save-dev typescript @types/node @types/aws-lambda ts-node -npx tsc --init - -# Install Serverless Framework -npm install --save-dev serverless serverless-offline serverless-plugin-typescript -npm install --save-dev @types/serverless - -# Initialize Prisma -npx prisma init -``` - -### 3. Essential Dependencies to Install - -#### Backend (Lambda Functions + Serverless) -```bash -cd backend - -# AWS Lambda Types -npm install --save-dev @types/aws-lambda - -# Firebase Admin SDK -npm install firebase-admin - -# Database -npm install @prisma/client -npm install prisma --save-dev - -# AWS SDK v3 -npm install @aws-sdk/client-s3 @aws-sdk/client-iam @aws-sdk/client-ec2 @aws-sdk/client-rds @aws-sdk/client-sts - -# Validation (lightweight) -npm install zod - -# Encryption (for IAM role storage) -npm install crypto-js -npm install @types/crypto-js --save-dev - -# Serverless (dev dependencies) -npm install --save-dev serverless serverless-offline serverless-plugin-typescript -``` - -#### Frontend (Nuxt 3 + PrimeVue + Firebase) -```bash -cd frontend - -# Core Nuxt -npm install @pinia/nuxt pinia - -# PrimeVue -npm install primevue primeicons -npm install @primevue/themes - -# Firebase -npm install firebase - -# HTTP Client (optional, can use built-in $fetch) -npm install axios -``` - -### 4. Environment Variables Setup - -#### Backend `.env` (for local development) -```env -# Database -DATABASE_URL="mysql://user:password@localhost:3306/cloudsecurity" - -# Firebase Admin SDK -FIREBASE_PROJECT_ID=your-project-id -FIREBASE_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n" -FIREBASE_CLIENT_EMAIL=firebase-adminsdk-xxxxx@your-project.iam.gserviceaccount.com - -# AWS -AWS_REGION=us-east-1 -AWS_PARENT_ACCOUNT_ID=123456789012 - -# Encryption -ENCRYPTION_KEY=your-32-character-encryption-key - -# Serverless (for local) -IS_OFFLINE=true -``` - -#### Frontend `.env` -```env -# Firebase -NUXT_PUBLIC_FIREBASE_API_KEY=your-api-key -NUXT_PUBLIC_FIREBASE_AUTH_DOMAIN=your-project.firebaseapp.com -NUXT_PUBLIC_FIREBASE_PROJECT_ID=your-project-id -NUXT_PUBLIC_FIREBASE_STORAGE_BUCKET=your-project.appspot.com -NUXT_PUBLIC_FIREBASE_MESSAGING_SENDER_ID=123456789 -NUXT_PUBLIC_FIREBASE_APP_ID=1:123456789:web:abcdef - -# API (local development) -NUXT_PUBLIC_API_BASE_URL=http://localhost:3000/dev -``` - -### 5. Firebase Setup - -1. Create Firebase project at https://console.firebase.google.com -2. Enable Authentication: - - Email/Password - - Google (OAuth) - - Apple (OAuth) - - Microsoft (OAuth) -3. Create Service Account: - - Project Settings → Service Accounts - - Generate new private key - - Download JSON file - - Use values in backend `.env` - -### 6. Serverless Framework Configuration - -Create `backend/serverless.yml`: -```yaml -service: cloud-security-api - -frameworkVersion: '3' - -provider: - name: aws - runtime: nodejs18.x - region: us-east-1 - stage: ${opt:stage, 'dev'} - environment: - DATABASE_URL: ${env:DATABASE_URL} - FIREBASE_PROJECT_ID: ${env:FIREBASE_PROJECT_ID} - # ... other env vars - iam: - role: - statements: - - Effect: Allow - Action: - - sts:AssumeRole - - s3:* - - iam:* - - ec2:* - - rds:* - Resource: '*' - -functions: - # Authentication - verifyAuth: - handler: src/handlers/auth.handler.verify - events: - - http: - path: /auth/verify - method: POST - cors: true - - # AWS Accounts - getAwsAccounts: - handler: src/handlers/aws-accounts.handler.getAccounts - events: - - http: - path: /organizations/{orgId}/aws-accounts - method: GET - cors: true - - createAwsAccount: - handler: src/handlers/aws-accounts.handler.create - events: - - http: - path: /organizations/{orgId}/aws-accounts - method: POST - cors: true - - # Scans - createScan: - handler: src/handlers/scans.handler.create - events: - - http: - path: /organizations/{orgId}/scans - method: POST - cors: true - - getScan: - handler: src/handlers/scans.handler.get - events: - - http: - path: /organizations/{orgId}/scans/{scanId} - method: GET - cors: true - - listScans: - handler: src/handlers/scans.handler.list - events: - - http: - path: /organizations/{orgId}/scans - method: GET - cors: true - -plugins: - - serverless-plugin-typescript - - serverless-offline -``` - ---- - -## Confirmed Tech Stack - -- **Monorepo structure** ✅ -- **MySQL** database (AWS RDS) ✅ -- **Prisma** ORM ✅ -- **PrimeVue** for UI (Material Design) ✅ -- **Firebase Authentication** (OAuth preferred) ✅ -- **Serverless Framework** for backend deployment ✅ - ---- - -## What I Can Do Next - -Based on your requirements in `PLANNING.md`, I can now: - -1. ✅ Initialize monorepo structure with Nuxt 3 and Lambda functions -2. ✅ Set up Serverless Framework configuration -3. ✅ Configure Prisma with MySQL -4. ✅ Set up Firebase Authentication (frontend + backend) -5. ✅ Create Firebase Auth utility/middleware for Lambda functions -6. ✅ Set up PrimeVue in Nuxt 3 -7. ✅ Create multi-tenant organization structure -8. ✅ Build AWS account management with cross-account role workflow -9. ✅ Implement AWS scanning service (S3, IAM, EC2, RDS) -10. ✅ Set up GitHub Actions CI/CD -11. ✅ Create initial UI components with PrimeVue - -**Ready to proceed?** I'll start initializing the project structure with all the confirmed requirements! - diff --git a/user-docs/README.md b/user-docs/README.md new file mode 100644 index 00000000..d477119c --- /dev/null +++ b/user-docs/README.md @@ -0,0 +1,114 @@ +# TOPS user documentation + +The source for **docs.teemops.com** — documentation for the people *running* TOPS, as opposed +to `docs/`, which is the contributor and maintainer knowledge base. See +[D-10](../docs/roadmap.md#d-10-in-full-where-user-documentation-lives) for why the two are +kept apart. + +**Status: first slice.** Four pages so far. The structure it will grow into is designed in +[`design/docs-site/information-architecture.md`](../design/docs-site/information-architecture.md) +— 34 pages across 8 sections, with a content map showing what already exists elsewhere in +the repo and needs porting rather than writing. + +## What is here + +| Page | For | +| --- | --- | +| [How TOPS connects to AWS](start-here/how-tops-connects-to-aws.md) | Security reviewers approving TOPS for an AWS organisation | +| [What the IAM role can do](aws-accounts/what-the-iam-role-can-do.md) | The same reviewers, once they ask what the permissions actually grant | +| [Reading a finding](using-tops/reading-a-finding.md) | Whoever has just run their first scan and is looking at the results | +| [Resolving a finding](using-tops/resolving-a-finding.md) | The same person, once they have fixed something and want it off the list | + +## Conventions + +**Plain Markdown, no framework.** The renderer is deliberately undecided until there is +enough content to need navigation and search. Nothing here may depend on renderer-specific +syntax — no MDX, no components, no shortcodes. If a page cannot be written in CommonMark plus +tables, that is a signal about the page, not about the tooling. + +**Install instructions live in `README.md`, not here.** Linking beats copying; two canonical +copies drift. Pages reference it rather than restating commands. + +**No links to pages that do not exist yet.** The IA maps 34 pages; only some are written. A +page may link forward only once the target is real — otherwise the site ships dead links, +which cost more trust than a missing cross-reference. Until a target exists, link the +repository file it will be based on, or leave the link out. Check before committing: + +```bash +python3 - <<'EOF' +import pathlib, re +bad = [] +for md in pathlib.Path("user-docs").rglob("*.md"): + text = re.sub(r"```.*?```", "", md.read_text(), flags=re.S) + for m in re.finditer(r"\]\((?!https?:|#)([^)#]+)(#[^)]*)?\)", text): + if not (md.parent / m.group(1)).resolve().exists(): + bad.append(f"{md}: -> {m.group(1)}") +print("\n".join(bad) or "all relative links resolve") +EOF +``` + +**Do not document what does not exist.** Verified against `docs/PROGRESS.md`. Firebase +sign-in is built but off by default, MFA is gated, and report export, scheduled scans and +multi-cloud are not started — none get a page. + +### Diagrams + +Standalone SVG files in `assets/diagrams/`, referenced as linked images so one click opens +them full size: + +```markdown +[![Description of what the diagram shows](../assets/diagrams/name.svg)](../assets/diagrams/name.svg) +``` + +**Author at a 900px viewBox**, with body text at 12.5px, mono at 13px and titles at 15.5px. +That is what keeps a diagram readable when it is scaled into a docs column — at 900px it +renders near 1:1 on a laptop and stays above 10px down to an 820px viewport. Wider diagrams +do not survive: the same content at a 1160px viewBox scaled to 7.5px body text and was +unreadable. Stack zones vertically rather than side by side; that is what buys the room for +larger type. + +Four rules, learned the hard way when the first three diagrams were extracted from inline +HTML: + +1. **Numeric character references only.** `—`, not `—`. A standalone `.svg` is + parsed as XML, where HTML named entities are undefined and the file fails to render + entirely. +2. **No angle brackets anywhere in the SVG's CSS**, including inside comments — an XML parser + reads them as tags. +3. **Each SVG carries its own theme.** An image-embedded SVG cannot inherit page tokens, so + every diagram needs its own `@media (prefers-color-scheme: dark)` block. +4. **Check the geometry, do not eyeball it.** Both defects in the first set were a card + overlapping another card and a text run escaping its card — invisible in review, obvious + to a script that compares every `getBBox()` against its containing rectangle. + +Validate before committing — a malformed SVG shows as a broken image, which looks like a +missing file rather than a syntax error: + +```bash +python3 -c "import xml.etree.ElementTree as ET,sys; [ET.parse(f) for f in sys.argv[1:]]; print('ok')" user-docs/assets/diagrams/*.svg +``` + +### Writing a new page + +Copy the matching archetype and replace it. The guidance is inline, as comments that do not +render: + +| Archetype | Use for | Template | +| --- | --- | --- | +| **Task** | Someone doing a thing right now | [`task.md`](../design/docs-site/archetypes/task.md) | +| **Explainer** | Someone deciding whether to trust something | [`explainer.md`](../design/docs-site/archetypes/explainer.md) | +| **Reference** | Someone who arrived from search and wants to leave | [`reference.md`](../design/docs-site/archetypes/reference.md) | + +Seven rules, in full in the +[IA document](../design/docs-site/information-architecture.md#house-voice): + +1. **Depth increases down the page** — first screen for a non-technical reader, last third + for someone debugging. This is what lets one site serve three audiences. +2. **Write from the reader's side of the screen** — they manage accounts and findings, not + rows and rule evaluations. +3. **Symptom before cause** — troubleshooting is reached by searching an error message. +4. **Show real output**, not paraphrased output. +5. **Never claim more than the code does.** The child IAM role is not read-only, so no page + says it is. Every explainer carries a *What we do not claim* section. +6. **Link, do not restate.** Two canonical copies drift, and the stale one gets followed. +7. **No page for a feature that does not exist** — check `docs/PROGRESS.md`, not the roadmap. diff --git a/user-docs/assets/diagrams/aws-child-account-linking.svg b/user-docs/assets/diagrams/aws-child-account-linking.svg new file mode 100644 index 00000000..fb54b914 --- /dev/null +++ b/user-docs/assets/diagrams/aws-child-account-linking.svg @@ -0,0 +1,123 @@ + + Linking a child AWS account to TOPS + TOPS generates a CloudFormation quick-create URL. The child account's own administrator creates a stack that makes a cross-account IAM role plus a custom resource. The custom resource publishes the role ARN to your parent SNS topic, your worker long-polls SQS, matches it against a pending record and stores it, then replies to CloudFormation. + + + + + + + + + + + + Your TOPS install + + + Add AWS account + Creates a pending record, then + external_id · random + + + aws:process-sqs + Matches both ids, then stores + the role ARN in your database. + + + Any account you want scanned ×N + + + + + CloudFormation stack + tops-vendor-audit + Created by that account’s own admin, in their own console session. + + + IAM role + Trusts your parent account, + and only with sts:ExternalId + + + Custom resource + Fires at stack create, and + again at delete · sns:Publish + + + Your parent account + + + SQS · teemops_main + Long-polled from inside your + own network. + + + SNS · teemops-sns + Messaging only. No credentials, + no findings, no customer data. + + + + 1 + + + 2 + + + + 3 + + + + 4 + + + 5 + + The link is the only thing that travels outward. What comes back is one message, on one topic, that you own. + diff --git a/user-docs/assets/diagrams/aws-docker-runtime.svg b/user-docs/assets/diagrams/aws-docker-runtime.svg new file mode 100644 index 00000000..b43b43d5 --- /dev/null +++ b/user-docs/assets/diagrams/aws-docker-runtime.svg @@ -0,0 +1,136 @@ + + TOPS Docker Compose runtime and scan fan-out + Six containers on one host: app, mysql, worker, backup, db-restore and maildev. The worker's supervisord pools poll the database queue for scan jobs and SQS for account linking, and reach AWS outbound only, through sts:AssumeRole. + + + + + + + + + + + + One Docker Compose project + + + app + teem/tops:<tag> + nginx + php-fpm on :8080 + No AWS credentials at all + + + mysql 8.0 + volume: mysql-data + Orgs, accounts, findings + — and the job queue + + + worker + same image, different entrypoint + supervisord pools + tops-queue ×1 + default, teemops_audit — orchestrates a scan + tops-region-queue ×5 + teemops_audit_region — does the scanning + tops-account-queue · tops-scheduler + aws:process-sqs, off unless SQS is configured + + + backup + Full 17:00 · diff hourly + ~/.tops/backups + + + Also in the project + maildev on :8090 + db-restore, parked behind a profile + + + AWS — outbound only + + + SQS long-poll + teemops_main + Account linking only. + + + sts:AssumeRole + Role ARN + external id. + Temporary credentials, + held for one job only. + + + + + Child accounts ×N + Describe / List / Get + across every enabled + region. Results come + back to your database. + + + + 1 + + + 2 + + + 3 + + + 4 + + + + One full scan of one account + 1 orchestrator job → ~153 region jobs → 5 workers in parallel + + Nothing listens for AWS. Every arrow leaving this box was started from inside it. + diff --git a/user-docs/assets/diagrams/aws-parent-account-setup.svg b/user-docs/assets/diagrams/aws-parent-account-setup.svg new file mode 100644 index 00000000..bf00129f --- /dev/null +++ b/user-docs/assets/diagrams/aws-parent-account-setup.svg @@ -0,0 +1,122 @@ + + TOPS parent account setup + The installer container on your own machine deploys two CloudFormation stacks into an AWS account you choose, creating an SNS topic, four SQS queues and an S3 bucket, then writes the stack outputs back to generated/teemops.env. + + + + + + + + + + + + Your machine + + + Installer container + ./install.sh --aws + One-shot. Uses your own AWS CLI credentials. + + + Written back to disk + generated/teemops.env + Queue names, ARNs and the template URL. + + + Your parent AWS account + + + CloudFormation + teemops-core-docker + teemops-messaging + Two stacks, one region. + Re-runnable. + + + SNS topic + teemops-sns + The one inbound path. + + + SQS queues + teemops_main + teemops_main_dlq + teemops_audit + teemops_audit_region + + + S3 bucket + <env>-<id>-tops-deploy + Hosts the child-account + template. Public read on + /templates/* only. + + + Why four queues + teemops_main carries the + account-link callbacks, and + _dlq catches them after five + failures. The audit pair is + for SQS-mode scanning: idle. + + + + 1 + + + 2 + + + 3 + + + 4 + + Two stacks, one region, one account. Idle when you are not scanning. + diff --git a/user-docs/aws-accounts/what-the-iam-role-can-do.md b/user-docs/aws-accounts/what-the-iam-role-can-do.md new file mode 100644 index 00000000..4b960640 --- /dev/null +++ b/user-docs/aws-accounts/what-the-iam-role-can-do.md @@ -0,0 +1,153 @@ +# What the IAM role can do + +> **Written for security reviewers.** Every permission the TOPS CloudFormation stack grants in +> your AWS account, what uses it, and how to remove the parts you do not want. No prior +> knowledge of TOPS is assumed. + +**The role TOPS asks for is not read-only.** On top of two AWS-managed read policies it carries +21 inline policies, 18 of which are unscoped, spanning CloudFormation, CloudTrail, CloudWatch +Logs, AWS Config, Security Hub, GuardDuty, Macie and Inspector. + +Scanning uses none of them. Every scan TOPS performs is `list`, `get` or `describe`, and the +write permissions exist for operational features — **none of which are implemented as of +v0.4.0.** The detail is below, along with how to strip them. + +## At a glance + +| | Grants | Resource-scoped | Used by scanning | +| --- | --- | --- | --- | +| **2 managed policies** | Read across all services | n/a | **Yes — this is all scanning uses** | +| **3 inline policies** | Write, narrowly | Yes | No | +| **18 inline policies** | Write | No — `Resource: "*"` | No | + +The stack also creates a **second role** for CloudWatch Events, covered below. + +## The read half — everything a scan uses + +Two AWS-managed policies, attached to the role: + +| Policy | | +| --- | --- | +| `arn:aws:iam::aws:policy/ReadOnlyAccess` | AWS-managed read access across services | +| `arn:aws:iam::aws:policy/ResourceGroupsandTagEditorReadOnlyAccess` | Reads resource groups and tags | + +Scan behaviour is defined in JSON, not code — `app/rules/tasks//tasks.json` — which +means the complete set of API calls TOPS can make is enumerable rather than a matter of trust. +As of v0.4.0 that is **36 operations, of which 35 are `list`, `get` or `describe`, and none +mutate anything.** You can check this yourself; see [Verifying this](#verifying-this). + +## The write half — what none of it is for, yet + +Grouped by service. Nothing in this section is exercised by a scan. + +**Resource-scoped** — these three are constrained to TOPS-named resources: + +| Policy | Actions | Limited to | +| --- | --- | --- | +| `TopsPassRole` | `iam:PassRole` | `arn:aws:iam::*:role/tops*` | +| `TopsS3Bucket` | `s3:` CreateBucket, DeleteBucket, ListBucket, GetBucketLocation, ListAllMyBuckets | `arn:aws:s3:::tops-config-*` | +| `TopsCloudFormationAccess` | `cloudformation:` Create/Update/Delete stack, change sets, GetTemplate | `arn:aws:cloudformation:*:*:stack/tops*` | + +**Unscoped** — the following carry `Resource: "*"`: + +| Service | Actions | +| --- | --- | +| CloudFormation | `ListStacks`, `ListStackResources`, `DescribeStacks` (read; unscoped because these do not support resource-level permissions) | +| CloudTrail | Create, Update, Delete trails · Start/Stop logging · Describe, GetTrailStatus | +| CloudWatch Logs | Create/Delete log groups, streams and metric filters · `PutLogEvents` | +| AWS Config | Put/Delete config rules, configuration recorders, delivery channels, aggregators, conformance packs, remediation configurations and exceptions · Start/Stop the recorder | +| Security Hub | `securityhub:*` | +| GuardDuty | `guardduty:*` | +| Macie | `macie:*` | +| Inspector | `inspector:*` | + +The last four are full service control — enable, disable, configure and delete, including the +findings those services hold. + +## What we do not claim + +- **These write permissions are not used by anything.** TOPS instantiates only STS, SNS and SQS + clients directly, plus read-only clients built from the scan definitions. There is no code in + the repository that calls Security Hub, GuardDuty, Macie, Inspector, AWS Config or CloudTrail. + They are provisioned ahead of features that do not exist yet. **If you would rather not grant + a permission for a feature nobody has built, remove them** — the next section shows how, and + scanning is unaffected. +- **`securityhub:*`, `guardduty:*`, `macie:*` and `inspector:*` are broad by any standard.** + They allow disabling those services and deleting their findings. We are not going to describe + that as least privilege. +- **`s3:ListAllMyBuckets` in `TopsS3Bucket` can never authorise.** It does not support + resource-level permissions, and the statement scopes it to `arn:aws:s3:::tops-config-*`. It is + inert rather than dangerous — but it is there, and you would have found it. +- **We do not claim the role cannot be misused if the parent account is compromised.** Anyone + who can assume it gets everything above. That is what the `ExternalId` condition and your own + control of the parent account are protecting. + +## The second role + +The stack creates `TopsCWEventRole` as well, which is easy to miss: + +- **Trusted by** `events.amazonaws.com`, not by TOPS +- **Grants** `events:PutEvents` to the default event bus in the TOPS parent account only +- **Purpose** forwarding EC2 state-change events + +It holds no read access to your account and cannot be assumed by TOPS. + +## How the role is protected + +The trust policy allows exactly one principal — the TOPS parent account you were given — and +only when the caller presents a matching `ExternalId`: + +```yaml +Principal: + AWS: +Condition: + StringEquals: + "sts:ExternalId": +``` + +The `ExternalId` is generated per account and is what stops a different TOPS install, or anyone +who learns your account id, from assuming the role. It is the +[AWS-recommended defence](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html) +against the confused deputy problem. + +**You can revoke access at any time** by deleting the CloudFormation stack. TOPS is told, and +the account unlinks itself. + +## Removing permissions you do not want + +The template is a plain CloudFormation file in an S3 bucket **in your own account**, put there +by your own installer. Nothing stops you editing it before you hand out the link. + +1. Download it from the URL in the quick-create link, or take it from + [`templates/iam.role.child.account.cfn.yaml`](https://github.com/teemops/tops/blob/develop/templates/iam.role.child.account.cfn.yaml). +2. Delete the `AWS::IAM::Policy` resources you do not want. For scanning only, that is every one + of them — keep the `TeemOps` role, its two managed policies and `TopsCustomNotifier`. +3. Upload it back over the same S3 key, or point `TOPS_CFN_TEMPLATE_URL` at your edited copy. + +Scanning keeps working. Anything that later needs the write permissions will fail loudly rather +than silently, because AWS returns an explicit `AccessDenied`. + +## Verifying this + +Do not take the page's word for it. Against a linked account: + +```bash +aws iam list-attached-role-policies --role-name +aws iam list-role-policies --role-name +``` + +And to confirm the claim that scanning only reads — every API call TOPS can make is declared in +JSON, so grep for a mutating verb and expect nothing: + +```bash +grep -rhoE '"(task|start)": "[a-z][a-zA-Z]+"' app/rules/tasks/*/tasks.json | sort -u +``` + +## Related + +- [How TOPS connects to AWS](../start-here/how-tops-connects-to-aws.md) — the wider picture, and what crosses each boundary +- [Reporting a security issue](https://github.com/teemops/tops/blob/develop/SECURITY.md) — including what is already known +- [The template itself](https://github.com/teemops/tops/blob/develop/templates/iam.role.child.account.cfn.yaml) + +*Source of truth for this page: `templates/iam.role.child.account.cfn.yaml` and +`app/rules/tasks/*/tasks.json`. Checked against v0.4.0.* diff --git a/user-docs/start-here/how-tops-connects-to-aws.md b/user-docs/start-here/how-tops-connects-to-aws.md new file mode 100644 index 00000000..9db4adc4 --- /dev/null +++ b/user-docs/start-here/how-tops-connects-to-aws.md @@ -0,0 +1,169 @@ +# How TOPS connects to AWS + +> **Written for security reviewers.** If you have been asked to approve TOPS for your AWS +> organisation, this page is the whole answer. No prior knowledge of TOPS is assumed. + +TOPS runs on your infrastructure and reads your AWS accounts through cross-account IAM roles. +There are no stored access keys, no inbound network path to the install, and no data path to +us. This page shows exactly what gets created, who creates it, and what crosses each boundary. + +The diagrams use three zones throughout: + +| Zone | What it is | +| --- | --- | +| **Your infrastructure** | The server you run TOPS on. Owns the database, the findings, and the only copy of the role ARNs | +| **Your parent AWS account** | One account you nominate. Holds an SNS topic, SQS queues and one S3 bucket. Deployed once | +| **Child accounts** | Every account you want scanned. Each gets one CloudFormation stack and one IAM role | + +--- + +## Stage 1 — Standing up the parent account + +*Run once · `./install.sh --aws` · one region* + +The installer runs as a throwaway container on your own machine, using your own AWS CLI +credentials. It deploys two CloudFormation stacks into an account you choose, then writes the +resulting queue names and ARNs back to disk. Nothing here is shared with other installs. + +[![Diagram: the installer container on your machine deploys two CloudFormation stacks into your parent AWS account, creating an SNS topic, four SQS queues and an S3 bucket, then writes the outputs back to generated/teemops.env.](../assets/diagrams/aws-parent-account-setup.svg)](../assets/diagrams/aws-parent-account-setup.svg) + +1. **Deploy.** The installer checks `sts:get-caller-identity`, confirms the account and region + with you, then runs `sam deploy`. +2. **Create.** CloudFormation creates the topic, the queues, the dead-letter queue and the + bucket. You own all of it. +3. **Subscribe.** `teemops-sns` is subscribed to `teemops_main`, so a notification becomes a + queue message. +4. **Report back.** Stack outputs are written to `generated/teemops.env` and loaded by the + containers on restart. + +Two of the four queues — `teemops_audit` and `teemops_audit_region` — are created but stay +idle. Scans run on the database queue by default; those two exist for operators who want to +move scanning onto SQS. + +**This step is optional.** TOPS runs without it — you just cannot link an account yet. You can +come back later with `./install.sh --aws-only`. Everything above is removed by deleting two +CloudFormation stacks. + +--- + +## Stage 2 — Connecting child accounts + +*Repeat per account · no limit · no per-account infrastructure in the parent* + +TOPS never asks for credentials to a child account. It hands the account's own administrator a +CloudFormation quick-create link, pre-filled with an `ExternalId` generated for that account +alone. The administrator creates the stack in their own console, under their own session. + +[![Diagram: TOPS generates a CloudFormation quick-create URL; the child account admin creates a stack that makes a cross-account IAM role plus a custom resource; the custom resource publishes the role ARN to the parent SNS topic; the TOPS worker long-polls SQS, matches it and stores it, then replies to CloudFormation.](../assets/diagrams/aws-child-account-linking.svg)](../assets/diagrams/aws-child-account-linking.svg) + +1. **Hand over a link.** A console quick-create URL carrying four parameters: your parent + account id, the region, the account's `ExternalId` and its `UniqueId`. +2. **The admin creates the stack.** In their account, with their permissions. TOPS has no + session there and cannot create it for them. +3. **The stack calls home.** A custom resource publishes the new role ARN, external id and + unique id to your SNS topic, which fans into `teemops_main`. +4. **Your worker picks it up.** Outbound long-poll from inside your network. The message is + accepted only if both ids match a record you already created. +5. **The stack completes.** TOPS replies to the CloudFormation `ResponseURL`; the admin sees + `CREATE_COMPLETE`, or a failure with a reason. + +**Why there is no account limit.** Linking adds one stack in the child account and one database +row in yours — no queue, no topic, no bucket, no per-account resource in the parent. The tenth +account and the thousandth cost the same to connect. Deleting the stack fires the same custom +resource in reverse and unlinks it. + +--- + +## Stage 3 — What actually runs + +*One `docker-compose.yml` · one host · `docker compose up -d`* + +TOPS is a Laravel monolith and a MySQL database, split across containers by role. The web tier +never talks to AWS. The worker tier does — outbound only, assuming a role per scan and holding +the credentials for the life of one job. + +[![Diagram: the Docker Compose project runs app, mysql, worker, backup, db-restore and maildev containers. The worker's supervisord pools poll the database queue for scan jobs and SQS for account linking, and reach AWS through sts:AssumeRole to make read calls against each child account.](../assets/diagrams/aws-docker-runtime.svg)](../assets/diagrams/aws-docker-runtime.svg) + +1. **The UI enqueues, it does not scan.** Starting a scan writes a job row. The web tier holds + no AWS credentials at all. +2. **Workers poll the database.** Scan jobs use the database queue driver, with + `SELECT … FOR UPDATE SKIP LOCKED` so five workers never collide. +3. **Account linking polls SQS.** The only pool that needs AWS messaging, and it stays switched + off until `TOPS_SQS_ARN` is set. +4. **Scanning assumes the role.** One `AssumeRole` per job, temporary credentials, read calls, + results written home. + +A full scan of one account fans out to roughly 153 region jobs across five worker processes. +Raise `TOPS_WORKER_PROCESSES` only with the memory and the AWS API rate limits to match. + +--- + +## What crosses the boundary + +| Direction | Traffic | Carries | When | +| --- | --- | --- | --- | +| **Outbound** | HTTPS to AWS API endpoints, signed with credentials from `sts:AssumeRole` | Describe / List / Get calls against the account being scanned | During a scan | +| **Outbound** | SQS `ReceiveMessage` / `DeleteMessage` long-poll | Nothing — an empty poll until a stack is created | Continuously, if AWS messaging is set up | +| **Into your AWS** | One `sns:Publish` from the child account's CloudFormation stack | Role ARN, external id, unique id | Once per account, at link and at unlink | +| **Never** | Inbound connections to the TOPS host | There is no listener. AWS is polled, not called back | — | +| **Never** | Long-lived AWS access keys | None are requested, entered, or stored | — | +| **Never** | Findings, inventory or account metadata leaving your install | Scan output is written to your MySQL and stays there | — | + +There is no telemetry endpoint, no licence check, and no hosted control plane in this path. The +container image is the only thing you pull from outside, and you can build it yourself with +`install-build.sh`. + +--- + +## What the child-account role can actually do + +Worth being precise about, because it is the first question every security reviewer asks. **The +role is not read-only.** Scanning uses only the read half — and as of v0.4.0 **nothing uses the +write half**, which is provisioned ahead of operational features that are not built yet. +[What the IAM role can do](../aws-accounts/what-the-iam-role-can-do.md) goes through it policy +by policy, including how to strip the parts you do not want. + +**Read — used by every scan** + +- `ReadOnlyAccess` (AWS managed) +- `ResourceGroupsandTagEditorReadOnlyAccess` + +This is the whole permission set the scanner exercises. + +**Write — only for the operational features** + +- `securityhub:*`, `guardduty:*`, `macie:*`, `inspector:*` +- `config:Put*` / `Delete*` — rules, recorders, delivery channels, conformance packs +- `cloudtrail:` create, update, delete, start/stop logging +- `logs:` create/delete log groups, streams, metric filters +- `cloudformation:` create, update, delete, change sets +- `s3:CreateBucket` / `DeleteBucket` (for AWS Config delivery) +- `iam:PassRole` → `arn:aws:iam::*:role/tops*` + +**If you only want scanning, remove the write policies.** The template is a plain +CloudFormation file in your own S3 bucket. Strip the policies you do not want before you +publish the link — the scanner keeps working, because it never used them. + +--- + +## The shape of it, in numbers + +| | | +| --- | --- | +| **2** stacks in the parent | Deployed once. Delete them and the integration is gone | +| **1** stack per child account | Created by that account's own admin. Revocable by them at any time | +| **0** stored AWS keys | Cross-account roles with a per-account external id, assumed per job | +| **0** inbound ports | Every connection is opened from inside your network | +| **~153** region jobs per full scan | Fanned across five worker processes by default | +| **6** containers on one host | app, worker, mysql, backup, maildev, and a restore container that stays parked | + +--- + +## Related + +- [Installing TOPS](https://github.com/teemops/tops#install) — the commands, kept in one place in the repository +- [The child-account CloudFormation template](https://github.com/teemops/tops/blob/develop/templates/iam.role.child.account.cfn.yaml) — every policy this page summarises, in full +- [Licence and trademark](https://github.com/teemops/tops/blob/develop/LICENSE) — Apache-2.0, with the name held separately + +*Source of truth for this page: `templates/iam.role.child.account.cfn.yaml`, +`infra/cloud-stack/`, and `docker-compose.yml` in the TOPS repository.* diff --git a/user-docs/using-tops/reading-a-finding.md b/user-docs/using-tops/reading-a-finding.md new file mode 100644 index 00000000..8203a2cf --- /dev/null +++ b/user-docs/using-tops/reading-a-finding.md @@ -0,0 +1,87 @@ +# Reading a finding + +What each part of a finding means, and how to get from a list of them to the handful that +matter today. + +A **finding** is one rule failing against one resource in one AWS account. It is a lasting +record, not a line in a scan report: the same misconfiguration on the same bucket is the same +finding next week, with the same history, whether you have scanned once or fifty times. + +## Before you start + +- At least one AWS account connected, and one completed scan + +## The anatomy of a finding + +| | | +| --- | --- | +| **Severity** | `critical`, `high`, `medium` or `low`. Set by the rule, not by your environment | +| **Title** | What failed, in a few words — *IAM User MFA Token* | +| **Service** | The AWS service the rule covers — `iam`, `s3`, `rds`, `ec2`, and seven others | +| **Resource** | The specific thing that failed: the resource id and its type. This is what makes it *your* finding rather than a generic warning | +| **Description** | What the rule checks, and why it matters | +| **Remediation** | What to do about it. Every rule has one | +| **Recommendation** | Sometimes: a grouped piece of guidance covering several related rules, with steps and links to AWS documentation | +| **Status** | `open`, `resolved` or `ignored` — see [Resolving a finding](resolving-a-finding.md) | + +## What the severities mean + +Severity comes from the rule and does not change based on your environment. TOPS ships 74 +rules: + +| | Rules | What it means | +| --- | --- | --- | +| **Critical** | 4 | Exploitable now, with wide blast radius. All four concern the root account — access keys existing, or MFA missing | +| **High** | 24 | A real weakness that an attacker could use as a step, or a control that is off when it should be on | +| **Medium** | 32 | Hygiene and hardening. Worth fixing, rarely worth paging anyone | +| **Low** | 14 | Best practice and tidiness | + +Severity is not the same as urgency. A `medium` on a resource that faces the internet may +matter more than a `high` on something isolated — the finding tells you which resource, so +you can make that call. + +## Narrowing the list + +The findings list opens on everything currently open, which for a first scan of a real +account is usually a lot. Four filters cut it down, and they combine: + +1. **Service** — pills across the top, each with a count. Clicking the active one clears it. +2. **Benchmark** — `basic`, `cis` or `pci`. Findings that belong to no benchmark are not + counted under any of them, so the benchmark counts will not add up to the total. +3. **AWS account** — when you have more than one connected. +4. **Status** — `open` by default. Resolved findings are hidden unless you ask for them. + +**Filters live in the address bar.** Whatever you have narrowed to can be bookmarked or +pasted to a colleague, and they will land on the same view. This is the fastest way to hand +someone their share of the work. + +## A reasonable first pass + +1. Filter to **critical**, and deal with those first. There are only four rules that can + produce one, and all four are about the root account. +2. Then **high**, filtered to one service at a time. Working service by service beats working + down the list, because the fix is often the same for every resource in a group. +3. Leave **medium** and **low** until the first two are empty. They will still be there. + +## When it does not say what you expect + +**A finding I fixed is still listed.** +Findings update when a scan looks again. Run a new scan for that account; if the rule now +passes, TOPS resolves the finding itself. See [Resolving a finding](resolving-a-finding.md). + +**The benchmark counts do not add up to the total.** +Expected. A finding only counts under a benchmark it belongs to, and many rules belong to +none. + +**A service I use has no findings at all.** +Either nothing failed, or TOPS does not cover that service yet — it currently has rules for +IAM, S3, RDS, EC2, CloudTrail, KMS, Lambda, SQS, SNS, DynamoDB and ELBv2. An uncovered +service looks exactly like a clean one, which is worth knowing. + +**Two findings look identical.** +Check the resource id. One rule failing against two resources is two findings, and they are +resolved independently. + +## Next + +- [Resolving a finding](resolving-a-finding.md) — fixing, ignoring, and what the next scan does diff --git a/user-docs/using-tops/resolving-a-finding.md b/user-docs/using-tops/resolving-a-finding.md new file mode 100644 index 00000000..34e18225 --- /dev/null +++ b/user-docs/using-tops/resolving-a-finding.md @@ -0,0 +1,98 @@ +# Resolving a finding + +How a finding leaves your open list, what the next scan does to that decision, and when TOPS +will overrule you. + +The short version: **fix it in AWS and let a scan confirm.** Marking things resolved by hand +works, but it is a claim rather than evidence, and evidence wins. + +## Before you start + +- A finding you have looked at — see [Reading a finding](reading-a-finding.md) +- Permission to change the resource in AWS, or someone who has it + +## The three statuses + +| Status | Means | Set by | +| --- | --- | --- | +| **Open** | Failing as of the last scan that looked | TOPS, when the rule first fails | +| **Resolved** | No longer failing, or you say it is dealt with | TOPS after a scan, or you | +| **Ignored** | You have decided this one does not apply | You | + +Resolved findings are hidden from the list by default. They are not deleted — filter by +status to see them. + +## Fixing it properly + +1. **Read the remediation on the finding.** Every rule has one, and for the critical and high + rules it is specific enough to act on rather than a restatement of the problem. +2. **Make the change in AWS.** TOPS does not change anything in your accounts — it has no + code that writes to them at all. +3. **Run a new scan** for that account. +4. **The finding resolves itself.** When a scan examines that resource and the rule now + passes, TOPS marks it resolved, records the reason as *fixed*, and timestamps it. You do + not have to tell it. + +This is the path worth taking. The finding's history then says the rule passed on a given +date, which is a different and better claim from someone having ticked a box. + +## Marking it resolved yourself + +Use this when the fix is real but TOPS cannot see it — a compensating control it does not +know about, or a resource being decommissioned this week. + +The finding is recorded as resolved with the reason *manual*, which stays distinguishable +from *fixed* forever. Nobody reading it later has to guess which kind of resolved it was. + +**A later scan can overrule you.** If a scan finds the rule still failing on that resource, +the finding is **reopened** — status back to `open`, reason cleared. This is deliberate: +evidence that something is still failing is more useful than a record of someone's earlier +opinion, and a silent stale "resolved" is exactly how a real problem gets lost. + +## Ignoring it + +Use this when the rule genuinely does not apply — a bucket that is public because it serves a +public website, an account that will never have MFA because it has no console users. + +**Ignored is respected.** A later scan finding it *still failing* leaves it ignored. That is +the difference between ignored and resolved: ignored is a standing judgement about the rule, +resolved is a claim about the state of the world, and only the second one can be contradicted +by evidence. + +If a scan later finds the resource **compliant**, the finding is resolved and leaves your +open list, which is usually what you want. + +## What a scan will and will not touch + +| Situation | What happens | +| --- | --- | +| Open, still failing | Stays open | +| Open, now passing | **Resolved**, reason *fixed* | +| Resolved by you, still failing | **Reopened** | +| Resolved by you, now passing | Stays resolved | +| Ignored, still failing | **Stays ignored** | +| Ignored, now passing | Resolved | +| The scan did not examine that service or region at all | **Untouched.** Not resolved, not reopened | + +That last row matters more than it looks. A scan that skipped a service tells you nothing +about it, so TOPS does not treat silence as good news. A finding only changes when a scan +actually looked. + +## When it does not behave + +**I fixed it, scanned, and it is still open.** +Check the scan covered the right service and region — a scan of a subset does not touch +findings elsewhere. Check the resource id on the finding matches the resource you changed. + +**A finding I resolved came back.** +It was reopened because a scan found the rule still failing on that resource. Look at the +resource rather than the finding: something has been reverted, or the fix did not apply to +the resource TOPS is checking. + +**I want it gone permanently.** +There is no delete. Ignore is the durable version of that decision, and it survives scans. + +## Next + +- [Reading a finding](reading-a-finding.md) — severities, filters, and the anatomy +- [How TOPS connects to AWS](../start-here/how-tops-connects-to-aws.md) — why TOPS cannot fix things for you