From 5d9e6d9fc6231db08c29402cb9e66b086fd36242 Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Mon, 3 Aug 2026 12:22:53 -0400 Subject: [PATCH 01/12] Add Environment Sync guide section --- app/app.config.ts | 4 + .../14.environment-sync/.navigation.yml | 1 + content/guides/14.environment-sync/0.index.md | 93 ++++++++++++ .../1.installation-and-profiles.md | 73 +++++++++ .../guides/14.environment-sync/2.pulling.md | 122 +++++++++++++++ .../3.diffing-and-pushing.md | 86 +++++++++++ .../4.ci-and-automation.md | 53 +++++++ .../5.secrets-and-limitations.md | 65 ++++++++ .../14.environment-sync/6.common-workflows.md | 143 ++++++++++++++++++ 9 files changed, 640 insertions(+) create mode 100644 content/guides/14.environment-sync/.navigation.yml create mode 100644 content/guides/14.environment-sync/0.index.md create mode 100644 content/guides/14.environment-sync/1.installation-and-profiles.md create mode 100644 content/guides/14.environment-sync/2.pulling.md create mode 100644 content/guides/14.environment-sync/3.diffing-and-pushing.md create mode 100644 content/guides/14.environment-sync/4.ci-and-automation.md create mode 100644 content/guides/14.environment-sync/5.secrets-and-limitations.md create mode 100644 content/guides/14.environment-sync/6.common-workflows.md diff --git a/app/app.config.ts b/app/app.config.ts index a74846f2..1f5e268a 100644 --- a/app/app.config.ts +++ b/app/app.config.ts @@ -168,6 +168,10 @@ export default defineAppConfig({ to: '/guides/deployments', icon: 'directus-deployments', }, + { + label: 'Environment Sync', + to: '/guides/environment-sync', + }, { label: 'Security', to: '/guides/security/best-practices', diff --git a/content/guides/14.environment-sync/.navigation.yml b/content/guides/14.environment-sync/.navigation.yml new file mode 100644 index 00000000..b8061f24 --- /dev/null +++ b/content/guides/14.environment-sync/.navigation.yml @@ -0,0 +1 @@ +title: Environment Sync diff --git a/content/guides/14.environment-sync/0.index.md b/content/guides/14.environment-sync/0.index.md new file mode 100644 index 00000000..6778af60 --- /dev/null +++ b/content/guides/14.environment-sync/0.index.md @@ -0,0 +1,93 @@ +--- +stableId: 393b999d-c4ed-4b83-9208-cf088a4918ab +title: Overview +description: Move schema and configuration between Directus instances through files you commit to git, using the Directus CLI. +--- + +Environment Sync moves **schema and configuration** between Directus instances (development to staging, staging to production) through JSON files you commit to git. It ships as part of the Directus CLI (`directus-cli`, or its short alias `d6s`). + +`pull` writes files from a source instance, `diff` previews what a push would change, and `push` applies the files to a target: + +```bash +d6s sync pull --from staging # snapshot schema + configuration into committed files +d6s sync diff --to production # read-only preview of what a push would change +d6s sync push --to production # apply schema, then import configuration records +d6s sync # interactive wizard: pull, then push +``` + +Because the files live in your repository, your normal review workflow applies: schema changes show up in pull requests, environments converge through git history, and a bad change is a revert away from being found. The files are the record; restoring a database still requires a backup. + +## What syncs + +A pull touches two axes, and the CLI reports each: + +``` +Schema 24 collections → directus/default/schema +Resources 206 records in 10 resources → directus/default/data +``` + +| Axis | What it covers | Default | Scope it with | +| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- | --------------------------------------------------------- | +| **Schema** | Every collection, field, and relation, including custom fields on system collections and collection folders. | Full snapshot | `--collections` / `--exclude-collections` / `--no-schema` | +| **Configuration resources** | Records of `directus_*` configuration tables: roles, policies, access, permissions, flows, operations, dashboards, panels, settings, and media-library folders. | 10 resource types | `--` / `--no-` / `--all` | +| **Users** | Accounts, with all secret columns stripped. | Opt-in (`--users`) | `--users` or `--all` | +| **Translations** | Custom translation strings. | Opt-in (`--translations`) | `--translations` or `--all` | + +Your own collections' **content**, the rows in the tables you create, is not synced. Environment Sync is for the _shape_ of a project and its configuration, not its data. + +::callout{icon="material-symbols:info-outline-rounded"} +Content sync is deferred to a future release. Cross-instance record identity for integer primary keys, file references, and user references needs its own architecture, and a wrong guess there means overwritten data on the target. +:: + +## The committed files + +Artifacts land in a directory you commit: one JSON file per collection, written deterministically so a re-pull with no changes is byte-identical and diffs only ever show real changes: + +``` +directus// + schema/ # the schema snapshot, split per collection + data/ # configuration records per resource + id_map.json # committed source → target record identity map +``` + +The `metadata.json` manifest in each directory records which files the CLI owns. The CLI never deletes a file it did not write. + +## Safety model + +Environment Sync is built around a small set of promises: + +- **`diff` applies nothing.** It previews the schema change and dry-runs the data import server-side, then rolls back. +- **Deletions are gated.** Only `mirror` mode deletes, and deleting always requires explicit consent: `--dangerously-allow-delete` in automation, or typing the profile name interactively. `--yes` never authorizes a deletion. +- **Identity is never guessed.** Records are matched across instances by the committed id map and natural keys. An ambiguous match prompts in a terminal and refuses in CI. It is never resolved by picking the first candidate. +- **Stored secrets never land in committed files.** Built-in secret columns and fields you mark concealed, hashed, or encrypted are stripped at export. One warned exception: custom headers in flow request operations export verbatim. +- **Failures are loud.** Hand-edited files, corrupt manifests, truncated fetches, and version mismatches stop the command with a named error rather than degrading silently; an export the source itself curtailed is marked incomplete and refused at mirror push. A full re-pull converges the files again. + +## Next steps + +::card-group + +:::card{title="Installation & Profiles" icon="i-ph-download-simple" to="/guides/environment-sync/installation-and-profiles"} +Install the CLI, define an instance profile, and store credentials safely. +::: + +:::card{title="Pulling" icon="i-ph-download" to="/guides/environment-sync/pulling"} +Snapshot schema and configuration, and scope what a pull exports. +::: + +:::card{title="Diffing & Pushing" icon="i-ph-upload" to="/guides/environment-sync/diffing-and-pushing"} +Preview and apply changes, choose a push mode, and resolve record identity. +::: + +:::card{title="CI & Automation" icon="i-ph-robot" to="/guides/environment-sync/ci-and-automation"} +Run sync non-interactively with JSON reports and explicit gates. +::: + +:::card{title="Secrets & Limitations" icon="i-ph-shield-check" to="/guides/environment-sync/secrets-and-limitations"} +How secret values are handled, and what Environment Sync deliberately does not do. +::: + +:::card{title="Common Workflows" icon="i-ph-map-trifold" to="/guides/environment-sync/common-workflows"} +Promote changes to production, recover from drift, and stand up a new environment. +::: + +:: diff --git a/content/guides/14.environment-sync/1.installation-and-profiles.md b/content/guides/14.environment-sync/1.installation-and-profiles.md new file mode 100644 index 00000000..21051d62 --- /dev/null +++ b/content/guides/14.environment-sync/1.installation-and-profiles.md @@ -0,0 +1,73 @@ +--- +stableId: 77e3530c-eb59-49be-acfc-0532942aebab +title: Installation & Profiles +description: Install the Directus CLI, define a profile for each instance you sync with, and store credentials safely outside your repository. +--- + +Environment Sync connects to your instances through **profiles**: named entries like `staging` or `production` that pair an instance URL with a credential. The URLs are project configuration you commit; the credentials never are. + +## Install the CLI + + + +```bash +npm install -g @directus/cli +``` + +Verify the install: + +```bash +d6s --version +``` + +## The project file + +Profiles live in `directus.config.json` in your project root, next to the directories a pull writes. Adding a profile records its name and URL there. The credential is stored separately. + +::callout{icon="material-symbols:info-outline-rounded"} +**Safe to commit** +`directus.config.json` never contains tokens, so committing it cannot leak credentials. Commit it so everyone on the project resolves the same profile names. +:: + +Because the URL is the part of a profile that gets committed, one rule is enforced: + +::callout{icon="material-symbols:warning-rounded" color="warning"} +**Credential-bearing URLs are refused** +A URL like `https://admin:secret@staging.example.com` embeds a credential in a value that lands in `directus.config.json`. The CLI refuses it. Tokens belong in the credential store or the environment, never in a committed file. +:: + +## Where credentials live + +When a command needs to authenticate against a profile, the credential resolves in order: + +1. A `--token` flag, on the commands that accept one (`profile add` and `profile test`). +2. A `DIRECTUS__TOKEN` environment variable. For a profile named `staging`, that's `DIRECTUS_STAGING_TOKEN`. +3. The saved credential store at `~/.directus/credentials.json`, written with owner-only file permissions (mode `0600`). + +In CI (any environment where the `CI` variable is set) the credential store is never consulted. The sync commands take no `--token` flag, so in CI their tokens come from the environment variables. + +## Adding a profile + +```bash +d6s profile add +``` + +The prompts walk you through it: name the profile, give it a URL, and authenticate by pasting a static token or logging in with your email and password to save a session. Saved sessions are refreshed automatically before requests when they are close to expiring. + +You can also pass everything directly: + +```bash +d6s profile add staging --url https://staging.example.com --token +``` + +## Testing a profile + +```bash +d6s profile test staging +``` + +This connects to the instance and prints who you are on it, confirming the URL is reachable and the credential works. Like the sync commands, it refreshes an expiring saved session. + +## Next step + +With a profile added and tested, snapshot your first instance: [Pulling](/guides/environment-sync/pulling). diff --git a/content/guides/14.environment-sync/2.pulling.md b/content/guides/14.environment-sync/2.pulling.md new file mode 100644 index 00000000..5b55978e --- /dev/null +++ b/content/guides/14.environment-sync/2.pulling.md @@ -0,0 +1,122 @@ +--- +stableId: 9ab17c77-208a-4db3-bbbb-fdb911f6130b +title: Pulling +description: Snapshot schema and configuration from a source instance into files you commit. Scope a pull by resource or by collection, or skip schema entirely for configuration-only projects. +--- + +Pulling snapshots a source instance into the committed files that `diff` and `push` later apply: + +```bash +d6s sync pull --from staging +``` + +## What a default pull exports + +A pull touches two axes, and the success output reports each: + +``` +◇ Pulled from staging — https://staging.example.com + Schema 24 collections → directus/default/schema + Resources 206 records in 10 resources → directus/default/data +``` + +Exact counts depend on your project. + +- **Schema**: a full snapshot of every collection, field, and relation, including custom fields on system collections and collection folders. +- **Resources**: records of 10 `directus_*` configuration resource types, listed below. + +**Users** and **translations** are excluded by default. Opt in with `--users` and `--translations`, or `--all`. + +## Configuration resources + +Resource selection follows a dependency graph: selecting a resource pulls in what it needs. + +| Resource | In default pull | Select directly | Pulls in | Notes | +| -------------- | ---------------------------- | ---------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | +| `roles` | Yes | `--roles` | `policies` | | +| `policies` | Yes | `--policies` | `access`, `permissions` | See the warning below before selecting policies on their own. | +| `access` | Yes, with roles and policies | — | — | Grants attached to users are dropped when users are out of scope; a mirror push does not delete them on the target. | +| `permissions` | Yes, with policies | — | — | Row counts are verified against the server. If the source hides rows (unlicensed custom permission rules), the export is marked incomplete. | +| `flows` | Yes | `--flows` | `operations` | | +| `operations` | Yes, with flows | — | — | | +| `dashboards` | Yes | `--dashboards` | `panels` | | +| `panels` | Yes, with dashboards | — | — | Panels have no natural key, so a first push into a matching target can duplicate once; the id map prevents repeats. | +| `settings` | Yes | `--settings` | — | Singleton. License and AI credentials, branding images (logos, backgrounds, favicon), and the default storage folder are stripped. | +| `folders` | Yes | `--folders` | — | The media-library folder tree. Distinct from collection folders (Data Studio sidebar groups), which sync as schema. | +| `users` | Opt-in | `--users` | `roles`, `policies` | Secret columns (`password`, `token`, `tfa_secret`, and others) are stripped. | +| `translations` | Opt-in | `--translations` | — | Mirror pushes of translations are not currently supported, which is why they are opt-in. | + +::callout{icon="material-symbols:warning-rounded" color="warning"} +**Select `--roles`, not `--policies` alone** +A selection that pulls policies without roles is not independently pushable when access rows reference roles: access rows carry role foreign keys, and with no roles in scope a push to a fresh target fails. Select `--roles` instead; it pulls policies and their children too. +:: + +## Selecting resources + +Three ways to change the default set: + +- **Positive selection**: `--flows --roles` narrows the pull to those resources plus their dependencies. +- **Subtraction**: `--no-flows` keeps the default set and removes flows. +- **Everything**: `--all` adds `users` and `translations` on top of the default set. + +Positive selections cannot be combined with `--all` or with `--no-` subtractions. + +Resource selection changes only the resources axis: a pull with `--flows` still snapshots the full schema. Schema has its own scoping. + +## Scoping the schema + +```bash +d6s sync pull --from staging --collections articles,authors +``` + +The Schema line reads `(scoped to: articles, authors)` and only those collections' schema files refresh. `--exclude-collections` inverts the scope: snapshot everything except the named collections. + +Two warnings a scoped pull can raise: + +- **Out-of-scope references.** If the scoped snapshot points at something you omitted (a relation target, a group parent, a many-to-any collection), the pull warns, because pushing that snapshot to a fresh target can fail. It warns; it never widens the scope for you. Add the missing collections to `--collections` yourself. +- **A name the server didn't return.** A `--collections` name absent from the returned snapshot (usually a typo) draws a warning naming the gap. The partial snapshot still commits, but never silently. + +## Skipping schema entirely + +Schema and resources are independent axes, and resource selection never narrows the schema snapshot. To make a configuration-only project explicit, opt out of schema: + +```bash +d6s sync pull --from staging --no-schema +``` + +To make it permanent, set `"schema": false` on the project in `directus.config.json`. Such a project carries no schema authority: pull skips the snapshot, and push and diff for that project never touch schema. Reports say `schemaSkipped`, so automation can tell a skipped phase from a matching one. Combining `"schema": false` with a collections scope is refused as a contradiction. + +## What a pull touches + +Two rules govern every pull, scoped or not: + +1. A pull only rewrites what it fetched. Everything it did not fetch keeps its committed bytes, untouched. +2. A push only applies what is committed. Work that never entered the repository cannot ship. + +Refreshed means the file is rewritten from the source; because writes are deterministic, an unchanged resource produces no git diff. Preserved means the file is not touched at all. + +| Pull | Schema files | Configuration files | +| ------------------------------------ | ------------------------------------------------------- | ------------------------------------------------ | +| `pull --from staging` | All refreshed | All refreshed | +| `... --collections posts` | `posts` refreshed, others preserved | All refreshed | +| `... --no-flows` | All refreshed | Flows preserved, others refreshed | +| `... --flows` | All refreshed (resource selection never narrows schema) | Flows and operations refreshed, others preserved | +| `... --flows --no-schema` | All preserved | Flows and operations refreshed, others preserved | +| `... --collections posts --no-flows` | `posts` refreshed, others preserved | Flows preserved, others refreshed | + +## Determinism + +Re-running a pull with no instance changes produces byte-identical files and a clean working tree, so `git diff` after a pull shows exactly what changed on the instance and nothing else: no timestamps, no reordering noise. + +Scoped pulls are just as predictable: a scoped or resource-selected pull refreshes only its subset and preserves every other committed file. Scope limits what a pull may replace; it never deletes the rest of your snapshot. + +## Pull before you push + +::callout{icon="material-symbols:warning-rounded" color="warning"} +**The committed tree is what a push applies** +A scoped pull refreshes only its scope, so the rest of the tree keeps whatever it last knew, and a later mirror push would apply that stale state to the target. Pull before you push. +:: + +## Next step + +With files committed, preview what they would change on a target: [Diffing & Pushing](/guides/environment-sync/diffing-and-pushing). diff --git a/content/guides/14.environment-sync/3.diffing-and-pushing.md b/content/guides/14.environment-sync/3.diffing-and-pushing.md new file mode 100644 index 00000000..587011ae --- /dev/null +++ b/content/guides/14.environment-sync/3.diffing-and-pushing.md @@ -0,0 +1,86 @@ +--- +stableId: a1896d05-d465-450c-b2d1-dd8ccef64ad5 +title: Diffing & Pushing +description: Preview what a push would change with a read-only diff, then apply it to a target instance by choosing a push mode, passing the deletion gates, and resolving record identity. +--- + +With a snapshot [pulled](/guides/environment-sync/pulling) and committed, `diff` previews what applying it to a target would change, and `push` applies it. Both commands read the committed files: what you push is what is in git, not what is currently on the source instance. + +## Previewing with diff + +```bash +d6s sync diff --to production +``` + +`diff` applies nothing. It previews the schema change, then has the target server dry-run the data import and roll it back, so the data plan is the server's own answer, not a client-side guess. + +Every preview opens by naming the target instance and what the mode would mean (`merge — additive, no deletions`), then lists schema changes line by line and the data plan per collection (`+N new ~N updated`). + +One thing a diff never does is guess: a committed record that could match more than one target record is reported as **unresolved**, not previewed as a create. An interactive push resolves it by asking you; a push in CI refuses. See [record identity](#record-identity) below. + +## Push modes + +```bash +d6s sync push --to production # merge (the default) +d6s sync push --to production --mode mirror # deletes, behind the gates below +``` + +| Mode | Schema | Data | Deletes? | +| ----------------- | ------------------- | ----------------------------------------------------------- | -------------- | +| `add` | Additive | Inserts only; existing records are never updated | No | +| `merge` (default) | Additive | Creates and updates | No | +| `mirror` | May delete in scope | Creates, updates, and deletes records absent from the files | **Yes, gated** | + +::callout{icon="material-symbols:warning-rounded" color="warning"} +**Pull before a mirror push.** The committed tree is what a push applies. A scoped pull refreshes only its scope, so the rest of the tree keeps whatever it last knew, and a `mirror` push applies that stale state to the target. Run a full pull first. +:: + +## Deletion gates + +Only `mirror` deletes, and deleting always requires its own explicit consent: + +| Context | To delete you must | +| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | +| Interactive terminal | Review the plan naming the losses, then type the profile name (unless you passed `--dangerously-allow-delete`, which is the consent) | +| Non-interactive / CI | Pass `--dangerously-allow-delete` | + +`--yes` skips the ordinary confirmation prompt, but it never authorizes a deletion. A `mirror` push in CI without `--dangerously-allow-delete` refuses before changing anything. + +## How a push runs + +A push applies in two phases, schema first: + +1. **Schema.** The schema change applies first, sealed with a hash of the target's schema. If the target changed between planning and applying, the push stops instead of applying a stale plan. +2. **Data.** Configuration records import once the schema is in place. + +The two phases are not one transaction. If the data import fails after the schema applied, run the push again: the schema now matches, so the re-run applies data alone. + +## Record identity + +The same role or flow carries different primary keys on each instance, so records have to be matched across them. Two mechanisms decide which target record a committed record is: + +- **The committed identity map.** Each push records source-to-target pairs in `directus//id_map.json`. Commit it; it is how the next push updates a record instead of creating a duplicate. +- **Natural keys.** A record not yet in the map is matched to a target record by a natural key: its name for most resources, its email for a user, its key for an operation. + +Ambiguity is never resolved by guessing. When two target records could both be the match, the CLI prompts you to choose in a terminal, and refuses in CI. + +The first push into a target seeded from the same template can ask a few of these identity questions about pairs of records that plausibly are the same one. That is expected: answer them, the answers land in the identity map, and the questions do not come back. It converges in one pass. + +## Version matching + +Schema changes require the snapshot's Directus version and the target's version to match exactly, patch release included. A mismatch refuses the command and names both versions. Align the instances (re-pull if the source was upgraded), or pass `--allow-version-drift` to override the gate, which proceeds with a loud warning. The CLI does not translate schema between versions. + +## Convergence + +A push that applied cleanly leaves nothing behind: + +```bash +d6s sync push --to production +# → schema and data match; nothing to push. +``` + +Re-running a completed push is safe, and the clean state is verified against the target, not assumed. If a connection drops mid-import and the result is unknown, the CLI says so. Run `d6s sync diff` before retrying rather than risking a blind retry. + +::callout{icon="material-symbols:info-outline-rounded" to="/guides/environment-sync/ci-and-automation"} +Running diff and push unattended? CI & Automation covers non-interactive behavior, tokens, and JSON reports. +:: diff --git a/content/guides/14.environment-sync/4.ci-and-automation.md b/content/guides/14.environment-sync/4.ci-and-automation.md new file mode 100644 index 00000000..c0e33fbf --- /dev/null +++ b/content/guides/14.environment-sync/4.ci-and-automation.md @@ -0,0 +1,53 @@ +--- +stableId: d5ad7a4c-dd80-4394-af25-333dabfeeaf1 +title: CI & Automation +description: Run Environment Sync unattended with tokens from environment variables, machine-readable JSON reports on stdout, and explicit flags in place of prompts. +--- + +Every sync command can run unattended. The contract in CI is deliberately strict: no prompts, no guessing, and nothing destructive without an explicit flag. + +## No prompts + +Where the interactive CLI would ask a question, a non-interactive run refuses instead: + +- An **ambiguous record match** (two target records that could both be the committed one) fails the command rather than picking a candidate. Resolve it once with an interactive push; the answer lands in the committed identity map and CI runs cleanly after that. +- **Deletions** happen only with `--dangerously-allow-delete`. A `mirror` push without it refuses before changing anything on the target. +- `--yes` confirms an ordinary, non-destructive apply. It never authorizes a deletion. + +## Credentials + +Pass tokens through environment variables named `DIRECTUS__TOKEN`: + +```bash +DIRECTUS_PRODUCTION_TOKEN=$PROD_TOKEN d6s sync push --to production --yes +``` + +The credential store saved on a developer machine is never read in CI; tokens come from the environment only. + +## JSON reports + +Add `--json` and stdout carries exactly one machine-readable report per command; pull, diff, and push each emit their own. Warnings (stripped secret fields, version drift, flow headers exported verbatim) still go to stderr, so your logs keep them while stdout stays parseable. + +A diff whose records are ambiguous reports them as `unresolved` and counts them into its `changes`. A non-interactive push refuses that state, so an unresolved diff is a real difference for your pipeline to surface, not noise. + +## A typical pipeline + +```bash +# On a schedule: refresh the committed snapshot from the source instance. +# Commit the result; a clean git status means nothing changed. +DIRECTUS_STAGING_TOKEN=$STAGING_TOKEN d6s sync pull --from staging --json + +# In pull request checks: preview what merging would apply to the target. +DIRECTUS_PRODUCTION_TOKEN=$PROD_TOKEN d6s sync diff --to production --json + +# On merge: apply. +DIRECTUS_PRODUCTION_TOKEN=$PROD_TOKEN d6s sync push --to production --yes --json +``` + +## Exit behavior + +Commands exit non-zero on any refusal or failure (an ambiguous match, a missing deletion flag, a version mismatch, a failed import), so a pipeline step fails visibly instead of continuing past a partial sync. + +::callout{icon="material-symbols:info-outline-rounded" to="/guides/environment-sync/diffing-and-pushing"} +The push modes and deletion gates these flags control are covered in Diffing & Pushing. +:: diff --git a/content/guides/14.environment-sync/5.secrets-and-limitations.md b/content/guides/14.environment-sync/5.secrets-and-limitations.md new file mode 100644 index 00000000..6404f397 --- /dev/null +++ b/content/guides/14.environment-sync/5.secrets-and-limitations.md @@ -0,0 +1,65 @@ +--- +stableId: ffd6edef-6cd2-499f-b807-5e9e6e1ef513 +title: Secrets & Limitations +description: How Environment Sync keeps secret values out of the files you commit, the one exception to review before publishing a repository, and what the tool deliberately does not do. +--- + +## How secrets are handled + +A [pull](/guides/environment-sync/pulling) exports real records: the settings row, user accounts, flow definitions. If any of those carried a secret value, it would land in JSON files you commit, and git history keeps it forever, in every clone of the repository. Environment Sync strips secret values at export: + +- **Built-in secret columns** (password hashes, tokens, 2FA seeds, license and AI keys) are always deleted from the export. +- **Fields you created and marked concealed, hashed, or encrypted** are deleted too. Every pull fetches the server's complete field list and drops any flagged value, printing a line naming each dropped field. A field missing from the export reads as protection, not data loss. + +Because the field list is fetched separately from the schema, the check also covers pulls scoped to a few collections and pulls that skip schema entirely. And if the check itself cannot run, the pull stops rather than continue unprotected. + +The field definition still syncs as schema: the column and its concealed setting arrive on the target. Only the value stays behind; set the real secret on each instance directly. + +### Why strip instead of sync + +The server never hands out the real value for these fields: concealed fields read back as `**********`, hashed fields as the hash. Exporting that and pushing it would overwrite the target's working secret with a mask. Stripping protects both the repository and the target. + +### The one blind spot + +::callout{icon="material-symbols:warning-rounded" color="warning"} +**Flow request headers export verbatim.** A secret pasted into free-form flow configuration (most commonly an Authorization header in a request operation) has no "this is secret" marker for the CLI to check, and legitimate headers have to sync. The pull warns you by operation name, and the value goes into the committed file as-is. Review those files before committing, and before publishing a repository. +:: + +## System collections that do not sync + +None of these sync today. Some are shared configuration that a future release could take on; the rest are per-instance data that a sync should never touch. + +| Collection | Why it doesn't sync | +| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | +| `directus_presets` | Bookmarks, saved layouts, and Insights presets mix shared configuration with personal preference. A future release may sync the shared part. | +| `directus_extensions` | The enabled/disabled state of installed extensions is tied to what is physically deployed on each instance. | +| `directus_files` | File-interface **fields** sync as schema; file rows and the binaries behind them are their own workstream. | +| `directus_comments` | Content comments; per-instance data. | +| `directus_activity` | The audit log; per-instance data. | +| `directus_revisions` | Change history; per-instance data. | +| `directus_versions` | Content-versioning drafts; per-instance data. | +| `directus_notifications` | User notifications; per-instance data. | +| `directus_shares` | Public share links; per-instance data. | +| `directus_sessions` | Active login sessions; per-instance data. | +| `directus_migrations` | The database migration ledger. The CLI never runs migrations or changes the Directus version. | +| `directus_webhooks` | Deprecated in Directus; superseded by flows. | + +## What it does not do + +| Won't | Because / instead | +| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Sync your collections' **content** (rows) | Environment Sync moves the shape of a project and its configuration. Content sync is deferred to a future release. | +| **Undo** a push | Git covers configuration; only a database backup covers data. Back up before big pushes. | +| **Auto-expand** a scoped snapshot | A scope pulls exactly what you name. Dangling references produce a warning; widen `--collections` yourself. | +| Translate schema **across versions** | A version mismatch refuses the command; `--allow-version-drift` overrides the gate but converts nothing. | +| Wrap schema and data in **one transaction** | Schema applies first, then data. A failed import re-runs data alone; see [Diffing & Pushing](/guides/environment-sync/diffing-and-pushing). | +| Select **individual records** | Resource selection is by type (`--roles`), never by row. | +| Model **code-first** | Model in the Data Studio and pull. Don't hand-author the JSON files. | +| **Seal the data plan** against target drift | Only the schema apply is sealed against the target changing between preview and apply. The data preview is the server's own dry-run answer, but it is advisory. | + +## Known limitations + +- **Translations mirror is opt-in.** A server limitation currently breaks `mirror` pushes of translations, so translations are excluded from pulls by default. Opt in with `--translations`; `merge` and `add` pushes work normally. +- **Unlicensed custom permission rules are invisible to the export.** On an instance without a license, the API hides custom permission rules, so a pull cannot export them. The pull detects the shortfall and marks the export incomplete: `merge` and `add` push normally, `mirror` refuses. License the source instance to export them. +- **Users without an email address fail import.** The server's import validation rejects a user with no email, so a `--users` push containing one fails before any data applies. Give the account an email on the source, or remove it, first. +- **Panels can duplicate on a first sync into a look-alike target.** Panels have no name to match on, so pushing into a target that already holds equivalent panels (seeded from the same template) can create them a second time. The identity map prevents repeats after that first push. diff --git a/content/guides/14.environment-sync/6.common-workflows.md b/content/guides/14.environment-sync/6.common-workflows.md new file mode 100644 index 00000000..3a7b792d --- /dev/null +++ b/content/guides/14.environment-sync/6.common-workflows.md @@ -0,0 +1,143 @@ +--- +stableId: b0bd4f6a-0708-4681-86dc-c482895372c1 +title: Common Workflows +description: End-to-end walkthroughs of the most common Environment Sync workflows, from promoting changes to production to recovering a drifted environment and standing up a new one. +--- + +These workflows cover most day-to-day use of Environment Sync: promoting changes to production (all of them, or just the ones that are ready), re-aligning an environment after out-of-band changes, and standing up a new environment from the committed files. The mechanics behind each step live in [Pulling](/guides/environment-sync/pulling), [Diffing & Pushing](/guides/environment-sync/diffing-and-pushing), and [CI & Automation](/guides/environment-sync/ci-and-automation). + +## Promote changes from development to production + +You model in the Data Studio on a development instance. Production only ever receives what git has reviewed. + +1. Make your changes on the development instance: collections, fields, flows, permissions. + +2. Pull them into the repository and commit: + + ```bash + d6s sync pull --from dev + git add directus/ + git commit -m "Add author bio fields" + ``` + + The files are deterministic, so the commit shows your change and nothing else. On a development instance other people also use, a scoped pull (`--collections posts`, or a resource flag like `--flows`) keeps their in-progress work out of your diff. + +3. Open a pull request. Reviewers read the change as plain JSON diffs. A CI check can add the target's view of the same change: + + ```bash + d6s sync diff --to production --json + ``` + +4. On merge, apply: + + ```bash + d6s sync push --to production --yes + ``` + + The default `merge` mode creates and updates but never deletes. A second run reports nothing to push. + +::callout{icon="material-symbols:warning-rounded" color="warning"} +**Removals need `mirror`.** A change that deletes a field or a record does not propagate under `merge`. Push with `--mode mirror` and pass its [deletion gate](/guides/environment-sync/diffing-and-pushing#deletion-gates), and run a full pull first so the mirror applies current state, not a stale tree. +:: + +## Promote only the changes that are ready + +A shared development or staging instance usually carries more than one piece of work at a time. Say the `posts` changes are ready to ship, and half-finished `authors` changes are not. Scope the pull to what ships: + +```bash +d6s sync pull --from staging --collections posts +``` + +What that pull just did: + +- The `posts` schema files were rewritten with the new state. +- The `authors` schema files were not touched. They still hold what the last full pull captured, which is the state production already runs. The half-finished `authors` work never entered the repository. +- Configuration resources refreshed too, because a collections scope only narrows schema. Nothing changed there, so those files came back byte-identical and `git status` shows only the `posts` files. If a flow had also changed on staging, its file would show up here; hold it back with `--no-flows` on the pull. + +Commit, then diff and push as usual: + +```bash +git add directus/ +git commit -m "Add post fields" +d6s sync diff --to production +d6s sync push --to production +``` + +A push always applies the whole committed folder. The `posts` change applies. The `authors` files already match production, so nothing happens there. And since the unfinished `authors` work is not in the repository, it cannot ship, no matter what state staging is in. + +For a configuration-only change, flip the scope: select the resource type and skip schema. + +```bash +d6s sync pull --from staging --flows --no-schema +``` + +The [What a pull touches](/guides/environment-sync/pulling#what-a-pull-touches) table shows exactly which files each combination refreshes and which it leaves alone. + +::callout{icon="material-symbols:warning-rounded" color="warning"} +**Selection is by collection and resource type, not by record.** `--collections posts` can separate finished posts work from unfinished authors work, but two changes inside the same collection travel together, and `--flows` takes every flow, not just one. If unrelated work shares a collection or a resource type, ship it together or wait. And while the committed tree holds a partial picture, avoid `mirror`: it would apply the stale remainder too. +:: + +## Rebase an environment from production after drift + +Sometimes production changes outside the deployment path, usually an urgent manual fix. Staging and the repository no longer reflect reality. Bring the fix into git, then re-align the lower environment. This is what `mirror` is for: converging an environment to the committed state exactly. + +1. Pull the full state from production. The out-of-band change appears as an ordinary git diff, which is your record of what the hotfix actually was: + + ```bash + d6s sync pull --from production + git diff + git add directus/ + git commit -m "Adopt production hotfix" + ``` + +2. Preview what re-aligning staging would mean: + + ```bash + d6s sync diff --to staging + ``` + + Read the deletions closely. Anything that exists only on staging and falls inside the sync's scope is on the list. + +3. Converge staging to the committed state: + + ```bash + d6s sync push --to staging --mode mirror + ``` + + Interactively, the push names the losses and asks you to type the profile name; in automation it requires `--dangerously-allow-delete`. See [deletion gates](/guides/environment-sync/diffing-and-pushing#deletion-gates). + +::callout{icon="material-symbols:warning-rounded" color="warning"} +**Mirror removes staging-only work in scope.** If staging holds experiments you want to keep, pull that work into a branch first, or use `merge` and clean up by hand. +:: + +## Stand up a new environment + +Going from an empty instance to a working copy of your project's shape: + +1. Provision a fresh Directus instance and create its admin account as usual. + +2. Add a profile for it: + + ```bash + d6s profile add staging --url https://staging.example.com --token + ``` + +3. Preview, then push the committed files: + + ```bash + d6s sync diff --to staging + d6s sync push --to staging + ``` + + Schema applies first, then the configuration records import. + +4. Commit the updated `id_map.json`. The push records which target record each committed record became, and that map is how every later push updates records instead of duplicating them. + +Two things to expect on a first push: + +- **Identity questions.** If the target already holds records the CLI cannot tell apart from the committed ones (two policies named "Administrator" is the classic case), an interactive push asks you to choose. Answer once, commit the map, and the questions do not come back. See [record identity](/guides/environment-sync/diffing-and-pushing#record-identity). +- **Secrets stay behind.** Stripped values (API keys in settings, concealed fields, flow credentials) never travel with the files. Set them on the new instance directly. + +::callout{icon="material-symbols:info-outline-rounded"} +Push the whole committed tree to a fresh target, not a scoped slice. A partial snapshot whose relations point at collections outside the scope can fail to apply on an instance that has nothing else yet. The pull warns about these references when it writes the files. +:: From 8a231105a7d27738d380d35dd9bd8fb6619b9c82 Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Tue, 4 Aug 2026 12:35:14 -0400 Subject: [PATCH 02/12] Restructure Environment Sync docs around the user journey --- content/guides/14.environment-sync/0.index.md | 80 ++-- .../1.installation-and-profiles.md | 73 ---- .../14.environment-sync/1.quickstart.md | 248 ++++++++++++ .../14.environment-sync/2.how-it-works.md | 100 +++++ .../guides/14.environment-sync/2.pulling.md | 122 ------ ...mon-workflows.md => 3.common-workflows.md} | 92 ++++- .../3.diffing-and-pushing.md | 86 ----- .../4.ci-and-automation.md | 126 ++++-- .../5.secrets-and-limitations.md | 51 +-- .../guides/14.environment-sync/6.reference.md | 358 ++++++++++++++++++ 10 files changed, 936 insertions(+), 400 deletions(-) delete mode 100644 content/guides/14.environment-sync/1.installation-and-profiles.md create mode 100644 content/guides/14.environment-sync/1.quickstart.md create mode 100644 content/guides/14.environment-sync/2.how-it-works.md delete mode 100644 content/guides/14.environment-sync/2.pulling.md rename content/guides/14.environment-sync/{6.common-workflows.md => 3.common-workflows.md} (50%) delete mode 100644 content/guides/14.environment-sync/3.diffing-and-pushing.md create mode 100644 content/guides/14.environment-sync/6.reference.md diff --git a/content/guides/14.environment-sync/0.index.md b/content/guides/14.environment-sync/0.index.md index 6778af60..66e725d8 100644 --- a/content/guides/14.environment-sync/0.index.md +++ b/content/guides/14.environment-sync/0.index.md @@ -4,90 +4,62 @@ title: Overview description: Move schema and configuration between Directus instances through files you commit to git, using the Directus CLI. --- -Environment Sync moves **schema and configuration** between Directus instances (development to staging, staging to production) through JSON files you commit to git. It ships as part of the Directus CLI (`directus-cli`, or its short alias `d6s`). - -`pull` writes files from a source instance, `diff` previews what a push would change, and `push` applies the files to a target: +You build your data model on a development instance, and at some point that work has to reach staging and production. Environment Sync makes that a git workflow: the Directus CLI (`directus-cli`, or its short alias `d6s`) writes an instance's **schema and configuration** to JSON files you commit, and applies those files to any instance you point it at. ```bash -d6s sync pull --from staging # snapshot schema + configuration into committed files -d6s sync diff --to production # read-only preview of what a push would change -d6s sync push --to production # apply schema, then import configuration records -d6s sync # interactive wizard: pull, then push +d6s sync pull --from staging # write the instance's schema + configuration to files +d6s sync diff --to production # preview what pushing those files would change +d6s sync push --to production # apply them ``` -Because the files live in your repository, your normal review workflow applies: schema changes show up in pull requests, environments converge through git history, and a bad change is a revert away from being found. The files are the record; restoring a database still requires a backup. - -## What syncs +Because the files live in your repository, your normal review workflow applies: schema changes show up in pull requests, environments are brought up to date through git history, and a bad change is a revert away from being found. (`d6s sync` on its own runs a small interactive wizard that pulls and pushes in one pass.) -A pull touches two axes, and the CLI reports each: +The CLI is built to be safe to point at production: `diff` never applies anything, deletions always require their own explicit consent, and a record that could match more than one target record is asked about, never guessed. [How It Works](/guides/environment-sync/how-it-works) covers the full safety model. -``` -Schema 24 collections → directus/default/schema -Resources 206 records in 10 resources → directus/default/data -``` +## What syncs -| Axis | What it covers | Default | Scope it with | -| --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------- | --------------------------------------------------------- | -| **Schema** | Every collection, field, and relation, including custom fields on system collections and collection folders. | Full snapshot | `--collections` / `--exclude-collections` / `--no-schema` | -| **Configuration resources** | Records of `directus_*` configuration tables: roles, policies, access, permissions, flows, operations, dashboards, panels, settings, and media-library folders. | 10 resource types | `--` / `--no-` / `--all` | -| **Users** | Accounts, with all secret columns stripped. | Opt-in (`--users`) | `--users` or `--all` | -| **Translations** | Custom translation strings. | Opt-in (`--translations`) | `--translations` or `--all` | +- **Schema**: every collection, field, and relation, including custom fields on system collections. +- **Configuration**: roles, policies, access, permissions, flows, operations, dashboards, panels, settings, and media-library folders. +- **Opt-in**: user accounts (with every secret column stripped) and translation strings. -Your own collections' **content**, the rows in the tables you create, is not synced. Environment Sync is for the _shape_ of a project and its configuration, not its data. +Your own collections' **content**, the rows in the tables you create, is not synced. Environment Sync moves the shape of a project and its configuration, not its data. ::callout{icon="material-symbols:info-outline-rounded"} Content sync is deferred to a future release. Cross-instance record identity for integer primary keys, file references, and user references needs its own architecture, and a wrong guess there means overwritten data on the target. :: -## The committed files +## Before you start -Artifacts land in a directory you commit: one JSON file per collection, written deterministically so a re-pull with no changes is byte-identical and diffs only ever show real changes: +- **Both instances must run the same Directus version, patch release included.** The server refuses cross-version schema changes because some patch releases change the schema format. Align your environments before you sync them. +- **An admin credential for each instance.** The schema and configuration endpoints the CLI uses are admin-only. A static token from an admin user is the usual choice. +- **A git repository.** The files the CLI writes are only useful committed. Any repository works; many teams use the one that already holds their Directus deployment configuration. -``` -directus// - schema/ # the schema snapshot, split per collection - data/ # configuration records per resource - id_map.json # committed source → target record identity map -``` - -The `metadata.json` manifest in each directory records which files the CLI owns. The CLI never deletes a file it did not write. - -## Safety model - -Environment Sync is built around a small set of promises: - -- **`diff` applies nothing.** It previews the schema change and dry-runs the data import server-side, then rolls back. -- **Deletions are gated.** Only `mirror` mode deletes, and deleting always requires explicit consent: `--dangerously-allow-delete` in automation, or typing the profile name interactively. `--yes` never authorizes a deletion. -- **Identity is never guessed.** Records are matched across instances by the committed id map and natural keys. An ambiguous match prompts in a terminal and refuses in CI. It is never resolved by picking the first candidate. -- **Stored secrets never land in committed files.** Built-in secret columns and fields you mark concealed, hashed, or encrypted are stripped at export. One warned exception: custom headers in flow request operations export verbatim. -- **Failures are loud.** Hand-edited files, corrupt manifests, truncated fetches, and version mismatches stop the command with a named error rather than degrading silently; an export the source itself curtailed is marked incomplete and refused at mirror push. A full re-pull converges the files again. - -## Next steps +## Where to go ::card-group -:::card{title="Installation & Profiles" icon="i-ph-download-simple" to="/guides/environment-sync/installation-and-profiles"} -Install the CLI, define an instance profile, and store credentials safely. +:::card{title="Quickstart" icon="i-ph-rocket-launch" to="/guides/environment-sync/quickstart"} +Run the full pull, diff, push loop against two throwaway instances and see every command's output. ::: -:::card{title="Pulling" icon="i-ph-download" to="/guides/environment-sync/pulling"} -Snapshot schema and configuration, and scope what a pull exports. +:::card{title="How It Works" icon="i-ph-lightbulb" to="/guides/environment-sync/how-it-works"} +The mental model: files as the source of truth, record identity, push phases, and the safety rules. ::: -:::card{title="Diffing & Pushing" icon="i-ph-upload" to="/guides/environment-sync/diffing-and-pushing"} -Preview and apply changes, choose a push mode, and resolve record identity. +:::card{title="Common Workflows" icon="i-ph-map-trifold" to="/guides/environment-sync/common-workflows"} +Promote changes, ship only what's ready, adopt sync on an existing project, roll back, recover from drift. ::: :::card{title="CI & Automation" icon="i-ph-robot" to="/guides/environment-sync/ci-and-automation"} -Run sync non-interactively with JSON reports and explicit gates. +Post the production diff on pull requests and push on merge, with tokens and JSON reports. ::: :::card{title="Secrets & Limitations" icon="i-ph-shield-check" to="/guides/environment-sync/secrets-and-limitations"} -How secret values are handled, and what Environment Sync deliberately does not do. +How secret values are kept out of your repository, and what Environment Sync deliberately does not do. ::: -:::card{title="Common Workflows" icon="i-ph-map-trifold" to="/guides/environment-sync/common-workflows"} -Promote changes to production, recover from drift, and stand up a new environment. +:::card{title="Reference" icon="i-ph-list-magnifying-glass" to="/guides/environment-sync/reference"} +Every command, flag, table, and report format in one place. ::: :: diff --git a/content/guides/14.environment-sync/1.installation-and-profiles.md b/content/guides/14.environment-sync/1.installation-and-profiles.md deleted file mode 100644 index 21051d62..00000000 --- a/content/guides/14.environment-sync/1.installation-and-profiles.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -stableId: 77e3530c-eb59-49be-acfc-0532942aebab -title: Installation & Profiles -description: Install the Directus CLI, define a profile for each instance you sync with, and store credentials safely outside your repository. ---- - -Environment Sync connects to your instances through **profiles**: named entries like `staging` or `production` that pair an instance URL with a credential. The URLs are project configuration you commit; the credentials never are. - -## Install the CLI - - - -```bash -npm install -g @directus/cli -``` - -Verify the install: - -```bash -d6s --version -``` - -## The project file - -Profiles live in `directus.config.json` in your project root, next to the directories a pull writes. Adding a profile records its name and URL there. The credential is stored separately. - -::callout{icon="material-symbols:info-outline-rounded"} -**Safe to commit** -`directus.config.json` never contains tokens, so committing it cannot leak credentials. Commit it so everyone on the project resolves the same profile names. -:: - -Because the URL is the part of a profile that gets committed, one rule is enforced: - -::callout{icon="material-symbols:warning-rounded" color="warning"} -**Credential-bearing URLs are refused** -A URL like `https://admin:secret@staging.example.com` embeds a credential in a value that lands in `directus.config.json`. The CLI refuses it. Tokens belong in the credential store or the environment, never in a committed file. -:: - -## Where credentials live - -When a command needs to authenticate against a profile, the credential resolves in order: - -1. A `--token` flag, on the commands that accept one (`profile add` and `profile test`). -2. A `DIRECTUS__TOKEN` environment variable. For a profile named `staging`, that's `DIRECTUS_STAGING_TOKEN`. -3. The saved credential store at `~/.directus/credentials.json`, written with owner-only file permissions (mode `0600`). - -In CI (any environment where the `CI` variable is set) the credential store is never consulted. The sync commands take no `--token` flag, so in CI their tokens come from the environment variables. - -## Adding a profile - -```bash -d6s profile add -``` - -The prompts walk you through it: name the profile, give it a URL, and authenticate by pasting a static token or logging in with your email and password to save a session. Saved sessions are refreshed automatically before requests when they are close to expiring. - -You can also pass everything directly: - -```bash -d6s profile add staging --url https://staging.example.com --token -``` - -## Testing a profile - -```bash -d6s profile test staging -``` - -This connects to the instance and prints who you are on it, confirming the URL is reachable and the credential works. Like the sync commands, it refreshes an expiring saved session. - -## Next step - -With a profile added and tested, snapshot your first instance: [Pulling](/guides/environment-sync/pulling). diff --git a/content/guides/14.environment-sync/1.quickstart.md b/content/guides/14.environment-sync/1.quickstart.md new file mode 100644 index 00000000..6f66dda3 --- /dev/null +++ b/content/guides/14.environment-sync/1.quickstart.md @@ -0,0 +1,248 @@ +--- +stableId: 04fa4fa3-2151-44c3-ba63-9433d180cbec +title: Quickstart +description: Run the full pull, diff, push loop against two throwaway Directus instances, and see what the CLI does at every step. +--- + +The fastest way to trust Environment Sync is to watch it work somewhere mistakes are free. In this guide you stand up two throwaway Directus instances with Docker, make a change on one, and move it to the other through git. Nothing here touches a real project, and at the end one command tears it all down. + +You need Docker, Node.js with npm, and git. + +## Start two instances + +Save this as `docker-compose.yml` in an empty directory: + +```yaml +services: + source-db: + image: postgres:16-alpine + environment: + POSTGRES_USER: "directus" + POSTGRES_PASSWORD: "directus" + POSTGRES_DB: "directus" + healthcheck: + test: ["CMD-SHELL", "pg_isready -U directus"] + interval: 5s + retries: 12 + + source: + image: directus/directus:latest + ports: + - 8055:8055 + environment: + SECRET: "quickstart-source-secret" + DB_CLIENT: "pg" + DB_HOST: "source-db" + DB_PORT: "5432" + DB_DATABASE: "directus" + DB_USER: "directus" + DB_PASSWORD: "directus" + ADMIN_EMAIL: "admin@example.com" + ADMIN_PASSWORD: "quickstart" + ADMIN_TOKEN: "source-token" + depends_on: + source-db: + condition: service_healthy + + target-db: + image: postgres:16-alpine + environment: + POSTGRES_USER: "directus" + POSTGRES_PASSWORD: "directus" + POSTGRES_DB: "directus" + healthcheck: + test: ["CMD-SHELL", "pg_isready -U directus"] + interval: 5s + retries: 12 + + target: + image: directus/directus:latest + ports: + - 8056:8055 + environment: + SECRET: "quickstart-target-secret" + DB_CLIENT: "pg" + DB_HOST: "target-db" + DB_PORT: "5432" + DB_DATABASE: "directus" + DB_USER: "directus" + DB_PASSWORD: "directus" + ADMIN_EMAIL: "admin@example.com" + ADMIN_PASSWORD: "quickstart" + ADMIN_TOKEN: "target-token" + depends_on: + target-db: + condition: service_healthy +``` + +```bash +docker compose up -d +``` + +After a minute, `http://localhost:8055` (the source) and `http://localhost:8056` (the target) both serve a fresh Data Studio. Log in with `admin@example.com` / `quickstart`. The `ADMIN_TOKEN` values are static admin tokens the CLI will use. + +::callout{icon="material-symbols:info-outline-rounded"} +Both services use the same image tag on purpose. Environment Sync requires the source and target to run the same Directus version, patch release included. In real projects, pin the same explicit version on every environment. +:: + +## Install the CLI + + + +```bash +npm install -g @directus/cli +d6s --version +``` + +The package installs two commands that do the same thing: `directus-cli` and the short alias `d6s`. These docs use `d6s`. + +## Add a profile for each instance + +A profile pairs a name with an instance URL. Run this in the same directory as the compose file: + +```bash +d6s profile add source --url http://localhost:8055 --token source-token +``` + +``` +◇ Saved profile "source" → http://localhost:8055 +◇ Saved a token for "source" to the credential store. +``` + +```bash +d6s profile add target --url http://localhost:8056 --token target-token +``` + +That first command created `directus.config.json` in the current directory: + +```json +{ + "profiles": { + "source": { "url": "http://localhost:8055", "auth": { "type": "token" } }, + "target": { "url": "http://localhost:8056", "auth": { "type": "token" } } + } +} +``` + +Notice what is not in it: the tokens. URLs are project configuration you commit; credentials go to `~/.directus/credentials.json`, readable only by you. Confirm both connections work: + +```bash +d6s profile test source +``` + +``` +◇ Authenticated to http://localhost:8055 as Admin User (Administrator). +``` + +## Make something to sync + +In the source Studio at `http://localhost:8055`, create a collection called `articles` with two fields: `title` and `status`. This plays the part of a day's modeling work on a development instance. + +## Pull it into files + +Make the directory a git repository, then pull: + +```bash +git init +d6s sync pull --from source +``` + +``` +◇ Pulled from source — http://localhost:8055 + Schema 1 collection → directus/default/schema + Resources 6 records in 10 resources → directus/default/data +``` + +Your record counts may differ slightly; a fresh instance carries a handful of configuration records (the admin role, its policy, the settings row) even before you touch it. + +Look at what appeared: + +``` +directus/default/ + schema/ # one JSON file per collection, plus metadata.json + data/ # one JSON file per configuration resource, plus metadata.json +``` + +Open the `articles` file under `schema/`. It's your collection, readable as JSON: the fields you just created, their types, their interface settings. This is what reviewers will see in pull requests. Commit it: + +```bash +git add directus/ directus.config.json +git commit -m "Baseline from source" +``` + +## Change something and pull again + +Back in the source Studio, add one more field to `articles`: `summary`. Then: + +```bash +d6s sync pull --from source +git diff +``` + +The diff touches one file, and inside it, only the new `summary` field. The CLI writes files deterministically: no timestamps, no reshuffling, nothing but your change. This is the property that makes the files reviewable, and it means a pull that finds nothing new leaves your working tree clean. Commit the field. + +```bash +git add directus/ +git commit -m "Add articles.summary" +``` + +## Preview against the target + +The target instance is still empty. Ask what pushing the committed files would do to it: + +```bash +d6s sync diff --to target +``` + +``` +● Comparing committed files with target — http://localhost:8056 (merge — additive, no deletions) +● Schema — 1 change: 1 added, 0 modified, 0 deleted ++ collection articles (3 fields) +● Data — no changes to import. +``` + +Read the plan: one collection to add, and the mode line tells you up front that `merge`, the default, never deletes anything. If your data section lists a few configuration records instead of "no changes", that's fine; fresh instances can differ slightly in their defaults. `diff` applied nothing; it's always safe to run. + +## Push + +```bash +d6s sync push --to target +``` + +The push prints the same plan, then asks before applying: `Apply 1 schema change to target — http://localhost:8056?`. Confirm it: + +``` +● Schema applied. +◇ Push complete. Applied 1 schema change to http://localhost:8056; schema hash verified. No data changes to import. +``` + +Open the target Studio at `http://localhost:8056`: the `articles` collection is there, fields and settings intact. That's the whole loop. Change on one instance, pull to files, review in git, push to another. + +If a push imports configuration records, it also writes `directus/default/id_map.json`, which remembers which target record each committed record became. Commit it when it changes; [How It Works](/guides/environment-sync/how-it-works#record-identity) explains why. + +## Push again + +```bash +d6s sync push --to target +``` + +``` +● Pushing to target — http://localhost:8056 (merge — additive, no deletions) +◇ target already matches the committed files — schema and data match; nothing to push. +``` + +Nothing to confirm, nothing applied. The CLI checked the target and proved it matches; it didn't assume. Re-running a completed push is safe, which is exactly what automation needs. + +## Clean up + +```bash +docker compose down +``` + +Both instances and their databases are gone. + +## Where to go next + +- [How It Works](/guides/environment-sync/how-it-works): the mental model behind what you just did, and the safety rules around deletions and record identity. +- [Common Workflows](/guides/environment-sync/common-workflows): the same loop applied to real situations, including adopting sync on an existing project. +- [Reference](/guides/environment-sync/reference): every flag the commands take, including how to scope a pull to specific collections or resources. diff --git a/content/guides/14.environment-sync/2.how-it-works.md b/content/guides/14.environment-sync/2.how-it-works.md new file mode 100644 index 00000000..53244f54 --- /dev/null +++ b/content/guides/14.environment-sync/2.how-it-works.md @@ -0,0 +1,100 @@ +--- +stableId: 5ec649aa-ca39-4d4c-b0f6-a10cbaf0cf38 +title: How It Works +description: The mental model behind Environment Sync, including files as the source of truth, record identity across instances, how a push applies, and the safety rules every command follows. +--- + +Environment Sync never moves anything directly between two instances. The files in your repository sit in the middle, and two rules govern everything the commands do: + +1. **A pull only rewrites what it fetched.** Everything it did not fetch keeps its committed state, untouched. +2. **A push only applies what is in the files.** A push reads your repository, not the source instance. Work that never entered the repository cannot ship. + +Everything else on this page is a consequence of those two rules. + +## The files + +A pull writes into a directory you commit (named `directus` by default, one subdirectory per [project](/guides/environment-sync/reference#directusconfigjson)): + +``` +directus/default/ + schema/ # the schema, one JSON file per collection + data/ # configuration records, one JSON file per resource + id_map.json # which target record each committed record became +``` + +The files are written deterministically: pulling twice with no instance changes produces byte-identical files and a clean working tree. `git diff` after a pull shows what changed on the instance and nothing else, so a schema change reads like any other code change in review. + +Each directory also holds a `metadata.json` that lists the files the CLI wrote. The CLI treats that list as ownership: it removes a stale file it wrote on an earlier pull, and it never deletes a file it did not write. Hand-edited or corrupt files stop the command with a named error rather than syncing something the instance never said. + +Profiles and per-project settings live in `directus.config.json` at the repository root. It contains URLs and scoping options, never credentials, so it is safe to commit. The [reference](/guides/environment-sync/reference#directusconfigjson) shows the full file. + +## Two independent axes + +A pull covers two independent things, and you can scope each without affecting the other: + +- **Schema**: collections, fields, relations. Scope it with `--collections` or `--exclude-collections`, or skip it with `--no-schema`. +- **Configuration resources**: records of ten `directus_*` resource types (flows, roles, settings, and the rest), plus opt-in users and translations. Scope it with resource flags like `--flows` or `--no-flows`. + +Scoping narrows what a pull _fetches_; combined with rule 1, that means a scoped pull refreshes its slice of the files and preserves every other file exactly as committed. The [pull scope matrix](/guides/environment-sync/reference#what-a-pull-touches) in the reference lists what each flag combination refreshes and preserves, and [Common Workflows](/guides/environment-sync/common-workflows#promote-only-the-changes-that-are-ready) shows why you'd want that: shipping finished work while half-finished work stays out of the repository. + +## Record identity + +The same role or flow carries a different primary key on every instance, so a push has to decide which target record a committed record _is_ before it can update rather than duplicate. Two mechanisms decide, in order: + +- **The identity map.** Each push records its decisions in `id_map.json`: this committed record became that target record. Later pushes look there first. +- **Identifying fields.** A record not yet in the map is matched by the field that names it: `name` for most resources, `email` for a user, `key` for an operation. Panels have no such field, which is why a first push into a look-alike target can duplicate them once. + +When two target records could both be the match, the CLI asks you to choose in a terminal and refuses in CI. Ambiguity is never resolved by guessing. Your answer lands in the identity map, so each question is asked exactly once. That's why the map belongs in git: commit it whenever a push changes it, and teammates and CI inherit every decision already made. + +One map file serves any number of instances. Internally it is keyed by source and target URL, so pushing the same files to staging and to production writes two independent sets of mappings; neither overwrites the other. Deleting the file is safe to the extent that matching starts over: named records re-match by their identifying fields, and records without one can duplicate on the next push. + +## How a push applies + +A push applies in two phases, schema first, and begins by showing you the full plan and asking (unless you pass `--yes`). + +1. **Schema.** Before applying, the push re-checks that the target's schema still matches what the plan was computed against. If someone changed the target between preview and apply, the push stops rather than apply a stale plan. +2. **Data.** Configuration records import in a single server-side transaction once the schema is in place. + +The two phases are not one transaction. If the data import fails after the schema applied, the target holds the new schema and the old configuration, and the CLI says so plainly: + +``` +▲ Schema was applied, but the data import did not complete. +✖ Could not reach https://cms.example.com. + The import may still have been applied on the server. Run d6s sync diff before retrying — a blind retry can duplicate records. +``` + +The guidance is the same in every partial-failure case: run `d6s sync diff` to see where the target actually stands, then push again. A re-run applies only what is still missing, and a completed push re-run reports "nothing to push" after verifying that against the target, not assuming it. + +## Push modes and deletions + +A push mode answers one question: what happens to things that exist on the target but not in your files? + +- **`merge`** (the default) creates and updates, and never deletes. Not in the schema phase, not in the data phase. +- **`add`** only creates; it never touches an existing record. +- **`mirror`** makes the target match the files exactly, which means deleting what the files no longer contain, within whatever scope the files cover. + +Deleting always requires its own explicit consent, separate from confirming the push. Interactively, a mirror push lists what would be lost and asks you to type the profile name. Non-interactively it requires the `--dangerously-allow-delete` flag; `--yes` never authorizes a deletion. The gate is a backstop as well as a policy: even if a supposedly additive push somehow carried a deletion, it would still be refused without consent. + +Because a mirror push makes the target match the files _exactly_, run it against a freshly pulled state. A mirror from stale files applies the stale state, including deleting things that only look obsolete because the files are old. + +## The version gate + +Schema changes require the files' Directus version and the target's version to match exactly, patch release included, because the server refuses cross-version schema diffs; historically, some patches change the schema format. + +``` +✖ Version mismatch: the snapshot was pulled from Directus 11.2.0, but the target runs 11.2.5. + The server requires an exact version match for schema diffs — historically some patches are breaking. Align both instances (re-pull if the source was upgraded), or pass --allow-version-drift to proceed anyway. +``` + +`--allow-version-drift` asks the server to proceed anyway and warns you loudly; the CLI never translates schema between versions. Projects configured with `"schema": false` skip the schema phase and this gate entirely. + +## The safety model + +The rules above add up to a small set of promises, each of which you can watch hold in the [Quickstart](/guides/environment-sync/quickstart): + +- **`pull` is read-only on the source.** Every request it makes is a read; it changes nothing on the instance it snapshots. (The one exception: a profile that authenticates with a saved login session refreshes that session when it is close to expiring.) +- **`diff` applies nothing.** The schema comparison is a preview, and the data plan comes from the target server dry-running the import inside a transaction and rolling it back, so the plan is the server's own answer, not a client-side guess. +- **Deletions are gated.** Only `mirror` deletes, always behind its own consent, and `--yes` never covers it. +- **Identity is never guessed.** An ambiguous record match prompts in a terminal and refuses in CI. +- **Stored secrets never enter your repository.** Built-in secret columns and fields marked concealed, hashed, or encrypted are stripped at export, with [one warned exception](/guides/environment-sync/secrets-and-limitations#the-one-blind-spot). +- **Failures are loud.** Hand-edited files, cut-short exports, version mismatches, and unreachable instances stop the command with a named error instead of degrading silently. An export the source itself left incomplete is marked as such and refused at mirror push. diff --git a/content/guides/14.environment-sync/2.pulling.md b/content/guides/14.environment-sync/2.pulling.md deleted file mode 100644 index 5b55978e..00000000 --- a/content/guides/14.environment-sync/2.pulling.md +++ /dev/null @@ -1,122 +0,0 @@ ---- -stableId: 9ab17c77-208a-4db3-bbbb-fdb911f6130b -title: Pulling -description: Snapshot schema and configuration from a source instance into files you commit. Scope a pull by resource or by collection, or skip schema entirely for configuration-only projects. ---- - -Pulling snapshots a source instance into the committed files that `diff` and `push` later apply: - -```bash -d6s sync pull --from staging -``` - -## What a default pull exports - -A pull touches two axes, and the success output reports each: - -``` -◇ Pulled from staging — https://staging.example.com - Schema 24 collections → directus/default/schema - Resources 206 records in 10 resources → directus/default/data -``` - -Exact counts depend on your project. - -- **Schema**: a full snapshot of every collection, field, and relation, including custom fields on system collections and collection folders. -- **Resources**: records of 10 `directus_*` configuration resource types, listed below. - -**Users** and **translations** are excluded by default. Opt in with `--users` and `--translations`, or `--all`. - -## Configuration resources - -Resource selection follows a dependency graph: selecting a resource pulls in what it needs. - -| Resource | In default pull | Select directly | Pulls in | Notes | -| -------------- | ---------------------------- | ---------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -| `roles` | Yes | `--roles` | `policies` | | -| `policies` | Yes | `--policies` | `access`, `permissions` | See the warning below before selecting policies on their own. | -| `access` | Yes, with roles and policies | — | — | Grants attached to users are dropped when users are out of scope; a mirror push does not delete them on the target. | -| `permissions` | Yes, with policies | — | — | Row counts are verified against the server. If the source hides rows (unlicensed custom permission rules), the export is marked incomplete. | -| `flows` | Yes | `--flows` | `operations` | | -| `operations` | Yes, with flows | — | — | | -| `dashboards` | Yes | `--dashboards` | `panels` | | -| `panels` | Yes, with dashboards | — | — | Panels have no natural key, so a first push into a matching target can duplicate once; the id map prevents repeats. | -| `settings` | Yes | `--settings` | — | Singleton. License and AI credentials, branding images (logos, backgrounds, favicon), and the default storage folder are stripped. | -| `folders` | Yes | `--folders` | — | The media-library folder tree. Distinct from collection folders (Data Studio sidebar groups), which sync as schema. | -| `users` | Opt-in | `--users` | `roles`, `policies` | Secret columns (`password`, `token`, `tfa_secret`, and others) are stripped. | -| `translations` | Opt-in | `--translations` | — | Mirror pushes of translations are not currently supported, which is why they are opt-in. | - -::callout{icon="material-symbols:warning-rounded" color="warning"} -**Select `--roles`, not `--policies` alone** -A selection that pulls policies without roles is not independently pushable when access rows reference roles: access rows carry role foreign keys, and with no roles in scope a push to a fresh target fails. Select `--roles` instead; it pulls policies and their children too. -:: - -## Selecting resources - -Three ways to change the default set: - -- **Positive selection**: `--flows --roles` narrows the pull to those resources plus their dependencies. -- **Subtraction**: `--no-flows` keeps the default set and removes flows. -- **Everything**: `--all` adds `users` and `translations` on top of the default set. - -Positive selections cannot be combined with `--all` or with `--no-` subtractions. - -Resource selection changes only the resources axis: a pull with `--flows` still snapshots the full schema. Schema has its own scoping. - -## Scoping the schema - -```bash -d6s sync pull --from staging --collections articles,authors -``` - -The Schema line reads `(scoped to: articles, authors)` and only those collections' schema files refresh. `--exclude-collections` inverts the scope: snapshot everything except the named collections. - -Two warnings a scoped pull can raise: - -- **Out-of-scope references.** If the scoped snapshot points at something you omitted (a relation target, a group parent, a many-to-any collection), the pull warns, because pushing that snapshot to a fresh target can fail. It warns; it never widens the scope for you. Add the missing collections to `--collections` yourself. -- **A name the server didn't return.** A `--collections` name absent from the returned snapshot (usually a typo) draws a warning naming the gap. The partial snapshot still commits, but never silently. - -## Skipping schema entirely - -Schema and resources are independent axes, and resource selection never narrows the schema snapshot. To make a configuration-only project explicit, opt out of schema: - -```bash -d6s sync pull --from staging --no-schema -``` - -To make it permanent, set `"schema": false` on the project in `directus.config.json`. Such a project carries no schema authority: pull skips the snapshot, and push and diff for that project never touch schema. Reports say `schemaSkipped`, so automation can tell a skipped phase from a matching one. Combining `"schema": false` with a collections scope is refused as a contradiction. - -## What a pull touches - -Two rules govern every pull, scoped or not: - -1. A pull only rewrites what it fetched. Everything it did not fetch keeps its committed bytes, untouched. -2. A push only applies what is committed. Work that never entered the repository cannot ship. - -Refreshed means the file is rewritten from the source; because writes are deterministic, an unchanged resource produces no git diff. Preserved means the file is not touched at all. - -| Pull | Schema files | Configuration files | -| ------------------------------------ | ------------------------------------------------------- | ------------------------------------------------ | -| `pull --from staging` | All refreshed | All refreshed | -| `... --collections posts` | `posts` refreshed, others preserved | All refreshed | -| `... --no-flows` | All refreshed | Flows preserved, others refreshed | -| `... --flows` | All refreshed (resource selection never narrows schema) | Flows and operations refreshed, others preserved | -| `... --flows --no-schema` | All preserved | Flows and operations refreshed, others preserved | -| `... --collections posts --no-flows` | `posts` refreshed, others preserved | Flows preserved, others refreshed | - -## Determinism - -Re-running a pull with no instance changes produces byte-identical files and a clean working tree, so `git diff` after a pull shows exactly what changed on the instance and nothing else: no timestamps, no reordering noise. - -Scoped pulls are just as predictable: a scoped or resource-selected pull refreshes only its subset and preserves every other committed file. Scope limits what a pull may replace; it never deletes the rest of your snapshot. - -## Pull before you push - -::callout{icon="material-symbols:warning-rounded" color="warning"} -**The committed tree is what a push applies** -A scoped pull refreshes only its scope, so the rest of the tree keeps whatever it last knew, and a later mirror push would apply that stale state to the target. Pull before you push. -:: - -## Next step - -With files committed, preview what they would change on a target: [Diffing & Pushing](/guides/environment-sync/diffing-and-pushing). diff --git a/content/guides/14.environment-sync/6.common-workflows.md b/content/guides/14.environment-sync/3.common-workflows.md similarity index 50% rename from content/guides/14.environment-sync/6.common-workflows.md rename to content/guides/14.environment-sync/3.common-workflows.md index 3a7b792d..5ab58760 100644 --- a/content/guides/14.environment-sync/6.common-workflows.md +++ b/content/guides/14.environment-sync/3.common-workflows.md @@ -1,10 +1,10 @@ --- stableId: b0bd4f6a-0708-4681-86dc-c482895372c1 title: Common Workflows -description: End-to-end walkthroughs of the most common Environment Sync workflows, from promoting changes to production to recovering a drifted environment and standing up a new one. +description: End-to-end walkthroughs of the most common Environment Sync workflows, from promoting changes to production to adopting sync on an existing project, rolling back, and recovering a drifted environment. --- -These workflows cover most day-to-day use of Environment Sync: promoting changes to production (all of them, or just the ones that are ready), re-aligning an environment after out-of-band changes, and standing up a new environment from the committed files. The mechanics behind each step live in [Pulling](/guides/environment-sync/pulling), [Diffing & Pushing](/guides/environment-sync/diffing-and-pushing), and [CI & Automation](/guides/environment-sync/ci-and-automation). +These workflows cover most day-to-day use of Environment Sync: promoting changes to production (all of them, or just the ones that are ready), starting to use sync on a project that already exists, undoing a bad push, re-aligning an environment after manual changes, and standing up a new environment from the committed files. The concepts behind each step live in [How It Works](/guides/environment-sync/how-it-works); every flag in [Reference](/guides/environment-sync/reference). ## Promote changes from development to production @@ -37,7 +37,7 @@ You model in the Data Studio on a development instance. Production only ever rec The default `merge` mode creates and updates but never deletes. A second run reports nothing to push. ::callout{icon="material-symbols:warning-rounded" color="warning"} -**Removals need `mirror`.** A change that deletes a field or a record does not propagate under `merge`. Push with `--mode mirror` and pass its [deletion gate](/guides/environment-sync/diffing-and-pushing#deletion-gates), and run a full pull first so the mirror applies current state, not a stale tree. +**Removals need `mirror`.** A change that deletes a field or a record does not propagate under `merge`. Push with `--mode mirror` and pass its [deletion gate](/guides/environment-sync/reference#deletion-gates), and run a full pull first so the mirror applies current state, not a stale tree. :: ## Promote only the changes that are ready @@ -71,17 +71,81 @@ For a configuration-only change, flip the scope: select the resource type and sk d6s sync pull --from staging --flows --no-schema ``` -The [What a pull touches](/guides/environment-sync/pulling#what-a-pull-touches) table shows exactly which files each combination refreshes and which it leaves alone. +The [What a pull touches](/guides/environment-sync/reference#what-a-pull-touches) table shows exactly which files each combination refreshes and which it leaves alone. ::callout{icon="material-symbols:warning-rounded" color="warning"} -**Selection is by collection and resource type, not by record.** `--collections posts` can separate finished posts work from unfinished authors work, but two changes inside the same collection travel together, and `--flows` takes every flow, not just one. If unrelated work shares a collection or a resource type, ship it together or wait. And while the committed tree holds a partial picture, avoid `mirror`: it would apply the stale remainder too. +**Selection is by collection and resource type, not by record.** `--collections posts` can separate finished posts work from unfinished authors work, but two changes inside the same collection travel together, and `--flows` takes every flow, not just one. If unrelated work shares a collection or a resource type, ship it together or wait. And while the committed files hold a partial picture, avoid `mirror`: it would apply the stale remainder too. :: +## Adopt Environment Sync on an existing project + +Most projects don't start from an empty instance. You have a development instance, a production instance, and months of changes applied to each by hand. Here's how to get from that to a synced setup. + +1. Decide which instance is closest to the state you want everywhere. Usually that's production: it's the environment you've been careful with. + +2. Pull it as your baseline and commit: + + ```bash + d6s profile add production --url https://cms.example.com + d6s sync pull --from production + git add directus/ directus.config.json + git commit -m "Baseline from production" + ``` + +3. See how far each other environment has drifted: + + ```bash + d6s profile add staging --url https://staging.example.com + d6s sync diff --to staging + ``` + + The first diff on a long-lived pair of instances can be long. Read it as an inventory of drift, not a to-do list: everything listed is a place where staging and production genuinely disagree, accumulated over however long the two were maintained by hand. + +4. Bring the environment in line gradually. A `merge` push adds and updates what staging is missing without deleting anything, so staging-only experiments survive: + + ```bash + d6s sync push --to staging + ``` + + Once nothing on staging is worth keeping outside the committed files, a `mirror` push finishes the job and makes it match exactly. + +5. Expect a few identity questions on the first push. Staging and production often hold records that are plausibly the same one (two roles named "Editor", two flows built from the same template). The CLI asks instead of guessing; answer once, then commit the updated `id_map.json`. The questions do not come back. + +From then on, the project runs the [promote workflow](#promote-changes-from-development-to-production): change on dev, pull, review, push. + +::callout{icon="material-symbols:info-outline-rounded"} +Nervous about the first push against a real instance? Rehearse the whole loop against throwaway instances first: the [Quickstart](/guides/environment-sync/quickstart) sandbox is exactly that, and adding a profile for a scratch instance to a real repository is harmless. +:: + +## Roll back a bad push + +A change made it to production and turned out to be wrong. The committed files are the record of every state production has been in, so rollback is a git operation followed by a push. One asymmetry matters: + +- If the bad change **modified** something, reverting the commit and pushing with `merge` restores the old values. +- If the bad change **added** something, `merge` cannot undo it: after the revert, the field or record is simply absent from your files, and merge never deletes what the files don't mention. Removal is a deletion, and deletions need `mirror`. + +The steps: + +```bash +git revert +d6s sync diff --to production +``` + +The diff now describes the undo. Read the deletions closely: they should name exactly what the bad change added, and nothing else. If anything else shows up as a deletion, your files are stale somewhere; stop and reconcile first (usually a fresh full pull on a branch to compare against). + +```bash +d6s sync push --to production --mode mirror +``` + +The [deletion gate](/guides/environment-sync/reference#deletion-gates) applies: the push lists the losses and asks you to type the profile name, or requires `--dangerously-allow-delete` in automation. + +When the bad change is tangled up with good ones, fixing forward is often simpler than reverting: correct it on the development instance, pull, and promote the fix like any other change. + ## Rebase an environment from production after drift -Sometimes production changes outside the deployment path, usually an urgent manual fix. Staging and the repository no longer reflect reality. Bring the fix into git, then re-align the lower environment. This is what `mirror` is for: converging an environment to the committed state exactly. +Sometimes production changes outside the deployment path, usually an urgent manual fix. Staging and the repository no longer reflect reality. Bring the fix into git, then re-align the lower environment. This is what `mirror` is for: making an environment match the committed state exactly. -1. Pull the full state from production. The out-of-band change appears as an ordinary git diff, which is your record of what the hotfix actually was: +1. Pull the full state from production. The manual change appears as an ordinary git diff, which is your record of what the hotfix actually was: ```bash d6s sync pull --from production @@ -98,13 +162,13 @@ Sometimes production changes outside the deployment path, usually an urgent manu Read the deletions closely. Anything that exists only on staging and falls inside the sync's scope is on the list. -3. Converge staging to the committed state: +3. Make staging match the committed state: ```bash d6s sync push --to staging --mode mirror ``` - Interactively, the push names the losses and asks you to type the profile name; in automation it requires `--dangerously-allow-delete`. See [deletion gates](/guides/environment-sync/diffing-and-pushing#deletion-gates). + Interactively, the push names the losses and asks you to type the profile name; in automation it requires `--dangerously-allow-delete`. See [deletion gates](/guides/environment-sync/reference#deletion-gates). ::callout{icon="material-symbols:warning-rounded" color="warning"} **Mirror removes staging-only work in scope.** If staging holds experiments you want to keep, pull that work into a branch first, or use `merge` and clean up by hand. @@ -133,11 +197,15 @@ Going from an empty instance to a working copy of your project's shape: 4. Commit the updated `id_map.json`. The push records which target record each committed record became, and that map is how every later push updates records instead of duplicating them. -Two things to expect on a first push: +Things to expect on a first push: -- **Identity questions.** If the target already holds records the CLI cannot tell apart from the committed ones (two policies named "Administrator" is the classic case), an interactive push asks you to choose. Answer once, commit the map, and the questions do not come back. See [record identity](/guides/environment-sync/diffing-and-pushing#record-identity). +- **Identity questions.** If the target already holds records the CLI cannot tell apart from the committed ones (two policies named "Administrator" is the classic case), an interactive push asks you to choose. Answer once, commit the map, and the questions do not come back. See [record identity](/guides/environment-sync/how-it-works#record-identity). - **Secrets stay behind.** Stripped values (API keys in settings, concealed fields, flow credentials) never travel with the files. Set them on the new instance directly. +::callout{icon="material-symbols:warning-rounded" color="warning"} +**Extensions do not sync, and nothing warns about them.** A field built on a custom interface, or a flow using a custom operation, pushes silently to a target that may not have that extension installed, and arrives broken in the Studio until it is. Deploy your extensions to the target before pushing schema or flows that depend on them. +:: + ::callout{icon="material-symbols:info-outline-rounded"} -Push the whole committed tree to a fresh target, not a scoped slice. A partial snapshot whose relations point at collections outside the scope can fail to apply on an instance that has nothing else yet. The pull warns about these references when it writes the files. +Push the whole committed folder to a fresh target, not a scoped slice. A partial snapshot whose relations point at collections outside the scope can fail to apply on an instance that has nothing else yet. The pull warns about these references when it writes the files. :: diff --git a/content/guides/14.environment-sync/3.diffing-and-pushing.md b/content/guides/14.environment-sync/3.diffing-and-pushing.md deleted file mode 100644 index 587011ae..00000000 --- a/content/guides/14.environment-sync/3.diffing-and-pushing.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -stableId: a1896d05-d465-450c-b2d1-dd8ccef64ad5 -title: Diffing & Pushing -description: Preview what a push would change with a read-only diff, then apply it to a target instance by choosing a push mode, passing the deletion gates, and resolving record identity. ---- - -With a snapshot [pulled](/guides/environment-sync/pulling) and committed, `diff` previews what applying it to a target would change, and `push` applies it. Both commands read the committed files: what you push is what is in git, not what is currently on the source instance. - -## Previewing with diff - -```bash -d6s sync diff --to production -``` - -`diff` applies nothing. It previews the schema change, then has the target server dry-run the data import and roll it back, so the data plan is the server's own answer, not a client-side guess. - -Every preview opens by naming the target instance and what the mode would mean (`merge — additive, no deletions`), then lists schema changes line by line and the data plan per collection (`+N new ~N updated`). - -One thing a diff never does is guess: a committed record that could match more than one target record is reported as **unresolved**, not previewed as a create. An interactive push resolves it by asking you; a push in CI refuses. See [record identity](#record-identity) below. - -## Push modes - -```bash -d6s sync push --to production # merge (the default) -d6s sync push --to production --mode mirror # deletes, behind the gates below -``` - -| Mode | Schema | Data | Deletes? | -| ----------------- | ------------------- | ----------------------------------------------------------- | -------------- | -| `add` | Additive | Inserts only; existing records are never updated | No | -| `merge` (default) | Additive | Creates and updates | No | -| `mirror` | May delete in scope | Creates, updates, and deletes records absent from the files | **Yes, gated** | - -::callout{icon="material-symbols:warning-rounded" color="warning"} -**Pull before a mirror push.** The committed tree is what a push applies. A scoped pull refreshes only its scope, so the rest of the tree keeps whatever it last knew, and a `mirror` push applies that stale state to the target. Run a full pull first. -:: - -## Deletion gates - -Only `mirror` deletes, and deleting always requires its own explicit consent: - -| Context | To delete you must | -| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | -| Interactive terminal | Review the plan naming the losses, then type the profile name (unless you passed `--dangerously-allow-delete`, which is the consent) | -| Non-interactive / CI | Pass `--dangerously-allow-delete` | - -`--yes` skips the ordinary confirmation prompt, but it never authorizes a deletion. A `mirror` push in CI without `--dangerously-allow-delete` refuses before changing anything. - -## How a push runs - -A push applies in two phases, schema first: - -1. **Schema.** The schema change applies first, sealed with a hash of the target's schema. If the target changed between planning and applying, the push stops instead of applying a stale plan. -2. **Data.** Configuration records import once the schema is in place. - -The two phases are not one transaction. If the data import fails after the schema applied, run the push again: the schema now matches, so the re-run applies data alone. - -## Record identity - -The same role or flow carries different primary keys on each instance, so records have to be matched across them. Two mechanisms decide which target record a committed record is: - -- **The committed identity map.** Each push records source-to-target pairs in `directus//id_map.json`. Commit it; it is how the next push updates a record instead of creating a duplicate. -- **Natural keys.** A record not yet in the map is matched to a target record by a natural key: its name for most resources, its email for a user, its key for an operation. - -Ambiguity is never resolved by guessing. When two target records could both be the match, the CLI prompts you to choose in a terminal, and refuses in CI. - -The first push into a target seeded from the same template can ask a few of these identity questions about pairs of records that plausibly are the same one. That is expected: answer them, the answers land in the identity map, and the questions do not come back. It converges in one pass. - -## Version matching - -Schema changes require the snapshot's Directus version and the target's version to match exactly, patch release included. A mismatch refuses the command and names both versions. Align the instances (re-pull if the source was upgraded), or pass `--allow-version-drift` to override the gate, which proceeds with a loud warning. The CLI does not translate schema between versions. - -## Convergence - -A push that applied cleanly leaves nothing behind: - -```bash -d6s sync push --to production -# → schema and data match; nothing to push. -``` - -Re-running a completed push is safe, and the clean state is verified against the target, not assumed. If a connection drops mid-import and the result is unknown, the CLI says so. Run `d6s sync diff` before retrying rather than risking a blind retry. - -::callout{icon="material-symbols:info-outline-rounded" to="/guides/environment-sync/ci-and-automation"} -Running diff and push unattended? CI & Automation covers non-interactive behavior, tokens, and JSON reports. -:: diff --git a/content/guides/14.environment-sync/4.ci-and-automation.md b/content/guides/14.environment-sync/4.ci-and-automation.md index c0e33fbf..23796583 100644 --- a/content/guides/14.environment-sync/4.ci-and-automation.md +++ b/content/guides/14.environment-sync/4.ci-and-automation.md @@ -1,53 +1,123 @@ --- stableId: d5ad7a4c-dd80-4394-af25-333dabfeeaf1 title: CI & Automation -description: Run Environment Sync unattended with tokens from environment variables, machine-readable JSON reports on stdout, and explicit flags in place of prompts. +description: Post the production diff on every pull request and push on merge, with tokens from environment variables, machine-readable JSON reports, and explicit flags in place of prompts. --- -Every sync command can run unattended. The contract in CI is deliberately strict: no prompts, no guessing, and nothing destructive without an explicit flag. +Automating Environment Sync buys you two things: every pull request shows what merging it would change on production, and merging applies it without anyone running a command. This page builds that pipeline and covers the rules unattended runs follow. -## No prompts +## The non-interactive contract -Where the interactive CLI would ask a question, a non-interactive run refuses instead: +The CLI treats any environment with the `CI` variable set as non-interactive (locally, `--no-interactive` simulates the same behavior). Where the interactive CLI would ask a question, a non-interactive run refuses instead: - An **ambiguous record match** (two target records that could both be the committed one) fails the command rather than picking a candidate. Resolve it once with an interactive push; the answer lands in the committed identity map and CI runs cleanly after that. - **Deletions** happen only with `--dangerously-allow-delete`. A `mirror` push without it refuses before changing anything on the target. - `--yes` confirms an ordinary, non-destructive apply. It never authorizes a deletion. +Commands exit `0` on success and `1` on any refusal or failure; there are no other exit codes. Anything finer-grained (which kind of failure, how many changes) comes from the JSON report, not the exit code. + ## Credentials -Pass tokens through environment variables named `DIRECTUS__TOKEN`: +Pass tokens through environment variables named `DIRECTUS__TOKEN`, the profile name uppercased. Profile names use letters, numbers, and underscores, so the mapping is mechanical: profile `production` reads `DIRECTUS_PRODUCTION_TOKEN`, profile `staging_eu` reads `DIRECTUS_STAGING_EU_TOKEN`. -```bash -DIRECTUS_PRODUCTION_TOKEN=$PROD_TOKEN d6s sync push --to production --yes -``` - -The credential store saved on a developer machine is never read in CI; tokens come from the environment only. +The credential store saved on a developer machine is never read when `CI` is set; tokens come from the environment only. ## JSON reports -Add `--json` and stdout carries exactly one machine-readable report per command; pull, diff, and push each emit their own. Warnings (stripped secret fields, version drift, flow headers exported verbatim) still go to stderr, so your logs keep them while stdout stays parseable. - -A diff whose records are ambiguous reports them as `unresolved` and counts them into its `changes`. A non-interactive push refuses that state, so an unresolved diff is a real difference for your pipeline to surface, not noise. +Add `--json` and stdout carries exactly one machine-readable report per command. Warnings (stripped secret fields, version drift, flow headers exported verbatim) still go to stderr, so your logs keep them while stdout stays parseable. A failure puts an error report on stdout instead, with a stable `code` naming the failure class. + +The fields automation usually keys on: + +- **`changes`** (diff): `true` when the push would do anything, including unresolved records. +- **`unresolved`** (diff): the count of ambiguous record matches. A non-interactive push refuses while this is non-zero, so an unresolved diff is a real difference for your pipeline to surface, not noise. +- **`applied`** (push): `true` when the push changed the target. + +`d6s sync diff` exits `0` whether or not differences exist; it fails only when it cannot produce an answer. Gate pipeline behavior on the report's `changes`, not the exit code. The [reference](/guides/environment-sync/reference#json-reports) documents every report field. + +## A GitHub Actions pipeline + +One workflow, two jobs: pull requests get the production diff as a comment, and merges to `main` apply the committed files. Store the token as an Actions secret. + +```yaml +name: environment-sync + +on: + pull_request: + push: + branches: [main] + +jobs: + diff: + if: github.event_name == 'pull_request' + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: write + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: 22 + - run: npm install -g @directus/cli + - name: Diff against production + run: d6s sync diff --to production --json > diff-report.json + env: + DIRECTUS_PRODUCTION_TOKEN: ${{ secrets.DIRECTUS_PRODUCTION_TOKEN }} + - name: Comment the result on the PR + uses: actions/github-script@v7 + with: + script: | + const report = require('./diff-report.json'); + const body = report.changes + ? `**Environment Sync**: merging changes production. ${report.added} added, ` + + `${report.modified} modified, ${report.deleted} deleted schema items; ` + + `${report.unresolved} unresolved records.` + : '**Environment Sync**: production already matches this branch.'; + await github.rest.issues.createComment({ + ...context.repo, + issue_number: context.issue.number, + body, + }); + + push: + if: github.event_name == 'push' + runs-on: ubuntu-latest + permissions: + contents: write + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: 22 + - run: npm install -g @directus/cli + - name: Push to production + run: d6s sync push --to production --yes --json > push-report.json + env: + DIRECTUS_PRODUCTION_TOKEN: ${{ secrets.DIRECTUS_PRODUCTION_TOKEN }} + - name: Commit the updated identity map + run: | + if [ -n "$(git status --porcelain -- 'directus/*/id_map.json')" ]; then + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + git add 'directus/*/id_map.json' + git commit -m "Update sync identity map" + git push + fi +``` -## A typical pipeline +Two things to know before enabling the push job: -```bash -# On a schedule: refresh the committed snapshot from the source instance. -# Commit the result; a clean git status means nothing changed. -DIRECTUS_STAGING_TOKEN=$STAGING_TOKEN d6s sync pull --from staging --json +- **Run the first push interactively, locally.** The first push into a target tends to raise the identity questions described in [How It Works](/guides/environment-sync/how-it-works#record-identity), and CI refuses them. Answer them once from a terminal, commit `id_map.json`, and CI is clean from then on. +- **The identity map commit-back step matters.** A push that creates records adds entries to `id_map.json`. If CI doesn't commit them, the next push re-matches those records from scratch, and records without an identifying field (panels) can duplicate. -# In pull request checks: preview what merging would apply to the target. -DIRECTUS_PRODUCTION_TOKEN=$PROD_TOKEN d6s sync diff --to production --json +A scheduled pull is the same pattern in reverse: run `d6s sync pull --from staging --json` on a cron trigger and commit the result. A clean working tree means nothing changed on the instance; a diff is drift, arriving as a reviewable commit or PR instead of a surprise. -# On merge: apply. -DIRECTUS_PRODUCTION_TOKEN=$PROD_TOKEN d6s sync push --to production --yes --json -``` +## Mirror pushes in automation -## Exit behavior +A `mirror` push deletes, so it additionally requires `--dangerously-allow-delete`: -Commands exit non-zero on any refusal or failure (an ambiguous match, a missing deletion flag, a version mismatch, a failed import), so a pipeline step fails visibly instead of continuing past a partial sync. +```bash +d6s sync push --to staging --mode mirror --yes --dangerously-allow-delete +``` -::callout{icon="material-symbols:info-outline-rounded" to="/guides/environment-sync/diffing-and-pushing"} -The push modes and deletion gates these flags control are covered in Diffing & Pushing. -:: +Reserve this for pipelines that rebuild disposable environments, and keep production pushes on the default `merge` unless a human reviewed the deletions in the diff. The flag name is deliberate. diff --git a/content/guides/14.environment-sync/5.secrets-and-limitations.md b/content/guides/14.environment-sync/5.secrets-and-limitations.md index 6404f397..de4a5ed9 100644 --- a/content/guides/14.environment-sync/5.secrets-and-limitations.md +++ b/content/guides/14.environment-sync/5.secrets-and-limitations.md @@ -6,7 +6,7 @@ description: How Environment Sync keeps secret values out of the files you commi ## How secrets are handled -A [pull](/guides/environment-sync/pulling) exports real records: the settings row, user accounts, flow definitions. If any of those carried a secret value, it would land in JSON files you commit, and git history keeps it forever, in every clone of the repository. Environment Sync strips secret values at export: +A pull exports real records: the settings row, user accounts, flow definitions. If any of those carried a secret value, it would land in JSON files you commit, and git history keeps it forever, in every clone of the repository. Environment Sync strips secret values at export: - **Built-in secret columns** (password hashes, tokens, 2FA seeds, license and AI keys) are always deleted from the export. - **Fields you created and marked concealed, hashed, or encrypted** are deleted too. Every pull fetches the server's complete field list and drops any flagged value, printing a line naming each dropped field. A field missing from the export reads as protection, not data loss. @@ -29,33 +29,33 @@ The server never hands out the real value for these fields: concealed fields rea None of these sync today. Some are shared configuration that a future release could take on; the rest are per-instance data that a sync should never touch. -| Collection | Why it doesn't sync | -| ------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | -| `directus_presets` | Bookmarks, saved layouts, and Insights presets mix shared configuration with personal preference. A future release may sync the shared part. | -| `directus_extensions` | The enabled/disabled state of installed extensions is tied to what is physically deployed on each instance. | -| `directus_files` | File-interface **fields** sync as schema; file rows and the binaries behind them are their own workstream. | -| `directus_comments` | Content comments; per-instance data. | -| `directus_activity` | The audit log; per-instance data. | -| `directus_revisions` | Change history; per-instance data. | -| `directus_versions` | Content-versioning drafts; per-instance data. | -| `directus_notifications` | User notifications; per-instance data. | -| `directus_shares` | Public share links; per-instance data. | -| `directus_sessions` | Active login sessions; per-instance data. | -| `directus_migrations` | The database migration ledger. The CLI never runs migrations or changes the Directus version. | -| `directus_webhooks` | Deprecated in Directus; superseded by flows. | +| Collection | Why it doesn't sync | +| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `directus_presets` | Bookmarks, saved layouts, and Insights presets mix shared configuration with personal preference. A future release may sync the shared part. | +| `directus_extensions` | The enabled/disabled state of installed extensions is tied to what is physically deployed on each instance. Deploy extensions to the target before pushing schema or flows that depend on them; the push itself does not warn. | +| `directus_files` | File-interface **fields** sync as schema; file rows and the binaries behind them are their own workstream. | +| `directus_comments` | Content comments; per-instance data. | +| `directus_activity` | The audit log; per-instance data. | +| `directus_revisions` | Change history; per-instance data. | +| `directus_versions` | Content-versioning drafts; per-instance data. | +| `directus_notifications` | User notifications; per-instance data. | +| `directus_shares` | Public share links; per-instance data. | +| `directus_sessions` | Active login sessions; per-instance data. | +| `directus_migrations` | The database migration ledger. The CLI never runs migrations or changes the Directus version. | +| `directus_webhooks` | Deprecated in Directus; superseded by flows. | ## What it does not do -| Won't | Because / instead | -| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Sync your collections' **content** (rows) | Environment Sync moves the shape of a project and its configuration. Content sync is deferred to a future release. | -| **Undo** a push | Git covers configuration; only a database backup covers data. Back up before big pushes. | -| **Auto-expand** a scoped snapshot | A scope pulls exactly what you name. Dangling references produce a warning; widen `--collections` yourself. | -| Translate schema **across versions** | A version mismatch refuses the command; `--allow-version-drift` overrides the gate but converts nothing. | -| Wrap schema and data in **one transaction** | Schema applies first, then data. A failed import re-runs data alone; see [Diffing & Pushing](/guides/environment-sync/diffing-and-pushing). | -| Select **individual records** | Resource selection is by type (`--roles`), never by row. | -| Model **code-first** | Model in the Data Studio and pull. Don't hand-author the JSON files. | -| **Seal the data plan** against target drift | Only the schema apply is sealed against the target changing between preview and apply. The data preview is the server's own dry-run answer, but it is advisory. | +| Won't | Because / instead | +| ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Sync your collections' **content** (rows) | Environment Sync moves the shape of a project and its configuration. Content sync is deferred to a future release. | +| **Undo** a push | Revert the commit and push again; removals need `mirror`. See [Roll back a bad push](/guides/environment-sync/common-workflows#roll-back-a-bad-push). Only a database backup covers content. | +| **Auto-expand** a scoped snapshot | A scope pulls exactly what you name. Dangling references produce a warning; widen `--collections` yourself. | +| Translate schema **across versions** | A version mismatch refuses the command; `--allow-version-drift` overrides the gate but converts nothing. | +| Wrap schema and data in **one transaction** | Schema applies first, then data. A failed import re-runs data alone; see [How It Works](/guides/environment-sync/how-it-works#how-a-push-applies). | +| Select **individual records** | Resource selection is by type (`--roles`), never by row. | +| Model **code-first** | Model in the Data Studio and pull. Don't hand-author the JSON files. | +| **Seal the data plan** against target drift | Only the schema apply is sealed against the target changing between preview and apply. The data preview is the server's own dry-run answer, but it is advisory. | ## Known limitations @@ -63,3 +63,4 @@ None of these sync today. Some are shared configuration that a future release co - **Unlicensed custom permission rules are invisible to the export.** On an instance without a license, the API hides custom permission rules, so a pull cannot export them. The pull detects the shortfall and marks the export incomplete: `merge` and `add` push normally, `mirror` refuses. License the source instance to export them. - **Users without an email address fail import.** The server's import validation rejects a user with no email, so a `--users` push containing one fails before any data applies. Give the account an email on the source, or remove it, first. - **Panels can duplicate on a first sync into a look-alike target.** Panels have no name to match on, so pushing into a target that already holds equivalent panels (seeded from the same template) can create them a second time. The identity map prevents repeats after that first push. +- **A mirror push of users does not protect your own account.** If users are committed and the account the push authenticates with is absent from them, a `mirror` push orders its deletion like any other record; the CLI has no self-protection, and whether the server refuses is up to the server. When you sync users, make sure the committed set includes the accounts your pushes run as. diff --git a/content/guides/14.environment-sync/6.reference.md b/content/guides/14.environment-sync/6.reference.md new file mode 100644 index 00000000..afd0a28a --- /dev/null +++ b/content/guides/14.environment-sync/6.reference.md @@ -0,0 +1,358 @@ +--- +stableId: 342cc194-f6cc-49f8-9455-b4ae10c2e9da +title: Reference +description: Every Environment Sync command and flag, the directus.config.json format, credential resolution, scope tables, push modes, deletion gates, and JSON report formats. +--- + +Everything on this page is lookup material. If you're learning the tool, start with the [Quickstart](/guides/environment-sync/quickstart) and [How It Works](/guides/environment-sync/how-it-works) instead. + +## Commands + +| Command | What it does | +| ------------------ | ---------------------------------------------------------------------------------------- | +| `d6s profile add` | Add or update a profile (name + URL) and optionally save a credential | +| `d6s profile test` | Connect with a profile and print who you are on the instance | +| `d6s sync pull` | Write a source instance's schema and configuration to committable files | +| `d6s sync diff` | Show what a push would change on the target; applies nothing | +| `d6s sync push` | Apply the committed files to a target instance | +| `d6s sync` | Interactive wizard: prompts for source, target, project, and mode, then pulls and pushes | + +The CLI installs as `directus-cli` with `d6s` as an equivalent short alias. + +Global flags, available on every command: + +| Flag | Effect | +| ------------------ | ---------------------------------------------------------------------------------------- | +| `--json` | One machine-readable report on stdout; human status stays off stdout | +| `--no-color` | Disable colored output | +| `--no-interactive` | Disable prompts; behave as in CI | +| `--config ` | Path to `directus.config.json` (default: found by walking up from the current directory) | + +## `d6s profile add` + +```bash +d6s profile add [name] [--url ] [--token ] [--yes] +``` + +| Flag | Effect | +| ----------------- | ---------------------------------------------------------------------- | +| `--url ` | Directus instance URL | +| `--token ` | Static token to save to the credential store for this profile | +| `--yes` | Skip the confirmation when repointing an existing profile to a new URL | + +Adding is an upsert: an existing name is updated. Run without arguments for prompts, which also offer to save a credential (paste a static token, or log in with email and password to save a session; saved sessions refresh themselves before they expire). Profile names use letters, numbers, and underscores. URLs with embedded credentials, query strings, or fragments are refused, because the URL is the part that lands in a committed file. + +## `d6s profile test` + +```bash +d6s profile test +``` + +| Flag | Effect | +| ----------------- | ----------------------------------------------------- | +| `--url ` | Test a URL directly, without a profile or config file | +| `--token ` | Override the resolved token | + +Connects and prints who the credential authenticates as: + +``` +◇ Authenticated to https://cms.example.com as Admin User (Administrator). +``` + +## `d6s sync pull` + +```bash +d6s sync pull --from [scope flags] +``` + +| Flag | Effect | +| ------------------------------ | ------------------------------------------------------------------------------------------------ | +| `--from ` (required) | Source profile name | +| `--collections ` | Schema scope: only these collections (comma-separated) | +| `--exclude-collections ` | Schema scope: all collections except these | +| `--no-schema` | Skip the schema entirely; configuration resources only | +| `--` | Select only the named resources, e.g. `--flows --roles` (plus their dependencies) | +| `--no-` | Keep the default resource set but exclude one, e.g. `--no-flows` | +| `--all` | Every configuration resource, including users and translations | +| `--no-deps` | Do not pull a selected resource's dependencies (dependent children still ride with their parent) | +| `--project ` | Project to sync (default: `default`) | + +The selectable resources are `roles`, `policies`, `flows`, `dashboards`, `settings`, `folders`, `users`, and `translations`; `access`, `permissions`, `operations`, and `panels` ride along with their parents (see [the dependency table](#configuration-resources)). Positive selection (`--flows`) cannot be combined with `--all` or with `--no-` flags. Resource selection never narrows the schema; the two axes are scoped independently. + +Two warnings a scoped pull can raise, neither of which widens the scope for you: + +- **Out-of-scope references.** The scoped snapshot points at something you omitted (a relation target, a group parent, a many-to-any collection). Pushing that snapshot to a fresh target can fail; add the missing collections to `--collections` yourself. +- **A name the server didn't return.** A `--collections` name absent from the returned snapshot (usually a typo) is named in a warning. The partial snapshot still lands, but never silently. + +## `d6s sync diff` + +```bash +d6s sync diff --to [--mode ] [--allow-version-drift] +``` + +| Flag | Effect | +| --------------------------- | ------------------------------------------------------------------------------ | +| `--to ` (required) | Target profile name | +| `--mode ` | `add`, `merge`, or `mirror`; changes what the preview plans for | +| `--allow-version-drift` | Preview despite a version mismatch (see [the version rule](#the-version-rule)) | +| `--project ` | Project to sync (default: `default`) | + +Applies nothing, and exits `0` whether or not differences exist; automation reads the report's `changes` field. + +## `d6s sync push` + +```bash +d6s sync push --to [--mode ] [--yes] [--dangerously-allow-delete] [--allow-version-drift] +``` + +| Flag | Effect | +| ---------------------------- | --------------------------------------------------------------------------- | +| `--to ` (required) | Target profile name | +| `--mode ` | `add`, `merge` (default), or `mirror` | +| `--yes` | Skip the apply confirmation; never authorizes deletions | +| `--dangerously-allow-delete` | Consent to deletions; required for non-interactive `mirror` | +| `--allow-version-drift` | Push despite a version mismatch (see [the version rule](#the-version-rule)) | +| `--project ` | Project to sync (default: `default`) | + +## `directus.config.json` + +Created and updated by `d6s profile add`; found by walking up from the current directory, like git finds `.git`. It never contains credentials, so commit it. The full shape: + +```json +{ + "profiles": { + "staging": { "url": "https://staging.example.com" }, + "production": { "url": "https://cms.example.com" } + }, + "directory": "directus", + "projects": { + "default": { + "schema": true, + "collections": ["pages", "posts"], + "resources": ["flows", "settings"], + "mode": "merge" + } + } +} +``` + +Top-level keys: + +| Key | Meaning | Default | +| ----------- | ------------------------------------------------------- | ------------ | +| `profiles` | Named instances: `{ "url": "https://..." }` per profile | `{}` | +| `directory` | The directory pulls write into and pushes read from | `"directus"` | +| `projects` | Per-project sync options (see below) | `{}` | + +Per-project keys, all optional. A project is a named slice of the sync with its own subdirectory (`//`); the `default` project exists without being declared. Flags on the command line override these per run: + +| Key | Meaning | +| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `schema` | `false` makes this a configuration-only project: pull, diff, and push never touch schema, and reports say `schemaSkipped` so automation can tell a skipped phase from a matching one | +| `collections` | Schema scope: only these collections | +| `excludeCollections` | Schema scope: all collections except these | +| `resources` | Only these configuration resources | +| `excludeResources` | The default resources except these | +| `mode` | Default push mode for this project: `add`, `merge`, or `mirror` | +| `deps` | `false` skips pulling selected resources' dependencies | + +`"schema": false` combined with a collections scope is refused as a contradiction, as is setting both an include and an exclude list for the same axis. Project names use letters, numbers, hyphens, and underscores. + +## Credentials + +When a command authenticates a profile, the token resolves in order: + +1. A `--token` flag, on the two `profile` commands that accept one. The sync commands take no token flag. +2. The `DIRECTUS__TOKEN` environment variable: the profile name uppercased, so `production` reads `DIRECTUS_PRODUCTION_TOKEN`. A `.env` file next to `directus.config.json` is loaded automatically without overriding real environment variables. +3. The credential store at `~/.directus/credentials.json`, written readable only by you (mode `0600`). Never consulted when the `CI` environment variable is set. + +Use an admin credential. The schema endpoints and the batch import the CLI relies on are admin-only on the server; the CLI does not check privileges up front, so a non-admin token fails with an authentication error, or produces an incomplete export that the completeness checks then flag. + +## Configuration resources + +Resource selection follows a dependency graph: selecting a resource pulls in what it needs (unless `--no-deps`). + +| Resource | In default pull | Select directly | Pulls in | Notes | +| -------------- | ---------------------------- | ---------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | +| `roles` | Yes | `--roles` | `policies` | | +| `policies` | Yes | `--policies` | `access`, `permissions` | See the warning below before selecting policies on their own. | +| `access` | Yes, with roles and policies | — | — | Grants attached to users are dropped when users are out of scope; a mirror push does not delete them on the target. | +| `permissions` | Yes, with policies | — | — | Row counts are verified against the server. If the source hides rows (unlicensed custom permission rules), the export is marked incomplete. | +| `flows` | Yes | `--flows` | `operations` | | +| `operations` | Yes, with flows | — | — | | +| `dashboards` | Yes | `--dashboards` | `panels` | | +| `panels` | Yes, with dashboards | — | — | Panels have no field to match on, so a first push into a matching target can duplicate once; the identity map prevents repeats. | +| `settings` | Yes | `--settings` | — | A single record. License and AI credentials, branding images (logos, backgrounds, favicon), and the default storage folder are stripped. | +| `folders` | Yes | `--folders` | — | The media-library folder tree. Distinct from collection folders (Data Studio sidebar groups), which sync as schema. | +| `users` | Opt-in | `--users` | `roles`, `policies` | Secret columns (`password`, `token`, `tfa_secret`, and others) are stripped. | +| `translations` | Opt-in | `--translations` | — | Mirror pushes of translations are not currently supported, which is why they are opt-in. | + +::callout{icon="material-symbols:warning-rounded" color="warning"} +**Select `--roles`, not `--policies` alone** +A selection that pulls policies without roles is not independently pushable when access rows reference roles: access rows carry references to roles, and with no roles in scope a push to a fresh target fails. Select `--roles` instead; it pulls policies and their children too. +:: + +## What a pull touches + +Two rules govern every pull, scoped or not: + +1. A pull only rewrites what it fetched. Everything it did not fetch keeps its committed state, untouched. +2. A push only applies what is committed. Work that never entered the repository cannot ship. + +Refreshed means the file is rewritten from the source; because writes are deterministic, an unchanged resource produces no git diff. Preserved means the file is not touched at all. + +| Pull | Schema files | Configuration files | +| ------------------------------------ | ------------------------------------------------------- | ------------------------------------------------ | +| `pull --from staging` | All refreshed | All refreshed | +| `... --collections posts` | `posts` refreshed, others preserved | All refreshed | +| `... --no-flows` | All refreshed | Flows preserved, others refreshed | +| `... --flows` | All refreshed (resource selection never narrows schema) | Flows and operations refreshed, others preserved | +| `... --flows --no-schema` | All preserved | Flows and operations refreshed, others preserved | +| `... --collections posts --no-flows` | `posts` refreshed, others preserved | Flows preserved, others refreshed | + +## Push modes + +| Mode | Schema | Data | Deletes? | +| ----------------- | ------------------- | ----------------------------------------------------------- | -------------- | +| `add` | Additive | Inserts only; existing records are never updated | No | +| `merge` (default) | Additive | Creates and updates | No | +| `mirror` | May delete in scope | Creates, updates, and deletes records absent from the files | **Yes, gated** | + +A `mirror` push deletes only within what the committed files cover: a snapshot scoped to some collections can delete fields inside those collections, never a collection it doesn't contain. An export the source itself left incomplete (hidden permission rows) is refused at mirror push outright. + +## Deletion gates + +Only `mirror` deletes, and deleting always requires its own explicit consent: + +| Context | To delete you must | +| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | +| Interactive terminal | Review the plan naming the losses, then type the profile name (unless you passed `--dangerously-allow-delete`, which is the consent) | +| Non-interactive / CI | Pass `--dangerously-allow-delete` | + +`--yes` skips the ordinary confirmation prompt, but it never authorizes a deletion; that holds even if a supposedly additive push unexpectedly carries one. A `mirror` push in CI without `--dangerously-allow-delete` refuses before changing anything: + +``` +✖ Refusing mirror mode in a non-interactive context without --dangerously-allow-delete. + mirror can delete schema and data rows absent from the import set; pass --dangerously-allow-delete to consent, or use --mode merge. +``` + +## Record identity + +Records are matched across instances first by the committed identity map (`//id_map.json`), then by an identifying field: + +| Resource | Matched by | +| -------------- | -------------------------- | +| Most resources | `name` | +| Users | `email` | +| Operations | `key` | +| Panels | Nothing; identity map only | + +The map is keyed internally by source and target instance URL, so one committed file serves any number of targets without conflicts. Commit it whenever a push changes it. An ambiguous match (two candidates) prompts interactively and refuses non-interactively: + +``` +✖ Ambiguous target matches: + directus_roles source "Editor" — sr1 → one of t1, t2 + Run d6s sync push interactively once to choose, then commit the updated id map. +``` + +## The version rule + +Schema changes require the snapshot's Directus version and the target's version to match exactly, patch release included; the mismatch error names both versions. `--allow-version-drift` proceeds anyway with a warning; the CLI never translates schema between versions. Projects with `"schema": false` skip the check entirely, and a target whose version cannot be read is left to the server's own gate rather than refused. + +## JSON reports + +With `--json`, stdout carries exactly one report per command; warnings still go to stderr so logs keep them while stdout stays parseable. Reports are emitted as a single line; they're formatted here for readability. + +`d6s sync pull --from staging --json`: + +```json +{ + "kind": "PullReport", + "formatVersion": 1, + "ok": true, + "source": "https://staging.example.com", + "profile": "staging", + "project": "default", + "schemaSkipped": false, + "dir": "directus/default/schema", + "collections": 12, + "fields": 87, + "systemFields": 2, + "relations": 14, + "files": 13, + "removed": [], + "scope": null, + "data": { + "resources": ["directus_flows", "directus_roles"], + "collections": 10, + "records": 57, + "files": 10, + "removed": [], + "incomplete": [] + } +} +``` + +The schema counters (`collections` through `files`) are `null` when the schema phase is skipped; `scope` echoes a `--collections`/`--exclude-collections` scope; `data.incomplete` names resources whose export the source cut short. + +`d6s sync diff --to production --json`: + +```json +{ + "kind": "DiffReport", + "formatVersion": 1, + "ok": true, + "target": "https://cms.example.com", + "profile": "production", + "project": "default", + "mode": "merge", + "changes": true, + "unresolved": 0, + "schemaSkipped": false, + "added": 1, + "modified": 1, + "deleted": 0, + "data": { + "mode": "merge", + "source": "https://staging.example.com", + "collections": { + "directus_flows": { + "existing": [], + "new": ["f1"], + "deleted": [], + "mapped": {} + } + }, + "matched": 1, + "ambiguous": 0, + "unmatched": 1, + "unchanged": 0, + "incomplete": [], + "skipped": false + } +} +``` + +`changes` is `true` when a push would do anything, including when records are `unresolved`; `added`/`modified`/`deleted` count schema items; `data.collections` is the target server's own per-collection dry-run answer. + +`d6s sync push --to production --yes --json` reports the same shape as a diff, with `applied` (`true` when the push changed the target) in place of `unresolved`, and `data.collections` reflecting what the import actually did. + +Failures put an error report on stdout: + +```json +{ + "kind": "ErrorReport", + "formatVersion": 1, + "error": { + "code": "STATE", + "message": "Version mismatch: the snapshot was pulled from Directus 11.2.0, but the target runs 11.2.5.", + "hint": "..." + } +} +``` + +The `code` is one of a small set of failure classes: `USAGE` (the command line needs fixing: a missing flag or missing consent), `CONFIG` (`directus.config.json` missing or invalid), `AUTH` (the credential was rejected), `HTTP` (the instance could not be reached or returned an error), `STATE` (the committed files and the instance disagree: version mismatch, changed target schema, incomplete export), or `UNKNOWN`. Exit codes are `0` for success and `1` for every failure; the code string is the finer-grained signal. + +## Output conventions + +Human-readable status lines go to stderr, prefixed `●` (info), `◇` (success), `▲` (warning), or `✖` (error, with its hint indented beneath). Plan lines go to stdout: `+` marks an addition, `~` a modification, and `✖ DELETE` a deletion, with data plans summarized per collection as `+N new ~N updated ✖N deleted`. `--no-color` disables coloring; `--json` replaces stdout output with the report while warnings stay on stderr. From 2c67ee99c6aa136fb0a6effc2ef99438fcdeb4fc Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Tue, 4 Aug 2026 13:12:42 -0400 Subject: [PATCH 03/12] Address review feedback: show deletion prompts, fix mirror diff mode --- .../14.environment-sync/1.quickstart.md | 4 ++- .../14.environment-sync/2.how-it-works.md | 27 ++++++++++---- .../14.environment-sync/3.common-workflows.md | 35 ++++++++++++++----- .../guides/14.environment-sync/6.reference.md | 12 +++---- 4 files changed, 57 insertions(+), 21 deletions(-) diff --git a/content/guides/14.environment-sync/1.quickstart.md b/content/guides/14.environment-sync/1.quickstart.md index 6f66dda3..f83764fd 100644 --- a/content/guides/14.environment-sync/1.quickstart.md +++ b/content/guides/14.environment-sync/1.quickstart.md @@ -150,7 +150,7 @@ d6s sync pull --from source ``` ◇ Pulled from source — http://localhost:8055 Schema 1 collection → directus/default/schema - Resources 6 records in 10 resources → directus/default/data + Resources 14 records in 10 resources → directus/default/data ``` Your record counts may differ slightly; a fresh instance carries a handful of configuration records (the admin role, its policy, the settings row) even before you touch it. @@ -216,6 +216,8 @@ The push prints the same plan, then asks before applying: `Apply 1 schema change ◇ Push complete. Applied 1 schema change to http://localhost:8056; schema hash verified. No data changes to import. ``` +The "schema hash verified" line is the push double-checking that the target's schema ended up matching the plan it previewed, rather than assuming it did. + Open the target Studio at `http://localhost:8056`: the `articles` collection is there, fields and settings intact. That's the whole loop. Change on one instance, pull to files, review in git, push to another. If a push imports configuration records, it also writes `directus/default/id_map.json`, which remembers which target record each committed record became. Commit it when it changes; [How It Works](/guides/environment-sync/how-it-works#record-identity) explains why. diff --git a/content/guides/14.environment-sync/2.how-it-works.md b/content/guides/14.environment-sync/2.how-it-works.md index 53244f54..935f9781 100644 --- a/content/guides/14.environment-sync/2.how-it-works.md +++ b/content/guides/14.environment-sync/2.how-it-works.md @@ -11,6 +11,8 @@ Environment Sync never moves anything directly between two instances. The files Everything else on this page is a consequence of those two rules. +One honest note about "the files": the CLI reads and writes the directory on disk, and it never checks your git state. Committing is the workflow that makes the files trustworthy, not something the CLI enforces; a push applies the directory as it stands, uncommitted edits included, which is also how the `d6s sync` wizard can pull and push in one pass. Git is how you control and review what enters the files. Keep the tree clean around pushes and the two views never differ. + ## The files A pull writes into a directory you commit (named `directus` by default, one subdirectory per [project](/guides/environment-sync/reference#directusconfigjson)): @@ -44,9 +46,17 @@ The same role or flow carries a different primary key on every instance, so a pu - **The identity map.** Each push records its decisions in `id_map.json`: this committed record became that target record. Later pushes look there first. - **Identifying fields.** A record not yet in the map is matched by the field that names it: `name` for most resources, `email` for a user, `key` for an operation. Panels have no such field, which is why a first push into a look-alike target can duplicate them once. -When two target records could both be the match, the CLI asks you to choose in a terminal and refuses in CI. Ambiguity is never resolved by guessing. Your answer lands in the identity map, so each question is asked exactly once. That's why the map belongs in git: commit it whenever a push changes it, and teammates and CI inherit every decision already made. +When two target records could both be the match, the CLI asks you to choose in a terminal and refuses in CI. Ambiguity is never resolved by guessing. The question looks like this, with the differences between candidates spelled out per option: + +```` +Resolve identity 1 of 1: directus_roles source "Editor" — sr1 matches multiple target records + Use "Editor" — t1 (Same synced values as source; only the ID differs) + Use "Editor" — t2 (icon: source "edit", target "star") + Create a separate record (Adds one record; leaves every existing match unchanged) + Abort the push (Applies no remote changes) +``` Your answer lands in the identity map, so each question is asked exactly once. That's why the map belongs in git: commit it whenever a push changes it, and teammates and CI inherit every decision already made. -One map file serves any number of instances. Internally it is keyed by source and target URL, so pushing the same files to staging and to production writes two independent sets of mappings; neither overwrites the other. Deleting the file is safe to the extent that matching starts over: named records re-match by their identifying fields, and records without one can duplicate on the next push. +One map file serves any number of instances. Internally it is keyed by source and target URL, so pushing the same files to staging and to production writes two independent sets of mappings; neither overwrites the other. Deleting the file is safe to the extent that matching starts over: named records re-match by their identifying fields, and records without one can duplicate on the next push. The same applies to repointing a profile at a new domain: the mappings are keyed by URL, so the old URL's decisions no longer apply and matching starts fresh for the new one. ## How a push applies @@ -57,10 +67,12 @@ A push applies in two phases, schema first, and begins by showing you the full p The two phases are not one transaction. If the data import fails after the schema applied, the target holds the new schema and the old configuration, and the CLI says so plainly: -``` +```` + ▲ Schema was applied, but the data import did not complete. ✖ Could not reach https://cms.example.com. - The import may still have been applied on the server. Run d6s sync diff before retrying — a blind retry can duplicate records. +The import may still have been applied on the server. Run d6s sync diff before retrying — a blind retry can duplicate records. + ``` The guidance is the same in every partial-failure case: run `d6s sync diff` to see where the target actually stands, then push again. A re-run applies only what is still missing, and a completed push re-run reports "nothing to push" after verifying that against the target, not assuming it. @@ -70,7 +82,7 @@ The guidance is the same in every partial-failure case: run `d6s sync diff` to s A push mode answers one question: what happens to things that exist on the target but not in your files? - **`merge`** (the default) creates and updates, and never deletes. Not in the schema phase, not in the data phase. -- **`add`** only creates; it never touches an existing record. +- **`add`** only creates records; it never touches an existing one. Its schema phase behaves exactly like `merge`: the mode only changes what happens to data. - **`mirror`** makes the target match the files exactly, which means deleting what the files no longer contain, within whatever scope the files cover. Deleting always requires its own explicit consent, separate from confirming the push. Interactively, a mirror push lists what would be lost and asks you to type the profile name. Non-interactively it requires the `--dangerously-allow-delete` flag; `--yes` never authorizes a deletion. The gate is a backstop as well as a policy: even if a supposedly additive push somehow carried a deletion, it would still be refused without consent. @@ -82,8 +94,10 @@ Because a mirror push makes the target match the files _exactly_, run it against Schema changes require the files' Directus version and the target's version to match exactly, patch release included, because the server refuses cross-version schema diffs; historically, some patches change the schema format. ``` + ✖ Version mismatch: the snapshot was pulled from Directus 11.2.0, but the target runs 11.2.5. - The server requires an exact version match for schema diffs — historically some patches are breaking. Align both instances (re-pull if the source was upgraded), or pass --allow-version-drift to proceed anyway. +The server requires an exact version match for schema diffs — historically some patches are breaking. Align both instances (re-pull if the source was upgraded), or pass --allow-version-drift to proceed anyway. + ``` `--allow-version-drift` asks the server to proceed anyway and warns you loudly; the CLI never translates schema between versions. Projects configured with `"schema": false` skip the schema phase and this gate entirely. @@ -98,3 +112,4 @@ The rules above add up to a small set of promises, each of which you can watch h - **Identity is never guessed.** An ambiguous record match prompts in a terminal and refuses in CI. - **Stored secrets never enter your repository.** Built-in secret columns and fields marked concealed, hashed, or encrypted are stripped at export, with [one warned exception](/guides/environment-sync/secrets-and-limitations#the-one-blind-spot). - **Failures are loud.** Hand-edited files, cut-short exports, version mismatches, and unreachable instances stop the command with a named error instead of degrading silently. An export the source itself left incomplete is marked as such and refused at mirror push. +``` diff --git a/content/guides/14.environment-sync/3.common-workflows.md b/content/guides/14.environment-sync/3.common-workflows.md index 5ab58760..629ede98 100644 --- a/content/guides/14.environment-sync/3.common-workflows.md +++ b/content/guides/14.environment-sync/3.common-workflows.md @@ -52,7 +52,7 @@ What that pull just did: - The `posts` schema files were rewritten with the new state. - The `authors` schema files were not touched. They still hold what the last full pull captured, which is the state production already runs. The half-finished `authors` work never entered the repository. -- Configuration resources refreshed too, because a collections scope only narrows schema. Nothing changed there, so those files came back byte-identical and `git status` shows only the `posts` files. If a flow had also changed on staging, its file would show up here; hold it back with `--no-flows` on the pull. +- Configuration resources refreshed too, because a collections scope only narrows schema. Nothing changed there, so those files came back byte-identical and `git status` shows only the `posts` files. If a flow had also changed on staging, its file would show up here; hold it back with `--no-flows` on the pull. One thing you cannot hold back on its own: the ride-along resources. Permissions travel with policies and operations with flows, and they have no `--no-` flags of their own, so a teammate's new permission rows come along with any pull that refreshes policies. Commit, then diff and push as usual: @@ -86,7 +86,7 @@ Most projects don't start from an empty instance. You have a development instanc 2. Pull it as your baseline and commit: ```bash - d6s profile add production --url https://cms.example.com + d6s profile add production --url https://cms.example.com # offers to save a credential; or pass --token d6s sync pull --from production git add directus/ directus.config.json git commit -m "Baseline from production" @@ -124,20 +124,39 @@ A change made it to production and turned out to be wrong. The committed files a - If the bad change **modified** something, reverting the commit and pushing with `merge` restores the old values. - If the bad change **added** something, `merge` cannot undo it: after the revert, the field or record is simply absent from your files, and merge never deletes what the files don't mention. Removal is a deletion, and deletions need `mirror`. -The steps: +Say the bad commit added a `summary` field and a notification flow, and tweaked the note on `articles.title`. Revert it, then preview the undo in mirror mode (the default `merge` preview plans no deletions, so it cannot show you a removal): ```bash git revert -d6s sync diff --to production +d6s sync diff --to production --mode mirror +``` + +``` +● Comparing committed files with production — https://cms.example.com (mirror — INCLUDES DELETIONS) +● Schema — 2 changes: 0 added, 1 modified, 1 deleted +✖ DELETE field articles.summary +~ field articles.title (meta.note) +● Data — 1 change: 0 created, 0 updated, 1 deleted +~ directus_flows +0 new ~0 updated ✖1 deleted (f7) ``` -The diff now describes the undo. Read the deletions closely: they should name exactly what the bad change added, and nothing else. If anything else shows up as a deletion, your files are stale somewhere; stop and reconcile first (usually a fresh full pull on a branch to compare against). +The `~` line restores the old note. The `✖ DELETE` and `✖1 deleted` lines are the undo, and they should name exactly what the bad change added, and nothing else. If anything else shows up for deletion, your files are stale somewhere: stop, run a full pull from production on a scratch branch, read that diff to see what production actually holds, and fold anything worth keeping into your files before mirroring. ```bash d6s sync push --to production --mode mirror ``` -The [deletion gate](/guides/environment-sync/reference#deletion-gates) applies: the push lists the losses and asks you to type the profile name, or requires `--dangerously-allow-delete` in automation. +The push repeats the plan, then the [deletion gate](/guides/environment-sync/reference#deletion-gates) demands typed consent: + +``` +This push permanently deletes 1 record and 1 schema item from production. Type "production" to confirm: +``` + +In automation, the same push requires `--dangerously-allow-delete` instead. + +::callout{icon="material-symbols:warning-rounded" color="warning"} +**Undoing an added field deletes its data.** Mirroring away `summary` drops the column and everything editors typed into it since the bad push. The committed files cover configuration; only a database backup covers data. Back up the target first if that data matters. +:: When the bad change is tangled up with good ones, fixing forward is often simpler than reverting: correct it on the development instance, pull, and promote the fix like any other change. @@ -157,10 +176,10 @@ Sometimes production changes outside the deployment path, usually an urgent manu 2. Preview what re-aligning staging would mean: ```bash - d6s sync diff --to staging + d6s sync diff --to staging --mode mirror ``` - Read the deletions closely. Anything that exists only on staging and falls inside the sync's scope is on the list. + Read the deletions closely. Anything that exists only on staging and falls inside the sync's scope is on the list. (The mode matters: a default `merge` preview plans no deletions, so only a mirror diff shows what re-aligning would remove.) 3. Make staging match the committed state: diff --git a/content/guides/14.environment-sync/6.reference.md b/content/guides/14.environment-sync/6.reference.md index afd0a28a..0405679e 100644 --- a/content/guides/14.environment-sync/6.reference.md +++ b/content/guides/14.environment-sync/6.reference.md @@ -197,7 +197,7 @@ A selection that pulls policies without roles is not independently pushable when Two rules govern every pull, scoped or not: 1. A pull only rewrites what it fetched. Everything it did not fetch keeps its committed state, untouched. -2. A push only applies what is committed. Work that never entered the repository cannot ship. +2. A push only applies what is in the files. Work that never entered them cannot ship. Refreshed means the file is rewritten from the source; because writes are deterministic, an unchanged resource produces no git diff. Preserved means the file is not touched at all. @@ -212,11 +212,11 @@ Refreshed means the file is rewritten from the source; because writes are determ ## Push modes -| Mode | Schema | Data | Deletes? | -| ----------------- | ------------------- | ----------------------------------------------------------- | -------------- | -| `add` | Additive | Inserts only; existing records are never updated | No | -| `merge` (default) | Additive | Creates and updates | No | -| `mirror` | May delete in scope | Creates, updates, and deletes records absent from the files | **Yes, gated** | +| Mode | Schema | Data | Deletes? | +| ----------------- | ---------------------------------- | ----------------------------------------------------------- | -------------- | +| `add` | Adds and modifies, same as `merge` | Inserts only; existing records are never updated | No | +| `merge` (default) | Adds and modifies | Creates and updates | No | +| `mirror` | May delete in scope | Creates, updates, and deletes records absent from the files | **Yes, gated** | A `mirror` push deletes only within what the committed files cover: a snapshot scoped to some collections can delete fields inside those collections, never a collection it doesn't contain. An export the source itself left incomplete (hidden permission rows) is refused at mirror push outright. From 82ae7325f0e074e1ff51094442b99aff79673c37 Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Thu, 6 Aug 2026 11:19:50 -0400 Subject: [PATCH 04/12] moar wip --- content/guides/14.environment-sync/0.index.md | 24 ++-- .../14.environment-sync/1.quickstart.md | 26 ++-- .../14.environment-sync/2.how-it-works.md | 75 +++++----- .../14.environment-sync/3.common-workflows.md | 51 +++---- .../4.ci-and-automation.md | 12 +- .../5.secrets-and-limitations.md | 61 ++++---- .../guides/14.environment-sync/6.reference.md | 134 +++++++++--------- 7 files changed, 194 insertions(+), 189 deletions(-) diff --git a/content/guides/14.environment-sync/0.index.md b/content/guides/14.environment-sync/0.index.md index 66e725d8..97e837ac 100644 --- a/content/guides/14.environment-sync/0.index.md +++ b/content/guides/14.environment-sync/0.index.md @@ -1,38 +1,38 @@ --- stableId: 393b999d-c4ed-4b83-9208-cf088a4918ab title: Overview -description: Move schema and configuration between Directus instances through files you commit to git, using the Directus CLI. +description: Move schema and configuration between Directus instances through sync files you commit to git, using the Directus CLI. --- -You build your data model on a development instance, and at some point that work has to reach staging and production. Environment Sync makes that a git workflow: the Directus CLI (`directus-cli`, or its short alias `d6s`) writes an instance's **schema and configuration** to JSON files you commit, and applies those files to any instance you point it at. +You build your data model on a development instance, and at some point that work has to reach staging and production. Environment Sync makes that a git workflow: the Directus CLI (`directus-cli`, or its short alias `d6s`) writes an instance's **Schema and Configuration** to JSON sync files you commit, then applies those files to any instance you point it at. ```bash -d6s sync pull --from staging # write the instance's schema + configuration to files -d6s sync diff --to production # preview what pushing those files would change +d6s sync pull --from staging # write the instance's Schema + Configuration to sync files +d6s sync diff --to production # preview what pushing the sync files would change d6s sync push --to production # apply them ``` -Because the files live in your repository, your normal review workflow applies: schema changes show up in pull requests, environments are brought up to date through git history, and a bad change is a revert away from being found. (`d6s sync` on its own runs a small interactive wizard that pulls and pushes in one pass.) +Because the sync files live in your repository, you can review changes in pull requests, promote them through git history, and roll them back with a revert. (`d6s sync` on its own runs a small interactive wizard that pulls and pushes in one pass.) -The CLI is built to be safe to point at production: `diff` never applies anything, deletions always require their own explicit consent, and a record that could match more than one target record is asked about, never guessed. [How It Works](/guides/environment-sync/how-it-works) covers the full safety model. +The CLI is built to be safe to point at production: `diff` never applies anything, deletions always require their own explicit consent, and the CLI asks you to resolve records that match more than one target record. It never guesses. [How It Works](/guides/environment-sync/how-it-works) covers the full safety model. ## What syncs - **Schema**: every collection, field, and relation, including custom fields on system collections. -- **Configuration**: roles, policies, access, permissions, flows, operations, dashboards, panels, settings, and media-library folders. -- **Opt-in**: user accounts (with every secret column stripped) and translation strings. +- **Configuration**: roles, policies, access, permissions, flows, operations, dashboards, panels, settings, media-library folders, and custom Data Studio translation strings. +- **Opt-in**: user accounts, with every secret field stripped. -Your own collections' **content**, the rows in the tables you create, is not synced. Environment Sync moves the shape of a project and its configuration, not its data. +Records in your own collections are **content**, and Environment Sync does not sync them. It moves the shape of a project and its configuration, not its content. ::callout{icon="material-symbols:info-outline-rounded"} -Content sync is deferred to a future release. Cross-instance record identity for integer primary keys, file references, and user references needs its own architecture, and a wrong guess there means overwritten data on the target. +Content sync is deferred to a future release. Cross-instance record identity for integer primary keys, file references, and user references needs its own architecture, and a wrong guess there could overwrite content on the target. :: ## Before you start - **Both instances must run the same Directus version, patch release included.** The server refuses cross-version schema changes because some patch releases change the schema format. Align your environments before you sync them. - **An admin credential for each instance.** The schema and configuration endpoints the CLI uses are admin-only. A static token from an admin user is the usual choice. -- **A git repository.** The files the CLI writes are only useful committed. Any repository works; many teams use the one that already holds their Directus deployment configuration. +- **A git repository.** The sync files are designed for review and versioning. Any repository works; many teams use the one that already holds their Directus deployment configuration. ## Where to go @@ -43,7 +43,7 @@ Run the full pull, diff, push loop against two throwaway instances and see every ::: :::card{title="How It Works" icon="i-ph-lightbulb" to="/guides/environment-sync/how-it-works"} -The mental model: files as the source of truth, record identity, push phases, and the safety rules. +The mental model: sync files as the source of truth, record identity, push phases, and the safety rules. ::: :::card{title="Common Workflows" icon="i-ph-map-trifold" to="/guides/environment-sync/common-workflows"} diff --git a/content/guides/14.environment-sync/1.quickstart.md b/content/guides/14.environment-sync/1.quickstart.md index f83764fd..7adb34de 100644 --- a/content/guides/14.environment-sync/1.quickstart.md +++ b/content/guides/14.environment-sync/1.quickstart.md @@ -4,7 +4,7 @@ title: Quickstart description: Run the full pull, diff, push loop against two throwaway Directus instances, and see what the CLI does at every step. --- -The fastest way to trust Environment Sync is to watch it work somewhere mistakes are free. In this guide you stand up two throwaway Directus instances with Docker, make a change on one, and move it to the other through git. Nothing here touches a real project, and at the end one command tears it all down. +The fastest way to trust Environment Sync is to watch it work somewhere mistakes are free. In this guide, you stand up two throwaway Directus instances with Docker, make a change on one, and move it to the other through git. Nothing here touches a real project, and one command tears it all down at the end. You need Docker, Node.js with npm, and git. @@ -138,7 +138,7 @@ d6s profile test source In the source Studio at `http://localhost:8055`, create a collection called `articles` with two fields: `title` and `status`. This plays the part of a day's modeling work on a development instance. -## Pull it into files +## Pull it into sync files Make the directory a git repository, then pull: @@ -150,10 +150,10 @@ d6s sync pull --from source ``` ◇ Pulled from source — http://localhost:8055 Schema 1 collection → directus/default/schema - Resources 14 records in 10 resources → directus/default/data + Resources 14 records in 11 resources → directus/default/data ``` -Your record counts may differ slightly; a fresh instance carries a handful of configuration records (the admin role, its policy, the settings row) even before you touch it. +Your record counts may differ slightly; a fresh instance carries a handful of configuration records (the admin role, its policy, and the settings record) even before you touch it. Look at what appeared: @@ -179,7 +179,7 @@ d6s sync pull --from source git diff ``` -The diff touches one file, and inside it, only the new `summary` field. The CLI writes files deterministically: no timestamps, no reshuffling, nothing but your change. This is the property that makes the files reviewable, and it means a pull that finds nothing new leaves your working tree clean. Commit the field. +The diff touches one file, and inside it, only the new `summary` field. The CLI writes sync files deterministically: no timestamps, no reshuffling, nothing but your change. This makes the sync files reviewable and means a pull that finds nothing new leaves your working tree clean. Commit the field. ```bash git add directus/ @@ -188,20 +188,20 @@ git commit -m "Add articles.summary" ## Preview against the target -The target instance is still empty. Ask what pushing the committed files would do to it: +The target instance is still empty. Ask what pushing the sync files would do to it: ```bash d6s sync diff --to target ``` ``` -● Comparing committed files with target — http://localhost:8056 (merge — additive, no deletions) +● Comparing commit-ready files with target — http://localhost:8056 (merge — additive, no deletions) ● Schema — 1 change: 1 added, 0 modified, 0 deleted + collection articles (3 fields) -● Data — no changes to import. +● Configuration — no changes to push. ``` -Read the plan: one collection to add, and the mode line tells you up front that `merge`, the default, never deletes anything. If your data section lists a few configuration records instead of "no changes", that's fine; fresh instances can differ slightly in their defaults. `diff` applied nothing; it's always safe to run. +Read the plan: one collection to add, and the mode line tells you up front that `merge`, the default, never deletes anything. If the Configuration section lists a few records instead of "no changes", that's fine; fresh instances can differ slightly in their defaults. `diff` applied nothing; it's always safe to run. ## Push @@ -213,14 +213,14 @@ The push prints the same plan, then asks before applying: `Apply 1 schema change ``` ● Schema applied. -◇ Push complete. Applied 1 schema change to http://localhost:8056; schema hash verified. No data changes to import. +◇ Push complete. Applied 1 schema change to http://localhost:8056; schema hash verified. No configuration changes to push. ``` The "schema hash verified" line is the push double-checking that the target's schema ended up matching the plan it previewed, rather than assuming it did. -Open the target Studio at `http://localhost:8056`: the `articles` collection is there, fields and settings intact. That's the whole loop. Change on one instance, pull to files, review in git, push to another. +Open the target Studio at `http://localhost:8056`: the `articles` collection is there, fields and settings intact. That's the whole loop. Change one instance, pull to sync files, review in git, and push to another instance. -If a push imports configuration records, it also writes `directus/default/id_map.json`, which remembers which target record each committed record became. Commit it when it changes; [How It Works](/guides/environment-sync/how-it-works#record-identity) explains why. +If a push imports configuration records, it also writes `directus/default/id_map.json`, which records the target record that corresponds to each source record. Commit it when it changes; [How It Works](/guides/environment-sync/how-it-works#record-identity) explains why. ## Push again @@ -230,7 +230,7 @@ d6s sync push --to target ``` ● Pushing to target — http://localhost:8056 (merge — additive, no deletions) -◇ target already matches the committed files — schema and data match; nothing to push. +◇ target already matches the commit-ready files — schema and configuration match; nothing to push. ``` Nothing to confirm, nothing applied. The CLI checked the target and proved it matches; it didn't assume. Re-running a completed push is safe, which is exactly what automation needs. diff --git a/content/guides/14.environment-sync/2.how-it-works.md b/content/guides/14.environment-sync/2.how-it-works.md index 935f9781..b264d699 100644 --- a/content/guides/14.environment-sync/2.how-it-works.md +++ b/content/guides/14.environment-sync/2.how-it-works.md @@ -1,32 +1,32 @@ --- stableId: 5ec649aa-ca39-4d4c-b0f6-a10cbaf0cf38 title: How It Works -description: The mental model behind Environment Sync, including files as the source of truth, record identity across instances, how a push applies, and the safety rules every command follows. +description: The mental model behind Environment Sync, including sync files as the source of truth, record identity across instances, how a push applies, and the safety rules every command follows. --- -Environment Sync never moves anything directly between two instances. The files in your repository sit in the middle, and two rules govern everything the commands do: +Environment Sync never moves anything directly between two instances. The sync files sit in the middle, and two rules govern everything the commands do: -1. **A pull only rewrites what it fetched.** Everything it did not fetch keeps its committed state, untouched. -2. **A push only applies what is in the files.** A push reads your repository, not the source instance. Work that never entered the repository cannot ship. +1. **A pull writes sync files only for what it fetched.** Sync files for anything it did not fetch stay unchanged. +2. **A push only applies what is in the sync files.** A push reads the files on disk, not the source instance. Work that never entered the sync files cannot ship. Everything else on this page is a consequence of those two rules. -One honest note about "the files": the CLI reads and writes the directory on disk, and it never checks your git state. Committing is the workflow that makes the files trustworthy, not something the CLI enforces; a push applies the directory as it stands, uncommitted edits included, which is also how the `d6s sync` wizard can pull and push in one pass. Git is how you control and review what enters the files. Keep the tree clean around pushes and the two views never differ. +The CLI reads and writes the sync directory on disk. It never checks your git state, so a push applies the sync files as they exist, including uncommitted edits. This also allows the `d6s sync` wizard to pull and push in one pass. Git provides review and history around the sync files. Keep the tree clean when you push so the files on disk match the reviewed commit. -## The files +## The sync files A pull writes into a directory you commit (named `directus` by default, one subdirectory per [project](/guides/environment-sync/reference#directusconfigjson)): ``` directus/default/ - schema/ # the schema, one JSON file per collection + schema/ # Schema, one JSON file per collection data/ # configuration records, one JSON file per resource - id_map.json # which target record each committed record became + id_map.json # which target record corresponds to each source record ``` -The files are written deterministically: pulling twice with no instance changes produces byte-identical files and a clean working tree. `git diff` after a pull shows what changed on the instance and nothing else, so a schema change reads like any other code change in review. +The sync files are written deterministically: pulling twice with no instance changes produces byte-identical files and a clean working tree. `git diff` after a pull shows what changed on the instance and nothing else, so a schema change reads like any other code change in review. -Each directory also holds a `metadata.json` that lists the files the CLI wrote. The CLI treats that list as ownership: it removes a stale file it wrote on an earlier pull, and it never deletes a file it did not write. Hand-edited or corrupt files stop the command with a named error rather than syncing something the instance never said. +Each directory also holds a `metadata.json` that lists the sync files the CLI wrote. The CLI treats that list as ownership: it removes a stale file it wrote on an earlier pull, and it never deletes a file it did not write. Hand-edited or corrupt sync files stop the command with a named error instead of applying invalid state. Profiles and per-project settings live in `directus.config.json` at the repository root. It contains URLs and scoping options, never credentials, so it is safe to commit. The [reference](/guides/environment-sync/reference#directusconfigjson) shows the full file. @@ -35,63 +35,65 @@ Profiles and per-project settings live in `directus.config.json` at the reposito A pull covers two independent things, and you can scope each without affecting the other: - **Schema**: collections, fields, relations. Scope it with `--collections` or `--exclude-collections`, or skip it with `--no-schema`. -- **Configuration resources**: records of ten `directus_*` resource types (flows, roles, settings, and the rest), plus opt-in users and translations. Scope it with resource flags like `--flows` or `--no-flows`. +- **Configuration resources**: records of eleven `directus_*` resource types (flows, roles, settings, translations, and the rest), plus opt-in users. Scope it with resource flags like `--flows` or `--no-flows`. -Scoping narrows what a pull _fetches_; combined with rule 1, that means a scoped pull refreshes its slice of the files and preserves every other file exactly as committed. The [pull scope matrix](/guides/environment-sync/reference#what-a-pull-touches) in the reference lists what each flag combination refreshes and preserves, and [Common Workflows](/guides/environment-sync/common-workflows#promote-only-the-changes-that-are-ready) shows why you'd want that: shipping finished work while half-finished work stays out of the repository. +Scoping narrows what a pull _fetches_. The CLI writes the current source state to the sync files for that scope and does not write any other sync files. The [pull scope matrix](/guides/environment-sync/reference#what-a-pull-touches) lists which sync files each flag combination writes, and [Common Workflows](/guides/environment-sync/common-workflows#promote-only-the-changes-that-are-ready) shows how to ship finished work while half-finished work stays out of the repository. ## Record identity -The same role or flow carries a different primary key on every instance, so a push has to decide which target record a committed record _is_ before it can update rather than duplicate. Two mechanisms decide, in order: +The same role or flow carries a different primary key on every instance, so a push has to decide which target record corresponds to each source record before it can update rather than duplicate. Two mechanisms decide, in order: -- **The identity map.** Each push records its decisions in `id_map.json`: this committed record became that target record. Later pushes look there first. -- **Identifying fields.** A record not yet in the map is matched by the field that names it: `name` for most resources, `email` for a user, `key` for an operation. Panels have no such field, which is why a first push into a look-alike target can duplicate them once. +- **The ID map.** Each push records its decisions in `id_map.json`: this source record corresponds to that target record. Later pushes look there first. +- **Identifying fields.** A record not yet in the map is matched by the field that names it: `name` for most resources, `email` for a user, `key` for an operation, and `language` plus `key` for a translation. Panels have no such field, which is why a first push into a look-alike target can duplicate them once. + +For an existing translation, the CLI replaces the source ID with the matching target ID and sends the complete record. Directus accepts an update that repeats that record's current `language` and `key`, so both `merge` and `mirror` can update translation strings. A pair already owned by another translation still fails as a duplicate. When two target records could both be the match, the CLI asks you to choose in a terminal and refuses in CI. Ambiguity is never resolved by guessing. The question looks like this, with the differences between candidates spelled out per option: -```` +```text Resolve identity 1 of 1: directus_roles source "Editor" — sr1 matches multiple target records Use "Editor" — t1 (Same synced values as source; only the ID differs) Use "Editor" — t2 (icon: source "edit", target "star") Create a separate record (Adds one record; leaves every existing match unchanged) Abort the push (Applies no remote changes) -``` Your answer lands in the identity map, so each question is asked exactly once. That's why the map belongs in git: commit it whenever a push changes it, and teammates and CI inherit every decision already made. +``` + +Your answer lands in the ID map, so each question is asked exactly once. That's why the map belongs in git: commit it whenever a push changes it, and teammates and CI inherit every decision already made. -One map file serves any number of instances. Internally it is keyed by source and target URL, so pushing the same files to staging and to production writes two independent sets of mappings; neither overwrites the other. Deleting the file is safe to the extent that matching starts over: named records re-match by their identifying fields, and records without one can duplicate on the next push. The same applies to repointing a profile at a new domain: the mappings are keyed by URL, so the old URL's decisions no longer apply and matching starts fresh for the new one. +One ID map serves any number of instances. Internally it is keyed by source and target URL, so pushing the same sync files to staging and production writes two independent sets of mappings; neither overwrites the other. Deleting the ID map is safe to the extent that matching starts over: named records re-match by their identifying fields, and records without one can duplicate on the next push. The same applies to repointing a profile at a new domain: the mappings are keyed by URL, so the old URL's decisions no longer apply and matching starts fresh for the new one. ## How a push applies -A push applies in two phases, schema first, and begins by showing you the full plan and asking (unless you pass `--yes`). +A push applies in two phases, Schema first, and begins by showing you the full plan and asking (unless you pass `--yes`). 1. **Schema.** Before applying, the push re-checks that the target's schema still matches what the plan was computed against. If someone changed the target between preview and apply, the push stops rather than apply a stale plan. -2. **Data.** Configuration records import in a single server-side transaction once the schema is in place. +2. **Configuration.** Configuration records import in a single server-side transaction once the schema is in place. -The two phases are not one transaction. If the data import fails after the schema applied, the target holds the new schema and the old configuration, and the CLI says so plainly: +The two phases are not one transaction. If the Configuration push fails after Schema applied, the two phases can be out of sync, and the CLI says so plainly: -```` - -▲ Schema was applied, but the data import did not complete. +```text +▲ Schema was applied, but the configuration push did not complete. ✖ Could not reach https://cms.example.com. -The import may still have been applied on the server. Run d6s sync diff before retrying — a blind retry can duplicate records. - + Schema is already applied — re-run d6s sync push to retry the configuration push against an empty schema diff. ``` The guidance is the same in every partial-failure case: run `d6s sync diff` to see where the target actually stands, then push again. A re-run applies only what is still missing, and a completed push re-run reports "nothing to push" after verifying that against the target, not assuming it. ## Push modes and deletions -A push mode answers one question: what happens to things that exist on the target but not in your files? +A push mode answers one question: what happens to things that exist on the target but not in your sync files? -- **`merge`** (the default) creates and updates, and never deletes. Not in the schema phase, not in the data phase. -- **`add`** only creates records; it never touches an existing one. Its schema phase behaves exactly like `merge`: the mode only changes what happens to data. -- **`mirror`** makes the target match the files exactly, which means deleting what the files no longer contain, within whatever scope the files cover. +- **`merge`** (the default) creates and updates, and never deletes. Not in the Schema phase, not in the Configuration phase. +- **`add`** only creates records; it never touches an existing one. Its Schema phase behaves exactly like `merge`: the mode only changes what happens to configuration records. +- **`mirror`** makes the target match the sync files exactly, which means deleting what the sync files no longer contain, within whatever scope they cover. Deleting always requires its own explicit consent, separate from confirming the push. Interactively, a mirror push lists what would be lost and asks you to type the profile name. Non-interactively it requires the `--dangerously-allow-delete` flag; `--yes` never authorizes a deletion. The gate is a backstop as well as a policy: even if a supposedly additive push somehow carried a deletion, it would still be refused without consent. -Because a mirror push makes the target match the files _exactly_, run it against a freshly pulled state. A mirror from stale files applies the stale state, including deleting things that only look obsolete because the files are old. +Because a mirror push makes the target match the sync files _exactly_, run it against a freshly pulled state. A mirror from stale sync files applies the stale state, including deleting things that only look obsolete because the files are old. ## The version gate -Schema changes require the files' Directus version and the target's version to match exactly, patch release included, because the server refuses cross-version schema diffs; historically, some patches change the schema format. +Schema changes require the version recorded in the sync files and the target's version to match exactly, patch release included, because the server refuses cross-version schema diffs; historically, some patches change the schema format. ``` @@ -100,16 +102,15 @@ The server requires an exact version match for schema diffs — historically som ``` -`--allow-version-drift` asks the server to proceed anyway and warns you loudly; the CLI never translates schema between versions. Projects configured with `"schema": false` skip the schema phase and this gate entirely. +`--allow-version-drift` asks the server to proceed anyway and warns you loudly; the CLI never translates schema between versions. Projects configured with `"schema": false` skip the Schema phase and this gate entirely. ## The safety model The rules above add up to a small set of promises, each of which you can watch hold in the [Quickstart](/guides/environment-sync/quickstart): - **`pull` is read-only on the source.** Every request it makes is a read; it changes nothing on the instance it snapshots. (The one exception: a profile that authenticates with a saved login session refreshes that session when it is close to expiring.) -- **`diff` applies nothing.** The schema comparison is a preview, and the data plan comes from the target server dry-running the import inside a transaction and rolling it back, so the plan is the server's own answer, not a client-side guess. +- **`diff` applies nothing.** The Schema comparison is a preview, and the Configuration plan comes from the target server dry-running the import inside a transaction and rolling it back, so the plan is the server's own answer, not a client-side guess. - **Deletions are gated.** Only `mirror` deletes, always behind its own consent, and `--yes` never covers it. - **Identity is never guessed.** An ambiguous record match prompts in a terminal and refuses in CI. -- **Stored secrets never enter your repository.** Built-in secret columns and fields marked concealed, hashed, or encrypted are stripped at export, with [one warned exception](/guides/environment-sync/secrets-and-limitations#the-one-blind-spot). -- **Failures are loud.** Hand-edited files, cut-short exports, version mismatches, and unreachable instances stop the command with a named error instead of degrading silently. An export the source itself left incomplete is marked as such and refused at mirror push. -``` +- **Stored secrets never enter your repository.** Built-in secret columns and fields marked concealed, hashed, or encrypted are stripped during pull, with [one warned exception](/guides/environment-sync/secrets-and-limitations#the-one-blind-spot). +- **Failures are loud.** Hand-edited sync files, incomplete pulls, version mismatches, and unreachable instances stop the command with a named error instead of degrading silently. A pull the source itself left incomplete is marked as such and refused at mirror push. diff --git a/content/guides/14.environment-sync/3.common-workflows.md b/content/guides/14.environment-sync/3.common-workflows.md index 629ede98..83589577 100644 --- a/content/guides/14.environment-sync/3.common-workflows.md +++ b/content/guides/14.environment-sync/3.common-workflows.md @@ -4,11 +4,11 @@ title: Common Workflows description: End-to-end walkthroughs of the most common Environment Sync workflows, from promoting changes to production to adopting sync on an existing project, rolling back, and recovering a drifted environment. --- -These workflows cover most day-to-day use of Environment Sync: promoting changes to production (all of them, or just the ones that are ready), starting to use sync on a project that already exists, undoing a bad push, re-aligning an environment after manual changes, and standing up a new environment from the committed files. The concepts behind each step live in [How It Works](/guides/environment-sync/how-it-works); every flag in [Reference](/guides/environment-sync/reference). +These workflows cover most day-to-day use of Environment Sync: promoting changes to production (all of them, or just the ones that are ready), starting to use sync on a project that already exists, undoing a bad push, re-aligning an instance after manual changes, and standing up a new instance from the sync files. The concepts behind each step live in [How It Works](/guides/environment-sync/how-it-works); every flag is in [Reference](/guides/environment-sync/reference). ## Promote changes from development to production -You model in the Data Studio on a development instance. Production only ever receives what git has reviewed. +You model in the Data Studio on a development instance. Production only receives sync files reviewed in git. 1. Make your changes on the development instance: collections, fields, flows, permissions. @@ -20,7 +20,7 @@ You model in the Data Studio on a development instance. Production only ever rec git commit -m "Add author bio fields" ``` - The files are deterministic, so the commit shows your change and nothing else. On a development instance other people also use, a scoped pull (`--collections posts`, or a resource flag like `--flows`) keeps their in-progress work out of your diff. + The sync files are deterministic, so the commit shows your change and nothing else. On a development instance other people also use, a scoped pull (`--collections posts`, or a resource flag like `--flows`) keeps their in-progress work out of your diff. 3. Open a pull request. Reviewers read the change as plain JSON diffs. A CI check can add the target's view of the same change: @@ -50,9 +50,10 @@ d6s sync pull --from staging --collections posts What that pull just did: -- The `posts` schema files were rewritten with the new state. -- The `authors` schema files were not touched. They still hold what the last full pull captured, which is the state production already runs. The half-finished `authors` work never entered the repository. -- Configuration resources refreshed too, because a collections scope only narrows schema. Nothing changed there, so those files came back byte-identical and `git status` shows only the `posts` files. If a flow had also changed on staging, its file would show up here; hold it back with `--no-flows` on the pull. One thing you cannot hold back on its own: the ride-along resources. Permissions travel with policies and operations with flows, and they have no `--no-` flags of their own, so a teammate's new permission rows come along with any pull that refreshes policies. +- The `posts` Schema sync files were written in this pull using the current source state. +- The `authors` Schema sync files were not written in this pull, so they stayed unchanged on disk. They still hold what the last full pull captured, which is the state production already runs. The half-finished `authors` work never entered the sync files. +- The Configuration sync files were also written in this pull because a collection scope only narrows Schema. Their source state had not changed, so the files remained byte-identical and `git status` shows only the `posts` files. If a flow had changed on staging, its file would appear too; exclude it with `--no-flows`. +- Dependent resources cannot be excluded individually. Permissions are included with policies, and operations are included with flows. A teammate's new permission records therefore appear whenever the pull includes policies. Commit, then diff and push as usual: @@ -63,7 +64,7 @@ d6s sync diff --to production d6s sync push --to production ``` -A push always applies the whole committed folder. The `posts` change applies. The `authors` files already match production, so nothing happens there. And since the unfinished `authors` work is not in the repository, it cannot ship, no matter what state staging is in. +A push always applies the full sync project on disk. The `posts` change applies. The `authors` files already match production, so nothing happens there. The unfinished `authors` work is absent from the sync files, so it cannot ship, no matter what state staging is in. For a configuration-only change, flip the scope: select the resource type and skip schema. @@ -71,10 +72,10 @@ For a configuration-only change, flip the scope: select the resource type and sk d6s sync pull --from staging --flows --no-schema ``` -The [What a pull touches](/guides/environment-sync/reference#what-a-pull-touches) table shows exactly which files each combination refreshes and which it leaves alone. +The [What a pull touches](/guides/environment-sync/reference#what-a-pull-touches) table shows exactly which sync files each combination writes and does not write. ::callout{icon="material-symbols:warning-rounded" color="warning"} -**Selection is by collection and resource type, not by record.** `--collections posts` can separate finished posts work from unfinished authors work, but two changes inside the same collection travel together, and `--flows` takes every flow, not just one. If unrelated work shares a collection or a resource type, ship it together or wait. And while the committed files hold a partial picture, avoid `mirror`: it would apply the stale remainder too. +**Selection is by collection and resource type, not by record.** `--collections posts` can separate finished posts work from unfinished authors work, but two changes inside the same collection travel together, and `--flows` takes every flow, not just one. If unrelated work shares a collection or a resource type, ship it together or wait. While the sync files hold a partial picture, avoid `mirror`: it would apply the stale remainder too. :: ## Adopt Environment Sync on an existing project @@ -107,7 +108,7 @@ Most projects don't start from an empty instance. You have a development instanc d6s sync push --to staging ``` - Once nothing on staging is worth keeping outside the committed files, a `mirror` push finishes the job and makes it match exactly. + Once nothing on staging is worth keeping outside the sync files, a `mirror` push finishes the job and makes it match exactly. 5. Expect a few identity questions on the first push. Staging and production often hold records that are plausibly the same one (two roles named "Editor", two flows built from the same template). The CLI asks instead of guessing; answer once, then commit the updated `id_map.json`. The questions do not come back. @@ -119,10 +120,10 @@ Nervous about the first push against a real instance? Rehearse the whole loop ag ## Roll back a bad push -A change made it to production and turned out to be wrong. The committed files are the record of every state production has been in, so rollback is a git operation followed by a push. One asymmetry matters: +A change made it to production and turned out to be wrong. Git history records every reviewed state of the sync files, so rollback is a git operation followed by a push. One asymmetry matters: - If the bad change **modified** something, reverting the commit and pushing with `merge` restores the old values. -- If the bad change **added** something, `merge` cannot undo it: after the revert, the field or record is simply absent from your files, and merge never deletes what the files don't mention. Removal is a deletion, and deletions need `mirror`. +- If the bad change **added** something, `merge` cannot undo it: after the revert, the field or record is absent from your sync files, and merge never deletes what the sync files don't mention. Removal is a deletion, and deletions need `mirror`. Say the bad commit added a `summary` field and a notification flow, and tweaked the note on `articles.title`. Revert it, then preview the undo in mirror mode (the default `merge` preview plans no deletions, so it cannot show you a removal): @@ -132,15 +133,15 @@ d6s sync diff --to production --mode mirror ``` ``` -● Comparing committed files with production — https://cms.example.com (mirror — INCLUDES DELETIONS) +● Comparing commit-ready files with production — https://cms.example.com (mirror — INCLUDES DELETIONS) ● Schema — 2 changes: 0 added, 1 modified, 1 deleted ✖ DELETE field articles.summary ~ field articles.title (meta.note) -● Data — 1 change: 0 created, 0 updated, 1 deleted +● Configuration — 1 change: 0 created, 0 updated, 1 deleted ~ directus_flows +0 new ~0 updated ✖1 deleted (f7) ``` -The `~` line restores the old note. The `✖ DELETE` and `✖1 deleted` lines are the undo, and they should name exactly what the bad change added, and nothing else. If anything else shows up for deletion, your files are stale somewhere: stop, run a full pull from production on a scratch branch, read that diff to see what production actually holds, and fold anything worth keeping into your files before mirroring. +The `~` line restores the old note. The `✖ DELETE` and `✖1 deleted` lines are the undo, and they should name exactly what the bad change added, and nothing else. If anything else shows up for deletion, your sync files are stale: stop, run a full pull from production on a scratch branch, read that diff to see what production actually holds, and fold anything worth keeping into the sync files before mirroring. ```bash d6s sync push --to production --mode mirror @@ -149,20 +150,20 @@ d6s sync push --to production --mode mirror The push repeats the plan, then the [deletion gate](/guides/environment-sync/reference#deletion-gates) demands typed consent: ``` -This push permanently deletes 1 record and 1 schema item from production. Type "production" to confirm: +This push permanently deletes 1 configuration record and 1 schema deletion from production. Type "production" to confirm: ``` In automation, the same push requires `--dangerously-allow-delete` instead. ::callout{icon="material-symbols:warning-rounded" color="warning"} -**Undoing an added field deletes its data.** Mirroring away `summary` drops the column and everything editors typed into it since the bad push. The committed files cover configuration; only a database backup covers data. Back up the target first if that data matters. +**Undoing an added field deletes its content.** Mirroring away `summary` drops the column and everything editors typed into it since the bad push. The sync files cover configuration; only a database backup covers content. Back up the target first if that content matters. :: When the bad change is tangled up with good ones, fixing forward is often simpler than reverting: correct it on the development instance, pull, and promote the fix like any other change. ## Rebase an environment from production after drift -Sometimes production changes outside the deployment path, usually an urgent manual fix. Staging and the repository no longer reflect reality. Bring the fix into git, then re-align the lower environment. This is what `mirror` is for: making an environment match the committed state exactly. +Sometimes production changes outside the deployment path, usually an urgent manual fix. Staging and the sync files no longer reflect reality. Bring the fix into git, then re-align the lower instance. This is what `mirror` is for: making an instance match the sync files exactly. 1. Pull the full state from production. The manual change appears as an ordinary git diff, which is your record of what the hotfix actually was: @@ -181,7 +182,7 @@ Sometimes production changes outside the deployment path, usually an urgent manu Read the deletions closely. Anything that exists only on staging and falls inside the sync's scope is on the list. (The mode matters: a default `merge` preview plans no deletions, so only a mirror diff shows what re-aligning would remove.) -3. Make staging match the committed state: +3. Make staging match the sync files: ```bash d6s sync push --to staging --mode mirror @@ -205,26 +206,26 @@ Going from an empty instance to a working copy of your project's shape: d6s profile add staging --url https://staging.example.com --token ``` -3. Preview, then push the committed files: +3. Preview, then push the sync files: ```bash d6s sync diff --to staging d6s sync push --to staging ``` - Schema applies first, then the configuration records import. + Schema applies first, then the Configuration records import. -4. Commit the updated `id_map.json`. The push records which target record each committed record became, and that map is how every later push updates records instead of duplicating them. +4. Commit the updated `id_map.json`. The push records which target record corresponds to each source record, and the ID map is how every later push updates records instead of duplicating them. Things to expect on a first push: -- **Identity questions.** If the target already holds records the CLI cannot tell apart from the committed ones (two policies named "Administrator" is the classic case), an interactive push asks you to choose. Answer once, commit the map, and the questions do not come back. See [record identity](/guides/environment-sync/how-it-works#record-identity). -- **Secrets stay behind.** Stripped values (API keys in settings, concealed fields, flow credentials) never travel with the files. Set them on the new instance directly. +- **Identity questions.** If the target already holds records the CLI cannot tell apart from the source records (two policies named "Administrator" is the classic case), an interactive push asks you to choose. Answer once, commit the ID map, and the questions do not come back. See [record identity](/guides/environment-sync/how-it-works#record-identity). +- **Secrets stay behind.** Stripped values (API keys in settings, concealed fields, flow credentials) never travel with the sync files. Set them on the new instance directly. ::callout{icon="material-symbols:warning-rounded" color="warning"} **Extensions do not sync, and nothing warns about them.** A field built on a custom interface, or a flow using a custom operation, pushes silently to a target that may not have that extension installed, and arrives broken in the Studio until it is. Deploy your extensions to the target before pushing schema or flows that depend on them. :: ::callout{icon="material-symbols:info-outline-rounded"} -Push the whole committed folder to a fresh target, not a scoped slice. A partial snapshot whose relations point at collections outside the scope can fail to apply on an instance that has nothing else yet. The pull warns about these references when it writes the files. +Push the full sync project to a fresh target, not a scoped slice. A partial sync project whose relations point at collections outside the scope can fail to apply on an instance that has nothing else yet. The pull warns about these references when it writes the sync files. :: diff --git a/content/guides/14.environment-sync/4.ci-and-automation.md b/content/guides/14.environment-sync/4.ci-and-automation.md index 23796583..c5bbc218 100644 --- a/content/guides/14.environment-sync/4.ci-and-automation.md +++ b/content/guides/14.environment-sync/4.ci-and-automation.md @@ -10,7 +10,7 @@ Automating Environment Sync buys you two things: every pull request shows what m The CLI treats any environment with the `CI` variable set as non-interactive (locally, `--no-interactive` simulates the same behavior). Where the interactive CLI would ask a question, a non-interactive run refuses instead: -- An **ambiguous record match** (two target records that could both be the committed one) fails the command rather than picking a candidate. Resolve it once with an interactive push; the answer lands in the committed identity map and CI runs cleanly after that. +- An **ambiguous record match** (two target records that could both correspond to the source record) fails the command rather than picking a candidate. Resolve it once with an interactive push; the answer lands in the ID map and CI runs cleanly after that. - **Deletions** happen only with `--dangerously-allow-delete`. A `mirror` push without it refuses before changing anything on the target. - `--yes` confirms an ordinary, non-destructive apply. It never authorizes a deletion. @@ -24,7 +24,7 @@ The credential store saved on a developer machine is never read when `CI` is set ## JSON reports -Add `--json` and stdout carries exactly one machine-readable report per command. Warnings (stripped secret fields, version drift, flow headers exported verbatim) still go to stderr, so your logs keep them while stdout stays parseable. A failure puts an error report on stdout instead, with a stable `code` naming the failure class. +Add `--json` and stdout carries exactly one machine-readable report per command. Warnings (stripped secret fields, version drift, flow headers written verbatim) still go to stderr, so your logs keep them while stdout stays parseable. A failure puts an error report on stdout instead, with a stable `code` naming the failure class. The fields automation usually keys on: @@ -36,7 +36,7 @@ The fields automation usually keys on: ## A GitHub Actions pipeline -One workflow, two jobs: pull requests get the production diff as a comment, and merges to `main` apply the committed files. Store the token as an Actions secret. +One workflow, two jobs: pull requests get the production diff as a comment, and merges to `main` apply the reviewed sync files. Store the token as an Actions secret. ```yaml name: environment-sync @@ -94,13 +94,13 @@ jobs: run: d6s sync push --to production --yes --json > push-report.json env: DIRECTUS_PRODUCTION_TOKEN: ${{ secrets.DIRECTUS_PRODUCTION_TOKEN }} - - name: Commit the updated identity map + - name: Commit the updated ID map run: | if [ -n "$(git status --porcelain -- 'directus/*/id_map.json')" ]; then git config user.name "github-actions[bot]" git config user.email "github-actions[bot]@users.noreply.github.com" git add 'directus/*/id_map.json' - git commit -m "Update sync identity map" + git commit -m "Update sync ID map" git push fi ``` @@ -108,7 +108,7 @@ jobs: Two things to know before enabling the push job: - **Run the first push interactively, locally.** The first push into a target tends to raise the identity questions described in [How It Works](/guides/environment-sync/how-it-works#record-identity), and CI refuses them. Answer them once from a terminal, commit `id_map.json`, and CI is clean from then on. -- **The identity map commit-back step matters.** A push that creates records adds entries to `id_map.json`. If CI doesn't commit them, the next push re-matches those records from scratch, and records without an identifying field (panels) can duplicate. +- **The ID map commit-back step matters.** A push that creates records adds entries to `id_map.json`. If CI doesn't commit them, the next push re-matches those records from scratch, and records without an identifying field (panels) can duplicate. A scheduled pull is the same pattern in reverse: run `d6s sync pull --from staging --json` on a cron trigger and commit the result. A clean working tree means nothing changed on the instance; a diff is drift, arriving as a reviewable commit or PR instead of a surprise. diff --git a/content/guides/14.environment-sync/5.secrets-and-limitations.md b/content/guides/14.environment-sync/5.secrets-and-limitations.md index de4a5ed9..41f6a878 100644 --- a/content/guides/14.environment-sync/5.secrets-and-limitations.md +++ b/content/guides/14.environment-sync/5.secrets-and-limitations.md @@ -6,61 +6,60 @@ description: How Environment Sync keeps secret values out of the files you commi ## How secrets are handled -A pull exports real records: the settings row, user accounts, flow definitions. If any of those carried a secret value, it would land in JSON files you commit, and git history keeps it forever, in every clone of the repository. Environment Sync strips secret values at export: +A pull writes real configuration records to sync files: settings, user accounts, flow definitions, and more. If any carried a secret value, it would land in JSON files you commit, and git history would keep it in every clone of the repository. Environment Sync strips secret values during pull: -- **Built-in secret columns** (password hashes, tokens, 2FA seeds, license and AI keys) are always deleted from the export. -- **Fields you created and marked concealed, hashed, or encrypted** are deleted too. Every pull fetches the server's complete field list and drops any flagged value, printing a line naming each dropped field. A field missing from the export reads as protection, not data loss. +- **Built-in secret fields** (password hashes, tokens, 2FA seeds, license and AI keys) are always removed from the sync files. +- **Fields you created and marked concealed, hashed, or encrypted** are removed too. Every pull fetches the server's complete field list and drops any flagged value, printing a line naming each dropped field. A value missing from a sync file is protection, not data loss. -Because the field list is fetched separately from the schema, the check also covers pulls scoped to a few collections and pulls that skip schema entirely. And if the check itself cannot run, the pull stops rather than continue unprotected. +Because the field list is fetched separately from Schema, the check also covers pulls scoped to a few collections and pulls that skip Schema entirely. If the check itself cannot run, the pull stops rather than continue unprotected. -The field definition still syncs as schema: the column and its concealed setting arrive on the target. Only the value stays behind; set the real secret on each instance directly. +The field definition still syncs as Schema: the field and its concealed setting arrive on the target. Only the value stays behind; set the real secret on each instance directly. ### Why strip instead of sync -The server never hands out the real value for these fields: concealed fields read back as `**********`, hashed fields as the hash. Exporting that and pushing it would overwrite the target's working secret with a mask. Stripping protects both the repository and the target. +The server never hands out the real value for these fields: concealed fields read back as `**********`, hashed fields as the hash. Writing that value to a sync file and pushing it would overwrite the target's working secret with a mask. Stripping protects both the repository and the target. ### The one blind spot ::callout{icon="material-symbols:warning-rounded" color="warning"} -**Flow request headers export verbatim.** A secret pasted into free-form flow configuration (most commonly an Authorization header in a request operation) has no "this is secret" marker for the CLI to check, and legitimate headers have to sync. The pull warns you by operation name, and the value goes into the committed file as-is. Review those files before committing, and before publishing a repository. +**Flow request headers are written verbatim.** A secret pasted into free-form flow configuration (most commonly an Authorization header in a request operation) has no "this is secret" marker for the CLI to check, and legitimate headers have to sync. The pull warns you by operation name, and the value goes into the sync file as-is. Review those files before committing and before publishing a repository. :: ## System collections that do not sync -None of these sync today. Some are shared configuration that a future release could take on; the rest are per-instance data that a sync should never touch. +None of these sync today. Some are shared configuration that a future release could take on; the rest are per-instance state that a sync should never touch. | Collection | Why it doesn't sync | | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `directus_presets` | Bookmarks, saved layouts, and Insights presets mix shared configuration with personal preference. A future release may sync the shared part. | | `directus_extensions` | The enabled/disabled state of installed extensions is tied to what is physically deployed on each instance. Deploy extensions to the target before pushing schema or flows that depend on them; the push itself does not warn. | -| `directus_files` | File-interface **fields** sync as schema; file rows and the binaries behind them are their own workstream. | -| `directus_comments` | Content comments; per-instance data. | -| `directus_activity` | The audit log; per-instance data. | -| `directus_revisions` | Change history; per-instance data. | -| `directus_versions` | Content-versioning drafts; per-instance data. | -| `directus_notifications` | User notifications; per-instance data. | -| `directus_shares` | Public share links; per-instance data. | -| `directus_sessions` | Active login sessions; per-instance data. | +| `directus_files` | File-interface **fields** sync as Schema; file records and the binaries behind them are their own workstream. | +| `directus_comments` | Content comments; per-instance state. | +| `directus_activity` | The audit log; per-instance state. | +| `directus_revisions` | Change history; per-instance state. | +| `directus_versions` | Content-versioning drafts; per-instance state. | +| `directus_notifications` | User notifications; per-instance state. | +| `directus_shares` | Public share links; per-instance state. | +| `directus_sessions` | Active login sessions; per-instance state. | | `directus_migrations` | The database migration ledger. The CLI never runs migrations or changes the Directus version. | | `directus_webhooks` | Deprecated in Directus; superseded by flows. | ## What it does not do -| Won't | Because / instead | -| ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Sync your collections' **content** (rows) | Environment Sync moves the shape of a project and its configuration. Content sync is deferred to a future release. | -| **Undo** a push | Revert the commit and push again; removals need `mirror`. See [Roll back a bad push](/guides/environment-sync/common-workflows#roll-back-a-bad-push). Only a database backup covers content. | -| **Auto-expand** a scoped snapshot | A scope pulls exactly what you name. Dangling references produce a warning; widen `--collections` yourself. | -| Translate schema **across versions** | A version mismatch refuses the command; `--allow-version-drift` overrides the gate but converts nothing. | -| Wrap schema and data in **one transaction** | Schema applies first, then data. A failed import re-runs data alone; see [How It Works](/guides/environment-sync/how-it-works#how-a-push-applies). | -| Select **individual records** | Resource selection is by type (`--roles`), never by row. | -| Model **code-first** | Model in the Data Studio and pull. Don't hand-author the JSON files. | -| **Seal the data plan** against target drift | Only the schema apply is sealed against the target changing between preview and apply. The data preview is the server's own dry-run answer, but it is advisory. | +| Won't | Because / instead | +| ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Sync **records in your own collections** | Environment Sync moves the shape of a project and its configuration. Content sync is deferred to a future release. | +| **Undo** a push | Revert the commit and push again; removals need `mirror`. See [Roll back a bad push](/guides/environment-sync/common-workflows#roll-back-a-bad-push). Only a database backup covers content. | +| **Auto-expand** a scoped pull | A scope pulls exactly what you name. Dangling references produce a warning; widen `--collections` yourself. | +| Translate schema **across versions** | A version mismatch refuses the command; `--allow-version-drift` overrides the gate but converts nothing. | +| Wrap Schema and Configuration in **one transaction** | Schema applies first, then Configuration. A failed import re-runs Configuration alone; see [How It Works](/guides/environment-sync/how-it-works#how-a-push-applies). | +| Select **individual records** | Resource selection is by type (`--roles`), never by record. | +| Model **code-first** | Model in the Data Studio and pull. Don't hand-author the JSON files. | +| **Seal the Configuration plan** against target drift | Only the Schema apply is sealed against the target changing between preview and apply. The Configuration preview is the server's own dry-run answer, but it is advisory. | ## Known limitations -- **Translations mirror is opt-in.** A server limitation currently breaks `mirror` pushes of translations, so translations are excluded from pulls by default. Opt in with `--translations`; `merge` and `add` pushes work normally. -- **Unlicensed custom permission rules are invisible to the export.** On an instance without a license, the API hides custom permission rules, so a pull cannot export them. The pull detects the shortfall and marks the export incomplete: `merge` and `add` push normally, `mirror` refuses. License the source instance to export them. -- **Users without an email address fail import.** The server's import validation rejects a user with no email, so a `--users` push containing one fails before any data applies. Give the account an email on the source, or remove it, first. -- **Panels can duplicate on a first sync into a look-alike target.** Panels have no name to match on, so pushing into a target that already holds equivalent panels (seeded from the same template) can create them a second time. The identity map prevents repeats after that first push. -- **A mirror push of users does not protect your own account.** If users are committed and the account the push authenticates with is absent from them, a `mirror` push orders its deletion like any other record; the CLI has no self-protection, and whether the server refuses is up to the server. When you sync users, make sure the committed set includes the accounts your pushes run as. +- **Unlicensed custom permission rules are unavailable to pull.** On an instance without a license, the API hides custom permission rules. The pull detects the missing records and marks the result incomplete: `merge` and `add` push normally, while `mirror` refuses. License the source instance to include them. +- **Users without an email address fail import.** The server's import validation rejects a user with no email, so a `--users` push containing one fails before any Configuration changes apply. Give the account an email on the source, or remove it, first. +- **Panels can duplicate on a first sync into a look-alike target.** Panels have no name to match on, so pushing into a target that already holds equivalent panels (seeded from the same template) can create them a second time. The ID map prevents repeats after that first push. +- **A mirror push of users does not protect your own account.** If users are in the sync files and the account the push authenticates with is absent from them, a `mirror` push orders its deletion like any other record; the CLI has no self-protection, and whether the server refuses is up to the server. When you sync users, make sure the sync files include the accounts your pushes run as. diff --git a/content/guides/14.environment-sync/6.reference.md b/content/guides/14.environment-sync/6.reference.md index 0405679e..0cb0935c 100644 --- a/content/guides/14.environment-sync/6.reference.md +++ b/content/guides/14.environment-sync/6.reference.md @@ -12,9 +12,9 @@ Everything on this page is lookup material. If you're learning the tool, start w | ------------------ | ---------------------------------------------------------------------------------------- | | `d6s profile add` | Add or update a profile (name + URL) and optionally save a credential | | `d6s profile test` | Connect with a profile and print who you are on the instance | -| `d6s sync pull` | Write a source instance's schema and configuration to committable files | +| `d6s sync pull` | Write a source instance's Schema and Configuration to sync files | | `d6s sync diff` | Show what a push would change on the target; applies nothing | -| `d6s sync push` | Apply the committed files to a target instance | +| `d6s sync push` | Apply the sync files to a target instance | | `d6s sync` | Interactive wizard: prompts for source, target, project, and mode, then pulls and pushes | The CLI installs as `directus-cli` with `d6s` as an equivalent short alias. @@ -40,7 +40,7 @@ d6s profile add [name] [--url ] [--token ] [--yes] | `--token ` | Static token to save to the credential store for this profile | | `--yes` | Skip the confirmation when repointing an existing profile to a new URL | -Adding is an upsert: an existing name is updated. Run without arguments for prompts, which also offer to save a credential (paste a static token, or log in with email and password to save a session; saved sessions refresh themselves before they expire). Profile names use letters, numbers, and underscores. URLs with embedded credentials, query strings, or fragments are refused, because the URL is the part that lands in a committed file. +Adding is an upsert: an existing name is updated. Run without arguments for prompts, which also offer to save a credential (paste a static token, or log in with email and password to save a session; saved sessions refresh themselves before they expire). Profile names use letters, numbers, and underscores. URLs with embedded credentials, query strings, or fragments are refused because the URL is stored in project configuration. ## `d6s profile test` @@ -65,24 +65,26 @@ Connects and prints who the credential authenticates as: d6s sync pull --from [scope flags] ``` -| Flag | Effect | -| ------------------------------ | ------------------------------------------------------------------------------------------------ | -| `--from ` (required) | Source profile name | -| `--collections ` | Schema scope: only these collections (comma-separated) | -| `--exclude-collections ` | Schema scope: all collections except these | -| `--no-schema` | Skip the schema entirely; configuration resources only | -| `--` | Select only the named resources, e.g. `--flows --roles` (plus their dependencies) | -| `--no-` | Keep the default resource set but exclude one, e.g. `--no-flows` | -| `--all` | Every configuration resource, including users and translations | -| `--no-deps` | Do not pull a selected resource's dependencies (dependent children still ride with their parent) | -| `--project ` | Project to sync (default: `default`) | +| Flag | Effect | +| ------------------------------ | --------------------------------------------------------------------------------- | +| `--from ` (required) | Source profile name | +| `--collections ` | Schema scope: only these collections (comma-separated) | +| `--exclude-collections ` | Schema scope: all collections except these | +| `--no-schema` | Skip Schema entirely; Configuration resources only | +| `--` | Select only the named resources, e.g. `--flows --roles` (plus their dependencies) | +| `--no-` | Keep the default resource set but exclude one, e.g. `--no-flows` | +| `--all` | Every configuration resource, including users | +| `--no-deps` | Do not add prerequisites; resources owned by a selected parent remain included | +| `--project ` | Project to sync (default: `default`) | -The selectable resources are `roles`, `policies`, `flows`, `dashboards`, `settings`, `folders`, `users`, and `translations`; `access`, `permissions`, `operations`, and `panels` ride along with their parents (see [the dependency table](#configuration-resources)). Positive selection (`--flows`) cannot be combined with `--all` or with `--no-` flags. Resource selection never narrows the schema; the two axes are scoped independently. +A bare pull includes every selectable resource except users. This means translations sync by default. Pass `--no-translations` to exclude them, or `--translations` to pull only translations. + +The selectable resources are `roles`, `policies`, `flows`, `dashboards`, `settings`, `folders`, `users`, and `translations`. The CLI automatically includes `access`, `permissions`, `operations`, and `panels` with their parent resources (see [the dependency table](#configuration-resources)). Positive selection (`--flows`) cannot be combined with `--all` or with `--no-` flags. Resource selection never narrows the schema; the two axes are scoped independently. Two warnings a scoped pull can raise, neither of which widens the scope for you: -- **Out-of-scope references.** The scoped snapshot points at something you omitted (a relation target, a group parent, a many-to-any collection). Pushing that snapshot to a fresh target can fail; add the missing collections to `--collections` yourself. -- **A name the server didn't return.** A `--collections` name absent from the returned snapshot (usually a typo) is named in a warning. The partial snapshot still lands, but never silently. +- **Out-of-scope references.** The scoped sync files point at something you omitted (a relation target, a group parent, a many-to-any collection). Pushing those files to a fresh target can fail; add the missing collections to `--collections` yourself. +- **A name the server didn't return.** A `--collections` name absent from the returned Schema (usually a typo) is named in a warning. The partial sync files are still written, but never silently. ## `d6s sync diff` @@ -166,59 +168,60 @@ When a command authenticates a profile, the token resolves in order: 2. The `DIRECTUS__TOKEN` environment variable: the profile name uppercased, so `production` reads `DIRECTUS_PRODUCTION_TOKEN`. A `.env` file next to `directus.config.json` is loaded automatically without overriding real environment variables. 3. The credential store at `~/.directus/credentials.json`, written readable only by you (mode `0600`). Never consulted when the `CI` environment variable is set. -Use an admin credential. The schema endpoints and the batch import the CLI relies on are admin-only on the server; the CLI does not check privileges up front, so a non-admin token fails with an authentication error, or produces an incomplete export that the completeness checks then flag. +Use an admin credential. The Schema endpoints and the batch import the CLI relies on are admin-only on the server; the CLI does not check privileges up front, so a non-admin token fails with an authentication error or produces an incomplete pull that the completeness checks then flag. ## Configuration resources Resource selection follows a dependency graph: selecting a resource pulls in what it needs (unless `--no-deps`). -| Resource | In default pull | Select directly | Pulls in | Notes | -| -------------- | ---------------------------- | ---------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -| `roles` | Yes | `--roles` | `policies` | | -| `policies` | Yes | `--policies` | `access`, `permissions` | See the warning below before selecting policies on their own. | -| `access` | Yes, with roles and policies | — | — | Grants attached to users are dropped when users are out of scope; a mirror push does not delete them on the target. | -| `permissions` | Yes, with policies | — | — | Row counts are verified against the server. If the source hides rows (unlicensed custom permission rules), the export is marked incomplete. | -| `flows` | Yes | `--flows` | `operations` | | -| `operations` | Yes, with flows | — | — | | -| `dashboards` | Yes | `--dashboards` | `panels` | | -| `panels` | Yes, with dashboards | — | — | Panels have no field to match on, so a first push into a matching target can duplicate once; the identity map prevents repeats. | -| `settings` | Yes | `--settings` | — | A single record. License and AI credentials, branding images (logos, backgrounds, favicon), and the default storage folder are stripped. | -| `folders` | Yes | `--folders` | — | The media-library folder tree. Distinct from collection folders (Data Studio sidebar groups), which sync as schema. | -| `users` | Opt-in | `--users` | `roles`, `policies` | Secret columns (`password`, `token`, `tfa_secret`, and others) are stripped. | -| `translations` | Opt-in | `--translations` | — | Mirror pushes of translations are not currently supported, which is why they are opt-in. | +| Resource | In default pull | Select directly | Also includes | Notes | +| -------------- | ---------------------------- | ---------------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | +| `roles` | Yes | `--roles` | `policies` | | +| `policies` | Yes | `--policies` | `access`, `permissions` | See the warning below before selecting policies on their own. | +| `access` | Yes, with roles and policies | — | — | Grants attached to users are dropped when users are out of scope; a mirror push does not delete them on the target. | +| `permissions` | Yes, with policies | — | — | Record counts are verified against the server. If the source hides records (unlicensed custom permission rules), the pull is incomplete. | +| `flows` | Yes | `--flows` | `operations` | | +| `operations` | Yes, with flows | — | — | | +| `dashboards` | Yes | `--dashboards` | `panels` | | +| `panels` | Yes, with dashboards | — | — | Panels have no field to match on, so a first push into a matching target can duplicate once; the ID map prevents repeats. | +| `settings` | Yes | `--settings` | — | A single record. License and AI credentials, branding images (logos, backgrounds, favicon), and the default storage folder are stripped. | +| `folders` | Yes | `--folders` | — | The media-library folder tree. Distinct from collection folders (Data Studio sidebar groups), which sync as Schema. | +| `translations` | Yes | `--translations` | — | Matched by `language` and `key`; `merge` and `mirror` can update existing strings. | +| `users` | Opt-in | `--users` | `roles`, `policies` | Secret fields (`password`, `token`, `tfa_secret`, and others) are stripped. | ::callout{icon="material-symbols:warning-rounded" color="warning"} **Select `--roles`, not `--policies` alone** -A selection that pulls policies without roles is not independently pushable when access rows reference roles: access rows carry references to roles, and with no roles in scope a push to a fresh target fails. Select `--roles` instead; it pulls policies and their children too. +A selection that pulls policies without roles is not independently pushable when access records reference roles. With no roles in scope, a push to a fresh target fails. Select `--roles` instead; it includes policies and their dependent resources too. :: ## What a pull touches Two rules govern every pull, scoped or not: -1. A pull only rewrites what it fetched. Everything it did not fetch keeps its committed state, untouched. -2. A push only applies what is in the files. Work that never entered them cannot ship. +1. A pull writes sync files only for what it fetched. Sync files for anything it did not fetch stay unchanged. +2. A push only applies what is in the sync files. Work that never entered them cannot ship. -Refreshed means the file is rewritten from the source; because writes are deterministic, an unchanged resource produces no git diff. Preserved means the file is not touched at all. +**Written in this pull** means the CLI fetched the current source state and wrote it to the corresponding sync files. Because writes are deterministic, an identical source state produces no git diff. **Not written** means the pull left those sync files unchanged on disk. -| Pull | Schema files | Configuration files | -| ------------------------------------ | ------------------------------------------------------- | ------------------------------------------------ | -| `pull --from staging` | All refreshed | All refreshed | -| `... --collections posts` | `posts` refreshed, others preserved | All refreshed | -| `... --no-flows` | All refreshed | Flows preserved, others refreshed | -| `... --flows` | All refreshed (resource selection never narrows schema) | Flows and operations refreshed, others preserved | -| `... --flows --no-schema` | All preserved | Flows and operations refreshed, others preserved | -| `... --collections posts --no-flows` | `posts` refreshed, others preserved | Flows preserved, others refreshed | +| Pull | Schema sync files | Configuration sync files | +| ------------------------------------ | ------------------------------------------------- | ---------------------------------------------------------------- | +| `pull --from staging` | All written in this pull | Default set written; users not written | +| `... --collections posts` | `posts` written; others not written | Default set written; users not written | +| `... --no-flows` | All written in this pull | Flows, operations, and users not written; other defaults written | +| `... --no-translations` | All written in this pull | Translations and users not written; other defaults written | +| `... --flows` | All written (resource flags do not narrow Schema) | Flows and operations written; all others not written | +| `... --flows --no-schema` | None written | Flows and operations written; all others not written | +| `... --collections posts --no-flows` | `posts` written; others not written | Flows, operations, and users not written; other defaults written | ## Push modes -| Mode | Schema | Data | Deletes? | -| ----------------- | ---------------------------------- | ----------------------------------------------------------- | -------------- | -| `add` | Adds and modifies, same as `merge` | Inserts only; existing records are never updated | No | -| `merge` (default) | Adds and modifies | Creates and updates | No | -| `mirror` | May delete in scope | Creates, updates, and deletes records absent from the files | **Yes, gated** | +| Mode | Schema | Configuration | Deletes? | +| ----------------- | ---------------------------------- | ---------------------------------------------------------------- | -------------- | +| `add` | Adds and modifies, same as `merge` | Creates only; existing records are never updated | No | +| `merge` (default) | Adds and modifies | Creates and updates | No | +| `mirror` | May delete in scope | Creates, updates, and deletes records absent from the sync files | **Yes, gated** | -A `mirror` push deletes only within what the committed files cover: a snapshot scoped to some collections can delete fields inside those collections, never a collection it doesn't contain. An export the source itself left incomplete (hidden permission rows) is refused at mirror push outright. +A `mirror` push deletes only within what the sync files cover: a pull scoped to some collections can delete fields inside those collections, never a collection it doesn't contain. A pull the source itself left incomplete (hidden permission records) is refused at mirror push outright. ## Deletion gates @@ -233,31 +236,32 @@ Only `mirror` deletes, and deleting always requires its own explicit consent: ``` ✖ Refusing mirror mode in a non-interactive context without --dangerously-allow-delete. - mirror can delete schema and data rows absent from the import set; pass --dangerously-allow-delete to consent, or use --mode merge. + mirror can delete schema and configuration records absent from the commit-ready files; pass --dangerously-allow-delete to consent, or use --mode merge. ``` ## Record identity -Records are matched across instances first by the committed identity map (`//id_map.json`), then by an identifying field: +Records are matched across instances first by the ID map (`//id_map.json`), then by an identifying field: -| Resource | Matched by | -| -------------- | -------------------------- | -| Most resources | `name` | -| Users | `email` | -| Operations | `key` | -| Panels | Nothing; identity map only | +| Resource | Matched by | +| -------------- | -------------------- | +| Most resources | `name` | +| Users | `email` | +| Operations | `key` | +| Translations | `language` and `key` | +| Panels | Nothing; ID map only | -The map is keyed internally by source and target instance URL, so one committed file serves any number of targets without conflicts. Commit it whenever a push changes it. An ambiguous match (two candidates) prompts interactively and refuses non-interactively: +The ID map is keyed internally by source and target instance URL, so one file serves any number of targets without conflicts. Commit it whenever a push changes it. An ambiguous match (two candidates) prompts interactively and refuses non-interactively: ``` ✖ Ambiguous target matches: directus_roles source "Editor" — sr1 → one of t1, t2 - Run d6s sync push interactively once to choose, then commit the updated id map. + Run d6s sync push interactively once to choose, then commit the updated ID map. ``` ## The version rule -Schema changes require the snapshot's Directus version and the target's version to match exactly, patch release included; the mismatch error names both versions. `--allow-version-drift` proceeds anyway with a warning; the CLI never translates schema between versions. Projects with `"schema": false` skip the check entirely, and a target whose version cannot be read is left to the server's own gate rather than refused. +Schema changes require the Directus version recorded in the sync files and the target's version to match exactly, patch release included; the mismatch error names both versions. `--allow-version-drift` proceeds anyway with a warning; the CLI never translates schema between versions. Projects with `"schema": false` skip the check entirely, and a target whose version cannot be read is left to the server's own gate rather than refused. ## JSON reports @@ -293,7 +297,7 @@ With `--json`, stdout carries exactly one report per command; warnings still go } ``` -The schema counters (`collections` through `files`) are `null` when the schema phase is skipped; `scope` echoes a `--collections`/`--exclude-collections` scope; `data.incomplete` names resources whose export the source cut short. +The Schema counters (`collections` through `files`) are `null` when the Schema phase is skipped; `scope` echoes a `--collections`/`--exclude-collections` scope; `data.incomplete` names resources whose pull the source cut short. The JSON API keeps `data` as its compatibility field name; it contains the Configuration report. `d6s sync diff --to production --json`: @@ -333,7 +337,7 @@ The schema counters (`collections` through `files`) are `null` when the schema p } ``` -`changes` is `true` when a push would do anything, including when records are `unresolved`; `added`/`modified`/`deleted` count schema items; `data.collections` is the target server's own per-collection dry-run answer. +`changes` is `true` when a push would do anything, including when records are `unresolved`; `added`/`modified`/`deleted` count Schema items; `data.collections` is the target server's own per-collection dry-run answer. `d6s sync push --to production --yes --json` reports the same shape as a diff, with `applied` (`true` when the push changed the target) in place of `unresolved`, and `data.collections` reflecting what the import actually did. @@ -351,8 +355,8 @@ Failures put an error report on stdout: } ``` -The `code` is one of a small set of failure classes: `USAGE` (the command line needs fixing: a missing flag or missing consent), `CONFIG` (`directus.config.json` missing or invalid), `AUTH` (the credential was rejected), `HTTP` (the instance could not be reached or returned an error), `STATE` (the committed files and the instance disagree: version mismatch, changed target schema, incomplete export), or `UNKNOWN`. Exit codes are `0` for success and `1` for every failure; the code string is the finer-grained signal. +The `code` is one of a small set of failure classes: `USAGE` (the command line needs fixing: a missing flag or missing consent), `CONFIG` (`directus.config.json` missing or invalid), `AUTH` (the credential was rejected), `HTTP` (the instance could not be reached or returned an error), `STATE` (the sync files and the instance disagree: version mismatch, changed target Schema, incomplete pull), or `UNKNOWN`. Exit codes are `0` for success and `1` for every failure; the code string is the finer-grained signal. ## Output conventions -Human-readable status lines go to stderr, prefixed `●` (info), `◇` (success), `▲` (warning), or `✖` (error, with its hint indented beneath). Plan lines go to stdout: `+` marks an addition, `~` a modification, and `✖ DELETE` a deletion, with data plans summarized per collection as `+N new ~N updated ✖N deleted`. `--no-color` disables coloring; `--json` replaces stdout output with the report while warnings stay on stderr. +Human-readable status lines go to stderr, prefixed `●` (info), `◇` (success), `▲` (warning), or `✖` (error, with its hint indented beneath). Plan lines go to stdout: `+` marks an addition, `~` a modification, and `✖ DELETE` a deletion, with Configuration plans summarized per collection as `+N new ~N updated ✖N deleted`. `--no-color` disables coloring; `--json` replaces stdout output with the report while warnings stay on stderr. From 15dc761850875dff5dcdee64b9018cce4ddbafd8 Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Thu, 6 Aug 2026 18:09:08 -0400 Subject: [PATCH 05/12] hopefully last wip --- content/guides/14.environment-sync/0.index.md | 18 +- .../14.environment-sync/1.quickstart.md | 32 ++- .../14.environment-sync/2.how-it-works.md | 63 +++-- .../14.environment-sync/3.common-workflows.md | 34 +-- .../4.ci-and-automation.md | 58 ++-- .../5.secrets-and-limitations.md | 10 +- .../guides/14.environment-sync/6.reference.md | 247 ++++++++++++------ 7 files changed, 286 insertions(+), 176 deletions(-) diff --git a/content/guides/14.environment-sync/0.index.md b/content/guides/14.environment-sync/0.index.md index 97e837ac..d3cce250 100644 --- a/content/guides/14.environment-sync/0.index.md +++ b/content/guides/14.environment-sync/0.index.md @@ -12,7 +12,7 @@ d6s sync diff --to production # preview what pushing the sync files would cha d6s sync push --to production # apply them ``` -Because the sync files live in your repository, you can review changes in pull requests, promote them through git history, and roll them back with a revert. (`d6s sync` on its own runs a small interactive wizard that pulls and pushes in one pass.) +Because the sync files live in your repository, you can review changes in pull requests, promote them through git history, and use a revert plus another push to restore an earlier state. Removals require `mirror`, even during a rollback. (`d6s sync` on its own runs a small interactive wizard that pulls and pushes in one pass.) The CLI is built to be safe to point at production: `diff` never applies anything, deletions always require their own explicit consent, and the CLI asks you to resolve records that match more than one target record. It never guesses. [How It Works](/guides/environment-sync/how-it-works) covers the full safety model. @@ -24,13 +24,13 @@ The CLI is built to be safe to point at production: `diff` never applies anythin Records in your own collections are **content**, and Environment Sync does not sync them. It moves the shape of a project and its configuration, not its content. -::callout{icon="material-symbols:info-outline-rounded"} +::callout{icon="i-lucide-info"} Content sync is deferred to a future release. Cross-instance record identity for integer primary keys, file references, and user references needs its own architecture, and a wrong guess there could overwrite content on the target. :: ## Before you start -- **Both instances must run the same Directus version, patch release included.** The server refuses cross-version schema changes because some patch releases change the schema format. Align your environments before you sync them. +- **Both instances must run Directus 12.2.0 or later, at the same exact version and on the same database vendor.** The patch release must match. The server refuses incompatible schema comparisons because some patches change the snapshot format and database vendors describe column types differently. - **An admin credential for each instance.** The schema and configuration endpoints the CLI uses are admin-only. A static token from an admin user is the usual choice. - **A git repository.** The sync files are designed for review and versioning. Any repository works; many teams use the one that already holds their Directus deployment configuration. @@ -38,27 +38,27 @@ Content sync is deferred to a future release. Cross-instance record identity for ::card-group -:::card{title="Quickstart" icon="i-ph-rocket-launch" to="/guides/environment-sync/quickstart"} +:::card{title="Quickstart" icon="i-lucide-rocket" to="/guides/environment-sync/quickstart"} Run the full pull, diff, push loop against two throwaway instances and see every command's output. ::: -:::card{title="How It Works" icon="i-ph-lightbulb" to="/guides/environment-sync/how-it-works"} +:::card{title="How It Works" icon="i-lucide-lightbulb" to="/guides/environment-sync/how-it-works"} The mental model: sync files as the source of truth, record identity, push phases, and the safety rules. ::: -:::card{title="Common Workflows" icon="i-ph-map-trifold" to="/guides/environment-sync/common-workflows"} +:::card{title="Common Workflows" icon="i-lucide-map" to="/guides/environment-sync/common-workflows"} Promote changes, ship only what's ready, adopt sync on an existing project, roll back, recover from drift. ::: -:::card{title="CI & Automation" icon="i-ph-robot" to="/guides/environment-sync/ci-and-automation"} +:::card{title="CI & Automation" icon="i-lucide-bot" to="/guides/environment-sync/ci-and-automation"} Post the production diff on pull requests and push on merge, with tokens and JSON reports. ::: -:::card{title="Secrets & Limitations" icon="i-ph-shield-check" to="/guides/environment-sync/secrets-and-limitations"} +:::card{title="Secrets & Limitations" icon="i-lucide-shield-check" to="/guides/environment-sync/secrets-and-limitations"} How secret values are kept out of your repository, and what Environment Sync deliberately does not do. ::: -:::card{title="Reference" icon="i-ph-list-magnifying-glass" to="/guides/environment-sync/reference"} +:::card{title="Reference" icon="i-lucide-list-checks" to="/guides/environment-sync/reference"} Every command, flag, table, and report format in one place. ::: diff --git a/content/guides/14.environment-sync/1.quickstart.md b/content/guides/14.environment-sync/1.quickstart.md index 7adb34de..40848c02 100644 --- a/content/guides/14.environment-sync/1.quickstart.md +++ b/content/guides/14.environment-sync/1.quickstart.md @@ -6,7 +6,7 @@ description: Run the full pull, diff, push loop against two throwaway Directus i The fastest way to trust Environment Sync is to watch it work somewhere mistakes are free. In this guide, you stand up two throwaway Directus instances with Docker, make a change on one, and move it to the other through git. Nothing here touches a real project, and one command tears it all down at the end. -You need Docker, Node.js with npm, and git. +You need Docker, Node.js 22 or later with npm, and git. ## Start two instances @@ -26,7 +26,7 @@ services: retries: 12 source: - image: directus/directus:latest + image: directus/directus:12.2.0 ports: - 8055:8055 environment: @@ -56,7 +56,7 @@ services: retries: 12 target: - image: directus/directus:latest + image: directus/directus:12.2.0 ports: - 8056:8055 environment: @@ -81,20 +81,18 @@ docker compose up -d After a minute, `http://localhost:8055` (the source) and `http://localhost:8056` (the target) both serve a fresh Data Studio. Log in with `admin@example.com` / `quickstart`. The `ADMIN_TOKEN` values are static admin tokens the CLI will use. -::callout{icon="material-symbols:info-outline-rounded"} -Both services use the same image tag on purpose. Environment Sync requires the source and target to run the same Directus version, patch release included. In real projects, pin the same explicit version on every environment. +::callout{icon="i-lucide-info"} +Both services use Directus 12.2.0 and PostgreSQL on purpose. Environment Sync requires the source and target to run the same Directus version, patch release included, and the same database vendor. In real projects, pin the same explicit version on every environment. :: ## Install the CLI - - ```bash -npm install -g @directus/cli +npm install -g @directus/cli@12 d6s --version ``` -The package installs two commands that do the same thing: `directus-cli` and the short alias `d6s`. These docs use `d6s`. +Pinning the package to major version 12 selects the current Environment Sync CLI instead of the deprecated 9.x package previously published under this name. The package installs two commands that do the same thing: `directus-cli` and the short alias `d6s`. These docs use `d6s`. ## Add a profile for each instance @@ -149,8 +147,8 @@ d6s sync pull --from source ``` ◇ Pulled from source — http://localhost:8055 - Schema 1 collection → directus/default/schema - Resources 14 records in 11 resources → directus/default/data + Schema 1 collection → directus/default/schema + Configuration 14 records across 11 collections → directus/default/data ``` Your record counts may differ slightly; a fresh instance carries a handful of configuration records (the admin role, its policy, and the settings record) even before you touch it. @@ -179,7 +177,7 @@ d6s sync pull --from source git diff ``` -The diff touches one file, and inside it, only the new `summary` field. The CLI writes sync files deterministically: no timestamps, no reshuffling, nothing but your change. This makes the sync files reviewable and means a pull that finds nothing new leaves your working tree clean. Commit the field. +The diff touches one file, and inside it, only the new `summary` field. The CLI writes sync files deterministically: no timestamps, no reshuffling, nothing but your change. This makes the sync files reviewable and means repeating the pull while the source is unchanged leaves your working tree clean. Commit the field. ```bash git add directus/ @@ -188,14 +186,14 @@ git commit -m "Add articles.summary" ## Preview against the target -The target instance is still empty. Ask what pushing the sync files would do to it: +The target still has only its initial Directus setup and does not have `articles`. Ask what pushing the sync files would do to it: ```bash d6s sync diff --to target ``` ``` -● Comparing commit-ready files with target — http://localhost:8056 (merge — additive, no deletions) +● Comparing ./directus/default with target — http://localhost:8056 (merge — creates and updates records, never deletes) ● Schema — 1 change: 1 added, 0 modified, 0 deleted + collection articles (3 fields) ● Configuration — no changes to push. @@ -216,7 +214,7 @@ The push prints the same plan, then asks before applying: `Apply 1 schema change ◇ Push complete. Applied 1 schema change to http://localhost:8056; schema hash verified. No configuration changes to push. ``` -The "schema hash verified" line is the push double-checking that the target's schema ended up matching the plan it previewed, rather than assuming it did. +The "schema hash verified" line means the server confirmed that the target still matched the schema hash used to build the plan before applying it. If someone changes the target between planning and apply, the push stops instead of applying a stale plan. Open the target Studio at `http://localhost:8056`: the `articles` collection is there, fields and settings intact. That's the whole loop. Change one instance, pull to sync files, review in git, and push to another instance. @@ -229,8 +227,8 @@ d6s sync push --to target ``` ``` -● Pushing to target — http://localhost:8056 (merge — additive, no deletions) -◇ target already matches the commit-ready files — schema and configuration match; nothing to push. +● Pushing ./directus/default to target — http://localhost:8056 (merge — creates and updates records, never deletes) +◇ target — http://localhost:8056 already matches ./directus/default — schema and configuration match; nothing to push. ``` Nothing to confirm, nothing applied. The CLI checked the target and proved it matches; it didn't assume. Re-running a completed push is safe, which is exactly what automation needs. diff --git a/content/guides/14.environment-sync/2.how-it-works.md b/content/guides/14.environment-sync/2.how-it-works.md index b264d699..4307bfd9 100644 --- a/content/guides/14.environment-sync/2.how-it-works.md +++ b/content/guides/14.environment-sync/2.how-it-works.md @@ -6,7 +6,7 @@ description: The mental model behind Environment Sync, including sync files as t Environment Sync never moves anything directly between two instances. The sync files sit in the middle, and two rules govern everything the commands do: -1. **A pull writes sync files only for what it fetched.** Sync files for anything it did not fetch stay unchanged. +1. **A pull overwrites the sync files in its requested scope with the source's current state.** Files outside that scope are left unchanged. 2. **A push only applies what is in the sync files.** A push reads the files on disk, not the source instance. Work that never entered the sync files cannot ship. Everything else on this page is a consequence of those two rules. @@ -20,13 +20,13 @@ A pull writes into a directory you commit (named `directus` by default, one subd ``` directus/default/ schema/ # Schema, one JSON file per collection - data/ # configuration records, one JSON file per resource + data/ # Configuration, one JSON file per resource id_map.json # which target record corresponds to each source record ``` The sync files are written deterministically: pulling twice with no instance changes produces byte-identical files and a clean working tree. `git diff` after a pull shows what changed on the instance and nothing else, so a schema change reads like any other code change in review. -Each directory also holds a `metadata.json` that lists the sync files the CLI wrote. The CLI treats that list as ownership: it removes a stale file it wrote on an earlier pull, and it never deletes a file it did not write. Hand-edited or corrupt sync files stop the command with a named error instead of applying invalid state. +Both `schema/` and `data/` also contain a `metadata.json` file listing the files the CLI owns. A later pull can remove an owned file that became stale, but it never deletes a file outside that list. Hand-edited or corrupt sync files stop the command with a named error instead of applying invalid state. Profiles and per-project settings live in `directus.config.json` at the repository root. It contains URLs and scoping options, never credentials, so it is safe to commit. The [reference](/guides/environment-sync/reference#directusconfigjson) shows the full file. @@ -35,36 +35,45 @@ Profiles and per-project settings live in `directus.config.json` at the reposito A pull covers two independent things, and you can scope each without affecting the other: - **Schema**: collections, fields, relations. Scope it with `--collections` or `--exclude-collections`, or skip it with `--no-schema`. -- **Configuration resources**: records of eleven `directus_*` resource types (flows, roles, settings, translations, and the rest), plus opt-in users. Scope it with resource flags like `--flows` or `--no-flows`. +- **Configuration resources**: records from supported `directus_*` collections, including flows, roles, settings, and translations. Users are opt-in. Scope Configuration with resource flags like `--flows` or `--no-flows`. -Scoping narrows what a pull _fetches_. The CLI writes the current source state to the sync files for that scope and does not write any other sync files. The [pull scope matrix](/guides/environment-sync/reference#what-a-pull-touches) lists which sync files each flag combination writes, and [Common Workflows](/guides/environment-sync/common-workflows#promote-only-the-changes-that-are-ready) shows how to ship finished work while half-finished work stays out of the repository. +Scoping narrows what a pull overwrites from the source. Sync files outside the scope are left unchanged. The [pull scope matrix](/guides/environment-sync/reference#what-a-pull-touches) lists the effect of each flag combination, and [Common Workflows](/guides/environment-sync/common-workflows#promote-only-the-changes-that-are-ready) shows how to ship finished work while half-finished work stays out of the repository. ## Record identity The same role or flow carries a different primary key on every instance, so a push has to decide which target record corresponds to each source record before it can update rather than duplicate. Two mechanisms decide, in order: - **The ID map.** Each push records its decisions in `id_map.json`: this source record corresponds to that target record. Later pushes look there first. -- **Identifying fields.** A record not yet in the map is matched by the field that names it: `name` for most resources, `email` for a user, `key` for an operation, and `language` plus `key` for a translation. Panels have no such field, which is why a first push into a look-alike target can duplicate them once. +- **Identifying fields.** A record not yet in the map is matched by stable fields: `name` for most named resources, `email` for a user, `flow` plus `key` for an operation, and `language` plus `key` for a translation. Access rules and permissions use their relationship fields. Panels have no such fields, which is why a first push into a look-alike target can duplicate them once. -For an existing translation, the CLI replaces the source ID with the matching target ID and sends the complete record. Directus accepts an update that repeats that record's current `language` and `key`, so both `merge` and `mirror` can update translation strings. A pair already owned by another translation still fails as a duplicate. +For an existing translation, the CLI replaces the source ID with the matching target ID and sends the complete record. Directus 12.2.0 and later accept an import update that repeats that record's current `language` and `key`, so both `merge` and `mirror` can update translation strings. A pair already owned by another translation still fails as a duplicate. -When two target records could both be the match, the CLI asks you to choose in a terminal and refuses in CI. Ambiguity is never resolved by guessing. The question looks like this, with the differences between candidates spelled out per option: +When two target records could both be the match, the CLI asks you to choose in a terminal and refuses in CI. Ambiguity is never resolved by guessing. The question names both sides, links to the records when the Data Studio has a stable route for them, and explains what every choice will do: ```text -Resolve identity 1 of 1: directus_roles source "Editor" — sr1 matches multiple target records - Use "Editor" — t1 (Same synced values as source; only the ID differs) - Use "Editor" — t2 (icon: source "edit", target "star") - Create a separate record (Adds one record; leaves every existing match unchanged) - Abort the push (Applies no remote changes) +directus_roles — 1 of 1 +./directus/default contains 1 role named "Editor". +production — https://cms.example.com contains 2 matching roles. + +Role: "Editor" — sr1 +Source UI: https://source.example.com/admin/settings/roles/sr1 +Target UI 1: https://cms.example.com/admin/settings/roles/t1 +Target UI 2: https://cms.example.com/admin/settings/roles/t2 + +Which target role does this represent? + Existing target role "Editor" — t1 (Same synced values; only the ID differs) + Existing target role "Editor" — t2 (Merge updates the target; icon: local "edit", target "star") + No existing role — create a new one on the target (Creates another "Editor" role) + Abort push (Applies no remote changes) ``` -Your answer lands in the ID map, so each question is asked exactly once. That's why the map belongs in git: commit it whenever a push changes it, and teammates and CI inherit every decision already made. +Your answer lands in the ID map and is reused for later pushes between the same source and target URLs. That's why the map belongs in git: commit it whenever a push changes it, and teammates and CI inherit the decisions already made. -One ID map serves any number of instances. Internally it is keyed by source and target URL, so pushing the same sync files to staging and production writes two independent sets of mappings; neither overwrites the other. Deleting the ID map is safe to the extent that matching starts over: named records re-match by their identifying fields, and records without one can duplicate on the next push. The same applies to repointing a profile at a new domain: the mappings are keyed by URL, so the old URL's decisions no longer apply and matching starts fresh for the new one. +One ID map serves any number of instances. Internally it is keyed by source and target URL, so pushing the same sync files to staging and production writes two independent sets of mappings; neither overwrites the other. Deleting the ID map does not change either instance, but matching starts over on the next push. Named records can usually match again by their identifying fields; records without them can duplicate. Repointing a profile at a new URL also starts a new set of mappings, because decisions stored for the old URL do not apply to the new one. ## How a push applies -A push applies in two phases, Schema first, and begins by showing you the full plan and asking (unless you pass `--yes`). +After record identity is settled, an interactive push dry-runs Configuration, shows the full plan, and asks before applying unless you pass `--yes`. It then applies in two phases, Schema first: 1. **Schema.** Before applying, the push re-checks that the target's schema still matches what the plan was computed against. If someone changed the target between preview and apply, the push stops rather than apply a stale plan. 2. **Configuration.** Configuration records import in a single server-side transaction once the schema is in place. @@ -87,30 +96,30 @@ A push mode answers one question: what happens to things that exist on the targe - **`add`** only creates records; it never touches an existing one. Its Schema phase behaves exactly like `merge`: the mode only changes what happens to configuration records. - **`mirror`** makes the target match the sync files exactly, which means deleting what the sync files no longer contain, within whatever scope they cover. -Deleting always requires its own explicit consent, separate from confirming the push. Interactively, a mirror push lists what would be lost and asks you to type the profile name. Non-interactively it requires the `--dangerously-allow-delete` flag; `--yes` never authorizes a deletion. The gate is a backstop as well as a policy: even if a supposedly additive push somehow carried a deletion, it would still be refused without consent. +Deleting always requires its own explicit consent, separate from confirming the push. Interactively, a mirror push lists what would be lost and asks you to type the profile name. Non-interactively it requires the `--dangerously-allow-delete` flag; `--yes` never authorizes a deletion. The gate is a backstop as well as a policy: even if a non-deleting push somehow carried a deletion, it would still be refused without consent. Because a mirror push makes the target match the sync files _exactly_, run it against a freshly pulled state. A mirror from stale sync files applies the stale state, including deleting things that only look obsolete because the files are old. -## The version gate +## The compatibility gate -Schema changes require the version recorded in the sync files and the target's version to match exactly, patch release included, because the server refuses cross-version schema diffs; historically, some patches change the schema format. +Schema comparison requires two things to match: the exact Directus version recorded in the sync files, patch release included, and the target's database vendor. The server refuses the comparison when either differs. Historically, some Directus patches change the schema format, while database vendors describe column types differently. +```text +✖ Version mismatch: the snapshot was pulled from Directus 12.1.1, but the target runs 12.2.0. + The server requires an exact version match for schema diffs — historically some patches are breaking. Align both instances (re-pull if the source was upgraded), or pass --allow-drift to proceed anyway. ``` -✖ Version mismatch: the snapshot was pulled from Directus 11.2.0, but the target runs 11.2.5. -The server requires an exact version match for schema diffs — historically some patches are breaking. Align both instances (re-pull if the source was upgraded), or pass --allow-version-drift to proceed anyway. - -``` +The CLI can name a known version mismatch before sending the snapshot. It cannot pre-check the target's database vendor, so a vendor mismatch comes back as an incompatible-snapshot refusal with the server's reason attached. -`--allow-version-drift` asks the server to proceed anyway and warns you loudly; the CLI never translates schema between versions. Projects configured with `"schema": false` skip the Schema phase and this gate entirely. +`--allow-drift` sends the server's `force` bypass for either mismatch and prints a `▲ Compatibility check bypassed` warning. It does not translate schema across Directus versions or database vendors, so read the plan closely. Projects configured with `"schema": false` skip the Schema phase and this gate entirely. ## The safety model The rules above add up to a small set of promises, each of which you can watch hold in the [Quickstart](/guides/environment-sync/quickstart): - **`pull` is read-only on the source.** Every request it makes is a read; it changes nothing on the instance it snapshots. (The one exception: a profile that authenticates with a saved login session refreshes that session when it is close to expiring.) -- **`diff` applies nothing.** The Schema comparison is a preview, and the Configuration plan comes from the target server dry-running the import inside a transaction and rolling it back, so the plan is the server's own answer, not a client-side guess. +- **`diff` changes no Schema or Configuration records.** The Schema comparison is a preview, and the Configuration plan comes from the target server dry-running the import inside a transaction and rolling it back, so the plan is the server's own answer, not a client-side guess. On PostgreSQL, that dry run can advance an integer primary-key sequence and leave harmless gaps. - **Deletions are gated.** Only `mirror` deletes, always behind its own consent, and `--yes` never covers it. - **Identity is never guessed.** An ambiguous record match prompts in a terminal and refuses in CI. -- **Stored secrets never enter your repository.** Built-in secret columns and fields marked concealed, hashed, or encrypted are stripped during pull, with [one warned exception](/guides/environment-sync/secrets-and-limitations#the-one-blind-spot). -- **Failures are loud.** Hand-edited sync files, incomplete pulls, version mismatches, and unreachable instances stop the command with a named error instead of degrading silently. A pull the source itself left incomplete is marked as such and refused at mirror push. +- **Recognized secret fields are stripped.** Built-in secret columns and fields marked concealed, hashed, or encrypted are removed during pull. Free-form flow configuration has [one warned blind spot](/guides/environment-sync/secrets-and-limitations#the-one-blind-spot). +- **Failures are loud.** Hand-edited sync files, incomplete pulls, compatibility mismatches, and unreachable instances stop the command with a named error instead of degrading silently. A pull the source itself left incomplete is marked as such and refused at mirror push. diff --git a/content/guides/14.environment-sync/3.common-workflows.md b/content/guides/14.environment-sync/3.common-workflows.md index 83589577..16fc9ea4 100644 --- a/content/guides/14.environment-sync/3.common-workflows.md +++ b/content/guides/14.environment-sync/3.common-workflows.md @@ -8,7 +8,7 @@ These workflows cover most day-to-day use of Environment Sync: promoting changes ## Promote changes from development to production -You model in the Data Studio on a development instance. Production only receives sync files reviewed in git. +You model in the Data Studio on a development instance. Production receives the state captured in sync files that were reviewed in git. 1. Make your changes on the development instance: collections, fields, flows, permissions. @@ -20,7 +20,7 @@ You model in the Data Studio on a development instance. Production only receives git commit -m "Add author bio fields" ``` - The sync files are deterministic, so the commit shows your change and nothing else. On a development instance other people also use, a scoped pull (`--collections posts`, or a resource flag like `--flows`) keeps their in-progress work out of your diff. + The sync files are deterministic, so the commit shows your change and nothing else. On a development instance other people also use, a scoped pull (`--collections posts`, or a resource flag like `--flows`) lets you overwrite only the part you are ready to review. 3. Open a pull request. Reviewers read the change as plain JSON diffs. A CI check can add the target's view of the same change: @@ -36,7 +36,7 @@ You model in the Data Studio on a development instance. Production only receives The default `merge` mode creates and updates but never deletes. A second run reports nothing to push. -::callout{icon="material-symbols:warning-rounded" color="warning"} +::callout{icon="i-lucide-triangle-alert" color="warning"} **Removals need `mirror`.** A change that deletes a field or a record does not propagate under `merge`. Push with `--mode mirror` and pass its [deletion gate](/guides/environment-sync/reference#deletion-gates), and run a full pull first so the mirror applies current state, not a stale tree. :: @@ -50,9 +50,9 @@ d6s sync pull --from staging --collections posts What that pull just did: -- The `posts` Schema sync files were written in this pull using the current source state. -- The `authors` Schema sync files were not written in this pull, so they stayed unchanged on disk. They still hold what the last full pull captured, which is the state production already runs. The half-finished `authors` work never entered the sync files. -- The Configuration sync files were also written in this pull because a collection scope only narrows Schema. Their source state had not changed, so the files remained byte-identical and `git status` shows only the `posts` files. If a flow had changed on staging, its file would appear too; exclude it with `--no-flows`. +- The `posts` Schema sync files were overwritten with the current source state. +- The `authors` Schema sync files were outside the pull scope and stayed unchanged on disk. They still hold what the last full pull captured, which is the state production already runs. The half-finished `authors` work never entered the sync files. +- The default Configuration sync files were also overwritten because a collection scope only narrows Schema. Their source state had not changed, so the overwritten files were byte-identical and `git status` shows only the `posts` files. If a flow had changed on staging, its file would appear too; exclude it with `--no-flows`. - Dependent resources cannot be excluded individually. Permissions are included with policies, and operations are included with flows. A teammate's new permission records therefore appear whenever the pull includes policies. Commit, then diff and push as usual: @@ -72,9 +72,9 @@ For a configuration-only change, flip the scope: select the resource type and sk d6s sync pull --from staging --flows --no-schema ``` -The [What a pull touches](/guides/environment-sync/reference#what-a-pull-touches) table shows exactly which sync files each combination writes and does not write. +The [What a pull touches](/guides/environment-sync/reference#what-a-pull-touches) table shows exactly which sync files each combination overwrites from the source and which it leaves unchanged. -::callout{icon="material-symbols:warning-rounded" color="warning"} +::callout{icon="i-lucide-triangle-alert" color="warning"} **Selection is by collection and resource type, not by record.** `--collections posts` can separate finished posts work from unfinished authors work, but two changes inside the same collection travel together, and `--flows` takes every flow, not just one. If unrelated work shares a collection or a resource type, ship it together or wait. While the sync files hold a partial picture, avoid `mirror`: it would apply the stale remainder too. :: @@ -110,11 +110,11 @@ Most projects don't start from an empty instance. You have a development instanc Once nothing on staging is worth keeping outside the sync files, a `mirror` push finishes the job and makes it match exactly. -5. Expect a few identity questions on the first push. Staging and production often hold records that are plausibly the same one (two roles named "Editor", two flows built from the same template). The CLI asks instead of guessing; answer once, then commit the updated `id_map.json`. The questions do not come back. +5. Expect a few identity questions on the first push. Staging and production often hold records that are plausibly the same one (two roles named "Editor", two flows built from the same template). The CLI asks instead of guessing; answer, then commit the updated `id_map.json`. Those decisions are reused on later pushes between the same source and target URLs. From then on, the project runs the [promote workflow](#promote-changes-from-development-to-production): change on dev, pull, review, push. -::callout{icon="material-symbols:info-outline-rounded"} +::callout{icon="i-lucide-info"} Nervous about the first push against a real instance? Rehearse the whole loop against throwaway instances first: the [Quickstart](/guides/environment-sync/quickstart) sandbox is exactly that, and adding a profile for a scratch instance to a real repository is harmless. :: @@ -133,7 +133,7 @@ d6s sync diff --to production --mode mirror ``` ``` -● Comparing commit-ready files with production — https://cms.example.com (mirror — INCLUDES DELETIONS) +● Comparing ./directus/default with production — https://cms.example.com (mirror — INCLUDES DELETIONS) ● Schema — 2 changes: 0 added, 1 modified, 1 deleted ✖ DELETE field articles.summary ~ field articles.title (meta.note) @@ -155,7 +155,7 @@ This push permanently deletes 1 configuration record and 1 schema deletion from In automation, the same push requires `--dangerously-allow-delete` instead. -::callout{icon="material-symbols:warning-rounded" color="warning"} +::callout{icon="i-lucide-triangle-alert" color="warning"} **Undoing an added field deletes its content.** Mirroring away `summary` drops the column and everything editors typed into it since the bad push. The sync files cover configuration; only a database backup covers content. Back up the target first if that content matters. :: @@ -190,7 +190,7 @@ Sometimes production changes outside the deployment path, usually an urgent manu Interactively, the push names the losses and asks you to type the profile name; in automation it requires `--dangerously-allow-delete`. See [deletion gates](/guides/environment-sync/reference#deletion-gates). -::callout{icon="material-symbols:warning-rounded" color="warning"} +::callout{icon="i-lucide-triangle-alert" color="warning"} **Mirror removes staging-only work in scope.** If staging holds experiments you want to keep, pull that work into a branch first, or use `merge` and clean up by hand. :: @@ -219,13 +219,13 @@ Going from an empty instance to a working copy of your project's shape: Things to expect on a first push: -- **Identity questions.** If the target already holds records the CLI cannot tell apart from the source records (two policies named "Administrator" is the classic case), an interactive push asks you to choose. Answer once, commit the ID map, and the questions do not come back. See [record identity](/guides/environment-sync/how-it-works#record-identity). -- **Secrets stay behind.** Stripped values (API keys in settings, concealed fields, flow credentials) never travel with the sync files. Set them on the new instance directly. +- **Identity questions.** If the target already holds records the CLI cannot tell apart from the source records (two policies named "Administrator" is the classic case), an interactive push asks you to choose. Commit the ID map so later pushes between the same source and target URLs reuse the answer. See [record identity](/guides/environment-sync/how-it-works#record-identity). +- **Recognized secrets stay behind.** API keys in settings and fields marked concealed, hashed, or encrypted are stripped. Set them on the new instance directly. Free-form flow request headers are not stripped, so review the warning and the operation files before committing them. -::callout{icon="material-symbols:warning-rounded" color="warning"} +::callout{icon="i-lucide-triangle-alert" color="warning"} **Extensions do not sync, and nothing warns about them.** A field built on a custom interface, or a flow using a custom operation, pushes silently to a target that may not have that extension installed, and arrives broken in the Studio until it is. Deploy your extensions to the target before pushing schema or flows that depend on them. :: -::callout{icon="material-symbols:info-outline-rounded"} +::callout{icon="i-lucide-info"} Push the full sync project to a fresh target, not a scoped slice. A partial sync project whose relations point at collections outside the scope can fail to apply on an instance that has nothing else yet. The pull warns about these references when it writes the sync files. :: diff --git a/content/guides/14.environment-sync/4.ci-and-automation.md b/content/guides/14.environment-sync/4.ci-and-automation.md index c5bbc218..f6f1825c 100644 --- a/content/guides/14.environment-sync/4.ci-and-automation.md +++ b/content/guides/14.environment-sync/4.ci-and-automation.md @@ -8,9 +8,9 @@ Automating Environment Sync buys you two things: every pull request shows what m ## The non-interactive contract -The CLI treats any environment with the `CI` variable set as non-interactive (locally, `--no-interactive` simulates the same behavior). Where the interactive CLI would ask a question, a non-interactive run refuses instead: +The CLI treats a non-empty `CI` variable as non-interactive, except for the literal value `false` (locally, `--no-interactive` forces the same behavior). Where the interactive CLI would ask a question, a non-interactive run refuses instead: -- An **ambiguous record match** (two target records that could both correspond to the source record) fails the command rather than picking a candidate. Resolve it once with an interactive push; the answer lands in the ID map and CI runs cleanly after that. +- An **ambiguous record match** (two target records that could both correspond to the source record) fails the command rather than picking a candidate. Resolve it with an interactive push and commit the ID map; later CI pushes between the same source and target URLs reuse the answer. - **Deletions** happen only with `--dangerously-allow-delete`. A `mirror` push without it refuses before changing anything on the target. - `--yes` confirms an ordinary, non-destructive apply. It never authorizes a deletion. @@ -20,23 +20,23 @@ Commands exit `0` on success and `1` on any refusal or failure; there are no oth Pass tokens through environment variables named `DIRECTUS__TOKEN`, the profile name uppercased. Profile names use letters, numbers, and underscores, so the mapping is mechanical: profile `production` reads `DIRECTUS_PRODUCTION_TOKEN`, profile `staging_eu` reads `DIRECTUS_STAGING_EU_TOKEN`. -The credential store saved on a developer machine is never read when `CI` is set; tokens come from the environment only. +The credential store saved on a developer machine is never read when `CI` is non-empty, except when its value is `false`; tokens come from the environment only. ## JSON reports -Add `--json` and stdout carries exactly one machine-readable report per command. Warnings (stripped secret fields, version drift, flow headers written verbatim) still go to stderr, so your logs keep them while stdout stays parseable. A failure puts an error report on stdout instead, with a stable `code` naming the failure class. +Add `--json` and stdout carries exactly one machine-readable report per command. Warnings (stripped secret fields, compatibility checks bypassed with `--allow-drift`, flow headers written verbatim) still go to stderr, so your logs keep them while stdout stays parseable. A failure puts an error report on stdout instead, with a stable `code` naming the failure class. The fields automation usually keys on: -- **`changes`** (diff): `true` when the push would do anything, including unresolved records. -- **`unresolved`** (diff): the count of ambiguous record matches. A non-interactive push refuses while this is non-zero, so an unresolved diff is a real difference for your pipeline to surface, not noise. +- **`changes`** (diff): `true` when the push would do anything, including when Configuration has ambiguous target matches. +- **`data.reconciliation.ambiguous`** (diff): the number of Configuration records that need an identity choice. **`data.reconciliation.dependent`** counts records waiting on those choices. A non-interactive push refuses this state, so it is a real difference for your pipeline to surface, not noise. - **`applied`** (push): `true` when the push changed the target. `d6s sync diff` exits `0` whether or not differences exist; it fails only when it cannot produce an answer. Gate pipeline behavior on the report's `changes`, not the exit code. The [reference](/guides/environment-sync/reference#json-reports) documents every report field. ## A GitHub Actions pipeline -One workflow, two jobs: pull requests get the production diff as a comment, and merges to `main` apply the reviewed sync files. Store the token as an Actions secret. +One workflow, two jobs: pull requests get the production diff as a comment, and merges to `main` apply the reviewed sync files. Store the token as the `DIRECTUS_PRODUCTION_TOKEN` Actions secret and the expected instance URL as the `DIRECTUS_PRODUCTION_URL` Actions variable. ```yaml name: environment-sync @@ -48,7 +48,7 @@ on: jobs: diff: - if: github.event_name == 'pull_request' + if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository runs-on: ubuntu-latest permissions: contents: read @@ -58,7 +58,19 @@ jobs: - uses: actions/setup-node@v4 with: node-version: 22 - - run: npm install -g @directus/cli + - run: npm install -g @directus/cli@12 + - name: Verify the production profile URL + env: + EXPECTED_DIRECTUS_URL: ${{ vars.DIRECTUS_PRODUCTION_URL }} + run: | + node - <<'NODE' + const fs = require('fs'); + const config = JSON.parse(fs.readFileSync('directus.config.json', 'utf8')); + const actual = config.profiles?.production?.url; + if (!process.env.EXPECTED_DIRECTUS_URL || actual !== process.env.EXPECTED_DIRECTUS_URL) { + throw new Error(`Unexpected production profile URL: ${actual ?? ''}`); + } + NODE - name: Diff against production run: d6s sync diff --to production --json > diff-report.json env: @@ -67,11 +79,22 @@ jobs: uses: actions/github-script@v7 with: script: | - const report = require('./diff-report.json'); + const fs = require('fs'); + const report = JSON.parse(fs.readFileSync('diff-report.json', 'utf8')); + const ambiguous = report.data.reconciliation?.ambiguous ?? 0; + const configuration = Object.values(report.data.resultsByCollection ?? {}).reduce( + (total, result) => ({ + created: total.created + result.new.length, + updated: total.updated + result.existing.length, + deleted: total.deleted + result.deleted.length, + }), + { created: 0, updated: 0, deleted: 0 }, + ); const body = report.changes - ? `**Environment Sync**: merging changes production. ${report.added} added, ` + - `${report.modified} modified, ${report.deleted} deleted schema items; ` + - `${report.unresolved} unresolved records.` + ? `**Environment Sync**: merging changes production. Schema: ${report.added} added, ` + + `${report.modified} modified, ${report.deleted} deleted. Configuration: ` + + `${configuration.created} created, ${configuration.updated} updated, ` + + `${configuration.deleted} deleted; ${ambiguous} ambiguous matches.` : '**Environment Sync**: production already matches this branch.'; await github.rest.issues.createComment({ ...context.repo, @@ -82,6 +105,9 @@ jobs: push: if: github.event_name == 'push' runs-on: ubuntu-latest + concurrency: + group: environment-sync-production + cancel-in-progress: false permissions: contents: write steps: @@ -89,7 +115,7 @@ jobs: - uses: actions/setup-node@v4 with: node-version: 22 - - run: npm install -g @directus/cli + - run: npm install -g @directus/cli@12 - name: Push to production run: d6s sync push --to production --yes --json > push-report.json env: @@ -107,9 +133,11 @@ jobs: Two things to know before enabling the push job: -- **Run the first push interactively, locally.** The first push into a target tends to raise the identity questions described in [How It Works](/guides/environment-sync/how-it-works#record-identity), and CI refuses them. Answer them once from a terminal, commit `id_map.json`, and CI is clean from then on. +- **Run the first push interactively, locally.** The first push into a target tends to raise the identity questions described in [How It Works](/guides/environment-sync/how-it-works#record-identity), and CI refuses them. Answer them from a terminal and commit `id_map.json` before enabling the push job. - **The ID map commit-back step matters.** A push that creates records adds entries to `id_map.json`. If CI doesn't commit them, the next push re-matches those records from scratch, and records without an identifying field (panels) can duplicate. +The URL check must run before any step receives the production token. It prevents a pull request from changing the `production` profile to another host and sending the token there. The sample skips pull requests from forks because GitHub does not expose repository secrets to them; run local or tokenless checks for those contributions instead. + A scheduled pull is the same pattern in reverse: run `d6s sync pull --from staging --json` on a cron trigger and commit the result. A clean working tree means nothing changed on the instance; a diff is drift, arriving as a reviewable commit or PR instead of a surprise. ## Mirror pushes in automation diff --git a/content/guides/14.environment-sync/5.secrets-and-limitations.md b/content/guides/14.environment-sync/5.secrets-and-limitations.md index 41f6a878..688851a6 100644 --- a/content/guides/14.environment-sync/5.secrets-and-limitations.md +++ b/content/guides/14.environment-sync/5.secrets-and-limitations.md @@ -9,7 +9,7 @@ description: How Environment Sync keeps secret values out of the files you commi A pull writes real configuration records to sync files: settings, user accounts, flow definitions, and more. If any carried a secret value, it would land in JSON files you commit, and git history would keep it in every clone of the repository. Environment Sync strips secret values during pull: - **Built-in secret fields** (password hashes, tokens, 2FA seeds, license and AI keys) are always removed from the sync files. -- **Fields you created and marked concealed, hashed, or encrypted** are removed too. Every pull fetches the server's complete field list and drops any flagged value, printing a line naming each dropped field. A value missing from a sync file is protection, not data loss. +- **Fields you created and marked concealed, hashed, or encrypted** are removed too. Every pull that includes Configuration fetches the server's complete field list and drops any flagged value, printing a line naming each dropped field. A value missing from a sync file is protection, not data loss. Because the field list is fetched separately from Schema, the check also covers pulls scoped to a few collections and pulls that skip Schema entirely. If the check itself cannot run, the pull stops rather than continue unprotected. @@ -21,7 +21,7 @@ The server never hands out the real value for these fields: concealed fields rea ### The one blind spot -::callout{icon="material-symbols:warning-rounded" color="warning"} +::callout{icon="i-lucide-triangle-alert" color="warning"} **Flow request headers are written verbatim.** A secret pasted into free-form flow configuration (most commonly an Authorization header in a request operation) has no "this is secret" marker for the CLI to check, and legitimate headers have to sync. The pull warns you by operation name, and the value goes into the sync file as-is. Review those files before committing and before publishing a repository. :: @@ -50,8 +50,9 @@ None of these sync today. Some are shared configuration that a future release co | ---------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Sync **records in your own collections** | Environment Sync moves the shape of a project and its configuration. Content sync is deferred to a future release. | | **Undo** a push | Revert the commit and push again; removals need `mirror`. See [Roll back a bad push](/guides/environment-sync/common-workflows#roll-back-a-bad-push). Only a database backup covers content. | -| **Auto-expand** a scoped pull | A scope pulls exactly what you name. Dangling references produce a warning; widen `--collections` yourself. | -| Translate schema **across versions** | A version mismatch refuses the command; `--allow-version-drift` overrides the gate but converts nothing. | +| **Auto-expand** a collection-scoped pull | A collection scope pulls exactly what you name. Dangling references produce a warning; widen `--collections` yourself. Resource dependencies are handled separately and included by default. | +| Translate schema **across versions** | A version mismatch refuses the command; `--allow-drift` bypasses the compatibility gate but converts nothing. | +| Translate schema **across database vendors** | A vendor mismatch refuses the command; `--allow-drift` bypasses the same gate but does not rewrite vendor-specific column types. | | Wrap Schema and Configuration in **one transaction** | Schema applies first, then Configuration. A failed import re-runs Configuration alone; see [How It Works](/guides/environment-sync/how-it-works#how-a-push-applies). | | Select **individual records** | Resource selection is by type (`--roles`), never by record. | | Model **code-first** | Model in the Data Studio and pull. Don't hand-author the JSON files. | @@ -63,3 +64,4 @@ None of these sync today. Some are shared configuration that a future release co - **Users without an email address fail import.** The server's import validation rejects a user with no email, so a `--users` push containing one fails before any Configuration changes apply. Give the account an email on the source, or remove it, first. - **Panels can duplicate on a first sync into a look-alike target.** Panels have no name to match on, so pushing into a target that already holds equivalent panels (seeded from the same template) can create them a second time. The ID map prevents repeats after that first push. - **A mirror push of users does not protect your own account.** If users are in the sync files and the account the push authenticates with is absent from them, a `mirror` push orders its deletion like any other record; the CLI has no self-protection, and whether the server refuses is up to the server. When you sync users, make sure the sync files include the accounts your pushes run as. +- **A Configuration diff can leave harmless gaps in PostgreSQL integer IDs.** The target server dry-runs the import and rolls it back, but PostgreSQL sequences do not roll back. No records are created or changed; only a sequence such as the one for `directus_permissions` can advance. diff --git a/content/guides/14.environment-sync/6.reference.md b/content/guides/14.environment-sync/6.reference.md index 0cb0935c..c1bcddc0 100644 --- a/content/guides/14.environment-sync/6.reference.md +++ b/content/guides/14.environment-sync/6.reference.md @@ -8,16 +8,19 @@ Everything on this page is lookup material. If you're learning the tool, start w ## Commands -| Command | What it does | -| ------------------ | ---------------------------------------------------------------------------------------- | -| `d6s profile add` | Add or update a profile (name + URL) and optionally save a credential | -| `d6s profile test` | Connect with a profile and print who you are on the instance | -| `d6s sync pull` | Write a source instance's Schema and Configuration to sync files | -| `d6s sync diff` | Show what a push would change on the target; applies nothing | -| `d6s sync push` | Apply the sync files to a target instance | -| `d6s sync` | Interactive wizard: prompts for source, target, project, and mode, then pulls and pushes | - -The CLI installs as `directus-cli` with `d6s` as an equivalent short alias. +| Command | What it does | +| -------------------- | ---------------------------------------------------------------------------------------- | +| `d6s profile add` | Create a profile and optionally save a credential | +| `d6s profile update` | Change a profile's name, URL, or saved credential | +| `d6s profile list` | List configured profile names and URLs | +| `d6s profile test` | Connect with a profile or URL and print who you are on the instance | +| `d6s profile remove` | Remove a profile and its saved credential | +| `d6s sync pull` | Write a source instance's Schema and Configuration to sync files | +| `d6s sync diff` | Show what a push would change on the target; applies nothing | +| `d6s sync push` | Apply the sync files to a target instance | +| `d6s sync` | Interactive wizard: prompts for source, target, project, and mode, then pulls and pushes | + +Install `@directus/cli@12`. It provides `directus-cli` and the equivalent short alias `d6s`. Global flags, available on every command: @@ -31,21 +34,43 @@ Global flags, available on every command: ## `d6s profile add` ```bash -d6s profile add [name] [--url ] [--token ] [--yes] +d6s profile add [name] [--url ] [--token ] ``` -| Flag | Effect | -| ----------------- | ---------------------------------------------------------------------- | -| `--url ` | Directus instance URL | -| `--token ` | Static token to save to the credential store for this profile | -| `--yes` | Skip the confirmation when repointing an existing profile to a new URL | +| Flag | Effect | +| ----------------- | ------------------------------------------------------------- | +| `--url ` | Directus instance URL | +| `--token ` | Static token to save to the credential store for this profile | + +An existing profile name is refused; use `profile update` to change it. Run without arguments for prompts, which also offer to save a credential (paste a static token, or log in with email and password to save a session; saved sessions refresh themselves before they expire). Profile names use letters, numbers, and underscores. URLs with embedded credentials, query strings, or fragments are refused because the URL is stored in project configuration. + +## `d6s profile update` + +```bash +d6s profile update [name] [--name ] [--url ] [--token ] [--yes] +``` + +| Flag | Effect | +| ----------------- | -------------------------------------------------------------------------------------- | +| `--name ` | Rename the profile; run this separately from URL or credential changes | +| `--url ` | Point the profile at a new Directus instance; keeps the current URL when omitted | +| `--token ` | Replace the saved credential with a static token | +| `--yes` | Skip confirmation when changing the profile name or URL | + +Renaming moves the saved credential to the new profile name and changes the environment variable the profile reads. Changing the URL clears the credential saved for the old URL; a `DIRECTUS__TOKEN` environment variable follows the profile to the new URL. The command states these effects before asking for confirmation. + +## `d6s profile list` + +```bash +d6s profile list +``` -Adding is an upsert: an existing name is updated. Run without arguments for prompts, which also offer to save a credential (paste a static token, or log in with email and password to save a session; saved sessions refresh themselves before they expire). Profile names use letters, numbers, and underscores. URLs with embedded credentials, query strings, or fragments are refused because the URL is stored in project configuration. +Prints each configured profile name and URL. Credentials are never shown. ## `d6s profile test` ```bash -d6s profile test +d6s profile test [name] [--url ] [--token ] ``` | Flag | Effect | @@ -59,6 +84,16 @@ Connects and prints who the credential authenticates as: ◇ Authenticated to https://cms.example.com as Admin User (Administrator). ``` +Pass either a profile name or `--url`, never both. Without a stored or environment credential, an interactive run asks for one; a non-interactive run tells you which token flag or environment variable to set. + +## `d6s profile remove` + +```bash +d6s profile remove [name] [--yes] +``` + +Removes the profile and clears its saved credential after confirmation. `--yes` skips that confirmation; non-interactive runs require it. + ## `d6s sync pull` ```bash @@ -84,49 +119,50 @@ The selectable resources are `roles`, `policies`, `flows`, `dashboards`, `settin Two warnings a scoped pull can raise, neither of which widens the scope for you: - **Out-of-scope references.** The scoped sync files point at something you omitted (a relation target, a group parent, a many-to-any collection). Pushing those files to a fresh target can fail; add the missing collections to `--collections` yourself. -- **A name the server didn't return.** A `--collections` name absent from the returned Schema (usually a typo) is named in a warning. The partial sync files are still written, but never silently. +- **A name the server didn't return.** A `--collections` name absent from the returned Schema (usually a typo) is named in a warning. The files for the rest of the requested scope are still overwritten, but never silently. ## `d6s sync diff` ```bash -d6s sync diff --to [--mode ] [--allow-version-drift] +d6s sync diff --to [--mode ] [--allow-drift] ``` -| Flag | Effect | -| --------------------------- | ------------------------------------------------------------------------------ | -| `--to ` (required) | Target profile name | -| `--mode ` | `add`, `merge`, or `mirror`; changes what the preview plans for | -| `--allow-version-drift` | Preview despite a version mismatch (see [the version rule](#the-version-rule)) | -| `--project ` | Project to sync (default: `default`) | +| Flag | Effect | +| --------------------------- | ---------------------------------------------------------------------------------------------------- | +| `--to ` (required) | Target profile name | +| `--mode ` | `add`, `merge`, or `mirror`; changes what the preview plans for | +| `--allow-drift` | Preview despite a Directus version or database vendor mismatch (see [the compatibility rule](#the-compatibility-rule)) | +| `--project ` | Project to sync (default: `default`) | Applies nothing, and exits `0` whether or not differences exist; automation reads the report's `changes` field. ## `d6s sync push` ```bash -d6s sync push --to [--mode ] [--yes] [--dangerously-allow-delete] [--allow-version-drift] +d6s sync push --to [--mode ] [--yes] [--dangerously-allow-delete] [--allow-drift] ``` -| Flag | Effect | -| ---------------------------- | --------------------------------------------------------------------------- | -| `--to ` (required) | Target profile name | -| `--mode ` | `add`, `merge` (default), or `mirror` | -| `--yes` | Skip the apply confirmation; never authorizes deletions | -| `--dangerously-allow-delete` | Consent to deletions; required for non-interactive `mirror` | -| `--allow-version-drift` | Push despite a version mismatch (see [the version rule](#the-version-rule)) | -| `--project ` | Project to sync (default: `default`) | +| Flag | Effect | +| ---------------------------- | --------------------------------------------------------------------------------------------------------- | +| `--to ` (required) | Target profile name | +| `--mode ` | `add`, `merge` (default), or `mirror` | +| `--yes` | Skip the apply confirmation; never authorizes deletions | +| `--dangerously-allow-delete` | Consent to deletions; required for non-interactive `mirror` | +| `--allow-drift` | Push despite a Directus version or database vendor mismatch (see [the compatibility rule](#the-compatibility-rule)) | +| `--project ` | Project to sync (default: `default`) | ## `directus.config.json` -Created and updated by `d6s profile add`; found by walking up from the current directory, like git finds `.git`. It never contains credentials, so commit it. The full shape: +Created by `d6s profile add` and changed by `d6s profile update`; found by walking up from the current directory, like git finds `.git`. It never contains credentials, so commit it. This example uses include scopes; each axis can use an exclude scope instead: ```json { "profiles": { - "staging": { "url": "https://staging.example.com" }, - "production": { "url": "https://cms.example.com" } + "staging": { "url": "https://staging.example.com", "auth": { "type": "token" } }, + "production": { "url": "https://cms.example.com", "auth": { "type": "token" } } }, "directory": "directus", + "format": "json", "projects": { "default": { "schema": true, @@ -140,11 +176,14 @@ Created and updated by `d6s profile add`; found by walking up from the current d Top-level keys: -| Key | Meaning | Default | -| ----------- | ------------------------------------------------------- | ------------ | -| `profiles` | Named instances: `{ "url": "https://..." }` per profile | `{}` | -| `directory` | The directory pulls write into and pushes read from | `"directus"` | -| `projects` | Per-project sync options (see below) | `{}` | +| Key | Meaning | Default | +| ----------- | --------------------------------------------------- | ------------ | +| `profiles` | Named instances and their URLs | `{}` | +| `directory` | The directory pulls write into and pushes read from | `"directus"` | +| `format` | Sync-file format; currently only `"json"` | `"json"` | +| `projects` | Per-project sync options (see below) | `{}` | + +`auth.type` is currently always `"token"`. The static token or saved login session itself stays outside this file. Per-project keys, all optional. A project is a named slice of the sync with its own subdirectory (`//`); the `default` project exists without being declared. Flags on the command line override these per run: @@ -162,11 +201,11 @@ Per-project keys, all optional. A project is a named slice of the sync with its ## Credentials -When a command authenticates a profile, the token resolves in order: +When a command authenticates a profile, its credential resolves in order: -1. A `--token` flag, on the two `profile` commands that accept one. The sync commands take no token flag. +1. A `--token` flag on `profile add`, `profile update`, or `profile test`. The sync commands take no token flag. 2. The `DIRECTUS__TOKEN` environment variable: the profile name uppercased, so `production` reads `DIRECTUS_PRODUCTION_TOKEN`. A `.env` file next to `directus.config.json` is loaded automatically without overriding real environment variables. -3. The credential store at `~/.directus/credentials.json`, written readable only by you (mode `0600`). Never consulted when the `CI` environment variable is set. +3. The static token or login session in `~/.directus/credentials.json`, written readable only by you (mode `0600`). Never consulted when `CI` is non-empty, except when its value is `false`. Use an admin credential. The Schema endpoints and the batch import the CLI relies on are admin-only on the server; the CLI does not check privileges up front, so a non-admin token fails with an authentication error or produces an incomplete pull that the completeness checks then flag. @@ -186,10 +225,10 @@ Resource selection follows a dependency graph: selecting a resource pulls in wha | `panels` | Yes, with dashboards | — | — | Panels have no field to match on, so a first push into a matching target can duplicate once; the ID map prevents repeats. | | `settings` | Yes | `--settings` | — | A single record. License and AI credentials, branding images (logos, backgrounds, favicon), and the default storage folder are stripped. | | `folders` | Yes | `--folders` | — | The media-library folder tree. Distinct from collection folders (Data Studio sidebar groups), which sync as Schema. | -| `translations` | Yes | `--translations` | — | Matched by `language` and `key`; `merge` and `mirror` can update existing strings. | +| `translations` | Yes | `--translations` | — | Matched by `language` and `key`; on Directus 12.2.0 and later, `merge` and `mirror` can update existing strings. | | `users` | Opt-in | `--users` | `roles`, `policies` | Secret fields (`password`, `token`, `tfa_secret`, and others) are stripped. | -::callout{icon="material-symbols:warning-rounded" color="warning"} +::callout{icon="i-lucide-triangle-alert" color="warning"} **Select `--roles`, not `--policies` alone** A selection that pulls policies without roles is not independently pushable when access records reference roles. With no roles in scope, a push to a fresh target fails. Select `--roles` instead; it includes policies and their dependent resources too. :: @@ -198,20 +237,20 @@ A selection that pulls policies without roles is not independently pushable when Two rules govern every pull, scoped or not: -1. A pull writes sync files only for what it fetched. Sync files for anything it did not fetch stay unchanged. +1. A pull overwrites the sync files in its requested scope with the source's current state. Files outside that scope are left unchanged. 2. A push only applies what is in the sync files. Work that never entered them cannot ship. -**Written in this pull** means the CLI fetched the current source state and wrote it to the corresponding sync files. Because writes are deterministic, an identical source state produces no git diff. **Not written** means the pull left those sync files unchanged on disk. +**Overwritten from source** means the CLI replaced the files in scope with the current source state. Because output is deterministic, an identical source state produces byte-identical files and no git diff. **Left unchanged** means the pull did not touch those files. -| Pull | Schema sync files | Configuration sync files | -| ------------------------------------ | ------------------------------------------------- | ---------------------------------------------------------------- | -| `pull --from staging` | All written in this pull | Default set written; users not written | -| `... --collections posts` | `posts` written; others not written | Default set written; users not written | -| `... --no-flows` | All written in this pull | Flows, operations, and users not written; other defaults written | -| `... --no-translations` | All written in this pull | Translations and users not written; other defaults written | -| `... --flows` | All written (resource flags do not narrow Schema) | Flows and operations written; all others not written | -| `... --flows --no-schema` | None written | Flows and operations written; all others not written | -| `... --collections posts --no-flows` | `posts` written; others not written | Flows, operations, and users not written; other defaults written | +| Pull | Schema sync files | Configuration sync files | +| ------------------------------------ | ------------------------------------------------------------- | ------------------------------------------------------------------------------ | +| `pull --from staging` | All overwritten from source | Default set overwritten; users left unchanged | +| `... --collections posts` | `posts` overwritten; all others left unchanged | Default set overwritten; users left unchanged | +| `... --no-flows` | All overwritten from source | Other defaults overwritten; flows, operations, and users left unchanged | +| `... --no-translations` | All overwritten from source | Other defaults overwritten; translations and users left unchanged | +| `... --flows` | All overwritten (resource flags do not narrow Schema) | Flows and operations overwritten; all others left unchanged | +| `... --flows --no-schema` | All left unchanged | Flows and operations overwritten; all others left unchanged | +| `... --collections posts --no-flows` | `posts` overwritten; all others left unchanged | Other defaults overwritten; flows, operations, and users left unchanged | ## Push modes @@ -232,36 +271,42 @@ Only `mirror` deletes, and deleting always requires its own explicit consent: | Interactive terminal | Review the plan naming the losses, then type the profile name (unless you passed `--dangerously-allow-delete`, which is the consent) | | Non-interactive / CI | Pass `--dangerously-allow-delete` | -`--yes` skips the ordinary confirmation prompt, but it never authorizes a deletion; that holds even if a supposedly additive push unexpectedly carries one. A `mirror` push in CI without `--dangerously-allow-delete` refuses before changing anything: +`--yes` skips the ordinary confirmation prompt, but it never authorizes a deletion; that holds even if a non-deleting push unexpectedly carries one. A `mirror` push in CI without `--dangerously-allow-delete` refuses before changing anything: ``` ✖ Refusing mirror mode in a non-interactive context without --dangerously-allow-delete. - mirror can delete schema and configuration records absent from the commit-ready files; pass --dangerously-allow-delete to consent, or use --mode merge. + mirror can delete schema and configuration records absent from ./directus/default; pass --dangerously-allow-delete to consent, or use --mode merge. ``` ## Record identity -Records are matched across instances first by the ID map (`//id_map.json`), then by an identifying field: +Records are matched across instances first by the ID map (`//id_map.json`), then by identifying fields: -| Resource | Matched by | -| -------------- | -------------------- | -| Most resources | `name` | -| Users | `email` | -| Operations | `key` | -| Translations | `language` and `key` | -| Panels | Nothing; ID map only | +| Resource | Matched by | +| ----------------------------------------------------- | ---------------------------------- | +| Roles, policies, flows, dashboards, and folders | `name` | +| Users | `email` | +| Access rules | `role`, `user`, and `policy` | +| Permissions | `policy`, `collection`, and `action` | +| Operations | `flow` and `key` | +| Translations | `language` and `key` | +| Settings | The single settings record | +| Panels | Nothing; ID map only | The ID map is keyed internally by source and target instance URL, so one file serves any number of targets without conflicts. Commit it whenever a push changes it. An ambiguous match (two candidates) prompts interactively and refuses non-interactively: ``` -✖ Ambiguous target matches: - directus_roles source "Editor" — sr1 → one of t1, t2 +✖ Push refused: 1 target match needs a choice. + directus_roles: ./directus/default contains 1 role named "Editor". + production — https://cms.example.com contains 2 matching roles. Run d6s sync push interactively once to choose, then commit the updated ID map. ``` -## The version rule +## The compatibility rule -Schema changes require the Directus version recorded in the sync files and the target's version to match exactly, patch release included; the mismatch error names both versions. `--allow-version-drift` proceeds anyway with a warning; the CLI never translates schema between versions. Projects with `"schema": false` skip the check entirely, and a target whose version cannot be read is left to the server's own gate rather than refused. +Environment Sync requires Directus 12.2.0 or later. Schema comparison requires the version recorded in the sync files and the target's version to match exactly, patch release included. The target server also requires the snapshot's database vendor to match its own. The CLI names both versions when it can detect a version mismatch; it cannot pre-check the target vendor, so it translates the server's refusal into an incompatible-snapshot error and keeps the server's reason as the detail. + +`--allow-drift` sends the server's `force` bypass for either mismatch and prints a `▲ Compatibility check bypassed` warning. It does not translate schema between Directus versions or database vendors. Projects with `"schema": false` skip the check entirely. When the target version cannot be read, the server makes the compatibility decision. ## JSON reports @@ -287,10 +332,35 @@ With `--json`, stdout carries exactly one report per command; warnings still go "removed": [], "scope": null, "data": { - "resources": ["directus_flows", "directus_roles"], - "collections": 10, - "records": 57, - "files": 10, + "resources": [ + "access", + "folders", + "operations", + "flows", + "panels", + "dashboards", + "permissions", + "policies", + "roles", + "settings", + "translations" + ], + "collections": [ + "directus_access", + "directus_folders", + "directus_operations", + "directus_flows", + "directus_panels", + "directus_dashboards", + "directus_permissions", + "directus_policies", + "directus_roles", + "directus_settings", + "directus_translations" + ], + "recordCount": 57, + "collectionCount": 11, + "fileCount": 12, "removed": [], "incomplete": [] } @@ -311,15 +381,15 @@ The Schema counters (`collections` through `files`) are `null` when the Schema p "project": "default", "mode": "merge", "changes": true, - "unresolved": 0, "schemaSkipped": false, "added": 1, "modified": 1, "deleted": 0, + "hash": "8a265a26cce1", "data": { "mode": "merge", "source": "https://staging.example.com", - "collections": { + "resultsByCollection": { "directus_flows": { "existing": [], "new": ["f1"], @@ -327,9 +397,12 @@ The Schema counters (`collections` through `files`) are `null` when the Schema p "mapped": {} } }, - "matched": 1, - "ambiguous": 0, - "unmatched": 1, + "reconciliation": { + "matched": 1, + "unmatched": 1, + "ambiguous": 0, + "dependent": 0 + }, "unchanged": 0, "incomplete": [], "skipped": false @@ -337,9 +410,9 @@ The Schema counters (`collections` through `files`) are `null` when the Schema p } ``` -`changes` is `true` when a push would do anything, including when records are `unresolved`; `added`/`modified`/`deleted` count Schema items; `data.collections` is the target server's own per-collection dry-run answer. +`changes` is `true` when a push would do anything, including when `data.reconciliation.ambiguous` is non-zero. `added`/`modified`/`deleted` count Schema items. `data.resultsByCollection` is the target server's own per-collection dry-run answer. Reconciliation separates records that matched, have no match, need a choice, or depend on a choice; `unchanged` counts matched records whose synced values already agree. -`d6s sync push --to production --yes --json` reports the same shape as a diff, with `applied` (`true` when the push changed the target) in place of `unresolved`, and `data.collections` reflecting what the import actually did. +`d6s sync push --to production --yes --json` reports the same top-level shape and adds `applied`, which is `true` when the push changed the target. Its `data.resultsByCollection` reflects what the import actually did; `data.reconciliation` and `data.unchanged` are `null` because push reports applied results rather than diff-only comparison counts. Failures put an error report on stdout: @@ -349,14 +422,14 @@ Failures put an error report on stdout: "formatVersion": 1, "error": { "code": "STATE", - "message": "Version mismatch: the snapshot was pulled from Directus 11.2.0, but the target runs 11.2.5.", + "message": "Version mismatch: the snapshot was pulled from Directus 12.1.1, but the target runs 12.2.0.", "hint": "..." } } ``` -The `code` is one of a small set of failure classes: `USAGE` (the command line needs fixing: a missing flag or missing consent), `CONFIG` (`directus.config.json` missing or invalid), `AUTH` (the credential was rejected), `HTTP` (the instance could not be reached or returned an error), `STATE` (the sync files and the instance disagree: version mismatch, changed target Schema, incomplete pull), or `UNKNOWN`. Exit codes are `0` for success and `1` for every failure; the code string is the finer-grained signal. +The `code` is one of a small set of failure classes: `USAGE` (the command line needs fixing: a missing flag or missing consent), `UNKNOWN_COMMAND`, `CONFIG` (saved configuration is missing, invalid, or conflicts with the request), `AUTH` (the credential was rejected), `HTTP` (the instance could not be reached or returned an error), `STATE` (the sync files and the instance disagree: version mismatch, changed target Schema, incomplete pull), or `UNKNOWN`. Exit codes are `0` for success and `1` for every failure; the code string is the finer-grained signal. ## Output conventions -Human-readable status lines go to stderr, prefixed `●` (info), `◇` (success), `▲` (warning), or `✖` (error, with its hint indented beneath). Plan lines go to stdout: `+` marks an addition, `~` a modification, and `✖ DELETE` a deletion, with Configuration plans summarized per collection as `+N new ~N updated ✖N deleted`. `--no-color` disables coloring; `--json` replaces stdout output with the report while warnings stay on stderr. +Human-readable status lines go to stderr, prefixed `●` (info), `◇` (success), `▲` (warning), or `✖` (error, with its hint indented beneath). Command results and plan lines go to stdout: `+` marks an addition, `~` a modification, and `✖ DELETE` a deletion, with Configuration plans summarized per collection as `+N new ~N updated ✖N deleted`. Legacy Windows consoles use the ASCII equivalents `i`, `+`, `!`, `x`, and `x DELETE`. `--no-color` disables coloring; `--json` replaces stdout output with the report while warnings stay on stderr. From 63af06b745fc5e6cd5e5ff455019fa5e09cf0c72 Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Tue, 11 Aug 2026 10:16:12 -0400 Subject: [PATCH 06/12] cleanup --- .../14.environment-sync/1.quickstart.md | 4 +- .../14.environment-sync/2.how-it-works.md | 6 +- .../4.ci-and-automation.md | 4 +- .../guides/14.environment-sync/6.reference.md | 143 +++++++++--------- 4 files changed, 77 insertions(+), 80 deletions(-) diff --git a/content/guides/14.environment-sync/1.quickstart.md b/content/guides/14.environment-sync/1.quickstart.md index 40848c02..c81a74bc 100644 --- a/content/guides/14.environment-sync/1.quickstart.md +++ b/content/guides/14.environment-sync/1.quickstart.md @@ -125,7 +125,7 @@ That first command created `directus.config.json` in the current directory: Notice what is not in it: the tokens. URLs are project configuration you commit; credentials go to `~/.directus/credentials.json`, readable only by you. Confirm both connections work: ```bash -d6s profile test source +d6s profile test-connection source ``` ``` @@ -161,7 +161,7 @@ directus/default/ data/ # one JSON file per configuration resource, plus metadata.json ``` -Open the `articles` file under `schema/`. It's your collection, readable as JSON: the fields you just created, their types, their interface settings. This is what reviewers will see in pull requests. Commit it: +Open the `articles` file under `schema/` (file names carry a short content-hash suffix, so it's `articles_.json`). It's your collection, readable as JSON: the fields you just created, their types, their interface settings. This is what reviewers will see in pull requests. Commit it: ```bash git add directus/ directus.config.json diff --git a/content/guides/14.environment-sync/2.how-it-works.md b/content/guides/14.environment-sync/2.how-it-works.md index 4307bfd9..1f5bbad4 100644 --- a/content/guides/14.environment-sync/2.how-it-works.md +++ b/content/guides/14.environment-sync/2.how-it-works.md @@ -19,8 +19,8 @@ A pull writes into a directory you commit (named `directus` by default, one subd ``` directus/default/ - schema/ # Schema, one JSON file per collection - data/ # Configuration, one JSON file per resource + schema/ # Schema, one JSON file per collection (_.json) + data/ # Configuration, one JSON file per resource (_.json) id_map.json # which target record corresponds to each source record ``` @@ -111,7 +111,7 @@ Schema comparison requires two things to match: the exact Directus version recor The CLI can name a known version mismatch before sending the snapshot. It cannot pre-check the target's database vendor, so a vendor mismatch comes back as an incompatible-snapshot refusal with the server's reason attached. -`--allow-drift` sends the server's `force` bypass for either mismatch and prints a `▲ Compatibility check bypassed` warning. It does not translate schema across Directus versions or database vendors, so read the plan closely. Projects configured with `"schema": false` skip the Schema phase and this gate entirely. +`--allow-drift` sends the server's `force` bypass for either mismatch and prints a `▲ Compatibility check bypassed` warning. It does not translate schema across Directus versions or database vendors, so read the plan closely. Projects configured with `"schema": false` skip the Schema phase and this comparison; the 12.2.0 minimum still applies. ## The safety model diff --git a/content/guides/14.environment-sync/4.ci-and-automation.md b/content/guides/14.environment-sync/4.ci-and-automation.md index f6f1825c..7f769dea 100644 --- a/content/guides/14.environment-sync/4.ci-and-automation.md +++ b/content/guides/14.environment-sync/4.ci-and-automation.md @@ -8,7 +8,7 @@ Automating Environment Sync buys you two things: every pull request shows what m ## The non-interactive contract -The CLI treats a non-empty `CI` variable as non-interactive, except for the literal value `false` (locally, `--no-interactive` forces the same behavior). Where the interactive CLI would ask a question, a non-interactive run refuses instead: +The CLI treats a non-empty `CI` variable as non-interactive (locally, `--no-interactive` forces the same behavior). Where the interactive CLI would ask a question, a non-interactive run refuses instead: - An **ambiguous record match** (two target records that could both correspond to the source record) fails the command rather than picking a candidate. Resolve it with an interactive push and commit the ID map; later CI pushes between the same source and target URLs reuse the answer. - **Deletions** happen only with `--dangerously-allow-delete`. A `mirror` push without it refuses before changing anything on the target. @@ -20,7 +20,7 @@ Commands exit `0` on success and `1` on any refusal or failure; there are no oth Pass tokens through environment variables named `DIRECTUS__TOKEN`, the profile name uppercased. Profile names use letters, numbers, and underscores, so the mapping is mechanical: profile `production` reads `DIRECTUS_PRODUCTION_TOKEN`, profile `staging_eu` reads `DIRECTUS_STAGING_EU_TOKEN`. -The credential store saved on a developer machine is never read when `CI` is non-empty, except when its value is `false`; tokens come from the environment only. +The credential store saved on a developer machine is never read when `CI` is non-empty; tokens come from the environment only. ## JSON reports diff --git a/content/guides/14.environment-sync/6.reference.md b/content/guides/14.environment-sync/6.reference.md index c1bcddc0..488ec71e 100644 --- a/content/guides/14.environment-sync/6.reference.md +++ b/content/guides/14.environment-sync/6.reference.md @@ -8,28 +8,28 @@ Everything on this page is lookup material. If you're learning the tool, start w ## Commands -| Command | What it does | -| -------------------- | ---------------------------------------------------------------------------------------- | -| `d6s profile add` | Create a profile and optionally save a credential | -| `d6s profile update` | Change a profile's name, URL, or saved credential | -| `d6s profile list` | List configured profile names and URLs | -| `d6s profile test` | Connect with a profile or URL and print who you are on the instance | -| `d6s profile remove` | Remove a profile and its saved credential | -| `d6s sync pull` | Write a source instance's Schema and Configuration to sync files | -| `d6s sync diff` | Show what a push would change on the target; applies nothing | -| `d6s sync push` | Apply the sync files to a target instance | -| `d6s sync` | Interactive wizard: prompts for source, target, project, and mode, then pulls and pushes | +| Command | What it does | +| ----------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | +| `d6s profile add` | Create a profile and optionally save a credential | +| `d6s profile update` | Change a profile's name, URL, or saved credential | +| `d6s profile list` | List configured profile names and URLs | +| `d6s profile test-connection` | Connect with a profile or URL and print who you are on the instance | +| `d6s profile remove` | Remove a profile and its saved credential | +| `d6s sync pull` | Write a source instance's Schema and Configuration to sync files | +| `d6s sync diff` | Show what a push would change on the target; applies nothing | +| `d6s sync push` | Apply the sync files to a target instance | +| `d6s sync` | Interactive wizard: prompts for source and target (plus project and mode when the configuration leaves them open), then pulls and pushes | Install `@directus/cli@12`. It provides `directus-cli` and the equivalent short alias `d6s`. Global flags, available on every command: -| Flag | Effect | -| ------------------ | ---------------------------------------------------------------------------------------- | -| `--json` | One machine-readable report on stdout; human status stays off stdout | -| `--no-color` | Disable colored output | -| `--no-interactive` | Disable prompts; behave as in CI | -| `--config ` | Path to `directus.config.json` (default: found by walking up from the current directory) | +| Flag | Effect | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------------- | +| `--json` | One machine-readable report on stdout; human status stays off stdout. Also disables prompts, so the non-interactive rules apply | +| `--no-color` | Disable colored output | +| `--no-interactive` | Disable prompts; behave as in CI | +| `--config ` | Path to `directus.config.json` (default: found by walking up from the current directory) | ## `d6s profile add` @@ -50,12 +50,12 @@ An existing profile name is refused; use `profile update` to change it. Run with d6s profile update [name] [--name ] [--url ] [--token ] [--yes] ``` -| Flag | Effect | -| ----------------- | -------------------------------------------------------------------------------------- | -| `--name ` | Rename the profile; run this separately from URL or credential changes | -| `--url ` | Point the profile at a new Directus instance; keeps the current URL when omitted | -| `--token ` | Replace the saved credential with a static token | -| `--yes` | Skip confirmation when changing the profile name or URL | +| Flag | Effect | +| ----------------- | -------------------------------------------------------------------------------- | +| `--name ` | Rename the profile; run this separately from URL or credential changes | +| `--url ` | Point the profile at a new Directus instance; keeps the current URL when omitted | +| `--token ` | Replace the saved credential with a static token | +| `--yes` | Skip confirmation when changing the profile name or URL | Renaming moves the saved credential to the new profile name and changes the environment variable the profile reads. Changing the URL clears the credential saved for the old URL; a `DIRECTUS__TOKEN` environment variable follows the profile to the new URL. The command states these effects before asking for confirmation. @@ -67,10 +67,10 @@ d6s profile list Prints each configured profile name and URL. Credentials are never shown. -## `d6s profile test` +## `d6s profile test-connection` ```bash -d6s profile test [name] [--url ] [--token ] +d6s profile test-connection [name] [--url ] [--token ] ``` | Flag | Effect | @@ -127,12 +127,12 @@ Two warnings a scoped pull can raise, neither of which widens the scope for you: d6s sync diff --to [--mode ] [--allow-drift] ``` -| Flag | Effect | -| --------------------------- | ---------------------------------------------------------------------------------------------------- | -| `--to ` (required) | Target profile name | -| `--mode ` | `add`, `merge`, or `mirror`; changes what the preview plans for | +| Flag | Effect | +| --------------------------- | ---------------------------------------------------------------------------------------------------------------------- | +| `--to ` (required) | Target profile name | +| `--mode ` | `add`, `merge`, or `mirror`; changes what the preview plans for | | `--allow-drift` | Preview despite a Directus version or database vendor mismatch (see [the compatibility rule](#the-compatibility-rule)) | -| `--project ` | Project to sync (default: `default`) | +| `--project ` | Project to sync (default: `default`) | Applies nothing, and exits `0` whether or not differences exist; automation reads the report's `changes` field. @@ -142,14 +142,14 @@ Applies nothing, and exits `0` whether or not differences exist; automation read d6s sync push --to [--mode ] [--yes] [--dangerously-allow-delete] [--allow-drift] ``` -| Flag | Effect | -| ---------------------------- | --------------------------------------------------------------------------------------------------------- | -| `--to ` (required) | Target profile name | -| `--mode ` | `add`, `merge` (default), or `mirror` | -| `--yes` | Skip the apply confirmation; never authorizes deletions | -| `--dangerously-allow-delete` | Consent to deletions; required for non-interactive `mirror` | +| Flag | Effect | +| ---------------------------- | ------------------------------------------------------------------------------------------------------------------- | +| `--to ` (required) | Target profile name | +| `--mode ` | `add`, `merge` (default), or `mirror` | +| `--yes` | Skip the apply confirmation; never authorizes deletions | +| `--dangerously-allow-delete` | Consent to deletions; required for non-interactive `mirror` | | `--allow-drift` | Push despite a Directus version or database vendor mismatch (see [the compatibility rule](#the-compatibility-rule)) | -| `--project ` | Project to sync (default: `default`) | +| `--project ` | Project to sync (default: `default`) | ## `directus.config.json` @@ -158,8 +158,14 @@ Created by `d6s profile add` and changed by `d6s profile update`; found by walki ```json { "profiles": { - "staging": { "url": "https://staging.example.com", "auth": { "type": "token" } }, - "production": { "url": "https://cms.example.com", "auth": { "type": "token" } } + "staging": { + "url": "https://staging.example.com", + "auth": { "type": "token" } + }, + "production": { + "url": "https://cms.example.com", + "auth": { "type": "token" } + } }, "directory": "directus", "format": "json", @@ -178,9 +184,9 @@ Top-level keys: | Key | Meaning | Default | | ----------- | --------------------------------------------------- | ------------ | -| `profiles` | Named instances and their URLs | `{}` | +| `profiles` | Named instances and their URLs | `{}` | | `directory` | The directory pulls write into and pushes read from | `"directus"` | -| `format` | Sync-file format; currently only `"json"` | `"json"` | +| `format` | Sync-file format; currently only `"json"` | `"json"` | | `projects` | Per-project sync options (see below) | `{}` | `auth.type` is currently always `"token"`. The static token or saved login session itself stays outside this file. @@ -203,9 +209,9 @@ Per-project keys, all optional. A project is a named slice of the sync with its When a command authenticates a profile, its credential resolves in order: -1. A `--token` flag on `profile add`, `profile update`, or `profile test`. The sync commands take no token flag. +1. A `--token` flag on `profile add`, `profile update`, or `profile test-connection`. The sync commands take no token flag. 2. The `DIRECTUS__TOKEN` environment variable: the profile name uppercased, so `production` reads `DIRECTUS_PRODUCTION_TOKEN`. A `.env` file next to `directus.config.json` is loaded automatically without overriding real environment variables. -3. The static token or login session in `~/.directus/credentials.json`, written readable only by you (mode `0600`). Never consulted when `CI` is non-empty, except when its value is `false`. +3. The static token or login session in `~/.directus/credentials.json`, written readable only by you (mode `0600`). Never consulted when `CI` is non-empty. Use an admin credential. The Schema endpoints and the batch import the CLI relies on are admin-only on the server; the CLI does not check privileges up front, so a non-admin token fails with an authentication error or produces an incomplete pull that the completeness checks then flag. @@ -242,15 +248,15 @@ Two rules govern every pull, scoped or not: **Overwritten from source** means the CLI replaced the files in scope with the current source state. Because output is deterministic, an identical source state produces byte-identical files and no git diff. **Left unchanged** means the pull did not touch those files. -| Pull | Schema sync files | Configuration sync files | -| ------------------------------------ | ------------------------------------------------------------- | ------------------------------------------------------------------------------ | -| `pull --from staging` | All overwritten from source | Default set overwritten; users left unchanged | -| `... --collections posts` | `posts` overwritten; all others left unchanged | Default set overwritten; users left unchanged | -| `... --no-flows` | All overwritten from source | Other defaults overwritten; flows, operations, and users left unchanged | -| `... --no-translations` | All overwritten from source | Other defaults overwritten; translations and users left unchanged | -| `... --flows` | All overwritten (resource flags do not narrow Schema) | Flows and operations overwritten; all others left unchanged | -| `... --flows --no-schema` | All left unchanged | Flows and operations overwritten; all others left unchanged | -| `... --collections posts --no-flows` | `posts` overwritten; all others left unchanged | Other defaults overwritten; flows, operations, and users left unchanged | +| Pull | Schema sync files | Configuration sync files | +| ------------------------------------ | ----------------------------------------------------- | ----------------------------------------------------------------------- | +| `pull --from staging` | All overwritten from source | Default set overwritten; users left unchanged | +| `... --collections posts` | `posts` overwritten; all others left unchanged | Default set overwritten; users left unchanged | +| `... --no-flows` | All overwritten from source | Other defaults overwritten; flows, operations, and users left unchanged | +| `... --no-translations` | All overwritten from source | Other defaults overwritten; translations and users left unchanged | +| `... --flows` | All overwritten (resource flags do not narrow Schema) | Flows and operations overwritten; all others left unchanged | +| `... --flows --no-schema` | All left unchanged | Flows and operations overwritten; all others left unchanged | +| `... --collections posts --no-flows` | `posts` overwritten; all others left unchanged | Other defaults overwritten; flows, operations, and users left unchanged | ## Push modes @@ -282,16 +288,16 @@ Only `mirror` deletes, and deleting always requires its own explicit consent: Records are matched across instances first by the ID map (`//id_map.json`), then by identifying fields: -| Resource | Matched by | -| ----------------------------------------------------- | ---------------------------------- | -| Roles, policies, flows, dashboards, and folders | `name` | -| Users | `email` | -| Access rules | `role`, `user`, and `policy` | -| Permissions | `policy`, `collection`, and `action` | -| Operations | `flow` and `key` | -| Translations | `language` and `key` | -| Settings | The single settings record | -| Panels | Nothing; ID map only | +| Resource | Matched by | +| ----------------------------------------------- | ------------------------------------ | +| Roles, policies, flows, dashboards, and folders | `name` | +| Users | `email` | +| Access rules | `role`, `user`, and `policy` | +| Permissions | `policy`, `collection`, and `action` | +| Operations | `flow` and `key` | +| Translations | `language` and `key` | +| Settings | The single settings record | +| Panels | Nothing; ID map only | The ID map is keyed internally by source and target instance URL, so one file serves any number of targets without conflicts. Commit it whenever a push changes it. An ambiguous match (two candidates) prompts interactively and refuses non-interactively: @@ -306,7 +312,7 @@ The ID map is keyed internally by source and target instance URL, so one file se Environment Sync requires Directus 12.2.0 or later. Schema comparison requires the version recorded in the sync files and the target's version to match exactly, patch release included. The target server also requires the snapshot's database vendor to match its own. The CLI names both versions when it can detect a version mismatch; it cannot pre-check the target vendor, so it translates the server's refusal into an incompatible-snapshot error and keeps the server's reason as the detail. -`--allow-drift` sends the server's `force` bypass for either mismatch and prints a `▲ Compatibility check bypassed` warning. It does not translate schema between Directus versions or database vendors. Projects with `"schema": false` skip the check entirely. When the target version cannot be read, the server makes the compatibility decision. +`--allow-drift` sends the server's `force` bypass for either mismatch and prints a `▲ Compatibility check bypassed` warning. It does not translate schema between Directus versions or database vendors. Projects with `"schema": false` skip the version-parity and vendor comparison; the 12.2.0 minimum still applies. When the target version cannot be read, the server makes the compatibility decision. ## JSON reports @@ -316,9 +322,6 @@ With `--json`, stdout carries exactly one report per command; warnings still go ```json { - "kind": "PullReport", - "formatVersion": 1, - "ok": true, "source": "https://staging.example.com", "profile": "staging", "project": "default", @@ -373,9 +376,6 @@ The Schema counters (`collections` through `files`) are `null` when the Schema p ```json { - "kind": "DiffReport", - "formatVersion": 1, - "ok": true, "target": "https://cms.example.com", "profile": "production", "project": "default", @@ -404,8 +404,7 @@ The Schema counters (`collections` through `files`) are `null` when the Schema p "dependent": 0 }, "unchanged": 0, - "incomplete": [], - "skipped": false + "incomplete": [] } } ``` @@ -418,8 +417,6 @@ Failures put an error report on stdout: ```json { - "kind": "ErrorReport", - "formatVersion": 1, "error": { "code": "STATE", "message": "Version mismatch: the snapshot was pulled from Directus 12.1.1, but the target runs 12.2.0.", From 5235d2c125b2cf648da732bfc6e176b4ebcf6ca4 Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Tue, 11 Aug 2026 23:03:56 -0400 Subject: [PATCH 07/12] tweak docs --- content/guides/14.environment-sync/0.index.md | 2 +- content/guides/14.environment-sync/6.reference.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/content/guides/14.environment-sync/0.index.md b/content/guides/14.environment-sync/0.index.md index d3cce250..91caf053 100644 --- a/content/guides/14.environment-sync/0.index.md +++ b/content/guides/14.environment-sync/0.index.md @@ -31,7 +31,7 @@ Content sync is deferred to a future release. Cross-instance record identity for ## Before you start - **Both instances must run Directus 12.2.0 or later, at the same exact version and on the same database vendor.** The patch release must match. The server refuses incompatible schema comparisons because some patches change the snapshot format and database vendors describe column types differently. -- **An admin credential for each instance.** The schema and configuration endpoints the CLI uses are admin-only. A static token from an admin user is the usual choice. +- **An admin credential for each instance.** A static token from an admin user is the usual choice. The CLI verifies this and refuses non-admin tokens: the server rejects non-admin schema and import writes, and non-admin reads are silently filtered by permissions, which would produce sync files that look complete but aren't. - **A git repository.** The sync files are designed for review and versioning. Any repository works; many teams use the one that already holds their Directus deployment configuration. ## Where to go diff --git a/content/guides/14.environment-sync/6.reference.md b/content/guides/14.environment-sync/6.reference.md index 488ec71e..476f2178 100644 --- a/content/guides/14.environment-sync/6.reference.md +++ b/content/guides/14.environment-sync/6.reference.md @@ -114,7 +114,7 @@ d6s sync pull --from [scope flags] A bare pull includes every selectable resource except users. This means translations sync by default. Pass `--no-translations` to exclude them, or `--translations` to pull only translations. -The selectable resources are `roles`, `policies`, `flows`, `dashboards`, `settings`, `folders`, `users`, and `translations`. The CLI automatically includes `access`, `permissions`, `operations`, and `panels` with their parent resources (see [the dependency table](#configuration-resources)). Positive selection (`--flows`) cannot be combined with `--all` or with `--no-` flags. Resource selection never narrows the schema; the two axes are scoped independently. +The selectable resources are `roles`, `policies`, `flows`, `dashboards`, `settings`, `folders`, `users`, and `translations`. The CLI automatically includes `access`, `permissions`, `operations`, and `panels` with their parent resources (see [the dependency table](#configuration-resources)). Positive selection (`--flows`) cannot be combined with `--all` or with `--no-` flags. An exclusion that a retained resource depends on is refused rather than silently overridden: `--no-policies` alone fails because `roles` requires policies and would pull them back — exclude `--no-roles` as well, or keep policies. Resource selection never narrows the schema; the two axes are scoped independently. Two warnings a scoped pull can raise, neither of which widens the scope for you: From b766511300a6455afc2fc0dd12984dc9c54e97f4 Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Thu, 13 Aug 2026 08:00:00 -0400 Subject: [PATCH 08/12] Scoped pulls up front, admin gate, and permission warning accuracy The index now says a pull is not all-or-nothing and anchors it in the promote workflow (review feedback). The reference reflects the CLI's new up-front admin check. The secrets blind spot covers URL query and body warnings plus the undetectable URL-path case, and the unlicensed permissions bullet explains hidden rows may just be recreated built-in permissions, in the same words the CLI warning uses. --- content/guides/14.environment-sync/0.index.md | 2 ++ .../5.secrets-and-limitations.md | 4 +-- .../guides/14.environment-sync/6.reference.md | 30 +++++++++---------- 3 files changed, 19 insertions(+), 17 deletions(-) diff --git a/content/guides/14.environment-sync/0.index.md b/content/guides/14.environment-sync/0.index.md index 91caf053..1cc1ff44 100644 --- a/content/guides/14.environment-sync/0.index.md +++ b/content/guides/14.environment-sync/0.index.md @@ -22,6 +22,8 @@ The CLI is built to be safe to point at production: `diff` never applies anythin - **Configuration**: roles, policies, access, permissions, flows, operations, dashboards, panels, settings, media-library folders, and custom Data Studio translation strings. - **Opt-in**: user accounts, with every secret field stripped. +Unlike a full schema snapshot, a pull is not all-or-nothing. `--collections posts` narrows which Schema files it overwrites, and resource flags like `--flows` or `--no-schema` choose which Configuration files it touches. That is how you promote finished work from a shared development instance while unfinished work stays out of the sync files entirely. Scope is by collection and resource type, not by individual record. See [Promote only the changes that are ready](/guides/environment-sync/common-workflows#promote-only-the-changes-that-are-ready). + Records in your own collections are **content**, and Environment Sync does not sync them. It moves the shape of a project and its configuration, not its content. ::callout{icon="i-lucide-info"} diff --git a/content/guides/14.environment-sync/5.secrets-and-limitations.md b/content/guides/14.environment-sync/5.secrets-and-limitations.md index 688851a6..4a6efba2 100644 --- a/content/guides/14.environment-sync/5.secrets-and-limitations.md +++ b/content/guides/14.environment-sync/5.secrets-and-limitations.md @@ -22,7 +22,7 @@ The server never hands out the real value for these fields: concealed fields rea ### The one blind spot ::callout{icon="i-lucide-triangle-alert" color="warning"} -**Flow request headers are written verbatim.** A secret pasted into free-form flow configuration (most commonly an Authorization header in a request operation) has no "this is secret" marker for the CLI to check, and legitimate headers have to sync. The pull warns you by operation name, and the value goes into the sync file as-is. Review those files before committing and before publishing a repository. +**Flow request options are written verbatim.** A secret pasted into free-form flow configuration (an Authorization header, a `?api_key=` URL parameter, a token in the request body) has no "this is secret" marker for the CLI to check, and legitimate values have to sync. The pull warns you by operation name when it sees headers, a URL query string, or a body — but it cannot recognize a secret embedded in a plain URL path, and the value always goes into the sync file as-is. Review those files before committing and before publishing a repository. :: ## System collections that do not sync @@ -60,7 +60,7 @@ None of these sync today. Some are shared configuration that a future release co ## Known limitations -- **Unlicensed custom permission rules are unavailable to pull.** On an instance without a license, the API hides custom permission rules. The pull detects the missing records and marks the result incomplete: `merge` and `add` push normally, while `mirror` refuses. License the source instance to include them. +- **Unlicensed custom permission rules are unavailable to pull.** On an instance without a license, the API hides stored permissions that use filters, field restrictions, validation, or presets. The pull detects the missing records and marks the result incomplete: `merge` and `add` push normally, while `mirror` refuses. Some hidden rows may just be stored copies of the built-in permissions every app-access policy gets automatically — harmless to lose, since the server recreates them — but the CLI cannot verify that, so the incomplete marking stands. License the source instance and re-pull to include authored rules. - **Users without an email address fail import.** The server's import validation rejects a user with no email, so a `--users` push containing one fails before any Configuration changes apply. Give the account an email on the source, or remove it, first. - **Panels can duplicate on a first sync into a look-alike target.** Panels have no name to match on, so pushing into a target that already holds equivalent panels (seeded from the same template) can create them a second time. The ID map prevents repeats after that first push. - **A mirror push of users does not protect your own account.** If users are in the sync files and the account the push authenticates with is absent from them, a `mirror` push orders its deletion like any other record; the CLI has no self-protection, and whether the server refuses is up to the server. When you sync users, make sure the sync files include the accounts your pushes run as. diff --git a/content/guides/14.environment-sync/6.reference.md b/content/guides/14.environment-sync/6.reference.md index 476f2178..5f27d494 100644 --- a/content/guides/14.environment-sync/6.reference.md +++ b/content/guides/14.environment-sync/6.reference.md @@ -213,26 +213,26 @@ When a command authenticates a profile, its credential resolves in order: 2. The `DIRECTUS__TOKEN` environment variable: the profile name uppercased, so `production` reads `DIRECTUS_PRODUCTION_TOKEN`. A `.env` file next to `directus.config.json` is loaded automatically without overriding real environment variables. 3. The static token or login session in `~/.directus/credentials.json`, written readable only by you (mode `0600`). Never consulted when `CI` is non-empty. -Use an admin credential. The Schema endpoints and the batch import the CLI relies on are admin-only on the server; the CLI does not check privileges up front, so a non-admin token fails with an authentication error or produces an incomplete pull that the completeness checks then flag. +Use an admin credential. The Schema endpoints and the batch import the CLI relies on are admin-only on the server, and non-admin reads are silently permission-filtered — so the CLI verifies admin access at the start of every pull, diff, and push and refuses a non-admin token outright. ## Configuration resources Resource selection follows a dependency graph: selecting a resource pulls in what it needs (unless `--no-deps`). -| Resource | In default pull | Select directly | Also includes | Notes | -| -------------- | ---------------------------- | ---------------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | -| `roles` | Yes | `--roles` | `policies` | | -| `policies` | Yes | `--policies` | `access`, `permissions` | See the warning below before selecting policies on their own. | -| `access` | Yes, with roles and policies | — | — | Grants attached to users are dropped when users are out of scope; a mirror push does not delete them on the target. | -| `permissions` | Yes, with policies | — | — | Record counts are verified against the server. If the source hides records (unlicensed custom permission rules), the pull is incomplete. | -| `flows` | Yes | `--flows` | `operations` | | -| `operations` | Yes, with flows | — | — | | -| `dashboards` | Yes | `--dashboards` | `panels` | | -| `panels` | Yes, with dashboards | — | — | Panels have no field to match on, so a first push into a matching target can duplicate once; the ID map prevents repeats. | -| `settings` | Yes | `--settings` | — | A single record. License and AI credentials, branding images (logos, backgrounds, favicon), and the default storage folder are stripped. | -| `folders` | Yes | `--folders` | — | The media-library folder tree. Distinct from collection folders (Data Studio sidebar groups), which sync as Schema. | -| `translations` | Yes | `--translations` | — | Matched by `language` and `key`; on Directus 12.2.0 and later, `merge` and `mirror` can update existing strings. | -| `users` | Opt-in | `--users` | `roles`, `policies` | Secret fields (`password`, `token`, `tfa_secret`, and others) are stripped. | +| Resource | In default pull | Select directly | Also includes | Notes | +| -------------- | ---------------------------- | ---------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `roles` | Yes | `--roles` | `policies` | | +| `policies` | Yes | `--policies` | `access`, `permissions` | See the warning below before selecting policies on their own. | +| `access` | Yes, with roles and policies | — | — | Grants attached to users are dropped when users are out of scope; a mirror push does not delete them on the target. | +| `permissions` | Yes, with policies | — | — | Record counts are verified against the server. If the source hides records (unlicensed custom permission rules), the pull is incomplete. Stored duplicates of the built-in app-access permissions are dropped and reported; the server recreates them. | +| `flows` | Yes | `--flows` | `operations` | | +| `operations` | Yes, with flows | — | — | | +| `dashboards` | Yes | `--dashboards` | `panels` | | +| `panels` | Yes, with dashboards | — | — | Panels have no field to match on, so a first push into a matching target can duplicate once; the ID map prevents repeats. | +| `settings` | Yes | `--settings` | — | A single record. License and AI credentials, branding images (logos, backgrounds, favicon), and the default storage folder are stripped. | +| `folders` | Yes | `--folders` | — | The media-library folder tree. Distinct from collection folders (Data Studio sidebar groups), which sync as Schema. | +| `translations` | Yes | `--translations` | — | Matched by `language` and `key`; on Directus 12.2.0 and later, `merge` and `mirror` can update existing strings. | +| `users` | Opt-in | `--users` | `roles`, `policies` | Secret fields (`password`, `token`, `tfa_secret`, and others) are stripped. | ::callout{icon="i-lucide-triangle-alert" color="warning"} **Select `--roles`, not `--policies` alone** From 0c5b9f391c1fe670176de1b7a036ff2516f93648 Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Thu, 13 Aug 2026 08:49:19 -0400 Subject: [PATCH 09/12] Clarity pass; move Environment Sync above Deployments in guides --- .../.navigation.yml | 0 .../0.index.md | 6 +++--- .../1.quickstart.md | 0 .../2.how-it-works.md | 8 +++++--- .../3.common-workflows.md | 2 +- .../4.ci-and-automation.md | 2 +- .../5.secrets-and-limitations.md | 2 +- .../6.reference.md | 10 +++++----- .../{10.deployments => 11.deployments}/.navigation.yml | 0 .../{10.deployments => 11.deployments}/0.index.md | 0 .../{10.deployments => 11.deployments}/1.security.md | 0 content/guides/{11.ai => 12.ai}/.navigation.yml | 0 content/guides/{11.ai => 12.ai}/0.index.md | 0 .../{11.ai => 12.ai}/1.assistant/.navigation.yml | 0 content/guides/{11.ai => 12.ai}/1.assistant/0.index.md | 0 content/guides/{11.ai => 12.ai}/1.assistant/1.setup.md | 0 content/guides/{11.ai => 12.ai}/1.assistant/2.usage.md | 0 content/guides/{11.ai => 12.ai}/1.assistant/3.tools.md | 0 content/guides/{11.ai => 12.ai}/1.assistant/4.tips.md | 0 .../guides/{11.ai => 12.ai}/1.assistant/5.security.md | 0 content/guides/{11.ai => 12.ai}/2.mcp/.navigation.yml | 0 content/guides/{11.ai => 12.ai}/2.mcp/0.index.md | 0 .../guides/{11.ai => 12.ai}/2.mcp/1.installation.md | 0 content/guides/{11.ai => 12.ai}/2.mcp/2.use-cases.md | 0 content/guides/{11.ai => 12.ai}/2.mcp/3.tools.md | 0 content/guides/{11.ai => 12.ai}/2.mcp/4.prompts.md | 0 .../guides/{11.ai => 12.ai}/2.mcp/5.troubleshooting.md | 0 content/guides/{11.ai => 12.ai}/2.mcp/6.oauth.md | 0 content/guides/{11.ai => 12.ai}/2.mcp/7.security.md | 0 .../{11.ai => 12.ai}/2.mcp/8.local-mcp/.navigation.yml | 0 .../{11.ai => 12.ai}/2.mcp/8.local-mcp/0.index.md | 0 .../{11.ai => 12.ai}/2.mcp/8.local-mcp/1.tools.md | 0 .../{11.ai => 12.ai}/2.mcp/8.local-mcp/2.prompts.md | 0 content/guides/{11.ai => 12.ai}/3.translations.md | 0 .../.navigation.yml | 0 .../1.n8n/.navigation.yml | 0 .../1.n8n/0.index.md | 0 .../1.n8n/directus-n8n-actions.md | 0 .../1.n8n/directus-n8n-advanced.md | 0 .../1.n8n/directus-n8n-triggers.md | 0 .../2.clay/.navigation.yml | 0 .../2.clay/0.index.md | 0 .../2.clay/directus-clay-data-operations.md | 0 .../2.clay/use-clay-templates-with-directus.md | 0 .../2.clay/use-directus-webhooks-with-clay.md | 0 .../3.zapier/.navigation.yml | 0 .../3.zapier/0.index.md | 0 .../3.zapier/actions.md | 0 .../3.zapier/advanced.md | 0 .../3.zapier/triggers.md | 0 .../4.vercel/.navigation.yml | 0 .../4.vercel/0.index.md | 0 .../4.vercel/deployments.md | 0 .../5.netlify/.navigation.yml | 0 .../5.netlify/0.index.md | 0 .../5.netlify/deployments.md | 0 .../6.airbyte/0.index.md | 0 .../6.framer/.navigation.yml | 0 .../6.framer/0.index.md | 0 .../6.supabase/.navigation.yml | 0 .../6.supabase/0.index.md | 0 .../connect-supabase-postgres-to-directus.md | 0 .../6.supabase/use-supabase-storage-with-directus.md | 0 .../{12.integrations => 13.integrations}/index.md | 0 .../{13.security => 14.security}/1.best-practices.md | 0 65 files changed, 16 insertions(+), 14 deletions(-) rename content/guides/{14.environment-sync => 10.environment-sync}/.navigation.yml (100%) rename content/guides/{14.environment-sync => 10.environment-sync}/0.index.md (86%) rename content/guides/{14.environment-sync => 10.environment-sync}/1.quickstart.md (100%) rename content/guides/{14.environment-sync => 10.environment-sync}/2.how-it-works.md (94%) rename content/guides/{14.environment-sync => 10.environment-sync}/3.common-workflows.md (99%) rename content/guides/{14.environment-sync => 10.environment-sync}/4.ci-and-automation.md (98%) rename content/guides/{14.environment-sync => 10.environment-sync}/5.secrets-and-limitations.md (97%) rename content/guides/{14.environment-sync => 10.environment-sync}/6.reference.md (98%) rename content/guides/{10.deployments => 11.deployments}/.navigation.yml (100%) rename content/guides/{10.deployments => 11.deployments}/0.index.md (100%) rename content/guides/{10.deployments => 11.deployments}/1.security.md (100%) rename content/guides/{11.ai => 12.ai}/.navigation.yml (100%) rename content/guides/{11.ai => 12.ai}/0.index.md (100%) rename content/guides/{11.ai => 12.ai}/1.assistant/.navigation.yml (100%) rename content/guides/{11.ai => 12.ai}/1.assistant/0.index.md (100%) rename content/guides/{11.ai => 12.ai}/1.assistant/1.setup.md (100%) rename content/guides/{11.ai => 12.ai}/1.assistant/2.usage.md (100%) rename content/guides/{11.ai => 12.ai}/1.assistant/3.tools.md (100%) rename content/guides/{11.ai => 12.ai}/1.assistant/4.tips.md (100%) rename content/guides/{11.ai => 12.ai}/1.assistant/5.security.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/.navigation.yml (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/0.index.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/1.installation.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/2.use-cases.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/3.tools.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/4.prompts.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/5.troubleshooting.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/6.oauth.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/7.security.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/8.local-mcp/.navigation.yml (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/8.local-mcp/0.index.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/8.local-mcp/1.tools.md (100%) rename content/guides/{11.ai => 12.ai}/2.mcp/8.local-mcp/2.prompts.md (100%) rename content/guides/{11.ai => 12.ai}/3.translations.md (100%) rename content/guides/{12.integrations => 13.integrations}/.navigation.yml (100%) rename content/guides/{12.integrations => 13.integrations}/1.n8n/.navigation.yml (100%) rename content/guides/{12.integrations => 13.integrations}/1.n8n/0.index.md (100%) rename content/guides/{12.integrations => 13.integrations}/1.n8n/directus-n8n-actions.md (100%) rename content/guides/{12.integrations => 13.integrations}/1.n8n/directus-n8n-advanced.md (100%) rename content/guides/{12.integrations => 13.integrations}/1.n8n/directus-n8n-triggers.md (100%) rename content/guides/{12.integrations => 13.integrations}/2.clay/.navigation.yml (100%) rename content/guides/{12.integrations => 13.integrations}/2.clay/0.index.md (100%) rename content/guides/{12.integrations => 13.integrations}/2.clay/directus-clay-data-operations.md (100%) rename content/guides/{12.integrations => 13.integrations}/2.clay/use-clay-templates-with-directus.md (100%) rename content/guides/{12.integrations => 13.integrations}/2.clay/use-directus-webhooks-with-clay.md (100%) rename content/guides/{12.integrations => 13.integrations}/3.zapier/.navigation.yml (100%) rename content/guides/{12.integrations => 13.integrations}/3.zapier/0.index.md (100%) rename content/guides/{12.integrations => 13.integrations}/3.zapier/actions.md (100%) rename content/guides/{12.integrations => 13.integrations}/3.zapier/advanced.md (100%) rename content/guides/{12.integrations => 13.integrations}/3.zapier/triggers.md (100%) rename content/guides/{12.integrations => 13.integrations}/4.vercel/.navigation.yml (100%) rename content/guides/{12.integrations => 13.integrations}/4.vercel/0.index.md (100%) rename content/guides/{12.integrations => 13.integrations}/4.vercel/deployments.md (100%) rename content/guides/{12.integrations => 13.integrations}/5.netlify/.navigation.yml (100%) rename content/guides/{12.integrations => 13.integrations}/5.netlify/0.index.md (100%) rename content/guides/{12.integrations => 13.integrations}/5.netlify/deployments.md (100%) rename content/guides/{12.integrations => 13.integrations}/6.airbyte/0.index.md (100%) rename content/guides/{12.integrations => 13.integrations}/6.framer/.navigation.yml (100%) rename content/guides/{12.integrations => 13.integrations}/6.framer/0.index.md (100%) rename content/guides/{12.integrations => 13.integrations}/6.supabase/.navigation.yml (100%) rename content/guides/{12.integrations => 13.integrations}/6.supabase/0.index.md (100%) rename content/guides/{12.integrations => 13.integrations}/6.supabase/connect-supabase-postgres-to-directus.md (100%) rename content/guides/{12.integrations => 13.integrations}/6.supabase/use-supabase-storage-with-directus.md (100%) rename content/guides/{12.integrations => 13.integrations}/index.md (100%) rename content/guides/{13.security => 14.security}/1.best-practices.md (100%) diff --git a/content/guides/14.environment-sync/.navigation.yml b/content/guides/10.environment-sync/.navigation.yml similarity index 100% rename from content/guides/14.environment-sync/.navigation.yml rename to content/guides/10.environment-sync/.navigation.yml diff --git a/content/guides/14.environment-sync/0.index.md b/content/guides/10.environment-sync/0.index.md similarity index 86% rename from content/guides/14.environment-sync/0.index.md rename to content/guides/10.environment-sync/0.index.md index 1cc1ff44..158fcd2a 100644 --- a/content/guides/14.environment-sync/0.index.md +++ b/content/guides/10.environment-sync/0.index.md @@ -12,9 +12,9 @@ d6s sync diff --to production # preview what pushing the sync files would cha d6s sync push --to production # apply them ``` -Because the sync files live in your repository, you can review changes in pull requests, promote them through git history, and use a revert plus another push to restore an earlier state. Removals require `mirror`, even during a rollback. (`d6s sync` on its own runs a small interactive wizard that pulls and pushes in one pass.) +Because the sync files live in your repository, you can review changes in pull requests, promote them through git history, and use a revert plus another push to restore an earlier state. `d6s sync` on its own runs a small interactive wizard that pulls and pushes in one pass. -The CLI is built to be safe to point at production: `diff` never applies anything, deletions always require their own explicit consent, and the CLI asks you to resolve records that match more than one target record. It never guesses. [How It Works](/guides/environment-sync/how-it-works) covers the full safety model. +The CLI is built to be safe to point at production: `diff` never applies anything, deletions happen only in `mirror` mode behind their own explicit consent (that includes removals during a rollback), and the CLI asks you to resolve records that match more than one target record. It never guesses. [How It Works](/guides/environment-sync/how-it-works) covers the full safety model. ## What syncs @@ -32,7 +32,7 @@ Content sync is deferred to a future release. Cross-instance record identity for ## Before you start -- **Both instances must run Directus 12.2.0 or later, at the same exact version and on the same database vendor.** The patch release must match. The server refuses incompatible schema comparisons because some patches change the snapshot format and database vendors describe column types differently. +- **Both instances must run Directus 12.2.0 or later, at the same exact version and on the same database vendor.** The patch release must match. The server refuses an incompatible schema comparison: some patches change the snapshot format, and database vendors describe column types differently. `--allow-drift` bypasses the check when you accept that risk; see [the compatibility gate](/guides/environment-sync/how-it-works#the-compatibility-gate). - **An admin credential for each instance.** A static token from an admin user is the usual choice. The CLI verifies this and refuses non-admin tokens: the server rejects non-admin schema and import writes, and non-admin reads are silently filtered by permissions, which would produce sync files that look complete but aren't. - **A git repository.** The sync files are designed for review and versioning. Any repository works; many teams use the one that already holds their Directus deployment configuration. diff --git a/content/guides/14.environment-sync/1.quickstart.md b/content/guides/10.environment-sync/1.quickstart.md similarity index 100% rename from content/guides/14.environment-sync/1.quickstart.md rename to content/guides/10.environment-sync/1.quickstart.md diff --git a/content/guides/14.environment-sync/2.how-it-works.md b/content/guides/10.environment-sync/2.how-it-works.md similarity index 94% rename from content/guides/14.environment-sync/2.how-it-works.md rename to content/guides/10.environment-sync/2.how-it-works.md index 1f5bbad4..3ce4a787 100644 --- a/content/guides/14.environment-sync/2.how-it-works.md +++ b/content/guides/10.environment-sync/2.how-it-works.md @@ -11,7 +11,7 @@ Environment Sync never moves anything directly between two instances. The sync f Everything else on this page is a consequence of those two rules. -The CLI reads and writes the sync directory on disk. It never checks your git state, so a push applies the sync files as they exist, including uncommitted edits. This also allows the `d6s sync` wizard to pull and push in one pass. Git provides review and history around the sync files. Keep the tree clean when you push so the files on disk match the reviewed commit. +The CLI reads and writes the sync directory on disk. It never checks your git state, so a push applies the sync files as they exist, including uncommitted edits. That is also what lets the `d6s sync` wizard pull and push in one pass: the freshly pulled files push without a commit in between. Git provides review and history around the sync files. Keep the tree clean when you push so the files on disk match the reviewed commit. ## The sync files @@ -21,7 +21,7 @@ A pull writes into a directory you commit (named `directus` by default, one subd directus/default/ schema/ # Schema, one JSON file per collection (_.json) data/ # Configuration, one JSON file per resource (_.json) - id_map.json # which target record corresponds to each source record + id_map.json # written by pushes: which target record corresponds to each source record ``` The sync files are written deterministically: pulling twice with no instance changes produces byte-identical files and a clean working tree. `git diff` after a pull shows what changed on the instance and nothing else, so a schema change reads like any other code change in review. @@ -67,6 +67,8 @@ Which target role does this represent? Abort push (Applies no remote changes) ``` +The parenthetical after each existing candidate compares it to the sync files: identical values, or the fields that differ (here `icon`). + Your answer lands in the ID map and is reused for later pushes between the same source and target URLs. That's why the map belongs in git: commit it whenever a push changes it, and teammates and CI inherit the decisions already made. One ID map serves any number of instances. Internally it is keyed by source and target URL, so pushing the same sync files to staging and production writes two independent sets of mappings; neither overwrites the other. Deleting the ID map does not change either instance, but matching starts over on the next push. Named records can usually match again by their identifying fields; records without them can duplicate. Repointing a profile at a new URL also starts a new set of mappings, because decisions stored for the old URL do not apply to the new one. @@ -102,7 +104,7 @@ Because a mirror push makes the target match the sync files _exactly_, run it ag ## The compatibility gate -Schema comparison requires two things to match: the exact Directus version recorded in the sync files, patch release included, and the target's database vendor. The server refuses the comparison when either differs. Historically, some Directus patches change the schema format, while database vendors describe column types differently. +Schema comparison requires two things to match: the exact Directus version recorded in the sync files, patch release included, and the target's database vendor. The server refuses the comparison when either differs. Some Directus patches have changed the snapshot format, and database vendors describe column types differently. ```text ✖ Version mismatch: the snapshot was pulled from Directus 12.1.1, but the target runs 12.2.0. diff --git a/content/guides/14.environment-sync/3.common-workflows.md b/content/guides/10.environment-sync/3.common-workflows.md similarity index 99% rename from content/guides/14.environment-sync/3.common-workflows.md rename to content/guides/10.environment-sync/3.common-workflows.md index 16fc9ea4..130e4def 100644 --- a/content/guides/14.environment-sync/3.common-workflows.md +++ b/content/guides/10.environment-sync/3.common-workflows.md @@ -150,7 +150,7 @@ d6s sync push --to production --mode mirror The push repeats the plan, then the [deletion gate](/guides/environment-sync/reference#deletion-gates) demands typed consent: ``` -This push permanently deletes 1 configuration record and 1 schema deletion from production. Type "production" to confirm: +This push permanently deletes 1 configuration record and 1 schema item from production. Type "production" to confirm: ``` In automation, the same push requires `--dangerously-allow-delete` instead. diff --git a/content/guides/14.environment-sync/4.ci-and-automation.md b/content/guides/10.environment-sync/4.ci-and-automation.md similarity index 98% rename from content/guides/14.environment-sync/4.ci-and-automation.md rename to content/guides/10.environment-sync/4.ci-and-automation.md index 7f769dea..6b1d3dbe 100644 --- a/content/guides/14.environment-sync/4.ci-and-automation.md +++ b/content/guides/10.environment-sync/4.ci-and-automation.md @@ -91,7 +91,7 @@ jobs: { created: 0, updated: 0, deleted: 0 }, ); const body = report.changes - ? `**Environment Sync**: merging changes production. Schema: ${report.added} added, ` + + ? `**Environment Sync**: merging this PR changes production. Schema: ${report.added} added, ` + `${report.modified} modified, ${report.deleted} deleted. Configuration: ` + `${configuration.created} created, ${configuration.updated} updated, ` + `${configuration.deleted} deleted; ${ambiguous} ambiguous matches.` diff --git a/content/guides/14.environment-sync/5.secrets-and-limitations.md b/content/guides/10.environment-sync/5.secrets-and-limitations.md similarity index 97% rename from content/guides/14.environment-sync/5.secrets-and-limitations.md rename to content/guides/10.environment-sync/5.secrets-and-limitations.md index 4a6efba2..c4b663a3 100644 --- a/content/guides/14.environment-sync/5.secrets-and-limitations.md +++ b/content/guides/10.environment-sync/5.secrets-and-limitations.md @@ -22,7 +22,7 @@ The server never hands out the real value for these fields: concealed fields rea ### The one blind spot ::callout{icon="i-lucide-triangle-alert" color="warning"} -**Flow request options are written verbatim.** A secret pasted into free-form flow configuration (an Authorization header, a `?api_key=` URL parameter, a token in the request body) has no "this is secret" marker for the CLI to check, and legitimate values have to sync. The pull warns you by operation name when it sees headers, a URL query string, or a body — but it cannot recognize a secret embedded in a plain URL path, and the value always goes into the sync file as-is. Review those files before committing and before publishing a repository. +**Flow request options are written verbatim.** A secret pasted into free-form flow configuration (an Authorization header, a `?api_key=` URL parameter, a token in the request body) has no "this is secret" marker for the CLI to check, and legitimate values have to sync. The pull warns you by operation name when it sees headers, a URL carrying a query string or embedded credentials, or a body — but it cannot recognize a secret embedded in a plain URL path, and the value always goes into the sync file as-is. Review those files before committing and before publishing a repository. :: ## System collections that do not sync diff --git a/content/guides/14.environment-sync/6.reference.md b/content/guides/10.environment-sync/6.reference.md similarity index 98% rename from content/guides/14.environment-sync/6.reference.md rename to content/guides/10.environment-sync/6.reference.md index 5f27d494..237965b1 100644 --- a/content/guides/14.environment-sync/6.reference.md +++ b/content/guides/10.environment-sync/6.reference.md @@ -114,7 +114,7 @@ d6s sync pull --from [scope flags] A bare pull includes every selectable resource except users. This means translations sync by default. Pass `--no-translations` to exclude them, or `--translations` to pull only translations. -The selectable resources are `roles`, `policies`, `flows`, `dashboards`, `settings`, `folders`, `users`, and `translations`. The CLI automatically includes `access`, `permissions`, `operations`, and `panels` with their parent resources (see [the dependency table](#configuration-resources)). Positive selection (`--flows`) cannot be combined with `--all` or with `--no-` flags. An exclusion that a retained resource depends on is refused rather than silently overridden: `--no-policies` alone fails because `roles` requires policies and would pull them back — exclude `--no-roles` as well, or keep policies. Resource selection never narrows the schema; the two axes are scoped independently. +The selectable resources are `roles`, `policies`, `flows`, `dashboards`, `settings`, `folders`, `users`, and `translations`. The CLI automatically includes `access`, `permissions`, `operations`, and `panels` with their parent resources (see [the dependency table](#configuration-resources)). Positive selection (`--flows`) cannot be combined with `--all` or with `--no-` flags. An exclusion that a retained resource depends on is refused rather than silently overridden: `--no-policies` alone fails because `roles` requires policies and would pull them back — pass `--no-roles` as well, or keep policies. Resource selection never narrows the schema; the two axes are scoped independently. Two warnings a scoped pull can raise, neither of which widens the scope for you: @@ -131,7 +131,7 @@ d6s sync diff --to [--mode ] [--allow-drift] | --------------------------- | ---------------------------------------------------------------------------------------------------------------------- | | `--to ` (required) | Target profile name | | `--mode ` | `add`, `merge`, or `mirror`; changes what the preview plans for | -| `--allow-drift` | Preview despite a Directus version or database vendor mismatch (see [the compatibility rule](#the-compatibility-rule)) | +| `--allow-drift` | Preview despite a Directus version or database vendor mismatch (see [the compatibility gate](#the-compatibility-gate)) | | `--project ` | Project to sync (default: `default`) | Applies nothing, and exits `0` whether or not differences exist; automation reads the report's `changes` field. @@ -148,7 +148,7 @@ d6s sync push --to [--mode ] [--yes] [--dangerously-allow-delete | `--mode ` | `add`, `merge` (default), or `mirror` | | `--yes` | Skip the apply confirmation; never authorizes deletions | | `--dangerously-allow-delete` | Consent to deletions; required for non-interactive `mirror` | -| `--allow-drift` | Push despite a Directus version or database vendor mismatch (see [the compatibility rule](#the-compatibility-rule)) | +| `--allow-drift` | Push despite a Directus version or database vendor mismatch (see [the compatibility gate](#the-compatibility-gate)) | | `--project ` | Project to sync (default: `default`) | ## `directus.config.json` @@ -308,7 +308,7 @@ The ID map is keyed internally by source and target instance URL, so one file se Run d6s sync push interactively once to choose, then commit the updated ID map. ``` -## The compatibility rule +## The compatibility gate Environment Sync requires Directus 12.2.0 or later. Schema comparison requires the version recorded in the sync files and the target's version to match exactly, patch release included. The target server also requires the snapshot's database vendor to match its own. The CLI names both versions when it can detect a version mismatch; it cannot pre-check the target vendor, so it translates the server's refusal into an incompatible-snapshot error and keeps the server's reason as the detail. @@ -370,7 +370,7 @@ With `--json`, stdout carries exactly one report per command; warnings still go } ``` -The Schema counters (`collections` through `files`) are `null` when the Schema phase is skipped; `scope` echoes a `--collections`/`--exclude-collections` scope; `data.incomplete` names resources whose pull the source cut short. The JSON API keeps `data` as its compatibility field name; it contains the Configuration report. +The Schema counters (`collections` through `files`) are `null` when the Schema phase is skipped; `scope` echoes a `--collections`/`--exclude-collections` scope; `data.incomplete` names resources whose pull the source cut short. The Configuration report lives under `data`; the name is kept for backward compatibility. `d6s sync diff --to production --json`: diff --git a/content/guides/10.deployments/.navigation.yml b/content/guides/11.deployments/.navigation.yml similarity index 100% rename from content/guides/10.deployments/.navigation.yml rename to content/guides/11.deployments/.navigation.yml diff --git a/content/guides/10.deployments/0.index.md b/content/guides/11.deployments/0.index.md similarity index 100% rename from content/guides/10.deployments/0.index.md rename to content/guides/11.deployments/0.index.md diff --git a/content/guides/10.deployments/1.security.md b/content/guides/11.deployments/1.security.md similarity index 100% rename from content/guides/10.deployments/1.security.md rename to content/guides/11.deployments/1.security.md diff --git a/content/guides/11.ai/.navigation.yml b/content/guides/12.ai/.navigation.yml similarity index 100% rename from content/guides/11.ai/.navigation.yml rename to content/guides/12.ai/.navigation.yml diff --git a/content/guides/11.ai/0.index.md b/content/guides/12.ai/0.index.md similarity index 100% rename from content/guides/11.ai/0.index.md rename to content/guides/12.ai/0.index.md diff --git a/content/guides/11.ai/1.assistant/.navigation.yml b/content/guides/12.ai/1.assistant/.navigation.yml similarity index 100% rename from content/guides/11.ai/1.assistant/.navigation.yml rename to content/guides/12.ai/1.assistant/.navigation.yml diff --git a/content/guides/11.ai/1.assistant/0.index.md b/content/guides/12.ai/1.assistant/0.index.md similarity index 100% rename from content/guides/11.ai/1.assistant/0.index.md rename to content/guides/12.ai/1.assistant/0.index.md diff --git a/content/guides/11.ai/1.assistant/1.setup.md b/content/guides/12.ai/1.assistant/1.setup.md similarity index 100% rename from content/guides/11.ai/1.assistant/1.setup.md rename to content/guides/12.ai/1.assistant/1.setup.md diff --git a/content/guides/11.ai/1.assistant/2.usage.md b/content/guides/12.ai/1.assistant/2.usage.md similarity index 100% rename from content/guides/11.ai/1.assistant/2.usage.md rename to content/guides/12.ai/1.assistant/2.usage.md diff --git a/content/guides/11.ai/1.assistant/3.tools.md b/content/guides/12.ai/1.assistant/3.tools.md similarity index 100% rename from content/guides/11.ai/1.assistant/3.tools.md rename to content/guides/12.ai/1.assistant/3.tools.md diff --git a/content/guides/11.ai/1.assistant/4.tips.md b/content/guides/12.ai/1.assistant/4.tips.md similarity index 100% rename from content/guides/11.ai/1.assistant/4.tips.md rename to content/guides/12.ai/1.assistant/4.tips.md diff --git a/content/guides/11.ai/1.assistant/5.security.md b/content/guides/12.ai/1.assistant/5.security.md similarity index 100% rename from content/guides/11.ai/1.assistant/5.security.md rename to content/guides/12.ai/1.assistant/5.security.md diff --git a/content/guides/11.ai/2.mcp/.navigation.yml b/content/guides/12.ai/2.mcp/.navigation.yml similarity index 100% rename from content/guides/11.ai/2.mcp/.navigation.yml rename to content/guides/12.ai/2.mcp/.navigation.yml diff --git a/content/guides/11.ai/2.mcp/0.index.md b/content/guides/12.ai/2.mcp/0.index.md similarity index 100% rename from content/guides/11.ai/2.mcp/0.index.md rename to content/guides/12.ai/2.mcp/0.index.md diff --git a/content/guides/11.ai/2.mcp/1.installation.md b/content/guides/12.ai/2.mcp/1.installation.md similarity index 100% rename from content/guides/11.ai/2.mcp/1.installation.md rename to content/guides/12.ai/2.mcp/1.installation.md diff --git a/content/guides/11.ai/2.mcp/2.use-cases.md b/content/guides/12.ai/2.mcp/2.use-cases.md similarity index 100% rename from content/guides/11.ai/2.mcp/2.use-cases.md rename to content/guides/12.ai/2.mcp/2.use-cases.md diff --git a/content/guides/11.ai/2.mcp/3.tools.md b/content/guides/12.ai/2.mcp/3.tools.md similarity index 100% rename from content/guides/11.ai/2.mcp/3.tools.md rename to content/guides/12.ai/2.mcp/3.tools.md diff --git a/content/guides/11.ai/2.mcp/4.prompts.md b/content/guides/12.ai/2.mcp/4.prompts.md similarity index 100% rename from content/guides/11.ai/2.mcp/4.prompts.md rename to content/guides/12.ai/2.mcp/4.prompts.md diff --git a/content/guides/11.ai/2.mcp/5.troubleshooting.md b/content/guides/12.ai/2.mcp/5.troubleshooting.md similarity index 100% rename from content/guides/11.ai/2.mcp/5.troubleshooting.md rename to content/guides/12.ai/2.mcp/5.troubleshooting.md diff --git a/content/guides/11.ai/2.mcp/6.oauth.md b/content/guides/12.ai/2.mcp/6.oauth.md similarity index 100% rename from content/guides/11.ai/2.mcp/6.oauth.md rename to content/guides/12.ai/2.mcp/6.oauth.md diff --git a/content/guides/11.ai/2.mcp/7.security.md b/content/guides/12.ai/2.mcp/7.security.md similarity index 100% rename from content/guides/11.ai/2.mcp/7.security.md rename to content/guides/12.ai/2.mcp/7.security.md diff --git a/content/guides/11.ai/2.mcp/8.local-mcp/.navigation.yml b/content/guides/12.ai/2.mcp/8.local-mcp/.navigation.yml similarity index 100% rename from content/guides/11.ai/2.mcp/8.local-mcp/.navigation.yml rename to content/guides/12.ai/2.mcp/8.local-mcp/.navigation.yml diff --git a/content/guides/11.ai/2.mcp/8.local-mcp/0.index.md b/content/guides/12.ai/2.mcp/8.local-mcp/0.index.md similarity index 100% rename from content/guides/11.ai/2.mcp/8.local-mcp/0.index.md rename to content/guides/12.ai/2.mcp/8.local-mcp/0.index.md diff --git a/content/guides/11.ai/2.mcp/8.local-mcp/1.tools.md b/content/guides/12.ai/2.mcp/8.local-mcp/1.tools.md similarity index 100% rename from content/guides/11.ai/2.mcp/8.local-mcp/1.tools.md rename to content/guides/12.ai/2.mcp/8.local-mcp/1.tools.md diff --git a/content/guides/11.ai/2.mcp/8.local-mcp/2.prompts.md b/content/guides/12.ai/2.mcp/8.local-mcp/2.prompts.md similarity index 100% rename from content/guides/11.ai/2.mcp/8.local-mcp/2.prompts.md rename to content/guides/12.ai/2.mcp/8.local-mcp/2.prompts.md diff --git a/content/guides/11.ai/3.translations.md b/content/guides/12.ai/3.translations.md similarity index 100% rename from content/guides/11.ai/3.translations.md rename to content/guides/12.ai/3.translations.md diff --git a/content/guides/12.integrations/.navigation.yml b/content/guides/13.integrations/.navigation.yml similarity index 100% rename from content/guides/12.integrations/.navigation.yml rename to content/guides/13.integrations/.navigation.yml diff --git a/content/guides/12.integrations/1.n8n/.navigation.yml b/content/guides/13.integrations/1.n8n/.navigation.yml similarity index 100% rename from content/guides/12.integrations/1.n8n/.navigation.yml rename to content/guides/13.integrations/1.n8n/.navigation.yml diff --git a/content/guides/12.integrations/1.n8n/0.index.md b/content/guides/13.integrations/1.n8n/0.index.md similarity index 100% rename from content/guides/12.integrations/1.n8n/0.index.md rename to content/guides/13.integrations/1.n8n/0.index.md diff --git a/content/guides/12.integrations/1.n8n/directus-n8n-actions.md b/content/guides/13.integrations/1.n8n/directus-n8n-actions.md similarity index 100% rename from content/guides/12.integrations/1.n8n/directus-n8n-actions.md rename to content/guides/13.integrations/1.n8n/directus-n8n-actions.md diff --git a/content/guides/12.integrations/1.n8n/directus-n8n-advanced.md b/content/guides/13.integrations/1.n8n/directus-n8n-advanced.md similarity index 100% rename from content/guides/12.integrations/1.n8n/directus-n8n-advanced.md rename to content/guides/13.integrations/1.n8n/directus-n8n-advanced.md diff --git a/content/guides/12.integrations/1.n8n/directus-n8n-triggers.md b/content/guides/13.integrations/1.n8n/directus-n8n-triggers.md similarity index 100% rename from content/guides/12.integrations/1.n8n/directus-n8n-triggers.md rename to content/guides/13.integrations/1.n8n/directus-n8n-triggers.md diff --git a/content/guides/12.integrations/2.clay/.navigation.yml b/content/guides/13.integrations/2.clay/.navigation.yml similarity index 100% rename from content/guides/12.integrations/2.clay/.navigation.yml rename to content/guides/13.integrations/2.clay/.navigation.yml diff --git a/content/guides/12.integrations/2.clay/0.index.md b/content/guides/13.integrations/2.clay/0.index.md similarity index 100% rename from content/guides/12.integrations/2.clay/0.index.md rename to content/guides/13.integrations/2.clay/0.index.md diff --git a/content/guides/12.integrations/2.clay/directus-clay-data-operations.md b/content/guides/13.integrations/2.clay/directus-clay-data-operations.md similarity index 100% rename from content/guides/12.integrations/2.clay/directus-clay-data-operations.md rename to content/guides/13.integrations/2.clay/directus-clay-data-operations.md diff --git a/content/guides/12.integrations/2.clay/use-clay-templates-with-directus.md b/content/guides/13.integrations/2.clay/use-clay-templates-with-directus.md similarity index 100% rename from content/guides/12.integrations/2.clay/use-clay-templates-with-directus.md rename to content/guides/13.integrations/2.clay/use-clay-templates-with-directus.md diff --git a/content/guides/12.integrations/2.clay/use-directus-webhooks-with-clay.md b/content/guides/13.integrations/2.clay/use-directus-webhooks-with-clay.md similarity index 100% rename from content/guides/12.integrations/2.clay/use-directus-webhooks-with-clay.md rename to content/guides/13.integrations/2.clay/use-directus-webhooks-with-clay.md diff --git a/content/guides/12.integrations/3.zapier/.navigation.yml b/content/guides/13.integrations/3.zapier/.navigation.yml similarity index 100% rename from content/guides/12.integrations/3.zapier/.navigation.yml rename to content/guides/13.integrations/3.zapier/.navigation.yml diff --git a/content/guides/12.integrations/3.zapier/0.index.md b/content/guides/13.integrations/3.zapier/0.index.md similarity index 100% rename from content/guides/12.integrations/3.zapier/0.index.md rename to content/guides/13.integrations/3.zapier/0.index.md diff --git a/content/guides/12.integrations/3.zapier/actions.md b/content/guides/13.integrations/3.zapier/actions.md similarity index 100% rename from content/guides/12.integrations/3.zapier/actions.md rename to content/guides/13.integrations/3.zapier/actions.md diff --git a/content/guides/12.integrations/3.zapier/advanced.md b/content/guides/13.integrations/3.zapier/advanced.md similarity index 100% rename from content/guides/12.integrations/3.zapier/advanced.md rename to content/guides/13.integrations/3.zapier/advanced.md diff --git a/content/guides/12.integrations/3.zapier/triggers.md b/content/guides/13.integrations/3.zapier/triggers.md similarity index 100% rename from content/guides/12.integrations/3.zapier/triggers.md rename to content/guides/13.integrations/3.zapier/triggers.md diff --git a/content/guides/12.integrations/4.vercel/.navigation.yml b/content/guides/13.integrations/4.vercel/.navigation.yml similarity index 100% rename from content/guides/12.integrations/4.vercel/.navigation.yml rename to content/guides/13.integrations/4.vercel/.navigation.yml diff --git a/content/guides/12.integrations/4.vercel/0.index.md b/content/guides/13.integrations/4.vercel/0.index.md similarity index 100% rename from content/guides/12.integrations/4.vercel/0.index.md rename to content/guides/13.integrations/4.vercel/0.index.md diff --git a/content/guides/12.integrations/4.vercel/deployments.md b/content/guides/13.integrations/4.vercel/deployments.md similarity index 100% rename from content/guides/12.integrations/4.vercel/deployments.md rename to content/guides/13.integrations/4.vercel/deployments.md diff --git a/content/guides/12.integrations/5.netlify/.navigation.yml b/content/guides/13.integrations/5.netlify/.navigation.yml similarity index 100% rename from content/guides/12.integrations/5.netlify/.navigation.yml rename to content/guides/13.integrations/5.netlify/.navigation.yml diff --git a/content/guides/12.integrations/5.netlify/0.index.md b/content/guides/13.integrations/5.netlify/0.index.md similarity index 100% rename from content/guides/12.integrations/5.netlify/0.index.md rename to content/guides/13.integrations/5.netlify/0.index.md diff --git a/content/guides/12.integrations/5.netlify/deployments.md b/content/guides/13.integrations/5.netlify/deployments.md similarity index 100% rename from content/guides/12.integrations/5.netlify/deployments.md rename to content/guides/13.integrations/5.netlify/deployments.md diff --git a/content/guides/12.integrations/6.airbyte/0.index.md b/content/guides/13.integrations/6.airbyte/0.index.md similarity index 100% rename from content/guides/12.integrations/6.airbyte/0.index.md rename to content/guides/13.integrations/6.airbyte/0.index.md diff --git a/content/guides/12.integrations/6.framer/.navigation.yml b/content/guides/13.integrations/6.framer/.navigation.yml similarity index 100% rename from content/guides/12.integrations/6.framer/.navigation.yml rename to content/guides/13.integrations/6.framer/.navigation.yml diff --git a/content/guides/12.integrations/6.framer/0.index.md b/content/guides/13.integrations/6.framer/0.index.md similarity index 100% rename from content/guides/12.integrations/6.framer/0.index.md rename to content/guides/13.integrations/6.framer/0.index.md diff --git a/content/guides/12.integrations/6.supabase/.navigation.yml b/content/guides/13.integrations/6.supabase/.navigation.yml similarity index 100% rename from content/guides/12.integrations/6.supabase/.navigation.yml rename to content/guides/13.integrations/6.supabase/.navigation.yml diff --git a/content/guides/12.integrations/6.supabase/0.index.md b/content/guides/13.integrations/6.supabase/0.index.md similarity index 100% rename from content/guides/12.integrations/6.supabase/0.index.md rename to content/guides/13.integrations/6.supabase/0.index.md diff --git a/content/guides/12.integrations/6.supabase/connect-supabase-postgres-to-directus.md b/content/guides/13.integrations/6.supabase/connect-supabase-postgres-to-directus.md similarity index 100% rename from content/guides/12.integrations/6.supabase/connect-supabase-postgres-to-directus.md rename to content/guides/13.integrations/6.supabase/connect-supabase-postgres-to-directus.md diff --git a/content/guides/12.integrations/6.supabase/use-supabase-storage-with-directus.md b/content/guides/13.integrations/6.supabase/use-supabase-storage-with-directus.md similarity index 100% rename from content/guides/12.integrations/6.supabase/use-supabase-storage-with-directus.md rename to content/guides/13.integrations/6.supabase/use-supabase-storage-with-directus.md diff --git a/content/guides/12.integrations/index.md b/content/guides/13.integrations/index.md similarity index 100% rename from content/guides/12.integrations/index.md rename to content/guides/13.integrations/index.md diff --git a/content/guides/13.security/1.best-practices.md b/content/guides/14.security/1.best-practices.md similarity index 100% rename from content/guides/13.security/1.best-practices.md rename to content/guides/14.security/1.best-practices.md From 61e0ac09798433af7123af0c051780c4ab8aadbd Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Thu, 13 Aug 2026 11:00:15 -0400 Subject: [PATCH 10/12] cli output component --- app/components/content/ProsePre.global.vue | 120 ++++++++++++++++++ .../10.environment-sync/1.quickstart.md | 12 +- .../10.environment-sync/2.how-it-works.md | 6 +- .../10.environment-sync/3.common-workflows.md | 4 +- .../guides/10.environment-sync/6.reference.md | 6 +- 5 files changed, 134 insertions(+), 14 deletions(-) create mode 100644 app/components/content/ProsePre.global.vue diff --git a/app/components/content/ProsePre.global.vue b/app/components/content/ProsePre.global.vue new file mode 100644 index 00000000..b5754b4f --- /dev/null +++ b/app/components/content/ProsePre.global.vue @@ -0,0 +1,120 @@ + + + diff --git a/content/guides/10.environment-sync/1.quickstart.md b/content/guides/10.environment-sync/1.quickstart.md index c81a74bc..49f6cc09 100644 --- a/content/guides/10.environment-sync/1.quickstart.md +++ b/content/guides/10.environment-sync/1.quickstart.md @@ -102,7 +102,7 @@ A profile pairs a name with an instance URL. Run this in the same directory as t d6s profile add source --url http://localhost:8055 --token source-token ``` -``` +```cli ◇ Saved profile "source" → http://localhost:8055 ◇ Saved a token for "source" to the credential store. ``` @@ -128,7 +128,7 @@ Notice what is not in it: the tokens. URLs are project configuration you commit; d6s profile test-connection source ``` -``` +```cli ◇ Authenticated to http://localhost:8055 as Admin User (Administrator). ``` @@ -145,7 +145,7 @@ git init d6s sync pull --from source ``` -``` +```cli ◇ Pulled from source — http://localhost:8055 Schema 1 collection → directus/default/schema Configuration 14 records across 11 collections → directus/default/data @@ -192,7 +192,7 @@ The target still has only its initial Directus setup and does not have `articles d6s sync diff --to target ``` -``` +```cli ● Comparing ./directus/default with target — http://localhost:8056 (merge — creates and updates records, never deletes) ● Schema — 1 change: 1 added, 0 modified, 0 deleted + collection articles (3 fields) @@ -209,7 +209,7 @@ d6s sync push --to target The push prints the same plan, then asks before applying: `Apply 1 schema change to target — http://localhost:8056?`. Confirm it: -``` +```cli ● Schema applied. ◇ Push complete. Applied 1 schema change to http://localhost:8056; schema hash verified. No configuration changes to push. ``` @@ -226,7 +226,7 @@ If a push imports configuration records, it also writes `directus/default/id_map d6s sync push --to target ``` -``` +```cli ● Pushing ./directus/default to target — http://localhost:8056 (merge — creates and updates records, never deletes) ◇ target — http://localhost:8056 already matches ./directus/default — schema and configuration match; nothing to push. ``` diff --git a/content/guides/10.environment-sync/2.how-it-works.md b/content/guides/10.environment-sync/2.how-it-works.md index 3ce4a787..eb966100 100644 --- a/content/guides/10.environment-sync/2.how-it-works.md +++ b/content/guides/10.environment-sync/2.how-it-works.md @@ -50,7 +50,7 @@ For an existing translation, the CLI replaces the source ID with the matching ta When two target records could both be the match, the CLI asks you to choose in a terminal and refuses in CI. Ambiguity is never resolved by guessing. The question names both sides, links to the records when the Data Studio has a stable route for them, and explains what every choice will do: -```text +```cli directus_roles — 1 of 1 ./directus/default contains 1 role named "Editor". production — https://cms.example.com contains 2 matching roles. @@ -82,7 +82,7 @@ After record identity is settled, an interactive push dry-runs Configuration, sh The two phases are not one transaction. If the Configuration push fails after Schema applied, the two phases can be out of sync, and the CLI says so plainly: -```text +```cli ▲ Schema was applied, but the configuration push did not complete. ✖ Could not reach https://cms.example.com. Schema is already applied — re-run d6s sync push to retry the configuration push against an empty schema diff. @@ -106,7 +106,7 @@ Because a mirror push makes the target match the sync files _exactly_, run it ag Schema comparison requires two things to match: the exact Directus version recorded in the sync files, patch release included, and the target's database vendor. The server refuses the comparison when either differs. Some Directus patches have changed the snapshot format, and database vendors describe column types differently. -```text +```cli ✖ Version mismatch: the snapshot was pulled from Directus 12.1.1, but the target runs 12.2.0. The server requires an exact version match for schema diffs — historically some patches are breaking. Align both instances (re-pull if the source was upgraded), or pass --allow-drift to proceed anyway. ``` diff --git a/content/guides/10.environment-sync/3.common-workflows.md b/content/guides/10.environment-sync/3.common-workflows.md index 130e4def..9e6829ff 100644 --- a/content/guides/10.environment-sync/3.common-workflows.md +++ b/content/guides/10.environment-sync/3.common-workflows.md @@ -132,7 +132,7 @@ git revert d6s sync diff --to production --mode mirror ``` -``` +```cli ● Comparing ./directus/default with production — https://cms.example.com (mirror — INCLUDES DELETIONS) ● Schema — 2 changes: 0 added, 1 modified, 1 deleted ✖ DELETE field articles.summary @@ -149,7 +149,7 @@ d6s sync push --to production --mode mirror The push repeats the plan, then the [deletion gate](/guides/environment-sync/reference#deletion-gates) demands typed consent: -``` +```cli This push permanently deletes 1 configuration record and 1 schema item from production. Type "production" to confirm: ``` diff --git a/content/guides/10.environment-sync/6.reference.md b/content/guides/10.environment-sync/6.reference.md index 237965b1..530ee73c 100644 --- a/content/guides/10.environment-sync/6.reference.md +++ b/content/guides/10.environment-sync/6.reference.md @@ -80,7 +80,7 @@ d6s profile test-connection [name] [--url ] [--token ] Connects and prints who the credential authenticates as: -``` +```cli ◇ Authenticated to https://cms.example.com as Admin User (Administrator). ``` @@ -279,7 +279,7 @@ Only `mirror` deletes, and deleting always requires its own explicit consent: `--yes` skips the ordinary confirmation prompt, but it never authorizes a deletion; that holds even if a non-deleting push unexpectedly carries one. A `mirror` push in CI without `--dangerously-allow-delete` refuses before changing anything: -``` +```cli ✖ Refusing mirror mode in a non-interactive context without --dangerously-allow-delete. mirror can delete schema and configuration records absent from ./directus/default; pass --dangerously-allow-delete to consent, or use --mode merge. ``` @@ -301,7 +301,7 @@ Records are matched across instances first by the ID map (`/ The ID map is keyed internally by source and target instance URL, so one file serves any number of targets without conflicts. Commit it whenever a push changes it. An ambiguous match (two candidates) prompts interactively and refuses non-interactively: -``` +```cli ✖ Push refused: 1 target match needs a choice. directus_roles: ./directus/default contains 1 role named "Editor". production — https://cms.example.com contains 2 matching roles. From fefe49f8067e0f523a55b8c7e9760a28b796b8b2 Mon Sep 17 00:00:00 2001 From: Bryant Gillespie Date: Fri, 14 Aug 2026 08:47:10 -0400 Subject: [PATCH 11/12] Update content/guides/10.environment-sync/5.secrets-and-limitations.md Co-authored-by: James White --- content/guides/10.environment-sync/5.secrets-and-limitations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/guides/10.environment-sync/5.secrets-and-limitations.md b/content/guides/10.environment-sync/5.secrets-and-limitations.md index c4b663a3..781ceb34 100644 --- a/content/guides/10.environment-sync/5.secrets-and-limitations.md +++ b/content/guides/10.environment-sync/5.secrets-and-limitations.md @@ -55,7 +55,7 @@ None of these sync today. Some are shared configuration that a future release co | Translate schema **across database vendors** | A vendor mismatch refuses the command; `--allow-drift` bypasses the same gate but does not rewrite vendor-specific column types. | | Wrap Schema and Configuration in **one transaction** | Schema applies first, then Configuration. A failed import re-runs Configuration alone; see [How It Works](/guides/environment-sync/how-it-works#how-a-push-applies). | | Select **individual records** | Resource selection is by type (`--roles`), never by record. | -| Model **code-first** | Model in the Data Studio and pull. Don't hand-author the JSON files. | +| Model **code-first** | Model in the Data Studio and pull. Hand edits to the JSON files are workable if you only push; a subsequent pull in scope overwrites them. | | **Seal the Configuration plan** against target drift | Only the Schema apply is sealed against the target changing between preview and apply. The Configuration preview is the server's own dry-run answer, but it is advisory. | ## Known limitations From ecda73ce322dd4ba5077a26bfc6f7e57583ab797 Mon Sep 17 00:00:00 2001 From: bryantgillespie Date: Fri, 14 Aug 2026 09:22:23 -0400 Subject: [PATCH 12/12] tweaks --- app/components/content/ProsePre.global.vue | 2 +- .../10.environment-sync/2.how-it-works.md | 24 +++++- .../4.ci-and-automation.md | 73 ++++++++++--------- 3 files changed, 59 insertions(+), 40 deletions(-) diff --git a/app/components/content/ProsePre.global.vue b/app/components/content/ProsePre.global.vue index b5754b4f..832cfc0e 100644 --- a/app/components/content/ProsePre.global.vue +++ b/app/components/content/ProsePre.global.vue @@ -99,7 +99,7 @@ const lines = computed(() => { }}
diff --git a/content/guides/10.environment-sync/2.how-it-works.md b/content/guides/10.environment-sync/2.how-it-works.md index eb966100..748b3e06 100644 --- a/content/guides/10.environment-sync/2.how-it-works.md +++ b/content/guides/10.environment-sync/2.how-it-works.md @@ -61,10 +61,10 @@ Target UI 1: https://cms.example.com/admin/settings/roles/t1 Target UI 2: https://cms.example.com/admin/settings/roles/t2 Which target role does this represent? - Existing target role "Editor" — t1 (Same synced values; only the ID differs) - Existing target role "Editor" — t2 (Merge updates the target; icon: local "edit", target "star") - No existing role — create a new one on the target (Creates another "Editor" role) - Abort push (Applies no remote changes) + ● Existing target role "Editor" — t1 (Same synced values; only the ID differs) + ○ Existing target role "Editor" — t2 (Merge updates the target; icon: local "edit", target "star") + ○ No existing role — create a new one on the target (Creates another "Editor" role) + ○ Abort push (Applies no remote changes) ``` The parenthetical after each existing candidate compares it to the sync files: identical values, or the fields that differ (here `icon`). @@ -100,6 +100,22 @@ A push mode answers one question: what happens to things that exist on the targe Deleting always requires its own explicit consent, separate from confirming the push. Interactively, a mirror push lists what would be lost and asks you to type the profile name. Non-interactively it requires the `--dangerously-allow-delete` flag; `--yes` never authorizes a deletion. The gate is a backstop as well as a policy: even if a non-deleting push somehow carried a deletion, it would still be refused without consent. +```bash +d6s sync push --to target --mode mirror --dangerously-allow-delete +``` + +```cli +● Pushing ./directus/default to target — http://localhost:8056 (mirror — INCLUDES DELETIONS) +● Configuration — 3 changes: 0 created, 0 updated, 3 deleted +~ directus_flows +0 new ~0 updated ✖1 deleted (dd7b3073-be2f-4dfa-aebf-79ab046fc619) +~ directus_roles +0 new ~0 updated ✖2 deleted (2121f751-31b2-4c34-93c0-64516f03b6f1, 86568388-e40d-4100-835c-f2238f57c95a) + +Apply 3 configuration changes to target — http://localhost:8056? + ● Yes / ○ No +``` + +The mode banner says `INCLUDES DELETIONS` up front. The plan names each record that would go, and the apply prompt still asks — `--dangerously-allow-delete` is consent to delete, not a skip of the ordinary confirmation. Pass `--yes` as well only when you mean both. + Because a mirror push makes the target match the sync files _exactly_, run it against a freshly pulled state. A mirror from stale sync files applies the stale state, including deleting things that only look obsolete because the files are old. ## The compatibility gate diff --git a/content/guides/10.environment-sync/4.ci-and-automation.md b/content/guides/10.environment-sync/4.ci-and-automation.md index 6b1d3dbe..896d333e 100644 --- a/content/guides/10.environment-sync/4.ci-and-automation.md +++ b/content/guides/10.environment-sync/4.ci-and-automation.md @@ -24,15 +24,17 @@ The credential store saved on a developer machine is never read when `CI` is non ## JSON reports -Add `--json` and stdout carries exactly one machine-readable report per command. Warnings (stripped secret fields, compatibility checks bypassed with `--allow-drift`, flow headers written verbatim) still go to stderr, so your logs keep them while stdout stays parseable. A failure puts an error report on stdout instead, with a stable `code` naming the failure class. +Add `--json` and stdout carries exactly one machine-readable report per command. Warnings (stripped secret fields, compatibility checks bypassed with `--allow-drift`, flow headers written verbatim) still go to stderr, so your logs keep them while stdout stays parseable. A failure puts an error report on stdout **in place of** the success report, shaped `{"error": {"code": …}}` with a stable `code` naming the failure class. Redirecting stdout to a file therefore captures the error instead of showing it; pipe through `tee` so the failure is visible in your logs too. The fields automation usually keys on: - **`changes`** (diff): `true` when the push would do anything, including when Configuration has ambiguous target matches. - **`data.reconciliation.ambiguous`** (diff): the number of Configuration records that need an identity choice. **`data.reconciliation.dependent`** counts records waiting on those choices. A non-interactive push refuses this state, so it is a real difference for your pipeline to surface, not noise. -- **`applied`** (push): `true` when the push changed the target. +- **`applied`** (push): `true` when the push sent an apply or import to the target. +- **`data`** is `null` when the project has no Configuration files at all, so reach through it (`report.data?.…`) rather than assuming an object. +- **`data.unchanged`** (diff) counts matched records whose values already agree. The server reports every matched record under `resultsByCollection[…].existing` whether or not it differs, so an honest "updated" count is `existing` minus `unchanged`. -`d6s sync diff` exits `0` whether or not differences exist; it fails only when it cannot produce an answer. Gate pipeline behavior on the report's `changes`, not the exit code. The [reference](/guides/environment-sync/reference#json-reports) documents every report field. +`d6s sync diff` exits `0` whether or not differences exist; it fails only when it cannot produce an answer. Distinguish those two: check for `error` first, then gate pipeline behavior on `changes` rather than the exit code. The [reference](/guides/environment-sync/reference#json-reports) documents every report field. ## A GitHub Actions pipeline @@ -61,46 +63,35 @@ jobs: - run: npm install -g @directus/cli@12 - name: Verify the production profile URL env: - EXPECTED_DIRECTUS_URL: ${{ vars.DIRECTUS_PRODUCTION_URL }} + EXPECTED_URL: ${{ vars.DIRECTUS_PRODUCTION_URL }} run: | - node - <<'NODE' - const fs = require('fs'); - const config = JSON.parse(fs.readFileSync('directus.config.json', 'utf8')); - const actual = config.profiles?.production?.url; - if (!process.env.EXPECTED_DIRECTUS_URL || actual !== process.env.EXPECTED_DIRECTUS_URL) { - throw new Error(`Unexpected production profile URL: ${actual ?? ''}`); - } - NODE + actual=$(jq -r '.profiles.production.url // ""' directus.config.json) + if [ -z "$EXPECTED_URL" ] || [ "$actual" != "$EXPECTED_URL" ]; then + echo "Unexpected production profile URL: $actual" + exit 1 + fi - name: Diff against production - run: d6s sync diff --to production --json > diff-report.json + shell: bash + run: d6s sync diff --to production --json | tee diff-report.json env: DIRECTUS_PRODUCTION_TOKEN: ${{ secrets.DIRECTUS_PRODUCTION_TOKEN }} - name: Comment the result on the PR uses: actions/github-script@v7 with: script: | - const fs = require('fs'); - const report = JSON.parse(fs.readFileSync('diff-report.json', 'utf8')); - const ambiguous = report.data.reconciliation?.ambiguous ?? 0; - const configuration = Object.values(report.data.resultsByCollection ?? {}).reduce( - (total, result) => ({ - created: total.created + result.new.length, - updated: total.updated + result.existing.length, - deleted: total.deleted + result.deleted.length, - }), - { created: 0, updated: 0, deleted: 0 }, - ); + const report = JSON.parse(require('fs').readFileSync('diff-report.json', 'utf8')); + const sum = (key) => + Object.values(report.data?.resultsByCollection ?? {}).reduce((total, result) => total + result[key].length, 0); + const updated = sum('existing') - (report.data?.unchanged ?? 0); + const schema = report.schemaSkipped + ? 'Schema: skipped.' + : `Schema: ${report.added} added, ${report.modified} modified, ${report.deleted} deleted.`; const body = report.changes - ? `**Environment Sync**: merging this PR changes production. Schema: ${report.added} added, ` + - `${report.modified} modified, ${report.deleted} deleted. Configuration: ` + - `${configuration.created} created, ${configuration.updated} updated, ` + - `${configuration.deleted} deleted; ${ambiguous} ambiguous matches.` + ? `**Environment Sync**: merging this PR changes production. ${schema} ` + + `Configuration: ${sum('new')} created, ${updated} updated, ${sum('deleted')} deleted; ` + + `${report.data?.reconciliation?.ambiguous ?? 0} ambiguous matches.` : '**Environment Sync**: production already matches this branch.'; - await github.rest.issues.createComment({ - ...context.repo, - issue_number: context.issue.number, - body, - }); + await github.rest.issues.createComment({ ...context.repo, issue_number: context.issue.number, body }); push: if: github.event_name == 'push' @@ -116,11 +107,22 @@ jobs: with: node-version: 22 - run: npm install -g @directus/cli@12 + - name: Verify the production profile URL + env: + EXPECTED_URL: ${{ vars.DIRECTUS_PRODUCTION_URL }} + run: | + actual=$(jq -r '.profiles.production.url // ""' directus.config.json) + if [ -z "$EXPECTED_URL" ] || [ "$actual" != "$EXPECTED_URL" ]; then + echo "Unexpected production profile URL: $actual" + exit 1 + fi - name: Push to production - run: d6s sync push --to production --yes --json > push-report.json + shell: bash + run: d6s sync push --to production --yes --json | tee push-report.json env: DIRECTUS_PRODUCTION_TOKEN: ${{ secrets.DIRECTUS_PRODUCTION_TOKEN }} - name: Commit the updated ID map + if: ${{ !cancelled() }} run: | if [ -n "$(git status --porcelain -- 'directus/*/id_map.json')" ]; then git config user.name "github-actions[bot]" @@ -134,7 +136,8 @@ jobs: Two things to know before enabling the push job: - **Run the first push interactively, locally.** The first push into a target tends to raise the identity questions described in [How It Works](/guides/environment-sync/how-it-works#record-identity), and CI refuses them. Answer them from a terminal and commit `id_map.json` before enabling the push job. -- **The ID map commit-back step matters.** A push that creates records adds entries to `id_map.json`. If CI doesn't commit them, the next push re-matches those records from scratch, and records without an identifying field (panels) can duplicate. +- **The ID map commit-back step matters.** A push that creates records adds entries to `id_map.json`. If CI doesn't commit them, the next push re-matches those records from scratch, and records without an identifying field (panels) can duplicate. It runs on `!cancelled()` rather than only on success, because a push that imports records and then fails still wrote real mappings worth keeping. +- **The bot needs somewhere to push.** Branch protection that requires pull requests blocks `github-actions[bot]` from committing the ID map; give it an exemption or have the step open a pull request instead. Commits made with `GITHUB_TOKEN` never trigger another workflow run, so this cannot loop. The URL check must run before any step receives the production token. It prevents a pull request from changing the `production` profile to another host and sending the token there. The sample skips pull requests from forks because GitHub does not expose repository secrets to them; run local or tokenless checks for those contributions instead.