From 89c54511ac9c55dafeb45538c8ad93529f26d26c Mon Sep 17 00:00:00 2001 From: Luke Knepper Date: Thu, 6 Aug 2026 16:04:54 -0700 Subject: [PATCH 1/2] Clarify Stable IP change policy and DNS redirect language Rewrite "Changes to Stable IP ranges" to state the actual commitments: once a region's ranges are published Temporal avoids adding to them and has no plans to remove one, and any unforeseen add or remove comes with six months of advance notice. Remove the "Don't depend on DNS resolution behavior" callout from the Stable IPs section. It named High Availability as the *only* case with guaranteed DNS resolution, which is wrong now that Stable IPs is a second case. The accurate version, listing both, already exists in "When Namespace Endpoint DNS resolution is predictable" on the same page and in the Namespaces page callout. Replace "routes to" with "redirects via DNS to" where the docs describe the Namespace Endpoint reaching an active region. The endpoint does not proxy traffic; it is a DNS redirect. Co-Authored-By: Claude Opus 5 --- docs/cloud/connectivity/ip-addresses.mdx | 11 ++++------- .../cloud/high-availability/architecture-patterns.mdx | 2 +- docs/cloud/high-availability/failovers/index.mdx | 2 +- 3 files changed, 6 insertions(+), 9 deletions(-) diff --git a/docs/cloud/connectivity/ip-addresses.mdx b/docs/cloud/connectivity/ip-addresses.mdx index 05fa2645e2..66fbcf8c71 100644 --- a/docs/cloud/connectivity/ip-addresses.mdx +++ b/docs/cloud/connectivity/ip-addresses.mdx @@ -341,11 +341,6 @@ IP addresses are provided in IPv4 or IPv6 format with CIDR notation. At launch, ::: -:::note Don't depend on DNS resolution behavior - -Take a dependency on Temporal's published endpoints (the hostnames themselves), not on what they resolve to. The underlying IP addresses, CNAME chains, and resolution behavior can change at any time without notice. The **only** exception is the Namespace Endpoint for a Namespace with [High Availability features](/cloud/high-availability), which deterministically CNAMEs to its active region's regional record — see [Connectivity for High Availability](/cloud/high-availability/ha-connectivity) for details. - -::: ### How to connect using Stable IPs To connect to a Namespace with Stable IPs enabled: @@ -432,9 +427,11 @@ Take note of the following: ### Changes to Stable IP ranges -Temporal does not intend to make changes to published Stable IP ranges. In the rare exception that a Stable IP range needs to be added to or removed from the list, Temporal will communicate via targeted communication to affected Temporal Cloud users and the [Temporal Cloud changelog](https://temporal.io/change-log), and will give ample time for users to update their firewalls with the changed IPs. +Once a region's Stable IP ranges are published, Temporal avoids adding new ranges to that region's list, and Temporal has no plans to ever remove a published range. + +If an unforeseen event requires Temporal to add or remove a published range, Temporal gives every Stable IPs user six months of advance notice, through both targeted communication and the [Temporal Cloud changelog](https://temporal.io/change-log). -Temporal will never route traffic to IP addresses that are not listed in the API response. +Temporal never routes traffic to IP addresses that are not on the published list. ### Pricing diff --git a/docs/cloud/high-availability/architecture-patterns.mdx b/docs/cloud/high-availability/architecture-patterns.mdx index 89fb561d87..7e1c2ba9ec 100644 --- a/docs/cloud/high-availability/architecture-patterns.mdx +++ b/docs/cloud/high-availability/architecture-patterns.mdx @@ -573,7 +573,7 @@ Here's how each component behaves during normal operation and after a failover: | Component | Normal operation | On failover | | --- | --- | --- | | **Workers** | Run in as many regions as you want — fleets don't have to match the Namespace's regions. Every fleet connects through the Namespace Endpoint. | Fleets in unaffected regions keep processing with no cold-start gap. Scale them up if needed to carry the full load. | -| **Namespace** | One active replica; every other region holds a passive replica that receives replicated state. The Namespace Endpoint always routes to whichever region currently holds the active Namespace. | Temporal Cloud promotes a passive replica in another region to active. Every fleet follows automatically — no reconfiguration, nothing to bring up. | +| **Namespace** | One active replica; every other region holds a passive replica that receives replicated state. The Namespace Endpoint always redirects via DNS to whichever region currently holds the active Namespace. | Temporal Cloud promotes a passive replica in another region to active. Every fleet follows automatically — no reconfiguration, nothing to bring up. | | **Workflow starters and Clients** | Run wherever convenient and connect through the Namespace Endpoint, like the Workers. | Automatically follow the Namespace Endpoint to the new active region, like the Workers. | | **Codec Servers and proxies** | Run in every region where Workers run. | Already running in every surviving region — no action needed. | | **Databases and queues** | Accessed from every Worker region, so you need a cross-region consistency story. | Promote the active region's copy, if needed, so the Workers there can read and write it. | diff --git a/docs/cloud/high-availability/failovers/index.mdx b/docs/cloud/high-availability/failovers/index.mdx index 5d1213d9d0..e61e3916bc 100644 --- a/docs/cloud/high-availability/failovers/index.mdx +++ b/docs/cloud/high-availability/failovers/index.mdx @@ -116,7 +116,7 @@ The failover process is the same whether it is triggered automatically by Tempor 3. This hybrid strategy balances consistency and availability. During the switch, Workflow operations are briefly paused, and Temporal Cloud returns a retryable "Service unavailable" error to SDKs. -4. **The Namespace Endpoint re-routes to the active region.** This DNS change can take a few minutes to fully propagate +4. **The Namespace Endpoint redirects via DNS to the active region.** This change can take a few minutes to fully propagate to all Clients and Workers. If your application has an extremely demanding Recovery Time, you can eliminate this stage by connecting through a [Regional Endpoint](/cloud/high-availability/ha-connectivity#regional-endpoint) instead of the Namespace Endpoint. From de72e04c1dc7bac4184e6a670fa0c528e90f6f6d Mon Sep 17 00:00:00 2001 From: Luke Knepper Date: Fri, 7 Aug 2026 10:59:26 -0700 Subject: [PATCH 2/2] Simplify and reorder the Stable IPs API examples Drop the temporal-cloud-api-version header from the curl and gRPC examples. Callers no longer need to pin the Cloud Ops API version, and carrying it through every command added noise. This removes the H_VER variable, the version entry in GRPC_HEADERS, the "two headers" preamble, and the two minimum-version bullets. Reorder both procedures to follow the order a caller actually works in. Creating the Connectivity Rule is now Step 1, including capturing the rule ID and waiting for the async operation to reach a terminal state. Attaching the rule to the Namespace is Step 2, folding in the GetNamespace read that the full-replace UpdateNamespace call depends on. Add a non-polling way to confirm each step. Step 1 can list the account's Connectivity Rules and read the new rule's state, which also reveals whether a public rule already exists. Step 2 can describe the Namespace and confirm connectivityRuleIds contains the new rule ID. Co-Authored-By: Claude Opus 5 --- docs/cloud/connectivity/ip-addresses.mdx | 127 +++++++++++++++-------- 1 file changed, 84 insertions(+), 43 deletions(-) diff --git a/docs/cloud/connectivity/ip-addresses.mdx b/docs/cloud/connectivity/ip-addresses.mdx index 66fbcf8c71..4bd06ff371 100644 --- a/docs/cloud/connectivity/ip-addresses.mdx +++ b/docs/cloud/connectivity/ip-addresses.mdx @@ -118,10 +118,9 @@ When enabling Stable IPs on an existing Namespace, Temporal updates DNS records If you cannot use `tcld` or Terraform, you can access the Cloud Ops API directly. The following procedure creates a public Connectivity Rule with Stable IPs enabled and attaches it to a Namespace. -All requests require two headers: +All requests require an authorization header: - `Authorization: Bearer $TEMPORAL_CLOUD_OPS_API_KEY` -- `temporal-cloud-api-version: v0.16.0` (or any version ≥ `v0.15.0`) Base URL: `https://saas-api.tmprl.cloud` @@ -131,26 +130,15 @@ Set up environment variables: export TEMPORAL_CLOUD_OPS_API_KEY='' export NS='.' # The Cloud-side Namespace identifier, e.g. "myns.a1b2c3" H_AUTH="Authorization: Bearer $TEMPORAL_CLOUD_OPS_API_KEY" -H_VER="temporal-cloud-api-version: v0.16.0" ``` -**Step 1. Read the current Namespace spec and resource version.** +**Step 1. Create a public Connectivity Rule with Stable IPs enabled.** -`UpdateNamespace` replaces the entire spec and requires the latest `resource_version`. Fetch them first: - -```bash -curl -sS "https://saas-api.tmprl.cloud/cloud/namespaces/$NS" \ - -H "$H_AUTH" -H "$H_VER" | tee /tmp/ns.json | \ - jq '{resource_version: .namespace.resourceVersion, connectivity_rule_ids: .namespace.spec.connectivityRuleIds}' - -RV=$(jq -r '.namespace.resourceVersion' /tmp/ns.json) -``` - -**Step 2. Create a public Connectivity Rule with Stable IPs enabled.** +Creating the rule returns both the new rule's ID and an async operation that tracks activation: ```bash curl -sS -X POST "https://saas-api.tmprl.cloud/cloud/connectivity-rules" \ - -H "$H_AUTH" -H "$H_VER" -H "Content-Type: application/json" \ + -H "$H_AUTH" -H "Content-Type: application/json" \ -d '{ "spec": { "publicRule": { "enableStableIps": true } @@ -163,18 +151,38 @@ OP_ID=$(jq -r '.asyncOperation.id // .asyncOperation.operationId' /tmp/cr.json) Field names follow proto3 JSON camelCase: `enable_stable_ips` becomes `enableStableIps`. -**Step 3. Wait for the create operation to reach a terminal state.** +Wait for the create operation to reach a terminal state before continuing: ```bash curl -sS "https://saas-api.tmprl.cloud/cloud/operations/$OP_ID" \ - -H "$H_AUTH" -H "$H_VER" | jq '.asyncOperation.state' + -H "$H_AUTH" | jq '.asyncOperation.state' ``` Re-run until `state` is `FULFILLED` (the rule is now `ACTIVE`). Any `*_FAILED` state means you should stop and inspect the operation. -**Step 4. Attach the rule to the Namespace.** +Alternatively, list every Connectivity Rule on the account and read the new rule's state directly: -`UpdateNamespace` posts the whole `NamespaceSpec` back. Update the spec from Step 1 by appending `$CR_ID` to `connectivityRuleIds`: +```bash +curl -sS "https://saas-api.tmprl.cloud/cloud/connectivity-rules" \ + -H "$H_AUTH" | \ + jq '.connectivityRules[] | {id, state, spec}' +``` + +The rule is ready when its `state` is `RESOURCE_STATE_ACTIVE`. This also shows you whether the account already has a public rule, which matters because only one is allowed. + +**Step 2. Attach the rule to the Namespace.** + +`UpdateNamespace` replaces the entire spec and requires the latest `resource_version`, so read the current Namespace first: + +```bash +curl -sS "https://saas-api.tmprl.cloud/cloud/namespaces/$NS" \ + -H "$H_AUTH" | tee /tmp/ns.json | \ + jq '{resource_version: .namespace.resourceVersion, connectivity_rule_ids: .namespace.spec.connectivityRuleIds}' + +RV=$(jq -r '.namespace.resourceVersion' /tmp/ns.json) +``` + +Then post the whole `NamespaceSpec` back, appending `$CR_ID` to `connectivityRuleIds`: ```bash jq --arg id "$CR_ID" --arg rv "$RV" --arg ns "$NS" ' @@ -186,18 +194,27 @@ jq --arg id "$CR_ID" --arg rv "$RV" --arg ns "$NS" ' }' /tmp/ns.json > /tmp/update.json curl -sS -X POST "https://saas-api.tmprl.cloud/cloud/namespaces/$NS" \ - -H "$H_AUTH" -H "$H_VER" -H "Content-Type: application/json" \ + -H "$H_AUTH" -H "Content-Type: application/json" \ -d @/tmp/update.json | tee /tmp/update-resp.json ``` -Poll the returned `asyncOperation` the same way as Step 3. +Poll the returned `asyncOperation` the same way as in Step 1. + +Alternatively, describe the Namespace and confirm the rule is attached: + +```bash +curl -sS "https://saas-api.tmprl.cloud/cloud/namespaces/$NS" \ + -H "$H_AUTH" | \ + jq '{state: .namespace.state, connectivity_rule_ids: .namespace.spec.connectivityRuleIds}' +``` + +The update is complete when `state` is `RESOURCE_STATE_ACTIVE` and `connectivityRuleIds` contains your `$CR_ID`. Take note of the following: - The `namespace` in the URL is the Cloud-side Namespace identifier (for example, `myns.a1b2c3`), not the SDK gRPC hostname (`myns.a1b2c3.tmprl.cloud`). Strip the trailing `.tmprl.cloud`. -- Do not omit fields from `spec` in Step 4. Proto3 update is full-replace — any field left out will be cleared. Building the body from the GET response avoids surprises. +- Do not omit fields from `spec` in Step 2. Proto3 update is full-replace — any field left out will be cleared. Building the body from the GET response avoids surprises. - A Namespace can have at most one public Connectivity Rule attached, and only one public rule exists per account. If `connectivityRuleIds` already references a public rule without Stable IPs, replace that entry instead of appending — you cannot create a second public rule. -- Stable IPs requires `temporal-cloud-api-version` of `v0.15.0` or later. #### How to enable Stable IPs with gRPC @@ -207,7 +224,6 @@ If you prefer to call the Cloud Ops API directly over gRPC (for example, with `g - Fully qualified service: `temporal.api.cloud.cloudservice.v1.CloudService` - Required metadata on every call: - `authorization: Bearer $TEMPORAL_CLOUD_OPS_API_KEY` - - `temporal-cloud-api-version: v0.16.0` (or any version ≥ `v0.15.0`) The examples below use `grpcurl`. They assume you have a local checkout of [`temporalio/api-cloud`](https://github.com/temporalio/api-cloud) at `./api-cloud` and a `proto/` directory containing the standard `google/api/annotations.proto` and associated files. @@ -216,22 +232,13 @@ Set up environment variables: ```bash export TEMPORAL_CLOUD_OPS_API_KEY='' export NS='.' # The Cloud-side Namespace identifier, e.g. "myns.a1b2c3" -GRPC_HEADERS=(-H "authorization: Bearer $TEMPORAL_CLOUD_OPS_API_KEY" -H "temporal-cloud-api-version: v0.16.0") +GRPC_HEADERS=(-H "authorization: Bearer $TEMPORAL_CLOUD_OPS_API_KEY") PROTO_FLAGS=(-import-path ./api-cloud -import-path ./proto -proto temporal/api/cloud/cloudservice/v1/service.proto) ``` -**Step 1. Read the current Namespace spec and resource version.** +**Step 1. Create a public Connectivity Rule with Stable IPs enabled.** -```bash -grpcurl "${PROTO_FLAGS[@]}" "${GRPC_HEADERS[@]}" \ - -d "{\"namespace\": \"$NS\"}" \ - saas-api.tmprl.cloud:443 \ - temporal.api.cloud.cloudservice.v1.CloudService/GetNamespace | tee /tmp/ns.json - -RV=$(jq -r '.namespace.resourceVersion' /tmp/ns.json) -``` - -**Step 2. Create a public Connectivity Rule with Stable IPs enabled.** +Creating the rule returns both the new rule's ID and an async operation that tracks activation: ```bash grpcurl "${PROTO_FLAGS[@]}" "${GRPC_HEADERS[@]}" \ @@ -249,7 +256,7 @@ OP_ID=$(jq -r '.asyncOperation.id' /tmp/cr.json) Field names in gRPC JSON requests can use either snake_case (proto field names) or camelCase — `grpcurl` accepts both. Responses come back in camelCase. -**Step 3. Wait for the create operation to reach a terminal state.** +Wait for the create operation to reach a terminal state before continuing: ```bash grpcurl "${PROTO_FLAGS[@]}" "${GRPC_HEADERS[@]}" \ @@ -260,9 +267,32 @@ grpcurl "${PROTO_FLAGS[@]}" "${GRPC_HEADERS[@]}" \ Re-run until `state` is `FULFILLED` (the rule is now `ACTIVE`). Any `*_FAILED` state means you should stop and inspect the operation. -**Step 4. Attach the rule to the Namespace.** +Alternatively, list every Connectivity Rule on the account and read the new rule's state directly: + +```bash +grpcurl "${PROTO_FLAGS[@]}" "${GRPC_HEADERS[@]}" \ + -d '{}' \ + saas-api.tmprl.cloud:443 \ + temporal.api.cloud.cloudservice.v1.CloudService/GetConnectivityRules | \ + jq '.connectivityRules[] | {id, state, spec}' +``` + +The rule is ready when its `state` is `RESOURCE_STATE_ACTIVE`. This also shows you whether the account already has a public rule, which matters because only one is allowed. + +**Step 2. Attach the rule to the Namespace.** + +`UpdateNamespace` replaces the entire spec and requires the latest `resource_version`, so read the current Namespace first: + +```bash +grpcurl "${PROTO_FLAGS[@]}" "${GRPC_HEADERS[@]}" \ + -d "{\"namespace\": \"$NS\"}" \ + saas-api.tmprl.cloud:443 \ + temporal.api.cloud.cloudservice.v1.CloudService/GetNamespace | tee /tmp/ns.json + +RV=$(jq -r '.namespace.resourceVersion' /tmp/ns.json) +``` -`UpdateNamespace` posts the whole `NamespaceSpec` back. Update the spec from Step 1 by appending `$CR_ID` to `connectivity_rule_ids`: +Then post the whole `NamespaceSpec` back, appending `$CR_ID` to `connectivity_rule_ids`: ```bash jq --arg id "$CR_ID" --arg rv "$RV" --arg ns "$NS" ' @@ -279,14 +309,25 @@ grpcurl "${PROTO_FLAGS[@]}" "${GRPC_HEADERS[@]}" \ temporal.api.cloud.cloudservice.v1.CloudService/UpdateNamespace < /tmp/update.json | tee /tmp/update-resp.json ``` -Poll the returned `asyncOperation` the same way as Step 3. +Poll the returned `asyncOperation` the same way as in Step 1. + +Alternatively, describe the Namespace and confirm the rule is attached: + +```bash +grpcurl "${PROTO_FLAGS[@]}" "${GRPC_HEADERS[@]}" \ + -d "{\"namespace\": \"$NS\"}" \ + saas-api.tmprl.cloud:443 \ + temporal.api.cloud.cloudservice.v1.CloudService/GetNamespace | \ + jq '{state: .namespace.state, connectivity_rule_ids: .namespace.spec.connectivityRuleIds}' +``` + +The update is complete when `state` is `RESOURCE_STATE_ACTIVE` and `connectivityRuleIds` contains your `$CR_ID`. Take note of the following: - The `namespace` field is the Cloud-side Namespace identifier (for example, `myns.a1b2c3`), not the SDK gRPC hostname (`myns.a1b2c3.tmprl.cloud`). Strip the trailing `.tmprl.cloud`. -- Do not omit fields from `spec` in Step 4. Proto3 update is full-replace — any field left out will be cleared. Building the body from the `GetNamespace` response avoids surprises. +- Do not omit fields from `spec` in Step 2. Proto3 update is full-replace — any field left out will be cleared. Building the body from the `GetNamespace` response avoids surprises. - A Namespace can have at most one public Connectivity Rule attached, and only one public rule exists per account. If `connectivity_rule_ids` already references a public rule without Stable IPs, replace that entry instead of appending — you cannot create a second public rule. -- Stable IPs requires `temporal-cloud-api-version` of `v0.15.0` or later. - Authentication uses Cloud API keys; mTLS client certificates are not accepted on `saas-api.tmprl.cloud`. The metadata key is `authorization` (lowercase), per the gRPC convention. ### How to view Stable IP ranges