diff --git a/docs/en/solutions/ecosystem/redis/How_to_Migrate_Redis_Sentinel_With_RDB_Snapshot.md b/docs/en/solutions/ecosystem/redis/How_to_Migrate_Redis_Sentinel_With_RDB_Snapshot.md new file mode 100644 index 000000000..4a047a532 --- /dev/null +++ b/docs/en/solutions/ecosystem/redis/How_to_Migrate_Redis_Sentinel_With_RDB_Snapshot.md @@ -0,0 +1,185 @@ +--- +products: + - Alauda Application Services +kind: + - Solution +ProductsVersion: + - 4.x +--- + +# How to Migrate a Redis Sentinel Instance Across Kubernetes Clusters With an RDB Snapshot + +## Introduction + +This guide describes how to move a Sentinel-mode Redis instance to a different Kubernetes cluster while also changing the StorageClass or the underlying storage. The dataset is carried across as a Redis database snapshot (RDB) file, `/data/dump.rdb`: writes are frozen on the source, a final snapshot is taken, the file is copied into a freshly created target instance, and only after the data is verified are replicas restored and the client application switched over. + +The target instance is deliberately created with a single Primary and no Replicas (`master: 1`, `slave: 0`) so that the imported snapshot is loaded exactly once and cannot be overwritten by a replication full sync before it has been checked. + +:::info Applicable Version +Sentinel-mode instances managed by redis-operator **4.0 and later**, on both the source and the target cluster. +::: + +:::note +This document uses "Primary" and "Replica" to refer to the main Redis node and its replicas. This is the current standard terminology, replacing the previously used "Master"/"Slave". The `master` and `slave` field names still appear in the `Redis` custom resource and in `redis-cli` output, and are kept verbatim where they are part of an API or command output. +::: + +### When to Use This Procedure + +Use this procedure when: + +- The S3 object-storage backup and restore path is not available. +- Writes from the client application can be stopped for the duration of the migration window. +- The Redis version and configuration on the source and the target are compatible. + +Do **not** use this procedure when a downtime window cannot be scheduled, or when the migration must be online and incremental. For a continuous-replication alternative, see [Migrate Redis Data Across Kubernetes Clusters](./How_to_Migrate_Redis_Across_Clusters.md), which uses RedisShake. + +## Prerequisites + +1. **Matching Redis version and configuration.** The source and target instances run the same Redis version with compatible configuration. +2. **Target capacity.** The target PVC is at least as large as the actual source dataset, and uses a StorageClass that exists in the new cluster. +3. **API access to both clusters.** The workstation running the procedure can reach the Kubernetes API of both the old and the new cluster, with a `kubectl` context configured for each. +4. **`tar` in the Redis container.** `kubectl cp` requires `tar` to be present inside the container. Verify this before the migration window starts. + +:::warning +Create the target instance with the **RDB parameter template**, not the AOF template. When `appendonly yes` is active, Redis loads the AOF file on startup and ignores `dump.rdb`, so the imported snapshot produces an empty dataset. See [Backup and Restore Compatibility With Parameter Templates](./Backup_Restore_Template_Compatibility.md) for the loading rules and for the procedure to re-enable AOF after the data is verified. +::: + +## Migration Procedure + +Replace the cluster context, namespace, pod names, and authentication arguments in the commands below to match your environment. The examples use `-a $REDIS_PASSWORD`, which resolves inside the Redis container; pass the password explicitly if the variable is not set in your environment. + +### 1. Stop Writes From the Client Application + +Stop every component that can write to Redis. For a GitLab deployment this includes Puma, Sidekiq, and any other worker that holds a Redis connection. Confirm that all of them are stopped **before** generating the final snapshot — a write that lands after `BGSAVE` will not be part of the migrated dataset. + +### 2. Record the Source Keyspace + +Use `INFO keyspace` to record the number of keys and the number of keys with an expiration for each logical database. This is the baseline you compare against after the import. + +```bash +kubectl --context -n exec -c redis -- \ + redis-cli -a $REDIS_PASSWORD INFO keyspace +``` + +### 3. Generate the Final RDB Snapshot + +```bash +kubectl --context -n exec -c redis -- \ + redis-cli -a $REDIS_PASSWORD BGSAVE + +kubectl --context -n exec -c redis -- \ + redis-cli -a $REDIS_PASSWORD INFO persistence +``` + +Do not continue until `INFO persistence` reports both of the following: + +```text +rdb_bgsave_in_progress:0 +rdb_last_bgsave_status:ok +``` + +### 4. Export the RDB File + +```bash +kubectl --context -n cp \ + -c redis ":/data/dump.rdb" ./dump.rdb + +sha256sum ./dump.rdb +``` + +Record the file size and the SHA-256 checksum. + +### 5. Create the Target Instance + +Install redis-operator 4.0 or later in the new cluster, then create a Sentinel-mode Redis instance using the RDB parameter template and the new StorageClass. The replica counts must be set as follows: + +```yaml +spec: + replicas: + sentinel: + master: 1 + slave: 0 +``` + +Wait until `rfr--0` is Ready. The target instance must not receive any application traffic at this point. + +### 6. Import the RDB File + +```bash +kubectl --context -n cp \ + -c redis ./dump.rdb "rfr--0:/data/dump.rdb" + +kubectl --context -n exec "rfr--0" -c redis -- \ + sha256sum /data/dump.rdb +``` + +The checksum inside the pod must match the one recorded in step 4. If it does not, copy the file again — do not proceed with a mismatched snapshot. + +### 7. Recreate the Target Pod + +Redis only reads `dump.rdb` at startup, so the pod must be restarted to load the imported file. Force-delete the pod to skip the `PreStop` hook, which would otherwise trigger a save and overwrite the file you just copied. The StatefulSet recreates `rfr--0` on the same PVC. + +```bash +kubectl --context -n delete pod "rfr--0" \ + --grace-period=0 --force +``` + +Wait until the old container has definitively exited and the PVC mount is released, then confirm that the new pod starts on the original PVC and becomes Ready. + +:::warning +Force deletion does not wait for the node to confirm that the old process has terminated. Do not run it when the node is unhealthy or the kubelet is unreachable — two processes writing the same PVC will corrupt the dataset. +::: + +### 8. Verify the Imported Data + +```bash +kubectl --context -n logs "rfr--0" -c redis + +kubectl --context -n exec "rfr--0" -c redis -- \ + redis-cli -a $REDIS_PASSWORD INFO keyspace +``` + +Check that: + +- The Redis log contains no RDB format, checksum, or load-failure errors. +- `INFO keyspace` matches the baseline recorded in step 2. +- Key expirations keep advancing during the migration, so the target may report slightly fewer keys than the source. A small shortfall accounted for by expired keys is expected; an unexplained gap is not. +- The application's critical Redis keys can be read back. + +### 9. Restore Replicas and Switch the Application Over + +Restore the replica count by editing the Redis custom resource. The example below returns the instance to two nodes: + +```bash +kubectl --context -n patch \ + redis.middleware.alauda.io "" --type=merge \ + -p '{"spec":{"replicas":{"sentinel":{"master":1,"slave":1}}}}' +``` + +The operator scales the Redis StatefulSet to `master + slave` pods. The new `rfr--1` pod is created with its own new PVC and performs a full sync from the current Primary. Wait for all pods to become Ready, then confirm the topology: + +```text +# On the Primary — INFO replication +role:master +connected_slaves: + +# On each Replica — INFO replication +master_link_status:up +``` + +Finally, update the client application's Redis/Sentinel address, credentials, and logical database number, start the application, and run basic validation such as login, project access, and background job execution. + +## Rollback + +- Keep the source instance and its PVCs until the migration has been formally accepted. +- As long as the target has not accepted any writes, roll back by stopping the target instance and pointing the application back at the source Redis. +- Once the target has accepted new writes, the two datasets have diverged and a direct switch back is no longer safe. Stop writes first and handle the reverse migration as a separate exercise. + +## Important Considerations + +- **Freeze writes before `BGSAVE`, not after.** The snapshot is a point-in-time copy; anything written after it is lost on cutover. +- **Verify the checksum on both sides.** A truncated `kubectl cp` produces a file that Redis may partially load without an obvious error. +- **Do not skip the forced pod deletion.** A graceful delete runs the `PreStop` hook, which saves the in-memory (empty) dataset over the imported `dump.rdb`. +- **Keep the target at zero replicas until the data is verified.** Scaling out before verification propagates a bad dataset to the new Replicas through a full sync. +- **Use the RDB parameter template.** With the AOF template the imported snapshot is ignored on startup. +- **The Replica PVCs are new.** Restoring the replica count provisions fresh PVCs from the new StorageClass and populates them by full sync; the old cluster's volumes are never reused.