From 2fa042539a85777f60a43d14123d8ec2041c85b0 Mon Sep 17 00:00:00 2001 From: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com> Date: Tue, 8 Sep 2026 14:53:33 +0530 Subject: [PATCH 01/11] PCSM-203 Implement copy of chunk distribution from source --- docs/sharding.md | 95 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 92 insertions(+), 3 deletions(-) diff --git a/docs/sharding.md b/docs/sharding.md index 34b13d88..d08235ff 100644 --- a/docs/sharding.md +++ b/docs/sharding.md @@ -38,15 +38,104 @@ For detailed information about authentication and connection string configuratio Before starting the initial sync, {{pcsm.short}} checks which collections are sharded on the source cluster and creates corresponding sharded collections on the destination cluster. The only sharding configuration preserved from the source cluster is the sharding key; all other sharding details are handled internally by the destination cluster. +Immediately after it shards a collection on the target, and before it copies any documents into it, {{pcsm.short}} pre-splits that collection so that the clone writes spread across all target shards. See Chunk distribution. + ### Balancer operation {{pcsm.full_name}} connects to source and target clusters via a `mongos` instance. Therefore, you do not need to disable the balancer on either the source or target cluster before starting replication. The target cluster's balancer continues to operate normally and manages chunk distribution according to its own sharding configuration and balancer settings. -### Chunk distribution +The target starts from the same chunk boundaries as the source, so a chunk migration on the source arrives where the target expects it. That makes it safe to leave the balancer running during the sync, which matters in write-heavy clusters where turning it off is not an option. See [Manage sharded cluster balancer :octicons-link-external-16:](https://www.mongodb.com/docs/manual/tutorial/manage-sharded-cluster-balancer/){:target="_blank"} in the MongoDB documentation. + +## Chunk distribution + +!!! admonition "Version added: 0.10.0" + +Sharding an empty collection on a ranged shard key gives you a single chunk that covers the whole key range, and the balancer only starts spreading data once documents arrive. For a clone, that is the worst possible starting point. Every document {{pcsm.short}} writes lands on one shard, that shard absorbs the entire write load, and when the clone finishes the balancer begins a long migration of data that never needed to be in one place. + +{{pcsm.short}} pre-splits the target collection before it copies anything. It reads the chunk layout of the source collection, recreates those boundaries on the target, and places the resulting chunks across the target shards. Clone writes then spread across every shard from the first document, and no rebalancing wave follows the clone. + +This happens automatically. There is no flag to set, nothing to enable, and no way to turn it off. {{pcsm.short}} identifies the source collection by UUID, keeps the chunk boundaries in order, and applies them with the standard MongoDB sharding commands. + +| **Source collection** | **Target shards** | **What {{pcsm.short}} does** | +|-----------------------|-------------------|------------------------------| +| Hashed shard key | Any number | Nothing. The target keeps the layout that `shardCollection` creates. | +| Ranged shard key | Same number as the source | Mirrors the source chunk boundaries and their ownership pattern. | +| Ranged shard key | Different number from the source | Replays the source boundaries and places the chunks so that each target shard holds roughly the same volume of data. | + +!!! note "The layout is a starting point, not a copy" + + {{pcsm.short}} reads the source chunk boundaries once, before the clone. It does not replicate sharding metadata afterwards, so later chunk migrations, splits, merges, and resharding on the source have no effect on the target layout. The two clusters drift apart as soon as either balancer moves data. A layout that no longer matches the source is expected and does not indicate a replication problem. + + + +### Hashed shard keys + +{{pcsm.short}} does no pre-splitting for hashed shard keys, and that is deliberate. The `shardCollection` command already produces an evenly distributed layout across all shards on every supported MongoDB version, so there is nothing to improve and {{pcsm.short}} keeps what MongoDB created. + +The number of initial chunks depends on the server version. With three target shards: + +* MongoDB 6.0 and 7.0 create six chunks, roughly two per shard. +* MongoDB 8.0 creates three chunks, roughly one per shard. + +See [Hashed sharding :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/hashed-sharding/){:target="_blank"} in the MongoDB documentation for how the server builds that initial layout. + +### Ranged shard keys with the same number of shards + +When both clusters have the same number of shards, {{pcsm.short}} reproduces the source layout directly. It sorts the shard IDs on each side, pairs them by position, replays every source chunk boundary on the target, and puts each target chunk on the shard paired with its source owner. + +```{.text .no-copy} +Source shards: src-a, src-b +Target shards: tgt-a, tgt-b + +Source layout: +[MinKey, 100) -> src-a +[100, MaxKey) -> src-b + +Target layout: +[MinKey, 100) -> tgt-a +[100, MaxKey) -> tgt-b +``` + +!!! note "Shard names are paired, not matched" + + Pairing is by sorted position, so the shard that owns a range on the target is not necessarily the one with a similar name on the source. What {{pcsm.short}} reproduces is the shape of the distribution, not the shard names. + +### Ranged shard keys with a different number of shards + +Source ownership cannot be mirrored when the shard counts differ, so {{pcsm.short}} aims for even data volume instead. It estimates the size of every source chunk, works through the chunks from largest to smallest, and assigns each one to the target shard holding the least estimated data so far. It then replays the source boundaries and places the chunks according to those assignments. + +```{.text .no-copy} +Target shards: tgt-a, tgt-b +Source chunk sizes: 100 MB, 60 MB, 40 MB + +100 MB -> tgt-a + 60 MB -> tgt-b + 40 MB -> tgt-b + +Estimated result: +tgt-a: 100 MB +tgt-b: 100 MB +``` + +The running size estimate carries across collections rather than resetting for each one, so a large chunk from one collection and a large chunk from the next do not both land on the same target shard. The source boundaries are preserved either way. Only the ownership changes. + +### If the pre-split fails + +A failed pre-split fails the clone for that instance. {{pcsm.short}} does not fall back to loading into an unsplit collection, because that would quietly reintroduce the single-shard bottleneck the pre-split exists to prevent. + +Fix the underlying problem on the target cluster, then restart replication with `pcsm resume --from-failure`. See [Resume the replication](install/usage.md#resume-the-replication), [Logging in {{pcsm.full_name}}](logging.md), and the [Troubleshooting guide](troubleshooting.md). + +### Check the layout on the target + +Connect to the target `mongos` and look at how a replicated collection is spread: + +```javascript +db.getSiblingDB('')..getShardDistribution() +``` -{{pcsm.short}} does not preserve chunk distribution information from the source cluster. The target cluster manages chunk distribution internally through its balancer. This means that after replication, chunks may be distributed differently on the target cluster compared to the source cluster, which is expected behavior. +For chunk counts per shard across the cluster, use `sh.status()`. See [db.collection.getShardDistribution() :octicons-link-external-16:](https://www.mongodb.com/docs/manual/reference/method/db.collection.getShardDistribution/){:target="_blank"} and [sh.status() :octicons-link-external-16:](https://www.mongodb.com/docs/manual/reference/method/sh.status/){:target="_blank"} in the MongoDB documentation. -Since the target cluster already has information about which collections are sharded, it handles sharding internally. {{pcsm.short}} does not interfere with the target cluster's sharding configuration or chunk distribution. +Look for data on every shard rather than an exact match with the source. Chunk counts and document counts per shard differ from the source even immediately after the clone, and they keep changing as the balancer works. ## Usage From 5b13dfcfa7f24661840cd8e87b594627df5e8948 Mon Sep 17 00:00:00 2001 From: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com> Date: Tue, 8 Sep 2026 15:04:30 +0530 Subject: [PATCH 02/11] Update sharding.md --- docs/sharding.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/sharding.md b/docs/sharding.md index d08235ff..96ad3641 100644 --- a/docs/sharding.md +++ b/docs/sharding.md @@ -66,7 +66,6 @@ This happens automatically. There is no flag to set, nothing to enable, and no w {{pcsm.short}} reads the source chunk boundaries once, before the clone. It does not replicate sharding metadata afterwards, so later chunk migrations, splits, merges, and resharding on the source have no effect on the target layout. The two clusters drift apart as soon as either balancer moves data. A layout that no longer matches the source is expected and does not indicate a replication problem. - ### Hashed shard keys From a52cedaeb295773be5e1a43e4202644a38f5d0c8 Mon Sep 17 00:00:00 2001 From: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com> Date: Tue, 8 Sep 2026 15:20:16 +0530 Subject: [PATCH 03/11] Update sharding.md --- docs/sharding.md | 86 +++++++++++++++++------------------------------- 1 file changed, 30 insertions(+), 56 deletions(-) diff --git a/docs/sharding.md b/docs/sharding.md index 96ad3641..ca2f48de 100644 --- a/docs/sharding.md +++ b/docs/sharding.md @@ -50,79 +50,55 @@ The target starts from the same chunk boundaries as the source, so a chunk migra !!! admonition "Version added: 0.10.0" -Sharding an empty collection on a ranged shard key gives you a single chunk that covers the whole key range, and the balancer only starts spreading data once documents arrive. For a clone, that is the worst possible starting point. Every document {{pcsm.short}} writes lands on one shard, that shard absorbs the entire write load, and when the clone finishes the balancer begins a long migration of data that never needed to be in one place. +When MongoDB shards an empty collection on a ranged shard key, it creates a single chunk covering the entire range of shard key values. See [Data partitioning with chunks :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/sharding-data-partitioning/){:target="_blank"} in the MongoDB documentation. A clone into that collection would therefore write to a single shard, and the target balancer would move the data afterwards. -{{pcsm.short}} pre-splits the target collection before it copies anything. It reads the chunk layout of the source collection, recreates those boundaries on the target, and places the resulting chunks across the target shards. Clone writes then spread across every shard from the first document, and no rebalancing wave follows the clone. +{{pcsm.short}} therefore recreates the source chunk boundaries on the target before copying any documents, so the clone writes to every shard from the start and no rebalancing wave follows. Matching boundaries are also what makes it safe to leave the balancer running on the source, as described in [Balancer operation](#balancer-operation). -This happens automatically. There is no flag to set, nothing to enable, and no way to turn it off. {{pcsm.short}} identifies the source collection by UUID, keeps the chunk boundaries in order, and applies them with the standard MongoDB sharding commands. +This runs automatically for every sharded collection, immediately after {{pcsm.short}} shards it on the target. There is no flag and nothing to configure. -| **Source collection** | **Target shards** | **What {{pcsm.short}} does** | -|-----------------------|-------------------|------------------------------| -| Hashed shard key | Any number | Nothing. The target keeps the layout that `shardCollection` creates. | -| Ranged shard key | Same number as the source | Mirrors the source chunk boundaries and their ownership pattern. | -| Ranged shard key | Different number from the source | Replays the source boundaries and places the chunks so that each target shard holds roughly the same volume of data. | +| **Source collection** | **Target shards** | **Result on the target** | +|-----------------------|-------------------|--------------------------| +| Hashed shard key | Any number | The layout that `shardCollection` creates, unchanged. | +| Ranged shard key | Same number as the source | The same chunk boundaries and the same ownership pattern as the source. | +| Ranged shard key | Different number from the source | The same chunk boundaries, with each target shard holding roughly the same volume of data. | !!! note "The layout is a starting point, not a copy" - {{pcsm.short}} reads the source chunk boundaries once, before the clone. It does not replicate sharding metadata afterwards, so later chunk migrations, splits, merges, and resharding on the source have no effect on the target layout. The two clusters drift apart as soon as either balancer moves data. A layout that no longer matches the source is expected and does not indicate a replication problem. - + {{pcsm.short}} reads the source boundaries once, before the clone, and does not replicate sharding metadata afterwards. Later migrations, splits, merges, and resharding on the source have no effect on the target, so the two layouts drift apart as the balancers work. That is expected and does not indicate a replication problem. ### Hashed shard keys -{{pcsm.short}} does no pre-splitting for hashed shard keys, and that is deliberate. The `shardCollection` command already produces an evenly distributed layout across all shards on every supported MongoDB version, so there is nothing to improve and {{pcsm.short}} keeps what MongoDB created. - -The number of initial chunks depends on the server version. With three target shards: - -* MongoDB 6.0 and 7.0 create six chunks, roughly two per shard. -* MongoDB 8.0 creates three chunks, roughly one per shard. - -See [Hashed sharding :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/hashed-sharding/){:target="_blank"} in the MongoDB documentation for how the server builds that initial layout. +{{pcsm.short}} does not pre-split hashed collections, and does not need to. MongoDB already spreads the initial chunks evenly across the shards for a hashed shard key, so {{pcsm.short}} keeps that layout. See [Hashed sharding :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/hashed-sharding/){:target="_blank"} in the MongoDB documentation. -### Ranged shard keys with the same number of shards +The number of chunks depends on your MongoDB version. With three target shards, MongoDB 6.0 and 7.0 create six chunks and MongoDB 8.0 creates three. -When both clusters have the same number of shards, {{pcsm.short}} reproduces the source layout directly. It sorts the shard IDs on each side, pairs them by position, replays every source chunk boundary on the target, and puts each target chunk on the shard paired with its source owner. +### Ranged shard keys -```{.text .no-copy} -Source shards: src-a, src-b -Target shards: tgt-a, tgt-b - -Source layout: -[MinKey, 100) -> src-a -[100, MaxKey) -> src-b +With the same number of shards on both sides, the target gets the source boundaries and the same ownership pattern. Shards are paired in sorted order, so a range does not necessarily land on the target shard whose name resembles its source shard. -Target layout: -[MinKey, 100) -> tgt-a -[100, MaxKey) -> tgt-b -``` +With different shard counts, the boundaries still come from the source, but the largest chunks are placed first, each on whichever target shard holds the least data at that point. Every shard ends up owning chunks and holding roughly the same volume. The estimate carries across collections, so the large chunks of several collections do not all collect on one shard. -!!! note "Shard names are paired, not matched" +??? example "How the two cases look" - Pairing is by sorted position, so the shard that owns a range on the target is not necessarily the one with a similar name on the source. What {{pcsm.short}} reproduces is the shape of the distribution, not the shard names. + ```{.text .no-copy} + Same number of shards + --------------------- + Source: [MinKey, 100) -> src-a Target: [MinKey, 100) -> tgt-a + [100, MaxKey) -> src-b [100, MaxKey) -> tgt-b -### Ranged shard keys with a different number of shards + Different number of shards + -------------------------- + Target shards: tgt-a, tgt-b + Source chunk sizes: 100 MB, 60 MB, 40 MB -Source ownership cannot be mirrored when the shard counts differ, so {{pcsm.short}} aims for even data volume instead. It estimates the size of every source chunk, works through the chunks from largest to smallest, and assigns each one to the target shard holding the least estimated data so far. It then replays the source boundaries and places the chunks according to those assignments. - -```{.text .no-copy} -Target shards: tgt-a, tgt-b -Source chunk sizes: 100 MB, 60 MB, 40 MB - -100 MB -> tgt-a - 60 MB -> tgt-b - 40 MB -> tgt-b - -Estimated result: -tgt-a: 100 MB -tgt-b: 100 MB -``` - -The running size estimate carries across collections rather than resetting for each one, so a large chunk from one collection and a large chunk from the next do not both land on the same target shard. The source boundaries are preserved either way. Only the ownership changes. + 100 MB -> tgt-a Estimated result: + 60 MB -> tgt-b tgt-a: 100 MB + 40 MB -> tgt-b tgt-b: 100 MB + ``` ### If the pre-split fails -A failed pre-split fails the clone for that instance. {{pcsm.short}} does not fall back to loading into an unsplit collection, because that would quietly reintroduce the single-shard bottleneck the pre-split exists to prevent. - -Fix the underlying problem on the target cluster, then restart replication with `pcsm resume --from-failure`. See [Resume the replication](install/usage.md#resume-the-replication), [Logging in {{pcsm.full_name}}](logging.md), and the [Troubleshooting guide](troubleshooting.md). +A failed pre-split fails the clone for that instance, and there is no fallback to loading into an unsplit collection. Check the log for the reported failure, resolve it on the target cluster, then restart replication with `pcsm resume --from-failure`. See [Resume the replication](install/usage.md#resume-the-replication), [Logging in {{pcsm.full_name}}](logging.md), and the [Troubleshooting guide](troubleshooting.md). ### Check the layout on the target @@ -132,9 +108,7 @@ Connect to the target `mongos` and look at how a replicated collection is spread db.getSiblingDB('')..getShardDistribution() ``` -For chunk counts per shard across the cluster, use `sh.status()`. See [db.collection.getShardDistribution() :octicons-link-external-16:](https://www.mongodb.com/docs/manual/reference/method/db.collection.getShardDistribution/){:target="_blank"} and [sh.status() :octicons-link-external-16:](https://www.mongodb.com/docs/manual/reference/method/sh.status/){:target="_blank"} in the MongoDB documentation. - -Look for data on every shard rather than an exact match with the source. Chunk counts and document counts per shard differ from the source even immediately after the clone, and they keep changing as the balancer works. +Look for data on every shard rather than an exact match with the source, since counts differ even immediately after the clone and keep changing as the balancer works. For chunk counts per shard across the cluster, use [sh.status() :octicons-link-external-16:](https://www.mongodb.com/docs/manual/reference/method/sh.status/){:target="_blank"}. ## Usage From 4633503aceb89bbb29cb4c83c80972c219480065 Mon Sep 17 00:00:00 2001 From: Rasika Chivate <95711051+rasika-chivate@users.noreply.github.com> Date: Tue, 8 Sep 2026 15:37:30 +0530 Subject: [PATCH 04/11] Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- docs/sharding.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sharding.md b/docs/sharding.md index ca2f48de..04e36e9d 100644 --- a/docs/sharding.md +++ b/docs/sharding.md @@ -38,7 +38,7 @@ For detailed information about authentication and connection string configuratio Before starting the initial sync, {{pcsm.short}} checks which collections are sharded on the source cluster and creates corresponding sharded collections on the destination cluster. The only sharding configuration preserved from the source cluster is the sharding key; all other sharding details are handled internally by the destination cluster. -Immediately after it shards a collection on the target, and before it copies any documents into it, {{pcsm.short}} pre-splits that collection so that the clone writes spread across all target shards. See Chunk distribution. +For a ranged shard key, immediately after it shards a collection on the target and before copying any documents, {{pcsm.short}} pre-splits the collection using the source chunk boundaries. Hashed collections retain the layout created by `shardCollection`. See [Chunk distribution](#chunk-distribution). ### Balancer operation From a502680fe97935107b9da4694512aab3dc700eb2 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Tue, 8 Sep 2026 10:09:17 +0000 Subject: [PATCH 05/11] Clarify sharding metadata replication Co-authored-by: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com> --- docs/limitations.md | 4 ++-- docs/sharding.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/limitations.md b/docs/limitations.md index 6308ecd9..048353dd 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -28,10 +28,10 @@ This page lists known limitations for using {{pcsm.full_name}}. The following limitations apply specifically to sharded cluster replication: -* {{pcsm.short}} replicates the data and doesn't replicate metadata. This means that the following information is not preserved from the source cluster: +* {{pcsm.short}} replicates the data and doesn't continuously replicate metadata. For ranged shard keys, the initial chunk boundaries and ownership are copied during the initial sync, but subsequent sharding metadata changes are not replicated. The following information is therefore not preserved from the source cluster: * The primary shard name for a collection. The target cluster may have a different primary shard name. - * The chunk distribution information. The target cluster manages chunk distribution according to its own sharding configuration. See [Sharding support](sharding.md#limitations) for more information. + * Ongoing chunk distribution information. The target cluster manages chunk distribution according to its own sharding configuration. See [Chunk distribution](sharding.md#chunk-distribution) for more information. * The configuration of [zones for sharded data :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/zone-sharding/). * During data replication, the following commands are not supported: `movePrimary`, `reshardCollecton`, `unshardCollection`, `refineCollectionShardKey`. Running them results in failed replication and you must start it anew, from the initial data sync stage. diff --git a/docs/sharding.md b/docs/sharding.md index 04e36e9d..576f8477 100644 --- a/docs/sharding.md +++ b/docs/sharding.md @@ -36,7 +36,7 @@ For detailed information about authentication and connection string configuratio ### Initial sync preparation -Before starting the initial sync, {{pcsm.short}} checks which collections are sharded on the source cluster and creates corresponding sharded collections on the destination cluster. The only sharding configuration preserved from the source cluster is the sharding key; all other sharding details are handled internally by the destination cluster. +Before starting the initial sync, {{pcsm.short}} checks which collections are sharded on the source cluster and creates corresponding sharded collections on the destination cluster. The sharding key is preserved from the source cluster. For ranged shard keys, {{pcsm.short}} also copies the initial chunk boundaries and ownership to the target; it does not replicate sharding metadata afterwards. For a ranged shard key, immediately after it shards a collection on the target and before copying any documents, {{pcsm.short}} pre-splits the collection using the source chunk boundaries. Hashed collections retain the layout created by `shardCollection`. See [Chunk distribution](#chunk-distribution). From e3f98863adfea5e1fc6f2b06431fc7a3db442757 Mon Sep 17 00:00:00 2001 From: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com> Date: Tue, 8 Sep 2026 16:22:05 +0530 Subject: [PATCH 06/11] Update sharding.md --- docs/sharding.md | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-) diff --git a/docs/sharding.md b/docs/sharding.md index 04e36e9d..26e51a24 100644 --- a/docs/sharding.md +++ b/docs/sharding.md @@ -12,7 +12,7 @@ The workflow for sharded clusters is similar to replica sets. See [How {{pcsm.fu Since {{pcsm.short}} connects through `mongos`, the cluster topology doesn't matter. This means the source and target clusters can have different numbers of shards. -Also, {{pcsm.short}} replicates data and not metadata. This means chunk distribution as well as the primary shard name for a collection may differ on source and target clusters. +{{pcsm.short}} replicates data and not sharding metadata. For a collection with a ranged shard key it copies the initial chunk boundaries to the target before the clone starts, but it does not replicate any sharding metadata changes that follow, and the primary shard name for a collection may differ on source and target clusters. See [Chunk distribution](#chunk-distribution). ## Prerequisites @@ -36,15 +36,15 @@ For detailed information about authentication and connection string configuratio ### Initial sync preparation -Before starting the initial sync, {{pcsm.short}} checks which collections are sharded on the source cluster and creates corresponding sharded collections on the destination cluster. The only sharding configuration preserved from the source cluster is the sharding key; all other sharding details are handled internally by the destination cluster. +Before starting the initial sync, {{pcsm.short}} checks which collections are sharded on the source cluster and creates corresponding sharded collections on the destination cluster. The shard key is the only sharding configuration carried over from the source; everything else is handled internally by the destination cluster. -For a ranged shard key, immediately after it shards a collection on the target and before copying any documents, {{pcsm.short}} pre-splits the collection using the source chunk boundaries. Hashed collections retain the layout created by `shardCollection`. See [Chunk distribution](#chunk-distribution). +For a ranged shard key, {{pcsm.short}} then pre-splits the collection using the source chunk boundaries, immediately after it shards the collection on the target and before it copies any documents. Collections with a hashed shard key keep the layout that `shardCollection` creates. See [Chunk distribution](#chunk-distribution). ### Balancer operation {{pcsm.full_name}} connects to source and target clusters via a `mongos` instance. Therefore, you do not need to disable the balancer on either the source or target cluster before starting replication. The target cluster's balancer continues to operate normally and manages chunk distribution according to its own sharding configuration and balancer settings. -The target starts from the same chunk boundaries as the source, so a chunk migration on the source arrives where the target expects it. That makes it safe to leave the balancer running during the sync, which matters in write-heavy clusters where turning it off is not an option. See [Manage sharded cluster balancer :octicons-link-external-16:](https://www.mongodb.com/docs/manual/tutorial/manage-sharded-cluster-balancer/){:target="_blank"} in the MongoDB documentation. +For ranged shard keys, the target also starts from the source chunk boundaries, which leaves the target balancer less data to move once the clone begins. Chunk migrations on either cluster are not replicated to the other, so both clusters keep managing their own layout throughout. See [Manage sharded cluster balancer :octicons-link-external-16:](https://www.mongodb.com/docs/manual/tutorial/manage-sharded-cluster-balancer/){:target="_blank"} in the MongoDB documentation. ## Chunk distribution @@ -52,26 +52,23 @@ The target starts from the same chunk boundaries as the source, so a chunk migra When MongoDB shards an empty collection on a ranged shard key, it creates a single chunk covering the entire range of shard key values. See [Data partitioning with chunks :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/sharding-data-partitioning/){:target="_blank"} in the MongoDB documentation. A clone into that collection would therefore write to a single shard, and the target balancer would move the data afterwards. -{{pcsm.short}} therefore recreates the source chunk boundaries on the target before copying any documents, so the clone writes to every shard from the start and no rebalancing wave follows. Matching boundaries are also what makes it safe to leave the balancer running on the source, as described in [Balancer operation](#balancer-operation). +{{pcsm.short}} therefore recreates the source chunk boundaries on the target before copying any documents. Clone writes follow the source layout instead of concentrating on one shard, which reduces how much the target balancer has to move afterwards. -This runs automatically for every sharded collection, immediately after {{pcsm.short}} shards it on the target. There is no flag and nothing to configure. +This runs automatically for every collection with a ranged shard key, immediately after {{pcsm.short}} shards it on the target. There is no flag and nothing to configure. Collections with a hashed shard key are not pre-split. | **Source collection** | **Target shards** | **Result on the target** | |-----------------------|-------------------|--------------------------| | Hashed shard key | Any number | The layout that `shardCollection` creates, unchanged. | | Ranged shard key | Same number as the source | The same chunk boundaries and the same ownership pattern as the source. | -| Ranged shard key | Different number from the source | The same chunk boundaries, with each target shard holding roughly the same volume of data. | +| Ranged shard key | Different number from the source | The same chunk boundaries, with chunks placed to even out the estimated data volume per shard.| !!! note "The layout is a starting point, not a copy" - {{pcsm.short}} reads the source boundaries once, before the clone, and does not replicate sharding metadata afterwards. Later migrations, splits, merges, and resharding on the source have no effect on the target, so the two layouts drift apart as the balancers work. That is expected and does not indicate a replication problem. ### Hashed shard keys {{pcsm.short}} does not pre-split hashed collections, and does not need to. MongoDB already spreads the initial chunks evenly across the shards for a hashed shard key, so {{pcsm.short}} keeps that layout. See [Hashed sharding :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/hashed-sharding/){:target="_blank"} in the MongoDB documentation. -The number of chunks depends on your MongoDB version. With three target shards, MongoDB 6.0 and 7.0 create six chunks and MongoDB 8.0 creates three. - ### Ranged shard keys With the same number of shards on both sides, the target gets the source boundaries and the same ownership pattern. Shards are paired in sorted order, so a range does not necessarily land on the target shard whose name resembles its source shard. @@ -105,7 +102,7 @@ A failed pre-split fails the clone for that instance, and there is no fallback t Connect to the target `mongos` and look at how a replicated collection is spread: ```javascript -db.getSiblingDB('')..getShardDistribution() +db.getSiblingDB('').getCollection('').getShardDistribution() ``` Look for data on every shard rather than an exact match with the source, since counts differ even immediately after the clone and keep changing as the balancer works. For chunk counts per shard across the cluster, use [sh.status() :octicons-link-external-16:](https://www.mongodb.com/docs/manual/reference/method/sh.status/){:target="_blank"}. From 960cb5df8dc62b9ed38db8bf9e20fefccb992b8b Mon Sep 17 00:00:00 2001 From: Rasika Chivate <95711051+rasika-chivate@users.noreply.github.com> Date: Tue, 8 Sep 2026 17:01:59 +0530 Subject: [PATCH 07/11] Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- docs/limitations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/limitations.md b/docs/limitations.md index 048353dd..66d515e3 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -28,7 +28,7 @@ This page lists known limitations for using {{pcsm.full_name}}. The following limitations apply specifically to sharded cluster replication: -* {{pcsm.short}} replicates the data and doesn't continuously replicate metadata. For ranged shard keys, the initial chunk boundaries and ownership are copied during the initial sync, but subsequent sharding metadata changes are not replicated. The following information is therefore not preserved from the source cluster: +* {{pcsm.short}} replicates the data and doesn't continuously replicate metadata. For ranged shard keys, the initial chunk boundaries are copied and ownership is initialized on the target during the initial sync, but subsequent sharding metadata changes are not replicated. The following information is therefore not preserved from the source cluster: * The primary shard name for a collection. The target cluster may have a different primary shard name. * Ongoing chunk distribution information. The target cluster manages chunk distribution according to its own sharding configuration. See [Chunk distribution](sharding.md#chunk-distribution) for more information. From 44b73b12fb78a23afe91c5485c639787880f8f2c Mon Sep 17 00:00:00 2001 From: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com> Date: Tue, 8 Sep 2026 17:05:25 +0530 Subject: [PATCH 08/11] Update sharding.md --- docs/sharding.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sharding.md b/docs/sharding.md index f33568cf..d4efe319 100644 --- a/docs/sharding.md +++ b/docs/sharding.md @@ -62,7 +62,7 @@ This runs automatically for every collection with a ranged shard key, immediatel | Ranged shard key | Same number as the source | The same chunk boundaries and the same ownership pattern as the source. | | Ranged shard key | Different number from the source | The same chunk boundaries, with chunks placed to even out the estimated data volume per shard.| -!!! note "The layout is a starting point, not a copy" +!!! note {{pcsm.short}} reads the source boundaries once, before the clone, and does not replicate sharding metadata afterwards. Later migrations, splits, merges, and resharding on the source have no effect on the target, so the two layouts drift apart as the balancers work. That is expected and does not indicate a replication problem. ### Hashed shard keys From 672af59567a81bffe18f242348eb97febd17df77 Mon Sep 17 00:00:00 2001 From: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com> Date: Tue, 8 Sep 2026 21:24:12 +0530 Subject: [PATCH 09/11] Update limitations.md --- docs/limitations.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/limitations.md b/docs/limitations.md index 66d515e3..e94cf224 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -30,9 +30,11 @@ The following limitations apply specifically to sharded cluster replication: * {{pcsm.short}} replicates the data and doesn't continuously replicate metadata. For ranged shard keys, the initial chunk boundaries are copied and ownership is initialized on the target during the initial sync, but subsequent sharding metadata changes are not replicated. The following information is therefore not preserved from the source cluster: - * The primary shard name for a collection. The target cluster may have a different primary shard name. - * Ongoing chunk distribution information. The target cluster manages chunk distribution according to its own sharding configuration. See [Chunk distribution](sharding.md#chunk-distribution) for more information. - * The configuration of [zones for sharded data :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/zone-sharding/). +* The primary shard name for a collection. The target cluster may have a different primary shard name. + +* Ongoing chunk distribution information. The target cluster manages chunk distribution according to its own sharding configuration. See [Chunk distribution](sharding.md#chunk-distribution) for more information. + +* The configuration of [zones for sharded data :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/zone-sharding/). * During data replication, the following commands are not supported: `movePrimary`, `reshardCollecton`, `unshardCollection`, `refineCollectionShardKey`. Running them results in failed replication and you must start it anew, from the initial data sync stage. From 9de90e2465f4a6c29545acfdec042368dc315fdc Mon Sep 17 00:00:00 2001 From: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com> Date: Tue, 8 Sep 2026 21:49:44 +0530 Subject: [PATCH 10/11] Update sharding.md --- docs/sharding.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sharding.md b/docs/sharding.md index d4efe319..792d7d98 100644 --- a/docs/sharding.md +++ b/docs/sharding.md @@ -12,7 +12,7 @@ The workflow for sharded clusters is similar to replica sets. See [How {{pcsm.fu Since {{pcsm.short}} connects through `mongos`, the cluster topology doesn't matter. This means the source and target clusters can have different numbers of shards. -{{pcsm.short}} replicates data and not sharding metadata. For a collection with a ranged shard key it copies the initial chunk boundaries to the target before the clone starts, but it does not replicate any sharding metadata changes that follow, and the primary shard name for a collection may differ on source and target clusters. See [Chunk distribution](#chunk-distribution). +{{pcsm.short}} does not continuously replicate sharding metadata from the source to the target. For collections with a ranged shard key, it copies the initial chunk boundaries to the target before the clone starts. Any sharding metadata changes made after that are not replicated. The primary shard name for a collection may also differ between the source and target clusters. See [Chunk distribution](#chunk-distribution). ## Prerequisites From 1368c8470a2fadf9ce71f1e1326746c3cce8d886 Mon Sep 17 00:00:00 2001 From: rasika-chivate <95711051+rasika-chivate@users.noreply.github.com> Date: Tue, 8 Sep 2026 22:05:37 +0530 Subject: [PATCH 11/11] fixed indentation for sub bullet points --- docs/limitations.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/limitations.md b/docs/limitations.md index e94cf224..0d396ad5 100644 --- a/docs/limitations.md +++ b/docs/limitations.md @@ -28,15 +28,15 @@ This page lists known limitations for using {{pcsm.full_name}}. The following limitations apply specifically to sharded cluster replication: -* {{pcsm.short}} replicates the data and doesn't continuously replicate metadata. For ranged shard keys, the initial chunk boundaries are copied and ownership is initialized on the target during the initial sync, but subsequent sharding metadata changes are not replicated. The following information is therefore not preserved from the source cluster: +- {{pcsm.short}} replicates the data and doesn't continuously replicate metadata. For ranged shard keys, the initial chunk boundaries are copied and ownership is initialized on the target during the initial sync, but subsequent sharding metadata changes are not replicated. The following information is therefore not preserved from the source cluster: -* The primary shard name for a collection. The target cluster may have a different primary shard name. + * The primary shard name for a collection. The target cluster may have a different primary shard name. -* Ongoing chunk distribution information. The target cluster manages chunk distribution according to its own sharding configuration. See [Chunk distribution](sharding.md#chunk-distribution) for more information. + * Ongoing chunk distribution information. The target cluster manages chunk distribution according to its own sharding configuration. See [Chunk distribution](sharding.md#chunk-distribution) for more information. -* The configuration of [zones for sharded data :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/zone-sharding/). + * The configuration of [zones for sharded data :octicons-link-external-16:](https://www.mongodb.com/docs/manual/core/zone-sharding/). -* During data replication, the following commands are not supported: `movePrimary`, `reshardCollecton`, `unshardCollection`, `refineCollectionShardKey`. Running them results in failed replication and you must start it anew, from the initial data sync stage. + * During data replication, the following commands are not supported: `movePrimary`, `reshardCollecton`, `unshardCollection`, `refineCollectionShardKey`. Running them results in failed replication and you must start it anew, from the initial data sync stage. ## Data types