Allow checkpointing to be performed on the candidate leader node.

[yangsw26]

Currently, checkpointing is only allowed on the leader node. This can lead to a potential node recovery deadlock: a large amount of data log replay fills up memory, and since checkpointing is not performed on the candidate leader node, the data in memory is not persisted and therefore cannot be evicted, preventing log replay from completing.

To resolve this issue, checkpointing must be allowed on the candidate leader node. Additionally, the requests involved in this process need to match the candidate term.

[yangsw26]

1. Add the flag allow_run_on_candidate to TransactionExecution, and also check the candidate leader term in the relevant CC request.

[yangsw26]

Currently, DataSync operations can be completed normally on the candidate leader node; however, the key cannot be evicted from memory, causing log replay to fail to finish.

[yangsw26]

The shard#0 heap is still full after checkpoint, and there is no ccentries on shard#0:

cc_shard.cpp:1178] Clean finished: free_cnt: 0, yield: false, clean start page addr: 0x0x21cfc1c7b80, clean page cnt: 0, scan page cnt: 0, shard: 0
cc_shard.cpp:1186] Printing cc pages on lru list on shard: 0
cc_shard.cpp:1195] Printing cc pages on lru list on shard: 0 finished, ccpage count: 0
cc_shard.cpp:1200] Table: ./sbtest/customer, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/customer finished
cc_shard.cpp:1200] Table: __range_bucket, all records on this ccm: 1024
cc_shard.cpp:1206] Printing cc pages on table: __range_bucket finished
cc_shard.cpp:1200] Table: ./sbtest/stock, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/stock finished
cc_shard.cpp:1200] Table: ./sbtest/history, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/history finished
cc_shard.cpp:1200] Table: ./sbtest/warehouse, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/warehouse finished
cc_shard.cpp:1200] Table: __sequence_table, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: __sequence_table finished
cc_shard.cpp:1200] Table: ./sbtest/order_line*$$ol_supply_w_id, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/order_line*$$ol_supply_w_id finished
cc_shard.cpp:1200] Table: ./mysql/global_priv, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./mysql/global_priv finished
cc_shard.cpp:1200] Table: __cluster_config, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: __cluster_config finished
cc_shard.cpp:1200] Table: ./sbtest/customer*$$c_w_id, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/customer*$$c_w_id finished
cc_shard.cpp:1200] Table: __catalog, all records on this ccm: 11
cc_shard.cpp:1206] Printing cc pages on table: __catalog finished
cc_shard.cpp:1200] Table: ./sbtest/order_line, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/order_line finished
cc_shard.cpp:1200] Table: ./sbtest/item, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/item finished
cc_shard.cpp:1200] Table: ./sbtest/stock*$$s_i_id, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/stock*$$s_i_id finished
cc_shard.cpp:1200] Table: ./sbtest/orders, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/orders finished
cc_shard.cpp:1200] Table: ./sbtest/history, all records on this ccm: 34
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/history finished
cc_shard.cpp:1200] Table: ./sbtest/orders*$$o_w_id, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/orders*$$o_w_id finished
cc_shard.cpp:1200] Table: ./sbtest/new_orders, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/new_orders finished
cc_shard.cpp:1200] Table: ./sbtest/district, all records on this ccm: 0
cc_shard.cpp:1206] Printing cc pages on table: ./sbtest/district finished
cc_req_misc.cpp:1506] ShardCleanCc::Execute still full after clean shard, on shard: 0, heap committed: 1247505200, heap allocated: 1243742272, need yield: false, free count: 0

[yangsw26]

Maybe the heap was used by the replay log blob: