实战案例：Redis集群动态缩容
source link: https://blog.51cto.com/shone/5279179
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
实战案例：Redis集群动态缩容_无冬无夏的技术博客_51CTO博客
#### 从集群中删除下面这对（主从）节点 IP58，演示缩容过程
M: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa 172.16.0.58:6379
slots:[1093-2183],[6553-7644],[12014-13104] (3274 slots) master
1 additional replica(s)
S: aeee686e355fa7784b383fec3543232126dcfbad 172.16.0.158:6379
slots: (0 slots) slave
replicates c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
## 第一步先挪动了 slots:[1093-2183] 给IP18
###################################################################################
## IP172.16.0.58 上的slots:[1093-2183]迁移到 IP172.16.0.28 上去
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning --cluster reshard 172.16.0.18:6379
>>> Performing Cluster Check (using node 172.16.0.18:6379)
M: d5462f6961c0f45ecbdf12d6606e6993c33e3e29 172.16.0.18:6379
slots:[2184-5461] (3278 slots) master
1 additional replica(s)
S: 93a4ba65d181a756c08bd3b6c2a7a4d24cf5855d 172.16.0.148:6379
slots: (0 slots) slave
replicates aaa956c280b5d9fe18fca48a910c1085b5f22122
S: aeee686e355fa7784b383fec3543232126dcfbad 172.16.0.158:6379
slots: (0 slots) slave
replicates c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
M: 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e 172.16.0.28:6379
slots:[7645-10922] (3278 slots) master
1 additional replica(s)
M: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa 172.16.0.58:6379
slots:[1093-2183],[6553-7644],[12014-13104] (3274 slots) master
1 additional replica(s)
S: a7583f69703921c6b3d14a97f54b3015966155ab 172.16.0.118:6379
slots: (0 slots) slave
replicates d5462f6961c0f45ecbdf12d6606e6993c33e3e29
S: 3d69cddc61df9443ff7de9850c220fc9e9187c03 172.16.0.138:6379
slots: (0 slots) slave
replicates 4c429a48054a771cbc154319182a3d16cf4ce7a1
M: aaa956c280b5d9fe18fca48a910c1085b5f22122 172.16.0.48:6379
slots:[0-1092],[5462-6552],[10923-12013] (3275 slots) master
1 additional replica(s)
S: 3bbdbc3ab34b67161655974fed9de5667def8ed0 172.16.0.128:6379
slots: (0 slots) slave
replicates 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e
M: 4c429a48054a771cbc154319182a3d16cf4ce7a1 172.16.0.38:6379
slots:[13105-16383] (3279 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 1091
What is the receiving node ID? d5462f6961c0f45ecbdf12d6606e6993c33e3e29
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
Source node #2: done
Ready to move 1091 slots.
Source nodes:
M: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa 172.16.0.58:6379
slots:[1093-2183],[6553-7644],[12014-13104] (3274 slots) master
1 additional replica(s)
Destination node:
M: d5462f6961c0f45ecbdf12d6606e6993c33e3e29 172.16.0.18:6379
slots:[2184-5461] (3278 slots) master
1 additional replica(s)
Resharding plan:
Moving slot 1093 from c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
.....................
Moving slot 2183 from c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
Do you want to proceed with the proposed reshard plan (yes/no)? yes
Moving slot 1093 from 172.16.0.58:6379 to 172.16.0.18:6379:
.........................
###################################################################################
## 第二步 IP172.16.0.58 上的slots:[6553-7644] 迁移到 IP172.16.0.28 上去
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning --cluster check 172.16.0.18:6379
172.16.0.18:6379 (d5462f69...) -> 0 keys | 4369 slots | 1 slaves.
172.16.0.28:6379 (5163e9ab...) -> 0 keys | 3278 slots | 1 slaves.
172.16.0.58:6379 (c4af78bc...) -> 0 keys | 2183 slots | 1 slaves.
172.16.0.48:6379 (aaa956c2...) -> 1 keys | 3275 slots | 1 slaves.
172.16.0.38:6379 (4c429a48...) -> 0 keys | 3279 slots | 1 slaves.
[OK] 1 keys in 5 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.16.0.18:6379)
M: d5462f6961c0f45ecbdf12d6606e6993c33e3e29 172.16.0.18:6379
slots:[1093-5461] (4369 slots) master
1 additional replica(s)
S: 93a4ba65d181a756c08bd3b6c2a7a4d24cf5855d 172.16.0.148:6379
slots: (0 slots) slave
replicates aaa956c280b5d9fe18fca48a910c1085b5f22122
S: aeee686e355fa7784b383fec3543232126dcfbad 172.16.0.158:6379
slots: (0 slots) slave
replicates c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
M: 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e 172.16.0.28:6379
slots:[7645-10922] (3278 slots) master
1 additional replica(s)
M: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa 172.16.0.58:6379
slots:[6553-7644],[12014-13104] (2183 slots) master
1 additional replica(s)
S: a7583f69703921c6b3d14a97f54b3015966155ab 172.16.0.118:6379
slots: (0 slots) slave
replicates d5462f6961c0f45ecbdf12d6606e6993c33e3e29
S: 3d69cddc61df9443ff7de9850c220fc9e9187c03 172.16.0.138:6379
slots: (0 slots) slave
replicates 4c429a48054a771cbc154319182a3d16cf4ce7a1
M: aaa956c280b5d9fe18fca48a910c1085b5f22122 172.16.0.48:6379
slots:[0-1092],[5462-6552],[10923-12013] (3275 slots) master
1 additional replica(s)
S: 3bbdbc3ab34b67161655974fed9de5667def8ed0 172.16.0.128:6379
slots: (0 slots) slave
replicates 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e
M: 4c429a48054a771cbc154319182a3d16cf4ce7a1 172.16.0.38:6379
slots:[13105-16383] (3279 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@CentOS84-IP172-18 ]#
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning --cluster reshard 172.16.0.18:6379
>>> Performing Cluster Check (using node 172.16.0.18:6379)
M: d5462f6961c0f45ecbdf12d6606e6993c33e3e29 172.16.0.18:6379
slots:[1093-5461] (4369 slots) master
1 additional replica(s)
S: 93a4ba65d181a756c08bd3b6c2a7a4d24cf5855d 172.16.0.148:6379
slots: (0 slots) slave
replicates aaa956c280b5d9fe18fca48a910c1085b5f22122
S: aeee686e355fa7784b383fec3543232126dcfbad 172.16.0.158:6379
slots: (0 slots) slave
replicates c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
M: 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e 172.16.0.28:6379
slots:[7645-10922] (3278 slots) master
1 additional replica(s)
M: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa 172.16.0.58:6379
slots:[6553-7644],[12014-13104] (2183 slots) master
1 additional replica(s)
S: a7583f69703921c6b3d14a97f54b3015966155ab 172.16.0.118:6379
slots: (0 slots) slave
replicates d5462f6961c0f45ecbdf12d6606e6993c33e3e29
S: 3d69cddc61df9443ff7de9850c220fc9e9187c03 172.16.0.138:6379
slots: (0 slots) slave
replicates 4c429a48054a771cbc154319182a3d16cf4ce7a1
M: aaa956c280b5d9fe18fca48a910c1085b5f22122 172.16.0.48:6379
slots:[0-1092],[5462-6552],[10923-12013] (3275 slots) master
1 additional replica(s)
S: 3bbdbc3ab34b67161655974fed9de5667def8ed0 172.16.0.128:6379
slots: (0 slots) slave
replicates 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e
M: 4c429a48054a771cbc154319182a3d16cf4ce7a1 172.16.0.38:6379
slots:[13105-16383] (3279 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 1092
What is the receiving node ID? 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
Source node #2: done
......................
Do you want to proceed with the proposed reshard plan (yes/no)? yes
......................
###################################################################################
## 第三步 IP172.16.0.58 上的slots:[1093-5461] 迁移到 IP172.16.0.38 上去
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning --cluster check 172.16.0.18:6379
172.16.0.18:6379 (d5462f69...) -> 0 keys | 4369 slots | 1 slaves.
172.16.0.28:6379 (5163e9ab...) -> 0 keys | 4370 slots | 1 slaves.
172.16.0.58:6379 (c4af78bc...) -> 0 keys | 1091 slots | 1 slaves.
172.16.0.48:6379 (aaa956c2...) -> 1 keys | 3275 slots | 1 slaves.
172.16.0.38:6379 (4c429a48...) -> 0 keys | 3279 slots | 1 slaves.
[OK] 1 keys in 5 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.16.0.18:6379)
M: d5462f6961c0f45ecbdf12d6606e6993c33e3e29 172.16.0.18:6379
slots:[1093-5461] (4369 slots) master
1 additional replica(s)
S: 93a4ba65d181a756c08bd3b6c2a7a4d24cf5855d 172.16.0.148:6379
slots: (0 slots) slave
replicates aaa956c280b5d9fe18fca48a910c1085b5f22122
S: aeee686e355fa7784b383fec3543232126dcfbad 172.16.0.158:6379
slots: (0 slots) slave
replicates c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
M: 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e 172.16.0.28:6379
slots:[6553-10922] (4370 slots) master
1 additional replica(s)
M: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa 172.16.0.58:6379
slots:[12014-13104] (1091 slots) master
1 additional replica(s)
S: a7583f69703921c6b3d14a97f54b3015966155ab 172.16.0.118:6379
slots: (0 slots) slave
replicates d5462f6961c0f45ecbdf12d6606e6993c33e3e29
S: 3d69cddc61df9443ff7de9850c220fc9e9187c03 172.16.0.138:6379
slots: (0 slots) slave
replicates 4c429a48054a771cbc154319182a3d16cf4ce7a1
M: aaa956c280b5d9fe18fca48a910c1085b5f22122 172.16.0.48:6379
slots:[0-1092],[5462-6552],[10923-12013] (3275 slots) master
1 additional replica(s)
S: 3bbdbc3ab34b67161655974fed9de5667def8ed0 172.16.0.128:6379
slots: (0 slots) slave
replicates 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e
M: 4c429a48054a771cbc154319182a3d16cf4ce7a1 172.16.0.38:6379
slots:[13105-16383] (3279 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning --cluster reshard 172.16.0.18:6379
>>> Performing Cluster Check (using node 172.16.0.18:6379)
M: d5462f6961c0f45ecbdf12d6606e6993c33e3e29 172.16.0.18:6379
slots:[1093-5461] (4369 slots) master
1 additional replica(s)
S: 93a4ba65d181a756c08bd3b6c2a7a4d24cf5855d 172.16.0.148:6379
slots: (0 slots) slave
replicates aaa956c280b5d9fe18fca48a910c1085b5f22122
S: aeee686e355fa7784b383fec3543232126dcfbad 172.16.0.158:6379
slots: (0 slots) slave
replicates c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
M: 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e 172.16.0.28:6379
slots:[6553-10922] (4370 slots) master
1 additional replica(s)
M: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa 172.16.0.58:6379
slots:[12014-13104] (1091 slots) master
1 additional replica(s)
S: a7583f69703921c6b3d14a97f54b3015966155ab 172.16.0.118:6379
slots: (0 slots) slave
replicates d5462f6961c0f45ecbdf12d6606e6993c33e3e29
S: 3d69cddc61df9443ff7de9850c220fc9e9187c03 172.16.0.138:6379
slots: (0 slots) slave
replicates 4c429a48054a771cbc154319182a3d16cf4ce7a1
M: aaa956c280b5d9fe18fca48a910c1085b5f22122 172.16.0.48:6379
slots:[0-1092],[5462-6552],[10923-12013] (3275 slots) master
1 additional replica(s)
S: 3bbdbc3ab34b67161655974fed9de5667def8ed0 172.16.0.128:6379
slots: (0 slots) slave
replicates 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e
M: 4c429a48054a771cbc154319182a3d16cf4ce7a1 172.16.0.38:6379
slots:[13105-16383] (3279 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 1091
What is the receiving node ID? 4c429a48054a771cbc154319182a3d16cf4ce7a1
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
Source node #2: done
Ready to move 1091 slots.
Source nodes:
M: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa 172.16.0.58:6379
slots:[12014-13104] (1091 slots) master
1 additional replica(s)
Destination node:
M: 4c429a48054a771cbc154319182a3d16cf4ce7a1 172.16.0.38:6379
slots:[13105-16383] (3279 slots) master
1 additional replica(s)
Resharding plan:
Moving slot 12014 from c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
..........................
Moving slot 13104 from c4af78bc4a26490d51edc78b6c547a7abaf4e1aa
Do you want to proceed with the proposed reshard plan (yes/no)? yes
..............................
###################################################################################
# 确认172.16.0.58 的所有slot都移走了，上面的 slave 也自动删除，成为其它 master （IP38)的 slave
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning --cluster check 172.16.0.18:6379
172.16.0.18:6379 (d5462f69...) -> 0 keys | 4369 slots | 1 slaves.
172.16.0.28:6379 (5163e9ab...) -> 0 keys | 4370 slots | 1 slaves.
172.16.0.48:6379 (aaa956c2...) -> 1 keys | 3275 slots | 1 slaves.
172.16.0.38:6379 (4c429a48...) -> 0 keys | 4370 slots | 3 slaves. #自动并入
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.16.0.18:6379)
M: d5462f6961c0f45ecbdf12d6606e6993c33e3e29 172.16.0.18:6379
slots:[1093-5461] (4369 slots) master
1 additional replica(s)
S: 93a4ba65d181a756c08bd3b6c2a7a4d24cf5855d 172.16.0.148:6379
slots: (0 slots) slave
replicates aaa956c280b5d9fe18fca48a910c1085b5f22122
S: aeee686e355fa7784b383fec3543232126dcfbad 172.16.0.158:6379
slots: (0 slots) slave
replicates 4c429a48054a771cbc154319182a3d16cf4ce7a1
M: 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e 172.16.0.28:6379
slots:[6553-10922] (4370 slots) master
1 additional replica(s)
S: c4af78bc4a26490d51edc78b6c547a7abaf4e1aa 172.16.0.58:6379
slots: (0 slots) slave
replicates 4c429a48054a771cbc154319182a3d16cf4ce7a1
S: a7583f69703921c6b3d14a97f54b3015966155ab 172.16.0.118:6379
slots: (0 slots) slave
replicates d5462f6961c0f45ecbdf12d6606e6993c33e3e29
S: 3d69cddc61df9443ff7de9850c220fc9e9187c03 172.16.0.138:6379
slots: (0 slots) slave
replicates 4c429a48054a771cbc154319182a3d16cf4ce7a1
M: aaa956c280b5d9fe18fca48a910c1085b5f22122 172.16.0.48:6379
slots:[0-1092],[5462-6552],[10923-12013] (3275 slots) master
1 additional replica(s)
S: 3bbdbc3ab34b67161655974fed9de5667def8ed0 172.16.0.128:6379
slots: (0 slots) slave
replicates 5163e9abbf42bd3540d9c04f6fb384ea23a1f58e
M: 4c429a48054a771cbc154319182a3d16cf4ce7a1 172.16.0.38:6379
slots:[12014-16383] (4370 slots) master
3 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@CentOS84-IP172-18 ]#
###################################################################################
## 在IP58上也看到自动成为其他的主master redis 的从节点；IP158也自动将成了其他 主master redis 的从节点了
[root@CentOS84-IP172-58 ]#redis-cli -a 123456 --no-auth-warning info replication
# Replication
role:slave
master_host:172.16.0.38
master_port:6379
master_link_status:up
master_last_io_seconds_ago:9
master_sync_in_progress:0
slave_read_repl_offset:222208
slave_repl_offset:222208
slave_priority:100
slave_read_only:1
replica_announced:1
connected_slaves:0
master_failover_state:no-failover
master_replid:348918a6f1e9fc1b9eab3289584760767ed883b4
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:222208
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:221957
repl_backlog_histlen:252
[root@CentOS84-IP172-58 ]#
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:10 #集群中还有10个节点
cluster_size:4 #只有4对主从
cluster_current_epoch:14
cluster_my_epoch:12
cluster_stats_messages_ping_sent:160157
cluster_stats_messages_pong_sent:171005
cluster_stats_messages_sent:331162
cluster_stats_messages_ping_received:161182
cluster_stats_messages_pong_received:168889
cluster_stats_messages_meet_received:4
cluster_stats_messages_update_received:2
cluster_stats_messages_received:330077
total_cluster_links_buffer_limit_exceeded:0
[root@CentOS84-IP172-18 ]#
#### 至此IP58上的所有槽位已经全部迁移走了，可以从集群中删除主机了
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK