Docker集群部署Redis
2个月前 • 119次点击 • 来自 Linux
标签: Docker
1. 创建网络
--subnet
用于指定子网(可选)
docker network create redis --subnet 172.18.0.0/16
docker network inspect redis
输出结果显示成功
[
{
"Name": "redis",
"Id": "1a707b2b6b7bbd6ee2ed846b06b3aa5767197d7578fa373bde4a905f3c2bd26f",
"Created": "2022-03-10T11:46:21.462969636+08:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "172.18.0.0/16",
"Gateway": "172.18.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {},
"Options": {},
"Labels": {}
}
]
2. 使用脚本创建redis集群配置
Redis集群至少需要3个master节点,1个master至少对应1个slave
- 1master-1slave,redis集群需要6个节点
- 1master-2slave,redis集群需要9个节点
- 以此类推
验证方式:
./redis-trib.rb create --replicas 2 192.168.30.10:7000 192.168.30.11:7003 192.168.30.10:7002 192.168.30.10:7001 192.168.30.11:7004 192.168.30.11:7005
>>> Creating cluster
*** ERROR: Invalid configuration for cluster creation.
*** Redis Cluster requires at least 3 master nodes.
*** This is not possible with 6 nodes and 2 replicas per node. #解释了,对于6个节点,不可能是3个master,每个有2个副本节点。因为,那样的结构需要9个节点。
*** At least 9 nodes are required. #至少需要9个节点。
使用脚本创建配置:
for port in {1..6};
do
mkdir -p /mydata/redis/node-${port}/conf
touch /mydata/redis/node-${port}/conf/redis.conf
cat << EOF >/mydata/redis/node-${port}/conf/redis.conf
port 6379
bind 0.0.0.0
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.18.0.1${port}
cluster-announce-port 6379
cluster-announce-bus-port 16379
appendonly yes
EOF
done
通过cat /mydata/redis/node-1/conf/redis.conf
查看配置:
port 6379
bind 0.0.0.0
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.18.0.11
cluster-announce-port 6379
cluster-announce-bus-port 16379
appendonly yes
3. 启动redis
测试启动一台 redis-1
docker run -d \
-p 6371:6379 -p 16371:16379 --name redis-1 \
--net redis --ip 172.18.0.11 \
-v /mydata/redis/node-1/data:/data \
-v /mydata/redis/node-1/conf/redis.conf:/etc/redis/redis.conf \
redis:latest redis-server /etc/redis/redis.conf
启动正常
root@VM-4-8-ubuntu:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2f24b222c7f0 redis:latest "docker-entrypoint.s…" 37 seconds ago Up 36 seconds 0.0.0.0:6371->6379/tcp, :::6371->6379/tcp, 0.0.0.0:16371->16379/tcp, :::16371->16379/tcp redis-1
4. 使用脚本启动redis
步骤3成功启动redis-1
,再使用一下启动剩余的5台(redis-2
至 redis-6
)
for port in {2..6};\
do
docker run -d \
-p 637${port}:6379 -p 1637${port}:16379 --name redis-${port} \
--net redis --ip 172.18.0.1${port} \
-v /mydata/redis/node-${port}/data:/data \
-v /mydata/redis/node-${port}/conf/redis.conf:/etc/redis/redis.conf \
redis:latest redis-server /etc/redis/redis.conf
done
5. 建立集群
#进入redis-1容器
docker exec -it redis-1 /bin/sh
#创建集群
redis-cli --cluster create \
172.18.0.11:6379 172.18.0.12:6379 172.18.0.13:6379 172.18.0.14:6379 172.18.0.15:6379 172.18.0.16:6379 \
--cluster-replicas 1
提醒自动配置主从,输入yes即可
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.18.0.15:6379 to 172.18.0.11:6379
Adding replica 172.18.0.16:6379 to 172.18.0.12:6379
Adding replica 172.18.0.14:6379 to 172.18.0.13:6379
M: 14e368fedee90602614d654a53b6338c2a64eebe 172.18.0.11:6379
slots:[0-5460] (5461 slots) master
M: 455fed6c160891326b0cf0f056afb6dfad3e643f 172.18.0.12:6379
slots:[5461-10922] (5462 slots) master
M: 91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2 172.18.0.13:6379
slots:[10923-16383] (5461 slots) master
S: a5f5d1abd9504043f7e55afbafb56f15023ce86a 172.18.0.14:6379
replicates 91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2
S: 06340f407322d57a1b1dbd492c0735dadcd88689 172.18.0.15:6379
replicates 14e368fedee90602614d654a53b6338c2a64eebe
S: 97ef1ba882968a8abaf26e16d007bb73fbf0fb92 172.18.0.16:6379
replicates 455fed6c160891326b0cf0f056afb6dfad3e643f
Can I set the above configuration? (type 'yes' to accept):
显示成功建立集群
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 172.18.0.11:6379)
M: 14e368fedee90602614d654a53b6338c2a64eebe 172.18.0.11:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2 172.18.0.13:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 06340f407322d57a1b1dbd492c0735dadcd88689 172.18.0.15:6379
slots: (0 slots) slave
replicates 14e368fedee90602614d654a53b6338c2a64eebe
S: 97ef1ba882968a8abaf26e16d007bb73fbf0fb92 172.18.0.16:6379
slots: (0 slots) slave
replicates 455fed6c160891326b0cf0f056afb6dfad3e643f
M: 455fed6c160891326b0cf0f056afb6dfad3e643f 172.18.0.12:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: a5f5d1abd9504043f7e55afbafb56f15023ce86a 172.18.0.14:6379
slots: (0 slots) slave
replicates 91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
6. 集群查看
redis-cli -c
cluster info
#输出
127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:137
cluster_stats_messages_pong_sent:131
cluster_stats_messages_sent:268
cluster_stats_messages_ping_received:126
cluster_stats_messages_pong_received:137
cluster_stats_messages_meet_received
cluster nodes
#输出
127.0.0.1:6379> cluster nodes
91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2 172.18.0.13:6379@16379 master - 0 1646888713863 3 connected 10923-16383
06340f407322d57a1b1dbd492c0735dadcd88689 172.18.0.15:6379@16379 slave 14e368fedee90602614d654a53b6338c2a64eebe 0 1646888714364 1 connected
97ef1ba882968a8abaf26e16d007bb73fbf0fb92 172.18.0.16:6379@16379 slave 455fed6c160891326b0cf0f056afb6dfad3e643f 0 1646888713000 2 connected
455fed6c160891326b0cf0f056afb6dfad3e643f 172.18.0.12:6379@16379 master - 0 1646888713562 2 connected 5461-10922
a5f5d1abd9504043f7e55afbafb56f15023ce86a 172.18.0.14:6379@16379 slave 91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2 0 1646888714000 3 connected
14e368fedee90602614d654a53b6338c2a64eebe 172.18.0.11:6379@16379 myself,master - 0 1646888714000 1 connected 0-5460
7. 测试主从复制
127.0.0.1:6379> set k1 test1
-> Redirected to slot [12706] located at 172.18.0.13:6379 #自动重定向到了 172.18.0.13 即redis-3完成本次操作
OK
172.18.0.13:6379> get k1
"test1"
停止redis-3容器,模拟宕机
docker stop redis-3
测试
172.18.0.13:6379> get k1
Error: Server closed the connection #此时redis-3容器已经模拟宕机
再次进入
redis-cli -c
127.0.0.1:6379> get k1
-> Redirected to slot [12706] located at 172.18.0.14:6379 #重定向至redis-4
"test1"
172.18.0.14:6379> cluster nodes #当前依旧有3台master 因为redis-3状态依旧fail redis-4提升为master
a5f5d1abd9504043f7e55afbafb56f15023ce86a 172.18.0.14:6379@16379 myself,master - 0 1646889611000 7 connected 10923-16383
06340f407322d57a1b1dbd492c0735dadcd88689 172.18.0.15:6379@16379 slave 14e368fedee90602614d654a53b6338c2a64eebe 0 1646889612000 1 connected
455fed6c160891326b0cf0f056afb6dfad3e643f 172.18.0.12:6379@16379 master - 0 1646889612097 2 connected 5461-10922
14e368fedee90602614d654a53b6338c2a64eebe 172.18.0.11:6379@16379 master - 0 1646889611095 1 connected 0-5460
91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2 172.18.0.13:6379@16379 master,fail - 1646889404046 1646889401537 3 connected
97ef1ba882968a8abaf26e16d007bb73fbf0fb92 172.18.0.16:6379@16379 slave 455fed6c160891326b0cf0f056afb6dfad3e643f 0 1646889613100 2 connected