Docker集群部署Redis

2个月前 • 119次点击 • 来自 Linux

1. 创建网络

--subnet用于指定子网（可选）

docker network create redis --subnet 172.18.0.0/16
docker network inspect redis

输出结果显示成功

[
    {
        "Name": "redis",
        "Id": "1a707b2b6b7bbd6ee2ed846b06b3aa5767197d7578fa373bde4a905f3c2bd26f",
        "Created": "2022-03-10T11:46:21.462969636+08:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.18.0.0/16",
                    "Gateway": "172.18.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {},
        "Labels": {}
    }
]

2. 使用脚本创建redis集群配置

Redis集群至少需要3个master节点，1个master至少对应1个slave

1master-1slave，redis集群需要6个节点
1master-2slave，redis集群需要9个节点
以此类推
验证方式：

./redis-trib.rb create --replicas 2 192.168.30.10:7000 192.168.30.11:7003 192.168.30.10:7002 192.168.30.10:7001 192.168.30.11:7004 192.168.30.11:7005
>>> Creating cluster
*** ERROR: Invalid configuration for cluster creation.
*** Redis Cluster requires at least 3 master nodes.
*** This is not possible with 6 nodes and 2 replicas per node.  #解释了，对于6个节点，不可能是3个master，每个有2个副本节点。因为，那样的结构需要9个节点。
*** At least 9 nodes are required.                              #至少需要9个节点。

使用脚本创建配置：

for port in {1..6}; 
do
mkdir -p /mydata/redis/node-${port}/conf
touch /mydata/redis/node-${port}/conf/redis.conf
cat << EOF >/mydata/redis/node-${port}/conf/redis.conf
port 6379
bind 0.0.0.0
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.18.0.1${port}
cluster-announce-port 6379
cluster-announce-bus-port 16379
appendonly yes
EOF
done

通过cat /mydata/redis/node-1/conf/redis.conf查看配置：

port 6379
bind 0.0.0.0
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.18.0.11
cluster-announce-port 6379
cluster-announce-bus-port 16379
appendonly yes

3. 启动redis

测试启动一台 redis-1

docker run -d \
-p 6371:6379 -p 16371:16379 --name redis-1 \
--net redis --ip 172.18.0.11 \
-v /mydata/redis/node-1/data:/data \
-v /mydata/redis/node-1/conf/redis.conf:/etc/redis/redis.conf \
redis:latest redis-server /etc/redis/redis.conf

启动正常

root@VM-4-8-ubuntu:~# docker ps
CONTAINER ID   IMAGE          COMMAND                  CREATED          STATUS          PORTS                                                                                      NAMES
2f24b222c7f0   redis:latest   "docker-entrypoint.s…"   37 seconds ago   Up 36 seconds   0.0.0.0:6371->6379/tcp, :::6371->6379/tcp, 0.0.0.0:16371->16379/tcp, :::16371->16379/tcp   redis-1

4. 使用脚本启动redis

步骤3成功启动redis-1,再使用一下启动剩余的5台(redis-2 至 redis-6）

for port in {2..6};\
do
docker run -d \
-p 637${port}:6379 -p 1637${port}:16379 --name redis-${port} \
--net redis --ip 172.18.0.1${port} \
-v /mydata/redis/node-${port}/data:/data \
-v /mydata/redis/node-${port}/conf/redis.conf:/etc/redis/redis.conf \
redis:latest redis-server /etc/redis/redis.conf
done

5. 建立集群

#进入redis-1容器
docker exec -it redis-1 /bin/sh

#创建集群
redis-cli --cluster create \
172.18.0.11:6379 172.18.0.12:6379 172.18.0.13:6379 172.18.0.14:6379 172.18.0.15:6379 172.18.0.16:6379 \
--cluster-replicas 1

提醒自动配置主从，输入yes即可

>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.18.0.15:6379 to 172.18.0.11:6379
Adding replica 172.18.0.16:6379 to 172.18.0.12:6379
Adding replica 172.18.0.14:6379 to 172.18.0.13:6379
M: 14e368fedee90602614d654a53b6338c2a64eebe 172.18.0.11:6379
   slots:[0-5460] (5461 slots) master
M: 455fed6c160891326b0cf0f056afb6dfad3e643f 172.18.0.12:6379
   slots:[5461-10922] (5462 slots) master
M: 91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2 172.18.0.13:6379
   slots:[10923-16383] (5461 slots) master
S: a5f5d1abd9504043f7e55afbafb56f15023ce86a 172.18.0.14:6379
   replicates 91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2
S: 06340f407322d57a1b1dbd492c0735dadcd88689 172.18.0.15:6379
   replicates 14e368fedee90602614d654a53b6338c2a64eebe
S: 97ef1ba882968a8abaf26e16d007bb73fbf0fb92 172.18.0.16:6379
   replicates 455fed6c160891326b0cf0f056afb6dfad3e643f
Can I set the above configuration? (type 'yes' to accept):

显示成功建立集群

>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 172.18.0.11:6379)
M: 14e368fedee90602614d654a53b6338c2a64eebe 172.18.0.11:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2 172.18.0.13:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
S: 06340f407322d57a1b1dbd492c0735dadcd88689 172.18.0.15:6379
   slots: (0 slots) slave
   replicates 14e368fedee90602614d654a53b6338c2a64eebe
S: 97ef1ba882968a8abaf26e16d007bb73fbf0fb92 172.18.0.16:6379
   slots: (0 slots) slave
   replicates 455fed6c160891326b0cf0f056afb6dfad3e643f
M: 455fed6c160891326b0cf0f056afb6dfad3e643f 172.18.0.12:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: a5f5d1abd9504043f7e55afbafb56f15023ce86a 172.18.0.14:6379
   slots: (0 slots) slave
   replicates 91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

6. 集群查看

redis-cli -c
cluster info 
#输出
127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:137
cluster_stats_messages_pong_sent:131
cluster_stats_messages_sent:268
cluster_stats_messages_ping_received:126
cluster_stats_messages_pong_received:137
cluster_stats_messages_meet_received

cluster nodes
#输出
127.0.0.1:6379> cluster nodes
91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2 172.18.0.13:6379@16379 master - 0 1646888713863 3 connected 10923-16383
06340f407322d57a1b1dbd492c0735dadcd88689 172.18.0.15:6379@16379 slave 14e368fedee90602614d654a53b6338c2a64eebe 0 1646888714364 1 connected
97ef1ba882968a8abaf26e16d007bb73fbf0fb92 172.18.0.16:6379@16379 slave 455fed6c160891326b0cf0f056afb6dfad3e643f 0 1646888713000 2 connected
455fed6c160891326b0cf0f056afb6dfad3e643f 172.18.0.12:6379@16379 master - 0 1646888713562 2 connected 5461-10922
a5f5d1abd9504043f7e55afbafb56f15023ce86a 172.18.0.14:6379@16379 slave 91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2 0 1646888714000 3 connected
14e368fedee90602614d654a53b6338c2a64eebe 172.18.0.11:6379@16379 myself,master - 0 1646888714000 1 connected 0-5460

7. 测试主从复制

127.0.0.1:6379> set k1 test1
-> Redirected to slot [12706] located at 172.18.0.13:6379 #自动重定向到了 172.18.0.13 即redis-3完成本次操作
OK
172.18.0.13:6379> get k1
"test1"

停止redis-3容器，模拟宕机

docker stop redis-3

测试

172.18.0.13:6379> get k1
Error: Server closed the connection #此时redis-3容器已经模拟宕机

再次进入

redis-cli -c
127.0.0.1:6379> get k1
-> Redirected to slot [12706] located at 172.18.0.14:6379 #重定向至redis-4 
"test1"
172.18.0.14:6379> cluster nodes #当前依旧有3台master 因为redis-3状态依旧fail redis-4提升为master
a5f5d1abd9504043f7e55afbafb56f15023ce86a 172.18.0.14:6379@16379 myself,master - 0 1646889611000 7 connected 10923-16383
06340f407322d57a1b1dbd492c0735dadcd88689 172.18.0.15:6379@16379 slave 14e368fedee90602614d654a53b6338c2a64eebe 0 1646889612000 1 connected
455fed6c160891326b0cf0f056afb6dfad3e643f 172.18.0.12:6379@16379 master - 0 1646889612097 2 connected 5461-10922
14e368fedee90602614d654a53b6338c2a64eebe 172.18.0.11:6379@16379 master - 0 1646889611095 1 connected 0-5460
91e2a2d8fc3c2214a7553cf0731fb2e3a432d3f2 172.18.0.13:6379@16379 master,fail - 1646889404046 1646889401537 3 connected
97ef1ba882968a8abaf26e16d007bb73fbf0fb92 172.18.0.16:6379@16379 slave 455fed6c160891326b0cf0f056afb6dfad3e643f 0 1646889613100 2 connected