Docker实现Redis集群创建

柔光的暖阳◎ 2023-10-04 19:26 27阅读 0赞

**目录**

一，redis集群介绍

二，Docker搭建一个简单的cluster集群

2.1，创建一个独立网段

2.2，创建三主三从的redis配置

2.3，通过密码配置集群

2.4，登录集群

2.5，测试集群效果

--------------------

## 一，redis集群介绍 ##

redis集群是一个由多个主从节点群组成的分布式服务器群，它具有复制、高可用和分片特性。Redis集群不需 要sentinel哨兵∙也能完成节点移除和故障转移的功能。需要将每个节点设置成集群模式，这种集群模式没有中心节点，可水平扩展。

redis集群的性能和高可用性均优于之前版本的哨兵模式，且集群配置非常简单。一个Redis Cluster由多个Redis节点构成，不同节点组服务的数据没有交集，也就是每个一节点组对应数据sharding的一个分片。节点组内部分为主备两类节点，对应master和slave节点。两者数据准实时一致，通过异步化订的主从复制机制来保证。一个节点组有且只有一个master节点，同时可以有0到多个slave节点，在这个节点组中只有master节点对用户提供些服务，读服务可以由master或者slave提供。

![2e9687be80474231aca0a1666ae49dbd.png][]

## 二，Docker搭建一个简单的cluster集群 ##

### 2.1，创建一个独立网段 ###

由于当docker创建容器的时候，会自动给容器分配IP地址，容器得到的IP地址是动态的，当下次启动容器的时候，之前的IP就不能用了，为了解决这个问题，就要给容器分配固定的IP地址。

因此我们需要创建一个独立的网段。但是发现被目标网段已经被占用了。

[root@VM-12-4-centos ~]# docker network create redis-cluster --subnet 172.17.0.0/16
    Error response from daemon: Pool overlaps with other one on this address space

> ***\*注意：默认情况下，docker默认创建的网段为172.17.0.X，因此创建的容器都是放在这个网段之中。\****

> ***\*注意：172.18.0.1是网段的网关地址，是不能用的\****

更换一个网段继续创建，可以看到网段创建成功

[root@VM-12-4-centos ~]# docker network create redis-cluster --subnet 172.18.0.0/16
    d64d0e79c1b03b6c57e59eb27311c4f1c2fddc98036f45bd42a4d7e59800f903

> docker network create --subnet=172.18.0.0/16 mynet --->创建网段  
> docker network rm mynet -->删除网段

### 2.2，创建三主三从的redis配置 ###

为了避免从大量的重复操作，在这里我们直接通过脚本实现六个测试节点的构建

#通过脚本创建六个redis配置
    [root@VM-12-4-centos ~]# for port in $(seq 2  7); \
    do \
    mkdir -p /www/server/redis/redis-cluster-test/node-${port}/conf
    touch /www/server/redis/redis-cluster-test/node-${port}/conf/redis.conf
    cat << EOF >/www/server/redis/redis-cluster-test/node-${port}/conf/redis.conf
    port 6379
    bind 0.0.0.0
    cluster-enabled yes
    cluster-node-timeout 5000
    cluster-config-file nodes.conf
    cluster-announce-ip 172.18.0.2${port}
    cluster-announce-port 6379
    cluster-announce-bus-port 16379
    masterauth 123456
    requirepass 123456
    appendonly yes
    EOF
    done

进入redis目录查看一下节点和相应的文件是否创建好

[root@VM-12-4-centos redis]# ls
    00-RELEASENOTES  MANIFESTO           runtest-sentinel
    backup.db        README.md           sentinel.conf
    BUGS             redis-cluster-test  src
    CONDUCT          redis.conf          start.pl
    CONTRIBUTING     redis.log           tests
    COPYING          redis.pid           TLS.md
    deps             root                utils
    dump.rdb         runtest             version_check.pl
    INSTALL          runtest-cluster     version.pl
    Makefile         runtest-moduleapi
    [root@VM-12-4-centos redis]# cd redis-cluster-test
    [root@VM-12-4-centos redis-cluster-test]# ls
    node-2  node-3  node-4  node-5  node-6  node-7
    [root@VM-12-4-centos redis-cluster-test]# cd node-2
    [root@VM-12-4-centos node-2]# ls
    conf
    [root@VM-12-4-centos node-2]# cd conf
    [root@VM-12-4-centos conf]# ls
    redis.conf
    [root@VM-12-4-centos conf]# cat redis.conf
    port 6379
    bind 0.0.0.0
    cluster-enabled yes
    cluster-node-timeout 5000
    cluster-config-file nodes.conf
    cluster announce-ip 172.18.0.22
    cluster-announce-port 6379
    cluster-announce-bus-port 16379
    masterauth 123456
    requirepass 123456
    appendonly yes

可以看到创建完成了。

在逐个docker上启动每个节点并查看启动是否成功

[root@VM-12-4-centos ~]# docker run -p 6372:6379 -p 16372:16379 --name redis-2 -v /www/server/redis/redis-cluster-test/node-2/data:/data  -v /www/server/redis/redis-cluster-test/node-2/conf/redis.conf:/etc/redis/redis.conf  -d --net redis-cluster --ip 172.18.0.22 redis:6.2.6 redis-server /etc/redis/redis.conf
    Unable to find image 'redis:6.2.6' locally
    6.2.6: Pulling from library/redis
    1fe172e4850f: Pull complete 
    6fbcd347bf99: Pull complete 
    993114c67627: Pull complete 
    2a560260ca39: Pull complete 
    b7179886a292: Pull complete 
    8901ffe2be84: Pull complete 
    Digest: sha256:b7fd1a2c89d09a836f659d72c52d27b9f71202c97014a47639f87c992e8c0f1b
    Status: Downloaded newer image for redis:6.2.6
    d6c9afaddd503aed70cf139d53483fc61903366ea8e8fdbbe70f5eaccc87b6f3

最后检查redis6个节点的容器是否运行成功

通过/bin/sh名可以查看aop与节点的配置

[root@VM-12-4-centos ~]# docker exec -it redis-2 /bin/sh
    # ls
    appendonly.aof	nodes.conf

### 2.3，通过密码配置集群 ###

# redis-cli -a 123456 --cluster create 172.18.0.22:6379 172.18.0.23:6379 172.18.0.24:6379 172.18.0.25:6379 172.18.0.26:6379 172.18.0.27:6379 --cluster-replicas 1
    Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.

然后就可以看到集群正在配置，期间会询问我们是否建立集群，输入yes即可

>>> Performing hash slots allocation on 6 nodes...
    Master[0] -> Slots 0 - 5460
    Master[1] -> Slots 5461 - 10922
    Master[2] -> Slots 10923 - 16383
    Adding replica 172.18.0.26:6379 to 172.18.0.22:6379
    Adding replica 172.18.0.27:6379 to 172.18.0.23:6379
    Adding replica 172.18.0.25:6379 to 172.18.0.24:6379
    M: cb84a2c5dc854d80915c809d76c2923bc273c5d8 172.18.0.22:6379
       slots:[0-5460] (5461 slots) master
    M: d41fc0f4067a431a8791ab0e6ea0c508da6ed56c 172.18.0.23:6379
       slots:[5461-10922] (5462 slots) master
    M: 8dfc40acffbd43dc49499c632feaa613ee96eca7 172.18.0.24:6379
       slots:[10923-16383] (5461 slots) master
    S: 800c4e41e6dbd6d0dbbdcdf8fcbe634cf1e9a66d 172.18.0.25:6379
       replicates 8dfc40acffbd43dc49499c632feaa613ee96eca7
    S: acd531b63b5a0579e0d0b2e535ae49b0becd6526 172.18.0.26:6379
       replicates cb84a2c5dc854d80915c809d76c2923bc273c5d8
    S: 1070f77ee02e09a9e6e4b0cd6d0e7891b79906be 172.18.0.27:6379
       replicates d41fc0f4067a431a8791ab0e6ea0c508da6ed56c
    Can I set the above configuration? (type 'yes' to accept): yes  #输入yes
    >>> Nodes configuration updated
    >>> Assign a different config epoch to each node
    >>> Sending CLUSTER MEET messages to join the cluster
    Waiting for the cluster to join
    .
    >>> Performing Cluster Check (using node 172.18.0.22:6379)
    M: cb84a2c5dc854d80915c809d76c2923bc273c5d8 172.18.0.22:6379
       slots:[0-5460] (5461 slots) master
       1 additional replica(s)
    S: 1070f77ee02e09a9e6e4b0cd6d0e7891b79906be 172.18.0.27:6379
       slots: (0 slots) slave
       replicates d41fc0f4067a431a8791ab0e6ea0c508da6ed56c
    S: acd531b63b5a0579e0d0b2e535ae49b0becd6526 172.18.0.26:6379
       slots: (0 slots) slave
       replicates cb84a2c5dc854d80915c809d76c2923bc273c5d8
    S: 800c4e41e6dbd6d0dbbdcdf8fcbe634cf1e9a66d 172.18.0.25:6379
       slots: (0 slots) slave
       replicates 8dfc40acffbd43dc49499c632feaa613ee96eca7
    M: 8dfc40acffbd43dc49499c632feaa613ee96eca7 172.18.0.24:6379
       slots:[10923-16383] (5461 slots) master
       1 additional replica(s)
    M: d41fc0f4067a431a8791ab0e6ea0c508da6ed56c 172.18.0.23:6379
       slots:[5461-10922] (5462 slots) master
       1 additional replica(s)
    [OK] All nodes agree about slots configuration.
    >>> Check for open slots...
    >>> Check slots coverage...
    [OK] All 16384 slots covered.

### 2.4，登录集群 ###

通过redis-cli -c登录集群，注意**必须添上-c**否则登入的就不是集群

# redis-cli -a 123456 -c
    Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
    127.0.0.1:6379>

查看一下集群信息，可以发现当前三主三从已经建立好了。

172.18.0.23:6379> cluster info
    cluster_state:ok
    cluster_slots_assigned:16384
    cluster_slots_ok:16384
    cluster_slots_pfail:0
    cluster_slots_fail:0
    cluster_known_nodes:6
    cluster_size:3
    cluster_current_epoch:6
    cluster_my_epoch:2
    cluster_stats_messages_ping_sent:577
    cluster_stats_messages_pong_sent:578
    cluster_stats_messages_meet_sent:1
    cluster_stats_messages_sent:1156
    cluster_stats_messages_ping_received:578
    cluster_stats_messages_pong_received:578
    cluster_stats_messages_received:1156
    172.18.0.23:6379> cluster nodes
    8dfc40acffbd43dc49499c632feaa613ee96eca7 172.18.0.24:6379@16379 master - 0 1663488343214 3 connected 10923-16383 #
    d41fc0f4067a431a8791ab0e6ea0c508da6ed56c 172.18.0.23:6379@16379 myself,master - 0 1663488343000 2 connected 5461-10922
    cb84a2c5dc854d80915c809d76c2923bc273c5d8 172.18.0.22:6379@16379 master - 0 1663488342712 1 connected 0-5460
    1070f77ee02e09a9e6e4b0cd6d0e7891b79906be 172.18.0.27:6379@16379 slave d41fc0f4067a431a8791ab0e6ea0c508da6ed56c 0 1663488342211 2 connected #从机
    acd531b63b5a0579e0d0b2e535ae49b0becd6526 172.18.0.26:6379@16379 slave cb84a2c5dc854d80915c809d76c2923bc273c5d8 0 1663488342612 1 connected
    800c4e41e6dbd6d0dbbdcdf8fcbe634cf1e9a66d 172.18.0.25:6379@16379 slave 8dfc40acffbd43dc49499c632feaa613ee96eca7 0 1663488341000 3 connected

### 2.5，测试集群效果 ###

在redis里设置一些简单数据

127.0.0.1:6379> set name young
    -> Redirected to slot [5798] located at 172.18.0.23:6379
    OK

当我们set一个值后，发现目前是主机node-3处理的当前信息。如果我们此时node-3挂掉了，信息是否还能获取到呢。模拟挂掉node-3 。

[root@VM-12-4-centos redis]# docker stop redis-3
    redis-3
    [root@VM-12-4-centos redis]# docker ps -a
    CONTAINER ID   IMAGE                 COMMAND                  CREATED          STATUS                      PORTS                                                                                                                                                 NAMES
    694afc3d9ab6   redis:6.2.6           "docker-entrypoint.s…"   21 minutes ago   Up 21 minutes               0.0.0.0:6377->6379/tcp, :::6377->6379/tcp, 0.0.0.0:16377->16379/tcp, :::16377->16379/tcp                                                              redis-7
    8178f2cace42   redis:6.2.6           "docker-entrypoint.s…"   21 minutes ago   Up 21 minutes               0.0.0.0:6376->6379/tcp, :::6376->6379/tcp, 0.0.0.0:16376->16379/tcp, :::16376->16379/tcp                                                              redis-6
    95ce3c2d9422   redis:6.2.6           "docker-entrypoint.s…"   22 minutes ago   Up 22 minutes               0.0.0.0:6375->6379/tcp, :::6375->6379/tcp, 0.0.0.0:16375->16379/tcp, :::16375->16379/tcp                                                              redis-5
    42fac216bfff   redis:6.2.6           "docker-entrypoint.s…"   22 minutes ago   Up 22 minutes               0.0.0.0:6374->6379/tcp, :::6374->6379/tcp, 0.0.0.0:16374->16379/tcp, :::16374->16379/tcp                                                              redis-4
    3781f50872b7   redis:6.2.6           "docker-entrypoint.s…"   23 minutes ago   Exited (0) 17 seconds ago                                                                                                                                                         redis-3
    d813587b992a   redis:6.2.6           "docker-entrypoint.s…"   28 minutes ago   Up 28 minutes               0.0.0.0:6372->6379/tcp, :::6372->6379/tcp, 0.0.0.0:16372->16379/tcp, :::16372->16379/tcp                                                              redis-2
    24be9bbde112   rabbitmq:management   "docker-entrypoint.s…"   5 weeks ago      Up 5 weeks                  4369/tcp, 5671/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:5675->5672/tcp, :::5675->5672/tcp, 0.0.0.0:15675->15672/tcp, :::15675->15672/tcp   rabbit3
    6c21c2595eaa   rabbitmq:management   "docker-entrypoint.s…"   5 weeks ago      Up 5 weeks                  4369/tcp, 5671/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:5674->5672/tcp, :::5674->5672/tcp, 0.0.0.0:15674->15672/tcp, :::15674->15672/tcp   rabbit2
    226789a013be   rabbitmq:management   "docker-entrypoint.s…"   5 weeks ago      Up 5 weeks                  4369/tcp, 5671/tcp, 15671/tcp, 15691-15692/tcp, 25672/tcp, 0.0.0.0:5673->5672/tcp, :::5673->5672/tcp, 0.0.0.0:15673->15672/tcp, :::15673->15672/tcp   rabbit1

此时可以看到node-3的服务已经Exited！然后我们再去获取刚才set的值。可以发现值已经由从机node-7获取了。这样在业务中模拟搭建集群就能基本实现高可用的效果了。

127.0.0.1:6379> get name
    -> Redirected to slot [5798] located at 172.18.0.27:6379
    "young"
    172.18.0.27:6379> cluster nodes
    800c4e41e6dbd6d0dbbdcdf8fcbe634cf1e9a66d 172.18.0.25:6379@16379 slave 8dfc40acffbd43dc49499c632feaa613ee96eca7 0 1663489199146 3 connected
    1070f77ee02e09a9e6e4b0cd6d0e7891b79906be 172.18.0.27:6379@16379 myself,master - 0 1663489198000 7 connected 5461-10922 #经过选举后redis-7成为了主机
    cb84a2c5dc854d80915c809d76c2923bc273c5d8 172.18.0.22:6379@16379 master - 0 1663489200000 1 connected 0-5460
    d41fc0f4067a431a8791ab0e6ea0c508da6ed56c 172.18.0.23:6379@16379 master,fail - 1663488784891 1663488783000 2 connected  #redis-3已嗝屁
    8dfc40acffbd43dc49499c632feaa613ee96eca7 172.18.0.24:6379@16379 master - 0 1663489200551 3 connected 10923-16383
    acd531b63b5a0579e0d0b2e535ae49b0becd6526 172.18.0.26:6379@16379 slave cb84a2c5dc854d80915c809d76c2923bc273c5d8 0 1663489200148 1 connected

后续有更好的一些方案也会分享出来……

[2e9687be80474231aca0a1666ae49dbd.png]: https://img-blog.csdnimg.cn/2e9687be80474231aca0a1666ae49dbd.png