|
|
|
|
|
<h1><center>Redis集群技术</center></h1>
|
|
|
|
|
|
|
|
|
|
|
|
------
|
|
|
|
|
|
|
|
|
|
|
|
**作者：行癫（盗版必究）**
|
|
|
|
|
|
|
|
|
|
|
|
## 一：Python连接Redis
|
|
|
|
|
|
|
|
|
|
|
|
#### 1.安装包
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
pip3 install redis
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
#### 2.连接
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
In [1]: import redis
|
|
|
|
|
|
|
|
|
|
|
|
In [2]: r = redis.Redis(host='10.18.42.174', port=6379)
|
|
|
|
|
|
|
|
|
|
|
|
In [3]: r.set("QF", "www.qfedu.com")
|
|
|
|
|
|
Out[3]: True
|
|
|
|
|
|
|
|
|
|
|
|
In [4]: r.get("QF")
|
|
|
|
|
|
Out[4]: b'www.qfedu.com'
|
|
|
|
|
|
|
|
|
|
|
|
In [5]: ret = r.get("QF")
|
|
|
|
|
|
|
|
|
|
|
|
In [6]: ret.decode()
|
|
|
|
|
|
Out[6]: 'www.qfedu.com'
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
注意：
|
|
|
|
|
|
|
|
|
|
|
|
关闭redis的保护模式：protected-mode no
|
|
|
|
|
|
|
|
|
|
|
|
#### 3.连接池
|
|
|
|
|
|
|
|
|
|
|
|
redis-py使用connection pool来管理对一个redis server的所有连接，避免每次建立、释放连接的开销
|
|
|
|
|
|
|
|
|
|
|
|
默认，每个Redis实例都会维护一个自己的连接池
|
|
|
|
|
|
|
|
|
|
|
|
可以直接建立一个连接池，然后作为参数传给Redis，这样就可以实现多个 Redis 实例共享一个连接池
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
In [12]: pool = redis.ConnectionPool(host='172.16.153.134', port=6379)
|
|
|
|
|
|
|
|
|
|
|
|
In [13]: rs = redis.Redis(connection_pool=pool)
|
|
|
|
|
|
|
|
|
|
|
|
In [14]: rs.set("foo", "bar")
|
|
|
|
|
|
Out[14]: True
|
|
|
|
|
|
|
|
|
|
|
|
In [15]: rs.get("foo")
|
|
|
|
|
|
Out[15]: b'bar'
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## 二：Redis主从复制集群
|
|
|
|
|
|
|
|
|
|
|
|
#### 1.Redis复制特性
|
|
|
|
|
|
|
|
|
|
|
|
使用异步复制
|
|
|
|
|
|
|
|
|
|
|
|
一个主服务器可以有多个从服务器
|
|
|
|
|
|
|
|
|
|
|
|
从服务器也可以有自己的从服务器
|
|
|
|
|
|
|
|
|
|
|
|
复制功能不会阻塞主服务器
|
|
|
|
|
|
|
|
|
|
|
|
可以通过复制功能来让主服务器免于执行持久化操作，由从服务器去执行持久化操作即可
|
|
|
|
|
|
|
|
|
|
|
|
关闭主服务器持久化时，复制功能的数据是安全的
|
|
|
|
|
|
|
|
|
|
|
|
当配置Redis复制功能时，强烈建议打开主服务器的持久化功能。；否则的话，由于延迟等问题应该要避免部署的服务自动拉起
|
|
|
|
|
|
|
|
|
|
|
|
#### 2.Redis主从复制原理
|
|
|
|
|
|
|
|
|
|
|
|
redis 主从同步有两种方式（或者所两个阶段）：全同步和部分同步
|
|
|
|
|
|
|
|
|
|
|
|
从刚刚连接的时候，进行全同步；全同步结束后，进行部分同步。当然，如果有需要，slave 在任何时候都可以发起全同步
|
|
|
|
|
|
|
|
|
|
|
|
主从同步的机制：
|
|
|
|
|
|
|
|
|
|
|
|
<img src="https://xingdian-image.oss-cn-beijing.aliyuncs.com/xingdian-image/image-20230526094614286.png" alt="image-20230526094614286" style="zoom:50%;" />
|
|
|
|
|
|
|
|
|
|
|
|
从服务器向主服务器发送 SYNC 命令
|
|
|
|
|
|
|
|
|
|
|
|
接到 SYNC 命令的主服务器会调用BGSAVE 命令，创建一个 RDB 文件，并使用缓冲区记录接下来执行的所有写命令
|
|
|
|
|
|
|
|
|
|
|
|
当主服务器执行完 BGSAVE 命令时，它会向从服务器发送 RDB 文件，而从服务器则会接收并载入这个文件
|
|
|
|
|
|
|
|
|
|
|
|
主服务器将缓冲区储存的所有写命令发送给从服务器执行
|
|
|
|
|
|
|
|
|
|
|
|
#### 3.Redis命令传播
|
|
|
|
|
|
|
|
|
|
|
|
在主从服务器完成同步之后，主服务器每执行一个写命令，会将被执行的写命令发送给从服务器执行，这个操作被称为“命令传播”
|
|
|
|
|
|
|
|
|
|
|
|
命令传播是一个持续的过程：只要复制仍在继续，命令传播就会一直进行，使得主从服务器的状态可以一直保持一致
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#### 4.Redis复制一致性问题
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
在读写分离环境下，客户端向主服务器发送写命令 SET n 10086，主服务器在执行这个写命令之后，向客户端返回回复，并将这个写命令传播给从服务器
|
|
|
|
|
|
|
|
|
|
|
|
接到回复的客户端继续向从服务器发送读命令 GET n ，并且因为网络状态的原因，客户端的 GET命令比主服务器传播的SET 命令更快到达了从服务器
|
|
|
|
|
|
|
|
|
|
|
|
因为从服务器键 n 的值还未被更新，所以客户端在从服务器读取到的将是一个错误（过期）的 n值
|
|
|
|
|
|
|
|
|
|
|
|
5.Redis复制安全性提升
|
|
|
|
|
|
|
|
|
|
|
|
主服务器只在有至少 N 个从服务器的情况下，才执行写操作从 Redis 2.8 开始， 为了保证数据的安全性， 可以通过配置， 让主服务器只在有至少 N 个当前已连接从服务器的情况下， 才执行写命令
|
|
|
|
|
|
|
|
|
|
|
|
不过， 因为 Redis 使用异步复制， 所以主服务器发送的写数据并不一定会被从服务器接收到， 因此， 数据丢失的可能性仍然是存在的
|
|
|
|
|
|
|
|
|
|
|
|
通过以下两个参数保证数据的安全：
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
min-slaves-to-write <number of slaves>
|
|
|
|
|
|
min-slaves-max-lag <number of seconds>
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
要求至少有1个slave，数据复制和同步的延迟不能超过10秒，如果说一旦所有的slave，数据复制和同步的延迟都超过了10秒钟，那么这个时候，master就不会再接收任何请求了
|
|
|
|
|
|
|
|
|
|
|
|
减少异步复制的数据丢失：
|
|
|
|
|
|
|
|
|
|
|
|
有了min-slaves-max-lag这个配置，就可以确保说，一旦slave复制数据和ack延时太长，就认为可能master宕机后损失的数据太多了，那么就拒绝写请求，这样可以把master宕机时由于部分数据未同步到slave导致的数据丢失降低的可控范围内
|
|
|
|
|
|
|
|
|
|
|
|
减少脑裂的数据丢失：
|
|
|
|
|
|
|
|
|
|
|
|
如果一个master出现了脑裂，跟其他slave丢了连接，那么上面两个配置可以确保说，如果不能继续给指定数量的slave发送数据，而且slave超过10秒没有给自己ack消息，那么就直接拒绝客户端的写请求，这样脑裂后的旧master就不会接受client的新数据，也就避免了数据丢失
|
|
|
|
|
|
|
|
|
|
|
|
总结：
|
|
|
|
|
|
|
|
|
|
|
|
上面的配置就确保了，如果跟任何一个slave丢了连接，在10秒后发现没有slave给自己ack，那么就拒绝新的写请求。因此在脑裂场景下，最多就丢失10秒的数据
|
|
|
|
|
|
|
|
|
|
|
|
#### 5.主从复制实战
|
|
|
|
|
|
|
|
|
|
|
|
实验环境
|
|
|
|
|
|
|
|
|
|
|
|
三台服务器,一台master,两台slave
|
|
|
|
|
|
|
|
|
|
|
|
部署
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
[root@xingdian /]# yum -y install http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
|
|
|
|
|
|
[root@xingdian /]# yum -y --enablerepo=remi install redis
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
修改配置文件
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
主配置:
|
|
|
|
|
|
[root@xingdian-server-20 /]# grep -Ev "^$|#" /etc/redis.conf
|
|
|
|
|
|
bind 172.17.0.5 127.0.0.1
|
|
|
|
|
|
protected-mode yes
|
|
|
|
|
|
port 6379
|
|
|
|
|
|
tcp-backlog 511
|
|
|
|
|
|
timeout 0
|
|
|
|
|
|
tcp-keepalive 300
|
|
|
|
|
|
daemonize no
|
|
|
|
|
|
supervised no
|
|
|
|
|
|
pidfile /var/run/redis_6379.pid
|
|
|
|
|
|
loglevel notice
|
|
|
|
|
|
logfile /var/log/redis/redis.log
|
|
|
|
|
|
databases 16
|
|
|
|
|
|
always-show-logo yes
|
|
|
|
|
|
save 900 1
|
|
|
|
|
|
save 300 10
|
|
|
|
|
|
save 60 10000
|
|
|
|
|
|
stop-writes-on-bgsave-error yes
|
|
|
|
|
|
rdbcompression yes
|
|
|
|
|
|
rdbchecksum yes
|
|
|
|
|
|
dbfilename dump.rdb
|
|
|
|
|
|
rdb-del-sync-files no
|
|
|
|
|
|
dir /var/lib/redis
|
|
|
|
|
|
replica-serve-stale-data yes
|
|
|
|
|
|
replica-read-only yes
|
|
|
|
|
|
repl-diskless-sync no
|
|
|
|
|
|
repl-diskless-sync-delay 5
|
|
|
|
|
|
repl-diskless-load disabled
|
|
|
|
|
|
repl-disable-tcp-nodelay no
|
|
|
|
|
|
replica-priority 100
|
|
|
|
|
|
acllog-max-len 128
|
|
|
|
|
|
requirepass redis
|
|
|
|
|
|
lazyfree-lazy-eviction no
|
|
|
|
|
|
lazyfree-lazy-expire no
|
|
|
|
|
|
lazyfree-lazy-server-del no
|
|
|
|
|
|
replica-lazy-flush no
|
|
|
|
|
|
lazyfree-lazy-user-del no
|
|
|
|
|
|
oom-score-adj no
|
|
|
|
|
|
oom-score-adj-values 0 200 800
|
|
|
|
|
|
appendonly no
|
|
|
|
|
|
appendfilename "appendonly.aof"
|
|
|
|
|
|
appendfsync everysec
|
|
|
|
|
|
no-appendfsync-on-rewrite no
|
|
|
|
|
|
auto-aof-rewrite-percentage 100
|
|
|
|
|
|
auto-aof-rewrite-min-size 64mb
|
|
|
|
|
|
aof-load-truncated yes
|
|
|
|
|
|
aof-use-rdb-preamble yes
|
|
|
|
|
|
lua-time-limit 5000
|
|
|
|
|
|
slowlog-log-slower-than 10000
|
|
|
|
|
|
slowlog-max-len 128
|
|
|
|
|
|
latency-monitor-threshold 0
|
|
|
|
|
|
notify-keyspace-events ""
|
|
|
|
|
|
hash-max-ziplist-entries 512
|
|
|
|
|
|
hash-max-ziplist-value 64
|
|
|
|
|
|
list-max-ziplist-size -2
|
|
|
|
|
|
list-compress-depth 0
|
|
|
|
|
|
set-max-intset-entries 512
|
|
|
|
|
|
zset-max-ziplist-entries 128
|
|
|
|
|
|
zset-max-ziplist-value 64
|
|
|
|
|
|
hll-sparse-max-bytes 3000
|
|
|
|
|
|
stream-node-max-bytes 4096
|
|
|
|
|
|
stream-node-max-entries 100
|
|
|
|
|
|
activerehashing yes
|
|
|
|
|
|
client-output-buffer-limit normal 0 0 0
|
|
|
|
|
|
client-output-buffer-limit replica 256mb 64mb 60
|
|
|
|
|
|
client-output-buffer-limit pubsub 32mb 8mb 60
|
|
|
|
|
|
hz 10
|
|
|
|
|
|
dynamic-hz yes
|
|
|
|
|
|
aof-rewrite-incremental-fsync yes
|
|
|
|
|
|
rdb-save-incremental-fsync yes
|
|
|
|
|
|
jemalloc-bg-thread yes
|
|
|
|
|
|
slave-1配置文件:
|
|
|
|
|
|
[root@xingdian-server-21 /]# grep -Ev "^$|#" /etc/redis.conf
|
|
|
|
|
|
bind 172.17.0.6
|
|
|
|
|
|
protected-mode yes
|
|
|
|
|
|
port 6379
|
|
|
|
|
|
tcp-backlog 511
|
|
|
|
|
|
timeout 0
|
|
|
|
|
|
tcp-keepalive 300
|
|
|
|
|
|
daemonize no
|
|
|
|
|
|
supervised no
|
|
|
|
|
|
pidfile /var/run/redis_6379.pid
|
|
|
|
|
|
loglevel notice
|
|
|
|
|
|
logfile /var/log/redis/redis.log
|
|
|
|
|
|
databases 16
|
|
|
|
|
|
always-show-logo yes
|
|
|
|
|
|
save 900 1
|
|
|
|
|
|
save 300 10
|
|
|
|
|
|
save 60 10000
|
|
|
|
|
|
stop-writes-on-bgsave-error yes
|
|
|
|
|
|
rdbcompression yes
|
|
|
|
|
|
rdbchecksum yes
|
|
|
|
|
|
dbfilename dump.rdb
|
|
|
|
|
|
rdb-del-sync-files no
|
|
|
|
|
|
dir /var/lib/redis
|
|
|
|
|
|
replicaof 172.17.0.5 6379
|
|
|
|
|
|
masterauth redis
|
|
|
|
|
|
replica-serve-stale-data yes
|
|
|
|
|
|
replica-read-only yes
|
|
|
|
|
|
repl-diskless-sync no
|
|
|
|
|
|
repl-diskless-sync-delay 5
|
|
|
|
|
|
repl-diskless-load disabled
|
|
|
|
|
|
repl-disable-tcp-nodelay no
|
|
|
|
|
|
replica-priority 100
|
|
|
|
|
|
acllog-max-len 128
|
|
|
|
|
|
requirepass redis
|
|
|
|
|
|
lazyfree-lazy-eviction no
|
|
|
|
|
|
lazyfree-lazy-expire no
|
|
|
|
|
|
lazyfree-lazy-server-del no
|
|
|
|
|
|
replica-lazy-flush no
|
|
|
|
|
|
lazyfree-lazy-user-del no
|
|
|
|
|
|
oom-score-adj no
|
|
|
|
|
|
oom-score-adj-values 0 200 800
|
|
|
|
|
|
appendonly no
|
|
|
|
|
|
appendfilename "appendonly.aof"
|
|
|
|
|
|
appendfsync everysec
|
|
|
|
|
|
no-appendfsync-on-rewrite no
|
|
|
|
|
|
auto-aof-rewrite-percentage 100
|
|
|
|
|
|
auto-aof-rewrite-min-size 64mb
|
|
|
|
|
|
aof-load-truncated yes
|
|
|
|
|
|
aof-use-rdb-preamble yes
|
|
|
|
|
|
lua-time-limit 5000
|
|
|
|
|
|
slowlog-log-slower-than 10000
|
|
|
|
|
|
slowlog-max-len 128
|
|
|
|
|
|
latency-monitor-threshold 0
|
|
|
|
|
|
notify-keyspace-events ""
|
|
|
|
|
|
hash-max-ziplist-entries 512
|
|
|
|
|
|
hash-max-ziplist-value 64
|
|
|
|
|
|
list-max-ziplist-size -2
|
|
|
|
|
|
list-compress-depth 0
|
|
|
|
|
|
set-max-intset-entries 512
|
|
|
|
|
|
zset-max-ziplist-entries 128
|
|
|
|
|
|
zset-max-ziplist-value 64
|
|
|
|
|
|
hll-sparse-max-bytes 3000
|
|
|
|
|
|
stream-node-max-bytes 4096
|
|
|
|
|
|
stream-node-max-entries 100
|
|
|
|
|
|
activerehashing yes
|
|
|
|
|
|
client-output-buffer-limit normal 0 0 0
|
|
|
|
|
|
client-output-buffer-limit replica 256mb 64mb 60
|
|
|
|
|
|
client-output-buffer-limit pubsub 32mb 8mb 60
|
|
|
|
|
|
hz 10
|
|
|
|
|
|
dynamic-hz yes
|
|
|
|
|
|
aof-rewrite-incremental-fsync yes
|
|
|
|
|
|
rdb-save-incremental-fsync yes
|
|
|
|
|
|
jemalloc-bg-thread yes
|
|
|
|
|
|
slave-2配置文件:
|
|
|
|
|
|
[root@xingdian-server-22 /]# grep -Ev "^$|#" /etc/redis.conf
|
|
|
|
|
|
bind 172.17.0.7
|
|
|
|
|
|
protected-mode yes
|
|
|
|
|
|
port 6379
|
|
|
|
|
|
tcp-backlog 511
|
|
|
|
|
|
timeout 0
|
|
|
|
|
|
tcp-keepalive 300
|
|
|
|
|
|
daemonize no
|
|
|
|
|
|
supervised no
|
|
|
|
|
|
pidfile /var/run/redis_6379.pid
|
|
|
|
|
|
loglevel notice
|
|
|
|
|
|
logfile /var/log/redis/redis.log
|
|
|
|
|
|
databases 16
|
|
|
|
|
|
always-show-logo yes
|
|
|
|
|
|
save 900 1
|
|
|
|
|
|
save 300 10
|
|
|
|
|
|
save 60 10000
|
|
|
|
|
|
stop-writes-on-bgsave-error yes
|
|
|
|
|
|
rdbcompression yes
|
|
|
|
|
|
rdbchecksum yes
|
|
|
|
|
|
dbfilename dump.rdb
|
|
|
|
|
|
rdb-del-sync-files no
|
|
|
|
|
|
dir /var/lib/redis
|
|
|
|
|
|
replicaof 172.17.0.5 6379
|
|
|
|
|
|
masterauth redis
|
|
|
|
|
|
replica-serve-stale-data yes
|
|
|
|
|
|
replica-read-only yes
|
|
|
|
|
|
repl-diskless-sync no
|
|
|
|
|
|
repl-diskless-sync-delay 5
|
|
|
|
|
|
repl-diskless-load disabled
|
|
|
|
|
|
repl-disable-tcp-nodelay no
|
|
|
|
|
|
replica-priority 100
|
|
|
|
|
|
acllog-max-len 128
|
|
|
|
|
|
requirepass redis
|
|
|
|
|
|
lazyfree-lazy-eviction no
|
|
|
|
|
|
lazyfree-lazy-expire no
|
|
|
|
|
|
lazyfree-lazy-server-del no
|
|
|
|
|
|
replica-lazy-flush no
|
|
|
|
|
|
lazyfree-lazy-user-del no
|
|
|
|
|
|
oom-score-adj no
|
|
|
|
|
|
oom-score-adj-values 0 200 800
|
|
|
|
|
|
appendonly no
|
|
|
|
|
|
appendfilename "appendonly.aof"
|
|
|
|
|
|
appendfsync everysec
|
|
|
|
|
|
no-appendfsync-on-rewrite no
|
|
|
|
|
|
auto-aof-rewrite-percentage 100
|
|
|
|
|
|
auto-aof-rewrite-min-size 64mb
|
|
|
|
|
|
aof-load-truncated yes
|
|
|
|
|
|
aof-use-rdb-preamble yes
|
|
|
|
|
|
lua-time-limit 5000
|
|
|
|
|
|
slowlog-log-slower-than 10000
|
|
|
|
|
|
slowlog-max-len 128
|
|
|
|
|
|
latency-monitor-threshold 0
|
|
|
|
|
|
notify-keyspace-events ""
|
|
|
|
|
|
hash-max-ziplist-entries 512
|
|
|
|
|
|
hash-max-ziplist-value 64
|
|
|
|
|
|
list-max-ziplist-size -2
|
|
|
|
|
|
list-compress-depth 0
|
|
|
|
|
|
set-max-intset-entries 512
|
|
|
|
|
|
zset-max-ziplist-entries 128
|
|
|
|
|
|
zset-max-ziplist-value 64
|
|
|
|
|
|
hll-sparse-max-bytes 3000
|
|
|
|
|
|
stream-node-max-bytes 4096
|
|
|
|
|
|
stream-node-max-entries 100
|
|
|
|
|
|
activerehashing yes
|
|
|
|
|
|
client-output-buffer-limit normal 0 0 0
|
|
|
|
|
|
client-output-buffer-limit replica 256mb 64mb 60
|
|
|
|
|
|
client-output-buffer-limit pubsub 32mb 8mb 60
|
|
|
|
|
|
hz 10
|
|
|
|
|
|
dynamic-hz yes
|
|
|
|
|
|
aof-rewrite-incremental-fsync yes
|
|
|
|
|
|
rdb-save-incremental-fsync yes
|
|
|
|
|
|
jemalloc-bg-thread yes
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
注意：
|
|
|
|
|
|
|
|
|
|
|
|
配置文件参数地址：https://xingdian-file.oss-cn-hangzhou.aliyuncs.com/redis-config.txt
|
|
|
|
|
|
|
|
|
|
|
|
启动服务
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
[root@xingdian /]# systemctl start redis
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
验证服务
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
[root@xingdian-server-20 /]# redis-cli -a redis -h 10.0.0.3
|
|
|
|
|
|
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
|
|
|
|
|
|
127.0.0.1:6379> info replication
|
|
|
|
|
|
# Replication
|
|
|
|
|
|
role:master
|
|
|
|
|
|
connected_slaves:2
|
|
|
|
|
|
slave0:ip=172.17.0.6,port=6379,state=online,offset=70,lag=0
|
|
|
|
|
|
slave1:ip=172.17.0.7,port=6379,state=online,offset=70,lag=0
|
|
|
|
|
|
master_replid:12e24fe583640c88aff07733c7bcd1acb32c4dd2
|
|
|
|
|
|
master_replid2:0000000000000000000000000000000000000000
|
|
|
|
|
|
master_repl_offset:70
|
|
|
|
|
|
second_repl_offset:-1
|
|
|
|
|
|
repl_backlog_active:1
|
|
|
|
|
|
repl_backlog_size:1048576
|
|
|
|
|
|
repl_backlog_first_byte_offset:1
|
|
|
|
|
|
repl_backlog_histlen:70
|
|
|
|
|
|
127.0.0.1:6379> set t1 aa
|
|
|
|
|
|
OK
|
|
|
|
|
|
127.0.0.1:6379> quit
|
|
|
|
|
|
slave端:
|
|
|
|
|
|
[root@xingdian-server-21 /]# redis-cli -h 172.17.0.6 -a redis
|
|
|
|
|
|
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
|
|
|
|
|
|
172.17.0.6:6379> get t1
|
|
|
|
|
|
"aa"
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
## 三：Redis HA Sentinel 集群
|
|
|
|
|
|
|
|
|
|
|
|
Redis-Sentinel 是 Redis 官方推荐的高可用性(HA)解决方案，当用Redis做Master-slave的高可用方案时，假如master宕机了，Redis本身(包括它的很多客户端)都没有实现自动进行主备切换，而Redis-sentinel本身也是一个独立运行的进程，它能监控多个master-slave集群，发现master宕机后能进行自动切换
|
|
|
|
|
|
|
|
|
|
|
|
Sentinel 是一个监视器，它可以根据被监视实例的身份和状态来判断应该执行何种动作
|
|
|
|
|
|
|
|
|
|
|
|
<img src="https://xingdian-image.oss-cn-beijing.aliyuncs.com/xingdian-image/image-20230526150237928.png" alt="image-20230526150237928" style="zoom:50%;" />
|
|
|
|
|
|
|
|
|
|
|
|
#### 1.Redis Sentinel 功能
|
|
|
|
|
|
|
|
|
|
|
|
Sentinel的主要功能包括主节点存活检测、主从运行情况检测、自动故障转移（failover）、主从切换。Redis的Sentinel最小配置是一主一从；Redis的Sentinel系统可以用来管理多个Redis服务器
|
|
|
|
|
|
|
|
|
|
|
|
监控
|
|
|
|
|
|
|
|
|
|
|
|
Sentinel会不断的检查主服务器和从服务器是否正常运行
|
|
|
|
|
|
|
|
|
|
|
|
通知
|
|
|
|
|
|
|
|
|
|
|
|
当被监控的某个Redis服务器出现问题，Sentinel通过API脚本向管理员或者其他的应用程序发送通知
|
|
|
|
|
|
|
|
|
|
|
|
自动故障转移
|
|
|
|
|
|
|
|
|
|
|
|
当主节点不能正常工作时，Sentinel会开始一次自动的故障转移操作，它会将与失效主节点是主从关系的其中一个从节点升级为新的主节点， 并且将其他的从节点指向新的主节点
|
|
|
|
|
|
|
|
|
|
|
|
配置提供者
|
|
|
|
|
|
|
|
|
|
|
|
在Redis Sentinel模式下，客户端应用在初始化时连接的是Sentinel节点集合，从中获取主节点的信息
|
|
|
|
|
|
|
|
|
|
|
|
#### 2.Redis Sentinel的工作流程
|
|
|
|
|
|
|
|
|
|
|
|
由一个或多个Sentinel实例组成的Sentinel系统可以监视任意多个主服务器，以及所有从服务器，并在被监视的主服务器进入下线状态时，自动将下线主服务器属下的某个从服务器升级为新的主服务器，然后由新的主服务器代替已下线的主服务器继续处理命令请求
|
|
|
|
|
|
|
|
|
|
|
|
<img src="https://xingdian-image.oss-cn-beijing.aliyuncs.com/xingdian-image/image-20230228223209112.png" alt="image-20230228223209112" style="zoom:50%;" />
|
|
|
|
|
|
|
|
|
|
|
|
Sentinel负责监控集群中的所有主、从Redis，当发现主故障时，Sentinel会在所有的从中选一个成为新的主。并且会把其余的从变为新主的从。同时那台有问题的旧主也会变为新主的从，也就是说当旧的主即使恢复时，并不会恢复原来的主身份，而是作为新主的一个从
|
|
|
|
|
|
|
|
|
|
|
|
在Redis高可用架构中，Sentinel往往不是只有一个，而是有3个或者以上。目的是为了让其更加可靠，毕竟主和从切换角色这个过程还是蛮复杂的
|
|
|
|
|
|
|
|
|
|
|
|
#### 3.Redis Sentinel 服务器连接
|
|
|
|
|
|
|
|
|
|
|
|
**发现并连接主服务器**
|
|
|
|
|
|
|
|
|
|
|
|
Sentinel 通过用户给定的配置文件来发现主服务器
|
|
|
|
|
|
|
|
|
|
|
|
<img src="https://xingdian-image.oss-cn-beijing.aliyuncs.com/xingdian-image/image-20230228223309007.png" alt="image-20230228223309007" style="zoom:33%;" />
|
|
|
|
|
|
|
|
|
|
|
|
Sentinel 会与被监视的主服务器创建两个网络连接
|
|
|
|
|
|
|
|
|
|
|
|
命令连接用于向主服务器发送命令
|
|
|
|
|
|
|
|
|
|
|
|
订阅连接用于订阅指定的频道，从而发现监视同一主服务器的其他 Sentinel
|
|
|
|
|
|
|
|
|
|
|
|
**发现并连接从服务器**
|
|
|
|
|
|
|
|
|
|
|
|
Sentinel 通过向主服务器发送 INFO 命令来自动获得所有从服务器的地址
|
|
|
|
|
|
|
|
|
|
|
|
跟主服务器一样，Sentinel 会与每个被发现的从服务器创建命令连接和订阅连接
|
|
|
|
|
|
|
|
|
|
|
|
<img src="https://xingdian-image.oss-cn-beijing.aliyuncs.com/xingdian-image/image-20230228223352987.png" style="zoom:33%;" />
|
|
|
|
|
|
|
|
|
|
|
|
#### 4.Redis Sentinel 命令操作
|
|
|
|
|
|
|
|
|
|
|
|
| **命令** | **描述** |
|
|
|
|
|
|
| ---------------------------------------------- | ------------------------------------------------------------ |
|
|
|
|
|
|
| PING | 返回 PONG |
|
|
|
|
|
|
| SENTINEL masters | 列出所有被监视的主服务器 |
|
|
|
|
|
|
| SENTINEL slaves <master name> | 列出所有被监视的从服务器 |
|
|
|
|
|
|
| SENTINEL get-master-addr-by-name <master name> | 返回给定名字的主服务器的 IP 地址和端口号。 |
|
|
|
|
|
|
| SENTINEL reset <pattern> | 重置所有名字和给定模式 pattern 相匹配的主服务器 |
|
|
|
|
|
|
| SENTINEL failover <master name> | 当主服务器失效时， 在不询问其他 Sentinel 意见的情况下，强制开始一次自动故障迁移。 |
|
|
|
|
|
|
|
|
|
|
|
|
#### 5.Redis Sentinel（哨兵）
|
|
|
|
|
|
|
|
|
|
|
|
**环境说明**
|
|
|
|
|
|
|
|
|
|
|
|
| **主机名称** | **IP地址** | **redis版本和角色说明** |
|
|
|
|
|
|
| ------------- | --------------------- | ----------------------- |
|
|
|
|
|
|
| redis-master | 192.168.152.133:6379 | redis 6.0.5（主） |
|
|
|
|
|
|
| redis-slave01 | 192.168.152.134:6379 | redis 6.0.5（从） |
|
|
|
|
|
|
| redis-slave02 | 192.168.152.135:6379 | redis 6.0.5（从） |
|
|
|
|
|
|
| redis-master | 192.168.152.133:26379 | Sentinel01 |
|
|
|
|
|
|
| redis-slave01 | 192.168.152.134:26379 | Sentinel02 |
|
|
|
|
|
|
| redis-slave02 | 192.168.152.135:26379 | Sentinel03 |
|
|
|
|
|
|
|
|
|
|
|
|
**安装(所有节点安装)**
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
[root@xingdian-server-23 /]# yum -y install http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
|
|
|
|
|
|
[root@xingdian-server-23 /]# yum -y --enablerepo=remi install redis
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
**部署主从(略)**
|
|
|
|
|
|
|
|
|
|
|
|
**部署Sentinel**
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
[root@redis-master ~]# grep -Ev "^$|#" /etc/redis-sentinel.conf
|
|
|
|
|
|
port 26379
|
|
|
|
|
|
daemonize no
|
|
|
|
|
|
pidfile /var/run/redis-sentinel.pid
|
|
|
|
|
|
logfile /var/log/redis/sentinel.log
|
|
|
|
|
|
dir /tmp
|
|
|
|
|
|
sentinel monitor mymaster 127.0.0.1 6379 2
|
|
|
|
|
|
sentinel down-after-milliseconds mymaster 30000
|
|
|
|
|
|
sentinel parallel-syncs mymaster 1
|
|
|
|
|
|
sentinel failover-timeout mymaster 180000
|
|
|
|
|
|
sentinel deny-scripts-reconfig yes
|
|
|
|
|
|
|
|
|
|
|
|
# 端口
|
|
|
|
|
|
port 26379
|
|
|
|
|
|
# 是否后台启动
|
|
|
|
|
|
daemonize yes
|
|
|
|
|
|
# 保护模式如果开启只接受回环地址的ipv4和ipv6地址链接，拒绝外部链接，而且正常应该配置多个哨兵，避免一个哨兵出现独裁情况，如果配置多个哨兵那如果开启也会拒绝其他sentinel的连接。导致哨兵配置无法生效。
|
|
|
|
|
|
protected-mode no
|
|
|
|
|
|
# pid文件路径
|
|
|
|
|
|
pidfile /var/run/redis-sentinel.pid
|
|
|
|
|
|
# 日志文件路径
|
|
|
|
|
|
logfile "/var/log/redis/sentinel.log"
|
|
|
|
|
|
# 定义工作目录
|
|
|
|
|
|
dir /tmp
|
|
|
|
|
|
# 定义Redis主的别名, IP, 端口，这里的2指的是需要至少2个Sentinel认为主Redis挂了才最终会采取下一步行为
|
|
|
|
|
|
sentinel monitor mymaster 127.0.0.1 6379 2
|
|
|
|
|
|
#如果redis配置了密码，那这里必须配置认证，否则不能自动切换
|
|
|
|
|
|
sentinel auth-pass mymaster redispass
|
|
|
|
|
|
# 如果mymaster 30秒内没有响应，则认为其主观失效
|
|
|
|
|
|
sentinel down-after-milliseconds mymaster 30000
|
|
|
|
|
|
# 如果master重新选出来后，其它slave节点能同时并行从新master同步数据的台数有多少个，显然该值越大，所有slave节点完成同步切换的整体速度越快，但如果此时正好有人在访问这些slave，可能造成读取失败，影响面会更广。最保守的设置为1，同一时间，只能有一台干这件事，这样其它slave还能继续服务，但是所有slave全部完成缓存更新同步的进程将变慢。
|
|
|
|
|
|
sentinel parallel-syncs mymaster 1
|
|
|
|
|
|
# 该参数指定一个时间段，在该时间段内没有实现故障转移成功，则会再一次发起故障转移的操作，单位毫秒
|
|
|
|
|
|
sentinel failover-timeout mymaster 180000
|
|
|
|
|
|
# 不允许使用SENTINEL SET设置notification-script和client-reconfig-script。
|
|
|
|
|
|
sentinel deny-scripts-reconfig yes
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
配置文件：
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
[root@xingdian-server-23 /]# grep -Ev "^$|#" /etc/redis-sentinel.conf
|
|
|
|
|
|
port 26379
|
|
|
|
|
|
daemonize no
|
|
|
|
|
|
protected-mode no
|
|
|
|
|
|
pidfile /var/run/redis-sentinel.pid
|
|
|
|
|
|
logfile /var/log/redis/sentinel.log
|
|
|
|
|
|
dir /tmp
|
|
|
|
|
|
sentinel monitor mymaster 172.17.0.5 6379 2
|
|
|
|
|
|
sentinel auth-pass mymaster redis
|
|
|
|
|
|
sentinel down-after-milliseconds mymaster 30000
|
|
|
|
|
|
sentinel parallel-syncs mymaster 1
|
|
|
|
|
|
sentinel failover-timeout mymaster 180000
|
|
|
|
|
|
sentinel deny-scripts-reconfig yes
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
启动的顺序：主Redis --> 从Redis --> Sentinel1/2/3
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
[root@xingdian-server-23 /]# systemctl restart redis-sentinel.service
|
|
|
|
|
|
[root@xingdian-server-23 /]# systemctl status redis-sentinel.service
|
|
|
|
|
|
● redis-sentinel.service - Redis Sentinel
|
|
|
|
|
|
Loaded: loaded (/usr/lib/systemd/system/redis-sentinel.service; disabled; vendor preset: disabled)
|
|
|
|
|
|
Drop-In: /etc/systemd/system/redis-sentinel.service.d
|
|
|
|
|
|
└─limit.conf
|
|
|
|
|
|
Active: active (running) since Wed 2020-12-16 14:07:18 UTC; 24s ago
|
|
|
|
|
|
Main PID: 202 (redis-sentinel)
|
|
|
|
|
|
Status: "Ready to accept connections"
|
|
|
|
|
|
CGroup: /docker/3976cfca1cf87ae43370f78ab3e2d26b68c0808e1e24178b12cddd8b3bd1fef5/system.slice/redis-sentinel.service
|
|
|
|
|
|
└─202 /usr/bin/redis-sentinel *:26379 [sentinel]
|
|
|
|
|
|
|
|
|
|
|
|
Dec 16 14:07:17 xingdian-server-23 systemd[1]: Starting Redis Sentinel...
|
|
|
|
|
|
Dec 16 14:07:18 xingdian-server-23 systemd[1]: Started Redis Sentinel.
|
|
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
**哨兵模式下的主从测试**
|
|
|
|
|
|
|
|
|
|
|
|
模拟停止master上的Redis，查看Redis的主从变化，如下：
|
|
|
|
|
|
|
|
|
|
|
|
```shell
|
|
|
|
|
|
主服务器:
|
|
|
|
|
|
[root@xingdian-server-1 /]# systemctl stop redis
|
|
|
|
|
|
从服务器:
|
|
|
|
|
|
[root@xingdian-server-2 /]# tailf /var/log/redis/sentinel.log
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:11.961 # +promoted-slave slave 172.17.0.4:6379 172.17.0.4 6379 @ mymaster 172.17.0.2 6379
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:11.961 # +failover-state-reconf-slaves master mymaster 172.17.0.2 6379
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:11.964 * +slave-reconf-sent slave 172.17.0.3:6379 172.17.0.3 6379 @ mymaster 172.17.0.2 6379
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:12.936 * +slave-reconf-inprog slave 172.17.0.3:6379 172.17.0.3 6379 @ mymaster 172.17.0.2 6379
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:13.134 # -odown master mymaster 172.17.0.2 6379
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:13.964 * +slave-reconf-done slave 172.17.0.3:6379 172.17.0.3 6379 @ mymaster 172.17.0.2 6379
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:14.030 # +failover-end master mymaster 172.17.0.2 6379
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:14.030 # +switch-master mymaster 172.17.0.2 6379 172.17.0.4 6379
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:14.031 * +slave slave 172.17.0.3:6379 172.17.0.3 6379 @ mymaster 172.17.0.4 6379
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:14.031 * +slave slave 172.17.0.2:6379 172.17.0.2 6379 @ mymaster 172.17.0.4 6379
|
|
|
|
|
|
270:X 16 Dec 2020 15:46:44.105 # +sdown slave 172.17.0.2:6379 172.17.0.2 6379 @ mymaster 172.17.0.4 6379
|
|
|
|
|
|
从上面的日志可以看到master已经sdown，并切换为192.168.152.1343为master节点，下面查看slave01上的配置，会自动的更改replicaof配置项，如下：
|
|
|
|
|
|
[root@xingdian-server-2 /]# grep "replicaof" /etc/redis.conf |grep -vE "#"
|
|
|
|
|
|
replicaof 172.17.0.4 6379
|
|
|
|
|
|
回到master.查看是否切换成功
|
|
|
|
|
|
[root@xingdian-server-1 /]# redis-cli -p 26379
|
|
|
|
|
|
127.0.0.1:26379> sentinel master mymaster
|
|
|
|
|
|
1) "name"
|
|
|
|
|
|
2) "mymaster"
|
|
|
|
|
|
3) "ip"
|
|
|
|
|
|
4) "172.17.0.4"
|
|
|
|
|
|
5) "port"
|
|
|
|
|
|
6) "6379"
|
|
|
|
|
|
7) "runid"
|
|
|
|
|
|
8) "074997b30d9e08339cbd27f6f478e83ffd03d13d"
|
|
|
|
|
|
9) "flags"
|
|
|
|
|
|
10) "master"
|
|
|
|
|
|
11) "link-pending-commands"
|
|
|
|
|
|
12) "0"
|
|
|
|
|
|
13) "link-refcount"
|
|
|
|
|
|
14) "1"
|
|
|
|
|
|
127.0.0.1:26379> sentinel slaves mymaster
|
|
|
|
|
|
1) 1) "name"
|
|
|
|
|
|
2) "172.17.0.2:6379"
|
|
|
|
|
|
3) "ip"
|
|
|
|
|
|
4) "172.17.0.2"
|
|
|
|
|
|
5) "port"
|
|
|
|
|
|
6) "6379"
|
|
|
|
|
|
7) "runid"
|
|
|
|
|
|
8) ""
|
|
|
|
|
|
9) "flags"
|
|
|
|
|
|
10) "s_down,slave,disconnected" # 提示该节点为从，并且状态为s_down，无法链接的状态
|
|
|
|
|
|
11) "link-pending-commands"
|
|
|
|
|
|
12) "3"
|
|
|
|
|
|
13) "link-refcount"
|
|
|
|
|
|
14) "1"
|
|
|
|
|
|
15) "last-ping-sent"
|
|
|
|
|
|
16) "173956"
|
|
|
|
|
|
17) "last-ok-ping-reply"
|
|
|
|
|
|
18) "173956"
|
|
|
|
|
|
19) "last-ping-reply"
|
|
|
|
|
|
20) "173956"
|
|
|
|
|
|
21) "s-down-time"
|
|
|
|
|
|
22) "143918"
|
|
|
|
|
|
23) "down-after-milliseconds"
|
|
|
|
|
|
24) "30000"
|
|
|
|
|
|
25) "info-refresh"
|
|
|
|
|
|
26) "1608133745921"
|
|
|
|
|
|
27) "role-reported"
|
|
|
|
|
|
28) "slave"
|
|
|
|
|
|
29) "role-reported-time"
|
|
|
|
|
|
30) "173956"
|
|
|
|
|
|
31) "master-link-down-time"
|
|
|
|
|
|
32) "0"
|
|
|
|
|
|
33) "master-link-status"
|
|
|
|
|
|
34) "err"
|
|
|
|
|
|
35) "master-host"
|
|
|
|
|
|
36) "?"
|
|
|
|
|
|
37) "master-port"
|
|
|
|
|
|
38) "0"
|
|
|
|
|
|
39) "slave-priority"
|
|
|
|
|
|
40) "100"
|
|
|
|
|
|
41) "slave-repl-offset"
|
|
|
|
|
|
42) "0"
|
|
|
|
|
|
2) 1) "name"
|
|
|
|
|
|
2) "172.17.0.3:6379"
|
|
|
|
|
|
3) "ip"
|
|
|
|
|
|
4) "172.17.0.3"
|
|
|
|
|
|
5) "port"
|
|
|
|
|
|
6) "6379"
|
|
|
|
|
|
7) "runid"
|
|
|
|
|
|
8) "fdce07036b5a1b435ee5a7bbe4b39231e5554756"
|
|
|
|
|
|
9) "flags"
|
|
|
|
|
|
10) "slave"
|
|
|
|
|
|
11) "link-pending-commands"
|
|
|
|
|
|
12) "0"
|
|
|
|
|
|
13) "link-refcount"
|
|
|
|
|
|
14) "1"
|
|
|
|
|
|
15) "last-ping-sent"
|
|
|
|
|
|
16) "0"
|
|
|
|
|
|
17) "last-ok-ping-reply"
|
|
|
|
|
|
18) "1024"
|
|
|
|
|
|
19) "last-ping-reply"
|
|
|
|
|
|
20) "1024"
|
|
|
|
|
|
21) "down-after-milliseconds"
|
|
|
|
|
|
22) "30000"
|
|
|
|
|
|
23) "info-refresh"
|
|
|
|
|
|
24) "2200"
|
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
|
|
注意：
|
|
|
|
|
|
|
|
|
|
|
|
如果旧主重新加入后数据不同步解决方案
|
|
|
|
|
|
|
|
|
|
|
|
可能之前是主节点，没有配置从节点的连接信息 如：masterauth 连接密码，当master转变为slave后，由于他没有密码，所以他不能从新的master同步数据，随之导致 info replication 的时候，同步状态为 down ，所以只需要修改 redis.conf 中的 masterauth 为 对应的密码
|
