Flask缓存

Centos redis哨兵模式

使用docker + docker-compose来部署

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-AVjgqCrQ-1653465381232)(/Users/zhujian/Library/Application Support/typora-user-images/image-20220524103820552.png)]

Sentinel 配置

Redis-sentinel-1.conf

# 哨兵的端口号
# 因为各个哨兵节点会运行在单独的Docker容器中
# 所以无需担心端口重复使用
# 如果需要在单机
port 26379

# 配置哨兵的监控参数
# 格式：sentinel monitor    
# master-name是为这个被监控的master起的名字
# ip是被监控的master的IP或主机名。因为Docker容器之间可以使用容器名访问，所以这里写master节点的容器名
# redis-port是被监控节点所监听的端口号
# quorom设定了当几个哨兵判定这个节点失效后，才认为这个节点真的失效了
sentinel myid 248195cce565a0dc133f513be0e0aea13fd1efab

# 连接主节点的密码
# 格式：sentinel auth-pass  
sentinel deny-scripts-reconfig yes

# master在连续多长时间无法响应PING指令后，就会主观判定节点下线，默认是30秒
# 格式：sentinel down-after-milliseconds  
sentinel monitor local-master 127.0.0.1 6381 2

redis-sentinel-2.conf

# 哨兵的端口号
# 因为各个哨兵节点会运行在单独的Docker容器中
# 所以无需担心端口重复使用
# 如果需要在单机
port 26380

# 配置哨兵的监控参数
# 格式：sentinel monitor    
# master-name是为这个被监控的master起的名字
# ip是被监控的master的IP或主机名。因为Docker容器之间可以使用容器名访问，所以这里写master节点的容器名
# redis-port是被监控节点所监听的端口号
# quorom设定了当几个哨兵判定这个节点失效后，才认为这个节点真的失效了
sentinel monitor local-master 127.0.0.1 6379 2

# 连接主节点的密码
# 格式：sentinel auth-pass  
sentinel auth-pass local-master zj122900

# master在连续多长时间无法响应PING指令后，就会主观判定节点下线，默认是30秒
# 格式：sentinel down-after-milliseconds  
sentinel down-after-milliseconds local-master 30000

Redis-sentinel-3.conf

# 哨兵的端口号
# 因为各个哨兵节点会运行在单独的Docker容器中
# 所以无需担心端口重复使用
# 如果需要在单机
port 26381

# 配置哨兵的监控参数
# 格式：sentinel monitor    
# master-name是为这个被监控的master起的名字
# ip是被监控的master的IP或主机名。因为Docker容器之间可以使用容器名访问，所以这里写master节点的容器名
# redis-port是被监控节点所监听的端口号
# quorom设定了当几个哨兵判定这个节点失效后，才认为这个节点真的失效了
sentinel monitor local-master 127.0.0.1 6379 2

# 连接主节点的密码
# 格式：sentinel auth-pass  
sentinel auth-pass local-master zj122900

# master在连续多长时间无法响应PING指令后，就会主观判定节点下线，默认是30秒
# 格式：sentinel down-after-milliseconds  
sentinel down-after-milliseconds local-master 30000

Redis server配置

Redis-master.conf

# 监听端口
port 6379

# 设定密码认证
requirepass zj122900

cluster-enabled yes

redis-server-slave-1.conf

# 监听端口
port 6380

# 设定密码认证
requirepass zj122900

slaveof 127.0.0.1 6379

masterauth "zj122900"

Redis-server-slave-2.conf

# 监听端口
port 6381

# 设定密码认证
requirepass zj122900

slaveof 127.0.0.1 6379

masterauth "zj122900"

Docker-compose配置

Redis-server

version: '3'

services:
  redis-server-master:
    image: redis:latest
    container_name: redis-server-master
    restart: always
    network_mode: host
    environment:
      TZ: "Asia/Shanghai"
    volumes:
      - ./redis-master.conf:/usr/local/etc/redis/redis.conf
    command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
  redis-server-slave-1:
    image: redis:latest
    container_name: redis-server-slave-1
    restart: always
    network_mode: host
    depends_on:
      - redis-server-master
    environment:
      TZ: "Asia/Shanghai"
    volumes:
      - ./redis-slave-1.conf:/usr/local/etc/redis/redis.conf
    command: ["redis-server", "/usr/local/etc/redis/redis.conf"]
  redis-server-slave-2:
    image: redis:latest
    container_name: redis-server-slave-2
    restart: always
    network_mode: host
    depends_on:
      - redis-server-master
    environment:
      TZ: "Asia/Shanghai"
    volumes:
      - ./redis-slave-2.conf:/usr/local/etc/redis/redis.conf
    command: ["redis-server", "/usr/local/etc/redis/redis.conf"]

redis-sentinel

version: '3'

services:
  redis-sentinel-1:
    image: redis:4.0.12
    container_name: redis-sentinel-1
    restart: always
    network_mode: host
    volumes:
      - ./redis-sentinel-1.conf:/usr/local/etc/redis/redis-sentinel.conf
    environment:
      TZ: "Asia/Shanghai"
    command: ["redis-sentinel", "/usr/local/etc/redis/redis-sentinel.conf"]
  redis-sentinel-2:
    image: redis:4.0.12
    container_name: redis-sentinel-2
    restart: always
    network_mode: host
    volumes:
      - ./redis-sentinel-2.conf:/usr/local/etc/redis/redis-sentinel.conf
    environment:
      TZ: "Asia/Shanghai"
    command: [ "redis-sentinel", "/usr/local/etc/redis/redis-sentinel.conf" ]
  redis-sentinel-3:
    image: redis:4.0.12
    container_name: redis-sentinel-3
    restart: always
    network_mode: host
    volumes:
      - ./redis-sentinel-3.conf:/usr/local/etc/redis/redis-sentinel.conf
    environment:
      TZ: "Asia/Shanghai"
    command: [ "redis-sentinel", "/usr/local/etc/redis/redis-sentinel.conf" ]

解决槽节点问题

redis-cli --cluster check 127.0.0.1:6379
redis-cli --cluster fix 127.0.0.1:6379

flask 连接redis

from redis.sentinel import Sentinel
from rediscluster import StrictRedisCluster
_sentinel = Sentinel([
        ('xx.xx.xx.xx', '26379'),
        ('xx.xx.xx.xx', '26380'),
        ('xx.xx.xx.xx', '26381'),
    ])
app.redis_master = _sentinel.master_for("redis-sentinel-name", password="passwd")
app.redis_master = _sentinel.slave_for("redis-sentinel-name", password="passwd")

app.redis_cluster = StrictRedisCluster(startup_nodes=[
        {'host': 'xx.xx.xx.xx', 'port': '6379'},
        {'host': 'xx.xx.xx.xx', 'port': '6380'},
        {'host': 'xx.xx.xx.xx', 'port': '6381'},
    ], decode_responses=True, password="passwd")

flask 缓存工具

class UserCache:
    """
    用户基本信息的缓存工具类
    """

    def __init__(self, user_id):
        # 记录到redis中的key, 使用的是redis的string类型
        self.redis_key = f"user:{user_id}:profile"
        self.user_id = user_id

    def save(self):
        """
        保存缓存记录
        """

        # 从current_app中获取redis连接
        r = current_app.redis_cluster

        try:
            user = User.query.options(load_only(
                User.name,
                User.mobile,
                User.status,
                User.email
            )).filter(User.id == self.user_id).first()
        except DatabaseError as e:
            current_app.logger.error(e)
            # 对于数据库异常，抛给调用者，由调用者决定
            raise e

        # 如果用户不存在
        if user is None:
            try:
                # 防止缓存击穿，给不存在的用户缓存为-1
                r.setex(self.redis_key, constants.UserNotExistsCacheTTL.get_val(), -1)
            except RedisError as e:
                current_app.logger.error(e)

            return None
        else:
            # 初始化返回数据格式
            user_dict = {
                'mobile': user.mobile,
                'name': user.name,
                'status': user.status,
                'email': user.email
            }
            try:
                r.setex(self.redis_key, constants.UserProfileCacheTTL.get_val(), json.dumps(user_dict))
            except RedisError as e:
                current_app.logger.error(e)

            return user_dict

    def get(self):
        """
        获取缓存数据
        """

        # 先到redis查询缓存记录
        r = current_app.redis_cluster

        try:
            ret = r.get(self.redis_key)
        except RedisError as e:
            current_app.logger.error(e)
            ret = None

        if ret is not None:
            # 如果redis中有缓存记录，判断是否是-1
            if ret == b"-1":
                return None
            else:
                user_dict = json.loads(ret)
                return user_dict
        else:
            return self.save()

    def clear(self):
        """
        清除缓存
        """
        try:
            r = current_app.redis_cluster
            r.delete(self.redis_key)
        except RedisError as e:
            current_app.logger.error(e)

    def is_user_exists(self):
        """
        通过缓存判断用户是否存在
        """
        r = current_app.redis_cluster

        # 查询redis中是否有缓存记录
        try:
            ret = r.get(self.redis_key)
        except RedisError as e:
            current_app.logger.error(e)
            ret = None

        # 如果redis中存在缓存记录，就判断值是否为-1
        if ret is not None:
            if ret == b"-1":
                return False
            else:
                return True
        else:
            ret = self.save()
            if ret is not None:
                return True
            else:
                return False

flask使用缓存

@user_bp.route('/user/')
@login_required
def obtain_user_info(user_id):
    """
    获取用户的信息
    :user_id: 用户id
    """
    error_msg = {
        "data": None,
        "response": "failed",
        "msg": ""
    }

    # 使用缓存查询
    user_cache = UserCache(user_id)

    # 判读用户是否存在
    is_exists = user_cache.is_user_exists()
    if not is_exists:
        error_msg["msg"] = "Invalid user"
        return error_msg, 400

    user_data = user_cache.get()

    user_data["id"] = user_id

    return {
        "data": user_data,
        "response": "success",
        "msg": "0"
    }

Flask缓存

Python相关栏目本月热门文章