在项目中使用博主自己封装的计算限流工具已经很久了,慢慢发展已经比较完善通用,现在开源共享。
gitee地址:count-limit: 轻量级的java限流工具, 针对大容量、高并发的接口、mq或者其他计算方法,同一时间在运行的计算维度进行限制,相当于把资源到计算的对象维度。 不同于参数限制,工具针对的是服务所有线程对于该计算维度的限流。
相比较博主之前在java计算限流工具_tingmailang的博客-CSDN博客_限流工具封装的工具,开源的工具除了支持本地锁、本地存储之外还支持spring redis、redisson等进行加锁或者存储,注解配置更为灵活,可以看出工具演变的过程。
二、代码解析 1、注解@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface CountLimit {
String objectName();
String paramName() default "";
int limit() default 10000;
int waitTime() default 2;
//存储与加锁方式,默认本地存储本地锁
String countFactoryEnum() default CountLimitCommonUtil.LOCAL_LOCK_STORE;
}
2、注解配置加锁存储枚举
@Getter
public enum CountFactoryEnum {
LOCAL_LOCK_STORE("local_lock_store", "使用本地锁本地缓存"),
LOCAL_LOCK_REDISSON_STORE("local_lock_redisson_store", "使用本地锁redisson缓存"),
LOCAL_LOCK_SPRING_REDIS_STORE("local_lock_spring_redis_store", "使用本地锁spring redis缓存"),
REDISSON_LOCK_STORE("redisson_lock_store", "使用redisson锁本地缓存"),
REDISSON_LOCK_REDISSON_STORE("redisson_lock_redisson_store", "使用redisson锁redisson缓存"),
SPRING_REDIS_LOCK_STORE("spring_redis_lock_store", "使用spring redis锁本地缓存"),
SPRING_REDIS_LOCK_SPRING_REDIS_STORE("spring_redis_lock_spring_redis_store", "使用spring redis锁spring redis缓存"),
;
private String key;
private String value;
CountFactoryEnum(String key, String value) {
this.key = key;
this.value = value;
}
@JsonCreator
public static CountFactoryEnum of(String key) {
Optional systemTypeEnum = Arrays.stream(CountFactoryEnum.values())
.filter(c -> c.getKey().equals(key)).findFirst();
return systemTypeEnum.orElse(null);
}
}
3、工具类
@Data
public class CountLimitDTO {
private String Key;
private Integer count;
private Integer limit;
private Boolean isAdd;
}
public class CountLimitCommonUtil {
public static final String COUNT_LIMIT_LOCK = "COUNT_LIMIT_LOCK:";
public static final String COUNT_LIMIT_REDIS_NODE_LOCK = "COUNT_LIMIT_REDIS_NODE_LOCK:";
public static final String COUNT_LIMIT_REDIS_NODE_STORE = "COUNT_LIMIT_REDIS_NODE_STORE:";
public static final String COUNT_LIMIT_STORE = "COUNT_LIMIT_STORE:";
public static final String LOCAL_LOCK_STORE = "local_lock_store";
public static final String LOCAL_LOCK_REDISSON_STORE = "local_lock_redisson_store";
public static final String LOCAL_LOCK_SPRING_REDIS_STORE = "local_lock_spring_redis_store";
public static final String REDISSON_LOCK_STORE = "redisson_lock_store";
public static final String REDISSON_LOCK_REDISSON_STORE = "redisson_lock_redisson_store";
public static final String SPRING_REDIS_LOCK_STORE = "spring_redis_lock_store";
public static final String SPRING_REDIS_LOCK_SPRING_REDIS_STORE = "spring_redis_lock_spring_redis_store";
private static volatile String NODE_ID;
public static volatile int SPILT_SLEEP = 20;
public static final String GET = "get";
public static String getNodeId() {
return NODE_ID;
}
public static synchronized void setNodeId(String nodeId) {
NODE_ID = nodeId;
}
public static synchronized void setSpiltSleep(int spiltSleep) {
SPILT_SLEEP = spiltSleep;
}
public static Object getFieldValueByName(String fieldName, Object o) {
try {
String firstLetter = fieldName.substring(0, 1).toUpperCase();
String getter = GET + firstLetter + fieldName.substring(1);
Method method = o.getClass().getMethod(getter, new Class[]{});
Object value = method.invoke(o, new Object[]{});
return value;
} catch (Exception e) {
throw new CountLimitException("获取属性值失败", e);
}
}
}
4、工厂类
public interface CountLimitFacade5、工厂实现类extends MatchingBean { boolean process(T t); } @Component public class CountLimitFacadeFactory implements FactoryList , CountFactoryEnum> { @Resource private List countLimitFacades; @Override public CountLimitFacade getBean(CountFactoryEnum factory) { for (CountLimitFacade countLimitFacade : countLimitFacades) { if (countLimitFacade.matching(factory)) { return countLimitFacade; } } throw new CountLimitException("找不到计数限流实现类"); } } public interface FactoryList , K> { E getBean(K factory); } public interface MatchingBean { Boolean matching(T factory); }
①本地锁本地存储
@Component public class LocalLockLocalStore extends CountLimitCommonBusiness implements CountLimitFacade{ private static ReentrantLock lock = new ReentrantLock(); @Override public Boolean matching(CountFactoryEnum factory) { return Objects.equals(CountFactoryEnum.LOCAL_LOCK_STORE, factory); } @Override public boolean process(CountLimitDTO countLimitDTO) { if (countLimitDTO.getIsAdd()) { return this.checkExceed(countLimitDTO.getKey(), countLimitDTO.getCount(), countLimitDTO.getLimit()); } else { return this.reduce(countLimitDTO.getKey(), countLimitDTO.getCount()); } } public boolean checkExceed(String key, int count, int limit) { try { if (lock.tryLock()) { return super.localCheckExceed(key, count, limit); } else { return false; } } finally { if (lock.isHeldByCurrentThread()) { lock.unlock(); } } } public boolean reduce(String key, int count) { try { if (lock.tryLock(10, TimeUnit.SECONDS)) { return super.localReduce(key, count); } } catch (InterruptedException e) { e.printStackTrace(); } finally { if (lock.isHeldByCurrentThread()) { lock.unlock(); } } return false; } }
②本地锁redisson存储
@Component public class LocalLockRedissonStore extends CountLimitCommonBusiness implements CountLimitFacade{ private static ReentrantLock lock = new ReentrantLock(); @Override public Boolean matching(CountFactoryEnum factory) { return Objects.equals(CountFactoryEnum.LOCAL_LOCK_REDISSON_STORE, factory); } @Override public boolean process(CountLimitDTO countLimitDTO) { if (countLimitDTO.getIsAdd()) { return this.checkExceed(countLimitDTO.getKey(), countLimitDTO.getCount(), countLimitDTO.getLimit()); } else { return this.reduce(countLimitDTO.getKey(), countLimitDTO.getCount()); } } public boolean checkExceed(String key, int count, int limit) { try { if (lock.tryLock()) { return super.redissonCheckExceed(key, count, limit); } else { return false; } } finally { if (lock.isHeldByCurrentThread()) { lock.unlock(); } } } public boolean reduce(String key, int count) { try { if (lock.tryLock(10, TimeUnit.SECONDS)) { return super.redissonReduce(key, count); } } catch (InterruptedException e) { e.printStackTrace(); } finally { if (lock.isHeldByCurrentThread()) { lock.unlock(); } } return false; } }
③本地锁spring redis存储
@Component public class LocalLockSpringRedisStore extends CountLimitCommonBusiness implements CountLimitFacade{ private static ReentrantLock lock = new ReentrantLock(); @Override public Boolean matching(CountFactoryEnum factory) { return Objects.equals(CountFactoryEnum.LOCAL_LOCK_SPRING_REDIS_STORE, factory); } @Override public boolean process(CountLimitDTO countLimitDTO) { if (countLimitDTO.getIsAdd()) { return this.checkExceed(countLimitDTO.getKey(), countLimitDTO.getCount(), countLimitDTO.getLimit()); } else { return this.reduce(countLimitDTO.getKey(), countLimitDTO.getCount()); } } public boolean checkExceed(String key, int count, int limit) { try { if (lock.tryLock()) { return super.springRedisCheckExceed(key, count, limit); } else { return false; } } finally { if (lock.isHeldByCurrentThread()) { lock.unlock(); } } } public boolean reduce(String key, int count) { try { if (lock.tryLock(10, TimeUnit.SECONDS)) { return super.localReduce(key, count); } } catch (InterruptedException e) { e.printStackTrace(); } finally { if (lock.isHeldByCurrentThread()) { lock.unlock(); } } return false; } }
④redisson锁本地存储
这里使用了博主封装的分布式代理锁进行并发控制,后续使用分布式锁的地方都可以看到一个注解搞定加锁开源工具distributed-proxy-lock(分布式代理锁)_tingmailang的博客-CSDN博客
@Component public class RedissonLockLocalStore extends CountLimitCommonBusiness implements CountLimitFacade{ @Override public Boolean matching(CountFactoryEnum factory) { return Objects.equals(CountFactoryEnum.REDISSON_LOCK_STORE, factory); } @Override @DistributedProxyLock(key = CountLimitCommonUtil.COUNT_LIMIT_LOCK, suffixKeyTypeEnum = DistributedProxyLockCommonUtil.PARAM, objectName = "countLimitDTO", paramName = "LockKey") public boolean process(CountLimitDTO countLimitDTO) { if (countLimitDTO.getIsAdd()) { return super.localCheckExceed(countLimitDTO.getKey(), countLimitDTO.getCount(), countLimitDTO.getLimit()); } else { return super.localReduce(countLimitDTO.getKey(), countLimitDTO.getCount()); } } }
⑤redisson锁redisson存储
@Component public class RedissonLockRedissonStore extends CountLimitCommonBusiness implements CountLimitFacade{ @Override public Boolean matching(CountFactoryEnum factory) { return Objects.equals(CountFactoryEnum.REDISSON_LOCK_REDISSON_STORE, factory); } @Override @DistributedProxyLock(key = CountLimitCommonUtil.COUNT_LIMIT_LOCK, suffixKeyTypeEnum = DistributedProxyLockCommonUtil.PARAM, objectName = "countLimitDTO", paramName = "LockKey") public boolean process(CountLimitDTO countLimitDTO) { if (countLimitDTO.getIsAdd()) { return super.redissonCheckExceed(countLimitDTO.getKey(), countLimitDTO.getCount(), countLimitDTO.getLimit()); } else { return super.redissonReduce(countLimitDTO.getKey(), countLimitDTO.getCount()); } } }
⑥SpringRedis锁本地存储
@Component public class SpringRedisLockLocalStore extends CountLimitCommonBusiness implements CountLimitFacade{ @Override public Boolean matching(CountFactoryEnum factory) { return Objects.equals(CountFactoryEnum.SPRING_REDIS_LOCK_STORE, factory); } @Override @DistributedProxyLock(key = CountLimitCommonUtil.COUNT_LIMIT_LOCK, suffixKeyTypeEnum = DistributedProxyLockCommonUtil.PARAM, objectName = "countLimitDTO", paramName = "LockKey", lockConnectionEnum = DistributedProxyLockCommonUtil.SPRING_REDIS) public boolean process(CountLimitDTO countLimitDTO) { if (countLimitDTO.getIsAdd()) { return super.localCheckExceed(countLimitDTO.getKey(), countLimitDTO.getCount(), countLimitDTO.getLimit()); } else { return super.localReduce(countLimitDTO.getKey(), countLimitDTO.getCount()); } } }
⑦SpringRedis锁SpringRedis存储
@Component public class SpringRedisLockSpringRedisStore extends CountLimitCommonBusiness implements CountLimitFacade{ @Override public Boolean matching(CountFactoryEnum factory) { return Objects.equals(CountFactoryEnum.SPRING_REDIS_LOCK_SPRING_REDIS_STORE, factory); } @Override @DistributedProxyLock(key = CountLimitCommonUtil.COUNT_LIMIT_LOCK, suffixKeyTypeEnum = DistributedProxyLockCommonUtil.PARAM, objectName = "countLimitDTO", paramName = "LockKey", lockConnectionEnum = DistributedProxyLockCommonUtil.SPRING_REDIS) public boolean process(CountLimitDTO countLimitDTO) { if (countLimitDTO.getIsAdd()) { return super.springRedisCheckExceed(countLimitDTO.getKey(), countLimitDTO.getCount(), countLimitDTO.getLimit()); } else { return super.springRedisReduce(countLimitDTO.getKey(), countLimitDTO.getCount()); } } }
以上工具都继承了父类CountLimitCommonBusiness,下面来看一下父类中做了什么
@Service
@Slf4j
public class CountLimitCommonBusiness {
@Resource
private RedissonClient redissonClient;
@Resource
private RedisTemplate redisTemplate;
@Resource
private CountLimitNode countLimitNode;
private static volatile ConcurrentHashMap countMap = new ConcurrentHashMap<>();
public boolean redissonCheckExceed(String key, int count, int limit) {
this.checkNode(LockConnectionEnum.REDISSON);
RBucket bucket = redissonClient.getBucket(CountLimitCommonUtil.COUNT_LIMIT_STORE + CountLimitCommonUtil.getNodeId() + key);
int now = 0;
if (bucket.isExists()) {
now = bucket.get();
}
if (now + count > limit) {
return false;
} else {
log.debug("CountLimitAspect:{}增加计算量:{}", key, now + count);
bucket.set(now + count);
return true;
}
}
public boolean redissonReduce(String key, int count) {
this.checkNode(LockConnectionEnum.REDISSON);
RBucket bucket = redissonClient.getBucket(CountLimitCommonUtil.COUNT_LIMIT_STORE + CountLimitCommonUtil.getNodeId() + key);
int now = bucket.get();
log.debug("CountLimitAspect:{}减少计算量:{}", key, now - count);
bucket.set(now - count);
return true;
}
public boolean localCheckExceed(String key, int count, int limit) {
int now = countMap.getOrDefault(key, 0);
if (now + count > limit) {
return false;
} else {
log.debug("CountLimitAspect:{}增加计算量:{}", key, now + count);
countMap.put(key, now + count);
return true;
}
}
public boolean localReduce(String key, int count) {
int now = countMap.get(key);
log.debug("CountLimitAspect:{}减少计算量:{}", key, now - count);
countMap.put(key, now - count);
return true;
}
public boolean springRedisCheckExceed(String key, int count, int limit) {
this.checkNode(LockConnectionEnum.SPRING_REDIS);
Integer now = (Integer) redisTemplate.opsForValue().get(CountLimitCommonUtil.COUNT_LIMIT_STORE + CountLimitCommonUtil.getNodeId() + key);
if (now == null) {
now = 0;
}
if (now + count > limit) {
return false;
} else {
log.debug("CountLimitAspect:{}增加计算量:{}", key, now + count);
redisTemplate.opsForValue().set(CountLimitCommonUtil.COUNT_LIMIT_STORE + CountLimitCommonUtil.getNodeId() + key, now + count);
return true;
}
}
public boolean springRedisReduce(String key, int count) {
this.checkNode(LockConnectionEnum.SPRING_REDIS);
Integer now = (Integer) redisTemplate.opsForValue().get(CountLimitCommonUtil.COUNT_LIMIT_STORE + CountLimitCommonUtil.getNodeId() + key);
if (now == null) {
now = 0;
}
log.debug("CountLimitAspect:{}减少计算量:{}", key, now - count);
redisTemplate.opsForValue().set(CountLimitCommonUtil.COUNT_LIMIT_STORE + CountLimitCommonUtil.getNodeId() + key, now - count);
return true;
}
public void checkNode(LockConnectionEnum lockConnectionEnum) {
if (CountLimitCommonUtil.getNodeId() != null) {
return;
}
countLimitNode.setNode(lockConnectionEnum);
}
}
可以看出父类封装了共通的操作redis和本地缓存的方法,同时在使用redis的同时考虑了分布式唯一性,在CountLimitCommonUtil中NODE_ID是可以设置的,可以由使用者自己进行赋值操作,volatile保证可见性。
如果使用者不进行赋值操作也没有问题,工具自动在CountLimitNode中实现了分布式节点的唯一id,保证节点存储的独立性。
@Service
public class CountLimitNode {
@Resource
private RedissonClient redissonClient;
@Resource
private RedisTemplate redisTemplate;
@DistributedProxyLock(key = CountLimitCommonUtil.COUNT_LIMIT_REDIS_NODE_LOCK)
public void setNode(LockConnectionEnum lockConnectionEnum) {
if (CountLimitCommonUtil.getNodeId() != null) {
return;
}
switch (lockConnectionEnum) {
case SPRING_REDIS:
Integer temp = (Integer) redisTemplate.opsForValue().get(CountLimitCommonUtil.COUNT_LIMIT_REDIS_NODE_STORE);
if (temp == null) {
temp = 0;
}
temp++;
redisTemplate.opsForValue().set(CountLimitCommonUtil.COUNT_LIMIT_REDIS_NODE_STORE, temp);
CountLimitCommonUtil.setNodeId(temp.toString());
break;
case REDISSON:
RBucket bucket = redissonClient.getBucket(CountLimitCommonUtil.COUNT_LIMIT_REDIS_NODE_STORE);
Integer now = 0;
if (bucket.isExists()) {
now = bucket.get() + 1;
}
bucket.set(now);
CountLimitCommonUtil.setNodeId(now.toString());
break;
}
}
}
6、切面
@Aspect @Component public class CountLimitAspect三、使用 1、计算量的存储与并发控制主要分为redis和本地两种,目前支持链接redis的工具主要是redisson、spring redis{ @Resource private CountLimitFacadeFactory countLimitFacadeFactory; @Pointcut("@annotation(com.annotation.CountLimit)") public void lockPointCut() { } @Around("lockPointCut() && @annotation(countLimit)") public Object around(ProceedingJoinPoint joinPoint, CountLimit countLimit) throws Throwable { LocalDateTime start = LocalDateTime.now(); String inter = joinPoint.getTarget().getClass().getName() + joinPoint.getSignature().getName(); String objectName = countLimit.objectName(); Object[] args = joinPoint.getArgs(); String[] paramNames = ((CodeSignature) joinPoint.getSignature()).getParameterNames(); Map param = new HashMap<>(); String par = countLimit.paramName(); for (int i = 0; i < paramNames.length; i++) { param.put(paramNames[i], args[i]); } CountFactoryEnum countFactoryEnum = CountFactoryEnum.of(countLimit.countFactoryEnum()); //获取限制的参数 List queryPar; String key = inter + objectName; int count; CountLimitDTO countLimitDTO = new CountLimitDTO(); try { if (StringUtil.isBlank(par)) { //如果没有设置参数,说明在入参对象中 queryPar = (List ) param.get(objectName); } else { //说明在入参的某个对象中,有一个参数是进行限流 Object o = param.get(objectName); queryPar = (List ) CountLimitCommonUtil.getFieldValueByName(par, o); key += par; } count = queryPar.size(); countLimitDTO.setKey(key); countLimitDTO.setCount(count); countLimitDTO.setLimit(countLimit.limit()); countLimitDTO.setIsAdd(Boolean.TRUE); while (!countLimitFacadeFactory.getBean(countFactoryEnum).process(countLimitDTO)) { //是否超出等待时间 if (start.plusSeconds(countLimit.waitTime()).isBefore(LocalDateTime.now())) { throw new CountLimitException("超出等待时间" + key); } //将等待时长划分为等份时长 Thread.sleep(countLimit.waitTime() * 1000 / CountLimitCommonUtil.SPILT_SLEEP); } } catch (Exception e) { throw new CountLimitException("计算限流异常", e); } try { return joinPoint.proceed(); } finally { countLimitDTO.setIsAdd(Boolean.FALSE); boolean success = countLimitFacadeFactory.getBean(countFactoryEnum).process(countLimitDTO); if (!success) { throw new CountLimitException("计算量减少失败" + countLimitDTO.toString()); } } } }
2、因此有以下七种使用方式,默认使用ReentrantLock加锁,本地map缓存
3、如果使用工具的地方很多,存储比较适合使用redis
4、如果限流的方法qps很高,使用redis进行加锁处理可能是比ReentrantLock更好的选择
1、ReentrantLock加锁,本地map缓存
@CountLimit(objectName = "request",
paramName = "shopIdList",
limit = 20000)
public void audit(ShopOnlineDTO request) {
//业务处理
}
2、ReentrantLock加锁,redisson缓存
@CountLimit(objectName = "request",
paramName = "shopIdList",
limit = 20000,
countFactoryEnum = CountLimitCommonUtil.LOCAL_LOCK_REDISSON_STORE)
public void audit(ShopOnlineDTO request) {
//业务处理
}
3、ReentrantLock加锁,spring redis缓存
@CountLimit(objectName = "request",
paramName = "shopIdList",
limit = 20000,
countFactoryEnum = CountLimitCommonUtil.LOCAL_LOCK_SPRING_REDIS_STORE)
public void audit(ShopOnlineDTO request) {
//业务处理
}
4、redisson加锁,本地map缓存
@CountLimit(objectName = "request",
paramName = "shopIdList",
limit = 20000,
countFactoryEnum = CountLimitCommonUtil.REDISSON_LOCK_STORE)
public void audit(ShopOnlineDTO request) {
//业务处理
}
5、redisson加锁,redisson缓存
@CountLimit(objectName = "request",
paramName = "shopIdList",
limit = 20000,
countFactoryEnum = CountLimitCommonUtil.REDISSON_LOCK_REDISSON_STORE)
public void audit(ShopOnlineDTO request) {
//业务处理
}
6、spring redis加锁,本地map缓存
@CountLimit(objectName = "request",
paramName = "shopIdList",
limit = 20000,
countFactoryEnum = CountLimitCommonUtil.SPRING_REDIS_LOCK_STORE)
public void audit(ShopOnlineDTO request) {
//业务处理
}
7、spring redis加锁,spring redis缓存
@CountLimit(objectName = "request",
paramName = "shopIdList",
limit = 20000,
countFactoryEnum = CountLimitCommonUtil.SPRING_REDIS_LOCK_SPRING_REDIS_STORE)
public void audit(ShopOnlineDTO request) {
//业务处理
}
四、总结
新生的工具总是可以做到更多事情的,希望有兴趣的同学发挥想象力提出issue或者加入到工具开源的扩展中。



