批量导入3w条数据需要一分多钟,最终使用多线程分批导入mysql中,将时间压缩到10s以内。
网上比较好的方法就是下面这种,但是这种方式比较繁琐,我们可以根据自己的情况来优化。
@Transactional(rollbackFor = Exception.class) public int addFreshStudentsNew(List3、Future类详解list, String schoolNo) { if (list == null || list.isEmpty()) { return 0; } //这里弄三个list来把list的每个对象的字段分开存储,进一步将过大list缩小 List studentEntityList = new linkedList<>(); List enrollStudentEntityList = new linkedList<>(); List allusersEntityList = new linkedList<>(); //这里是循环制造三个list来拆分过大list,根据自己的实际情况看是否拆分 list.forEach(freshStudentAndStudentModel -> { EnrollStudentEntity enrollStudentEntity = new EnrollStudentEntity(); StudentEntity studentEntity = new StudentEntity(); BeanUtils.copyProperties(freshStudentAndStudentModel, studentEntity); BeanUtils.copyProperties(freshStudentAndStudentModel, enrollStudentEntity); String operator = TenancyContext.UserID.get(); String studentId = baseUuidUtils.base58Uuid(); enrollStudentEntity.setId(baseUuidUtils.base58Uuid()); enrollStudentEntity.setStudentId(studentId); enrollStudentEntity.setIdentityCardId(freshStudentAndStudentModel.getIdCard()); enrollStudentEntity.setOperator(operator); studentEntity.setId(studentId); studentEntity.setIdentityCardId(freshStudentAndStudentModel.getIdCard()); studentEntity.setOperator(operator); studentEntityList.add(studentEntity); enrollStudentEntityList.add(enrollStudentEntity); AllusersEntity allusersEntity = new AllusersEntity(); allusersEntity.setId(enrollStudentEntity.getId()); allusersEntity.setUserCode(enrollStudentEntity.getNemtCode()); allusersEntity.setUserName(enrollStudentEntity.getName()); allusersEntity.setSchoolNo(schoolNo); allusersEntity.setTelNum(enrollStudentEntity.getTelNum()); allusersEntity.setPassword(enrollStudentEntity.getNemtCode()); //密码设置为考生号 allusersEntityList.add(allusersEntity); }); //开启50个线程 int nThreads = 50; //过大list的数据总条数 int size = enrollStudentEntityList.size(); //开启一个有50个线程的线程池 ExecutorService executorService = Executors.newFixedThreadPool(nThreads); //通过实现Callback接口,并用Future可以来接收多线程的执行结果 List > futures = new ArrayList >(nThreads); //遍历生成50个线程来执行任务,每个线程处理50条数据 for (int i = 0; i < nThreads; i++) { final List EnrollStudentEntityImputList = enrollStudentEntityList.subList(size / nThreads * i, size / nThreads * (i + 1)); final List studentEntityimportList = studentEntityList.subList(size / nThreads * i, size / nThreads * (i + 1)); final List allusersEntityimportList = allusersEntityList.subList(size / nThreads * i, size / nThreads * (i + 1)); //线程执行,实现callable接口,批量保存 Callable task1 = () -> { studentSave.saveStudent(EnrollStudentEntityImputList,studentEntityimportList,allusersEntityimportList); return 1; }; //每50条数据提交一次任务,报异常则执行事务回滚,并将结果添加到furures中用于展示 futures.add(executorService.submit(task1)); } //停止接收新任务,原来的任务继续执行 executorService.shutdown(); if (!futures.isEmpty() && futures != null) { return 10; } return -10; }
在并发编程中,我们经常用到非阻塞的模型,在之前的多线程的三种实现中,不管是继承thread类还是实现runnable接口,都无法保证获取到之前的执行结果。通过实现Callback接口,并用Future可以来接收多线程的执行结果。
原文链接:添加链接描述
4、ExecutorService 中 shutdown()、shutdownNow()、awaitTermination() 含义和区别ExecutorService 是 Java 提供的线程池,也就是说,每次我们需要使用线程的时候,可以通过 ExecutorService
创建线程。使用 ExecutorService 类时,经常用到 shutdown() 、shutdownNow()、awaitTermination() 3个方法,下面我们来说说它们的含义和三者的区别 。
原文链接



