我想并行化调用，但是不确定是否应该将整个文件读入内存
您应该使用
ExecutorService与bounded的
BlockingQueue。在读入百万行时，您将作业提交到线程池，直到作业
BlockingQueue满为止。这样，您将能够同时运行100个（或最佳数量）的HTTP请求，而无需事先读取文件的所有行。
RejectedExecutionHandler如果队列已满，您将需要设置一个阻止的对象。这比调用方运行处理程序更好。
BlockingQueue<Runnable> queue = new ArrayBlockingQueue<Runnable>(100);// NOTE: you want the min and max thread numbers here to be the same valueThreadPoolExecutor threadPool = new ThreadPoolExecutor(nThreads, nThreads, 0L, TimeUnit.MILLISECONDS, queue);// we need our RejectedExecutionHandler to block if the queue is fullthreadPool.setRejectedExecutionHandler(new RejectedExecutionHandler() { @Override public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {try { // this will block the producer until there's room in the queue executor.getQueue().put(r);} catch (InterruptedException e) { throw new RejectedExecutionException( "Unexpected InterruptedException", e);} }});// now read in the urlswhile ((String url = urlReader.readLine()) != null) { // submit them to the thread-pool. this may block. threadPool.submit(new DownloadUrlRunnable(url));}// after we submit we have to shutdown the poolthreadPool.shutdown();// wait for them to completethreadPool.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS);...private class DownloadUrlRunnable implements Runnable { private final String url; public DownloadUrlRunnable(String url) { this.url = url; } public void run() { // download the URL }}
