我一次运行200个以上Spider的解决方案是为该项目创建一个自定义命令。有关实现自定义命令的更多信息，请参见http://doc.scrapy.org/en/latest/topics/commands.html#custom-
project-commands。
YOURPROJECTNAME / commands / allcrawl.py ：
from scrapy.command import ScrapyCommandimport urllibimport urllib2from scrapy import logclass AllCrawlCommand(ScrapyCommand): requires_project = True default_settings = {'LOG_ENABLED': False} def short_desc(self): return "Schedule a run for all available spiders" def run(self, args, opts): url = 'http://localhost:6800/schedule.json' for s in self.crawler.spiders.list(): values = {'project' : 'YOUR_PROJECT_NAME', 'spider' : s} data = urllib.urlenpre(values) req = urllib2.Request(url, data) response = urllib2.urlopen(req) log.msg(response)确保在您的settings.py中包含以下内容
COMMANDS_MODULE = 'YOURPROJECTNAME.commands'
然后，从命令行（在项目目录中），您只需键入
scrapy allcrawl
