动态生成URL的最佳方法是重写Spider 的
start_requests方法:
from scrapy.http.request import Requestdef start_requests(self): with open('urls.txt', 'rb') as urls: for url in urls: yield Request(url, self.parse)
动态生成URL的最佳方法是重写Spider 的
start_requests方法:
from scrapy.http.request import Requestdef start_requests(self): with open('urls.txt', 'rb') as urls: for url in urls: yield Request(url, self.parse)