下载scrapy的整页

代码将下载此页面并将其保存在文件中

download-a-full-page-with-scrapy.html

test_scr.py

import scrapyclass TestSpider(scrapy.Spider):    name = "test"    start_urls = [        "http://stackoverflow.com/questions/38233614/download-a-full-page-with-scrapy",    ]    def parse(self, response):        filename = response.url.split("/")[-1] + '.html'        with open(filename, 'wb') as f: f.write(response.body)

通过此命令运行

scrapy runspider test_scr.py