分分钟几万上下！用Python爬取包图网视频（附代码）切勿商用！

1. 分析网页数据结构
   

经分析我们可以发现总站数据我们可以从这四这选项下手

2. 分析网页数据格式
   

网页数据为静态

3. 抓取下一页链接

4. OK, 上代码!

import requests
from lxml import etree
import threading


class Spider(object):
    def __init__(self):
 self.headers = {"User-Agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36"}
 self.offset = 1

    def start_work(self, url):
 print("正在爬取第 %d 页......" % self.offset)
 self.offset += 1
 response = requests.get(url=url,headers=self.headers)
 html = response.content.decode()
 html = etree.HTML(html)

 video_src = html.xpath('//div[@class="video-play"]/video/@src')
 video_title = html.xpath('//span[@class="video-title"]/text()')
 next_page = "http:" + html.xpath('//a[@class="next"]/@href')[0]
 # 爬取完毕...
 if next_page == "http:":
     return

 self.write_file(video_src, video_title)
 self.start_work(next_page)

    def write_file(self, video_src, video_title):
 for src, title in zip(video_src, video_title):
     response = requests.get("http:"+ src, headers=self.headers)
     file_name = title + ".mp4"
     file_name = "".join(file_name.split("/"))
     print("正在抓取%s" % file_name)
     with open(file_name, "wb") as f:
  f.write(response.content)

if __name__ == "__main__":
    spider = Spider()
    for i in range(0,3):
 # spider.start_work(url="https://ibaotu.com/shipin/7-0-0-0-"+ str(i) +"-1.html")
 t = threading.Thread(target=spider.start_work, args=("https://ibaotu.com/shipin/7-0-0-0-"+ str(i) +"-1.html",))
 t.start()

5. 运行结果
   
   

(老师声明下, 仅限爬取绿色网站，其它颜色的不准的噢…)!
(原创文章，转载请注明出处)