Python爬虫——爬取壁纸

总是感觉桌面太单调，又不想下载第三方壁纸软件，于是我利用爬虫下载了几百张壁纸，保存在磁盘里，设置桌面背景为指定文件夹的幻灯片播放，这下解决了桌面壁纸自动更换的问题。

首先导入所需的库：

import requests
from bs4 import BeautifulSoup
import time

定义要爬取的网页网址及请求头：

pic_list_url = 'https://pic.netbian.com/4kdongman/index_2.html'
Myheaders = {
        'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 '
                      '(KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36'
    }

使用requests将网页获取下来，并设置编码格式：

pic_list_html = requests.get(pic_list_url, headers = Myheaders)
pic_list_html.encoding = 'gbk'

使用BeautifulSoup解析网页并获取图片的标签列表：

soup = BeautifulSoup(pic_list_html.text,'lxml')
pic_lists = soup.find('ul',{'class' : 'clearfix'}).find_all('li')

遍历图片标签列表，从标签中获取到图片的详情页，再获取详情页，解析详情页，找到详情页中图片的标签，从该标签中获取图片地址，使用request.get()获取图片字节内容，设置好图片保存地址：

for li in pic_lists:
        pic_url = 'https://pic.netbian.com/' + li.a.get('href')
        
        pic_html = requests.get(pic_url,headers = Myheaders)
        pic_html.encoding = 'gbk'
        
        sp = BeautifulSoup(pic_html.text,'lxml')
        
        pic_download = 'https://pic.netbian.com/' + sp.find('a',{'id' : 'img'}).img.get('src')
        
        img = requests.get(pic_download, headers=Myheaders).content
        path = 'G:\桌面壁纸\dm\'+ str(sp.find('a',{'id' : 'img'}).img.get('title')) + ".jpg"

将获取到的字节内容保存为本地图片：

with open(path, 'wb') as f:
                f.write(img)
                time.sleep(1)
                print("第【{}】页第【{}】张图片下载完成！".format(page,x))
                x += 1

遍历网站的多个页面，加入循环，完整代码如下：

import requests
from bs4 import BeautifulSoup
import time

page = 9
while True:    
    pic_list_url = 'https://pic.netbian.com/4kdongman/index_' + str(page) + '.html'
    Myheaders = {
        'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 '
                      '(KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36'
    }
    
    #req = requests.session()
    pic_list_html = requests.get(pic_list_url, headers = Myheaders)
    pic_list_html.encoding = 'gbk'
    
    soup = BeautifulSoup(pic_list_html.text,'lxml')
    pic_lists = soup.find('ul',{'class' : 'clearfix'}).find_all('li')
    
    x = 1
    for li in pic_lists:
        pic_url = 'https://pic.netbian.com/' + li.a.get('href')
        
        pic_html = requests.get(pic_url,headers = Myheaders)
        pic_html.encoding = 'gbk'
        
        sp = BeautifulSoup(pic_html.text,'lxml')
        
        pic_download = 'https://pic.netbian.com/' + sp.find('a',{'id' : 'img'}).img.get('src')
        #获取返回的字节类型
        img = requests.get(pic_download, headers=Myheaders).content
        path = 'G:\桌面壁纸\dm\'+ str(sp.find('a',{'id' : 'img'}).img.get('title')) + ".jpg"
        
        with open(path, 'wb') as f:
                f.write(img)
                time.sleep(1)
                print("第【{}】页第【{}】张图片下载完成！".format(page,x))
                x += 1
        
    page += 1
    if page == 15:
        print('下载结束！')
        break

下面是抓取在文件夹中的壁纸图片：