爬虫手册02 Requests的使用

目标： 列举Requests常用的功能，方便查阅。

import requests

r = requests.get('https://www.baidu.com/')
print(type(r))
print(r.status_code)
print(type(r.text))
print(r.text[:100])
print(r.cookies)

运行结果：

200


 ]>

import requests  

data = {  
    'name': 'germey',  
    'age': 25
}  
r = requests.get('https://httpbin.org/get', params=data)  
print(r.text)
print(type(r.text))
print(r.json())
print(type(r.json()))

运行结果：返回的是json字符串, 可以直接调用json()转为字典

{
  "args": {
    "age": "25",    
    "name": "germey"
  }, 
  "headers": {
    "Accept": "**', 'Accept-Encoding': 'gzip, deflate', 'Host': 'httpbin.org', 'User-Agent': 'python-requests/2.22.0', 'X-Amzn-Trace-Id': 'Root=1-620913df-3f5acf216ce4775c03354c66'}, 'origin': '192.168.1.1', 'url': 'https://httpbin.org/get?name=germey&age=25'}

import requests

r = requests.get('https://github.com/favicon.ico')
with open('favicon.ico', 'wb') as f:
    f.write(r.content)

往文件里写就可以保存二进制数据。

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36'
}
r = requests.get('https://httpbin.org/get', headers=headers)
print(r.text)

运行结果：

{
  "args": {},
  "headers": {
    "Accept": "**", 
    "Accept-Encoding": "gzip, deflate", 
    "Content-Length": "18", 
    "Content-Type": "application/x-www-form-urlencoded",         
    "Host": "www.httpbin.org", 
    "User-Agent": "python-requests/2.22.0", 
    "X-Amzn-Trace-Id": "Root=1-62091826-3152bf7a1515a77d072a49b8"
  }, 
  "json": null, 
  "origin": "192.168.1.1", 
  "url": "https://www.httpbin.org/post"
}

import requests

r = requests.get('https://httpbin.org/get')
print(type(r.status_code), r.status_code)
print(type(r.headers), r.headers)
print(type(r.cookies), r.cookies)
print(type(r.url), r.url)
print(type(r.history), r.history)

运行结果：

 200
 {'Date': 'Sun, 13 Feb 2022 14:48:27 GMT', 'Content-Type': 'application/json', 'Content-Length': '308', 'Connection': 'keep-alive', 'Server': 'gunicorn/19.9.0', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Credentials': 'true'}
 
 https://httpbin.org/get
 []

import requests

r = requests.get('https://ssr1.scrape.center/')
exit() if not r.status_code == requests.codes.ok else print('Request Successfully')

运行结果：三元表达式, 语句1 if 条件 else 语句2, 条件为真执行语句1, 条件为假执行语句2

Request Successfully

import requests

files = {'file': open('favicon.ico', 'rb')}
r = requests.post('https://www.httpbin.org/post', files=files)
print(r.text)

运行结果：file部分就是上传的文件信息, 二进制内容部分省略

{
  "args": {},
  "data": "",
  "files": {
    "file": "data:application/octet-stream;base64,AAABAAI..."
  },
  "form": {},
  "headers": {
    "Accept": "**", 
    "Accept-Encoding": "gzip, deflate", 
    "Content-Length": "11", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "httpbin.org", 
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36", 
    "X-Amzn-Trace-Id": "Root=1-62092798-72199d8761a339441af9b39c"
  },
  "json": null,
  "origin": "192.168.1.1",
  "url": "https://httpbin.org/post"
}