网页访问常用到库:
requests(网页请求)
BeautifulSoup(从网页抓取数据)
selenium(模拟浏览器行为)
PhantomJS(虚拟浏览器)
import requests from bs4 import BeautifulSoup from selenium import webdriver res = requests.get(url) res.encoding = ‘utf-8‘ soup = BeautifulSoup(res.text, ‘html.parser‘)
import requests from bs4 import BeautifulSoup from selenium import webdriver res = requests.get(url)res.encoding = ‘utf-8‘soup = BeautifulSoup(res.text, ‘html.parser‘)
定时爬取数据:
import time time.sleep(10)
断线重连解决方法:
引入新函数reloading()
def reloading() try: except: reloading()
时间: 2024-07-30 13:43:17