Python - cookielib [urllib2/requests]

cookie操作(读取,添加,删除,保存),结合urllib2和requests进行说明,此处只做局部演示,如果有疑问,请查阅: (Author: nixawk)

https://docs.python.org/2/library/cookielib.html

https://hg.python.org/cpython/file/2.7/Lib/cookielib.py

#!/usr/bin/env python

# -*- encoding: utf-8 -*-

from cookielib import Cookie

from cookielib import CookieJar

from cookielib import FileCookieJar

from cookielib import LWPCookieJar

import urllib2

# Author : nixawk

def make_cookie(name, value, domain, path=‘/‘):

return Cookie(version=0,

name=name,

value=value,

port=None,

port_specified=False,

domain=domain,

domain_specified=True,

domain_initial_dot=False,

path=path,

path_specified=True,

secure=False,

expires=None,

discard=False,

comment=None,

comment_url=None,

rest=None,

rfc2109=False)

def add_cookie(cookiejar, cookie):

if isinstance(cookiejar, CookieJar):

cookiejar.set_cookie(cookie)

def del_cookie(cookiejar, domain, path, cookiename):

if isinstance(cookiejar, CookieJar):

cookiejar.clear(domain, path, cookiename)

def get_cookies_from_response(url):

cookiejar = CookieJar()

opener = urllib2.build_opener(

urllib2.HTTPCookieProcessor(cookiejar))

opener.open(url)

# add a new cookie or replace a old one

newcookie = make_cookie(‘newcookie‘, ‘11111‘, ‘.baidu.com‘, ‘/‘)

# remove a cookie

cookiejar.clear(‘.baidu.com‘, ‘/‘, ‘newcookie‘)

cookiejar.set_cookie(newcookie)

import pdb

pdb.set_trace()

return cookiejar

def save_cookies_to_file1(url):

cookiefile = ‘cookies.log‘

filecookiejar = FileCookieJar(filename=cookiefile)

opener = urllib2.build_opener(

urllib2.HTTPCookieProcessor(filecookiejar))

opener.open(url)

# *******************************************

# please read code cookielib [FileCookieJar]

# filecookiejar.save(filename=cookiefile)

# raise NotImplementedError()

# ******************************************

try:

filecookiejar.save()

except NotImplementedError as e:

print e

return filecookiejar

def save_cookies_to_file2(url):

cookiefile = ‘cookies.log‘

lwpcookiejar = LWPCookieJar(filename=cookiefile)

opener = urllib2.build_opener(

urllib2.HTTPCookieProcessor(lwpcookiejar))

opener.open(url)

lwpcookiejar.save()

return lwpcookiejar

from pprint import pprint

pprint(get_cookies_from_response(‘http://www.baidu.com‘))

# pprint(save_cookies_to_file1(‘http://www.baidu.com‘))

# pprint(save_cookies_to_file2(‘http://www.baidu.com‘))

#!/usr/bin/env python

# -*- encoding: utf-8 -*-

import requests

# Author : nixawk

def make_request(method, url, headers={}, files=None, data={},

json=None, params={}, auth=None,

cookies=None, hooks={}):

req = requests.Request(method, url, headers={},

files=None, data={},

json=None, params={},

auth=None,

cookies=None, hooks={})

r = req.prepare()

s = requests.Session()

response = s.send(r)

return response

def get_cookies_from_response(response):

if hasattr(response, ‘cookies‘):

return response.cookies

return None

def save_cookies_to_file(url):

pass

from pprint import pprint

response = make_request(‘GET‘, ‘http://www.baidu.com/‘)

cookies = get_cookies_from_response(response)

pprint(dir(cookies))

时间: 2024-10-13 17:16:54

Python - cookielib [urllib2/requests]的相关文章

人生苦短之Python的urllib urllib2 requests

在Python中涉及到URL请求相关的操作涉及到模块有urllib,urllib2,requests,其中urllib和urllib2是Python自带的HTTP访问标准库,requsets是第三方库,需要自行安装.requests是第三方库,可以想到在使用起来它可能是最方便的一个. urllib和urllib2       urllib和urllib2模块都是跟url请求相关的,但是提供的功能是不同的.我们常用的urllib2的请求方式: response = urllib2.urlopen(

Python第三方包 requests还是urllib?

我注意到一个很奇怪的事情.在极客学院爬虫课视频里,老师说要用requests,但是在其他学习群里,很多同学都在讨论urllib/urllib2相关. 然后就迷惑了,为什么会有这三个东西呢?扒了扒国内的博客网站,大多都是urllib相关,并且不推荐使用requests包.于是去墙外搜了下,竟然发现国外大多推荐用requests.我完全愣到了,完全相反的意见. 再仔细八一下,发现国内很多博客博主,只是说不建议使用requests,然后就介绍urllib,并没有说为什么不选requests:国外的稍微

【python】urllib2

urllib2.urlopen(url[, data][, timeout]) Open the URL url, which can be either a string or a Request object. data may be a string specifying additional data to send to the server, or None if no such data is needed. Currently HTTP requests are the only

基于Python的urllib2模块的多线程网络爬虫程序

1 m Queue import Queue 2 from gzip import GzipFile 3 from StringIO import StringIO 4 import time 5 import socket 6 class ContentEncodingProcessor(urllib2.BaseHandler): 7 """A handler to add gzip capabilities to urllib2 requests ""

python--深入理解urllib;urllib2;requests

摘自:http://1oscar.github.io/blog/2015/07/05/%E6%B7%B1%E5%85%A5%E7%90%86%E8%A7%A3urllib;urllib2;requests.html urllib and urllib2 区别 urllib和urllib2模块都做与请求URL相关的操作,但他们提供不同的功能. urllib2.urlopen accepts an instance of the Request class or a url, (whereas ur

Python:防止urllib2 302自动跳转

例子 import urllib2 class RedirctHandler(urllib2.HTTPRedirectHandler):     """docstring for RedirctHandler"""     def http_error_301(self, req, fp, code, msg, headers):         pass     def http_error_302(self, req, fp, code, m

Python第三方库Requests学习

1.安装 1 git clone git://github.com/kennethreitz/requests.git 2 cd requests 3 python setup.py install 2.上点威力 (GET) 1 >>> import requests 2 >>> url = 'http://dict.baidu.com/s' 3 >>> payload = {'wd':'python'} 4 >>> r = requ

[python]如何在requests post时不做urlencode?

这两天要为公司一个项目的服务端写性能测试脚本,其中有一个请求订单号的接口,需要客户端用post请求向服务端发送加密过的token 和订单数据.代码如下: payload = {'token':token,'order_data':order_data} r = requests.post(url, data=payload} 但是,服务端返回错误,花了很长时间检查了token和order_data的加密算法,确认没错: 查了下文档,上面有这样一句话: There are many times t

python上安装requests

首先需要配置好python的环境变量. 测试是否成功配置,进入命令行,输入python. 如下图为配置成功. 去第三方库的网站下载安装包,解压在python的安装目录 ,下载地址:https://pypi.python.org/pypi/requests#downloads 运行cmd,进入命令行.利用cd命令进入第三方库文件夹的位置. 键入命令python  setup.py install进行第三方库的安装. 最后进入命令行,import库名称,观察第三方库是否安装成功.