1、map, filter, reduce
1) map(func, input_list)
将函数应用到输入列表上的每个元素, 如：
input_list = [1, 2, 3, 4, 5]
def pow_elem(x):
"""
将x做乘方运算
:param x:
:return:
"""
return x * x
def multi_x_y(x, y):
return x * y
print map(pow_elem, input_list) # output:[1, 4, 9, 16, 25]
print map(multi_x_y, input_list, input_list) # output:[1, 4, 9, 16, 25]
2) filter(func_or_none, sequence)
过滤筛选出sequence中满足函数返回True的值，组成新的sequence返回，如：
def is_odd(x):
"""
判断x是否为奇数
:param x:
:return:
"""
return True if x % 2 > 0 else False
print filter(is_odd, input_list) # output: [1, 3, 5]
3） reduce(function, sequence)
reduce()函数接收的参数和 map()类似，一个函数 f，一个list，但行为和 map()不同，reduce()传入的函数 f 必须接收两个参数，reduce()对list的每个元素反复调用函数f，并返回最终结果值。例如：reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) 等价于((((1+2)+3)+4)+5)
print reduce(lambda x, y: x * y, input_list) # output: 120
2、三元运算
下面两种写法等价：
"Yes" if 2==2 else "No"
("No", "Yes")[2==2]
即：
1) condition_is_true if condition else condition_is_false
2) (if_test_is_false, if_test_is_true)[test]
1)和2）都能实现三元运算， 但是2）较为少见，且不太优雅，同时2）并不是一个短路运算，如下：
5 if True else 5/0 # output: 5
(1/0, 5)[True] # throw exception-> ZeroDivisionError: integer division or modulo by zero
3、装饰器
1) Python中，我们可以在函数内部定义函数并调用，如：
def hi(name="patty"):
print("now you are inside the hi() function")
def greet():
return "now you are in the greet() function"
def welcome():
return "now you are in the welcome() function"
print(greet())
print(welcome())
print("now you are back in the hi() function")
输出结果为：
now you are inside the hi() function
now you are in the greet() function
now you are in the welcome() function
now you are back in the hi() function
2) 也可以将内部函数返回，利用外部函数进行调用， 如：
def hi(name="patty"):
def greet():
return "now you are in the greet() function"
def welcome():
return "now you are in the welcome() function"
return greet if name == ‘patty‘ else welcome
print hi() # <function greet at 0x109379a28>
print hi()() # now you are in the greet() function
上述代码中，hi()调用返回的是一个function对象，从if/else语句中可以判断出，返回的是greet()函数，当我们调用hi()()时，实际上是调用了内部函数greet()。
3）将函数作为参数传递给另一个函数， 如：
def hi():
return "hi patty!"
def doSomethingBeforeHi(func):
print("I am doing some boring work before executing hi()")
print(func())
doSomethingBeforeHi(hi)
输出结果：
I am doing some boring work before executing hi()
hi patty!
至此， 我们已经实现了一个简单的装饰器， 在调用hi()函数之前， 先输出一行，实际应用中可能是一些预处理操作。实际上，装饰器的功能就是在你的核心逻辑执行前后，加上一些通用的功能。
4） 简单装饰器的实现
def a_new_decorator(a_func):
def wrapTheFunction():
print("I am doing some boring work before executing a_func()")
a_func() # call this function
print("I am doing some boring work after executing a_func()")
return wrapTheFunction
def a_function_requiring_decoration():
print("I am the function which needs some decoration to remove my foul smell")
a_function_requiring_decoration()
#outputs: "I am the function which needs some decoration to remove my foul smell"
a_function_requiring_decoration = a_new_decorator(a_function_requiring_decoration)
#now a_function_requiring_decoration is wrapped by wrapTheFunction()
a_function_requiring_decoration()
# I am doing some boring work before executing a_func()
# I am the function which needs some decoration to remove my foul smell
# I am doing some boring work after executing a_func()
5) 注解形式
@a_new_decorator
def b_function_requiring_decoration():
print("I am the another function which needs some decoration to remove my foul smell")
b_function_requiring_decoration()
# I am doing some boring work before executing a_func()
# I am the another function which needs some decoration to remove my foul smell
# I am doing some boring work after executing a_func()
此处@a_new_decorator就等价于a_new_decorator(b_function_requiring_decoration)
6) 获取name
对于4）中的a_function_requiring_decoration， 我们打印print(a_function_requiring_decoration.__name__) 得到的结果是wrapTheFunction，而实际上我们希望得到的是a_func所对应的a_function_requiring_decoration函数名，Python为我们提供了wraps用来解决这个问题。
from functools import wraps
def a_new_decorator(a_func):
@wraps(a_func)
def wrapTheFunction():
print("I am doing some boring work before executing a_func()")
a_func()
print("I am doing some boring work after executing a_func()")
return wrapTheFunction
7） 装饰器的一些应用场景
用户认证
def requires_auth(f):
@wraps(f)
def decorated(*args, **kwargs):
auth = {"username": "patty", "password": "123456"}
if not check_auth(auth[‘username‘], auth[‘password‘]):
authenticate()
return f(*args, **kwargs)
def check_auth(username, password):
print "Starting check auth..."
return True if (username == ‘patty‘ and password == ‘123456‘) else False
def authenticate():
print "Already authenticate"
return decorated
@requires_auth
def welcome():
return "Welcome patty!"
print welcome()
日志记录
def logit(func):
@wraps(func)
def with_logging(*args, **kwargs):
print(func.__name__ + " was called")
return func(*args, **kwargs)
return with_logging
@logit
def addition_func(x):
"""Do some math."""
return x + x
result = addition_func(4)
将会打印：addition_func was called
8）带参数的装饰器
from functools import wraps
def logit(logfile=‘out.log‘):
def logging_decorator(func):
@wraps(func)
def wrapped_function(*args, **kwargs):
log_string = func.__name__ + " was called"
print(log_string)
# Open the logfile and append
with open(logfile, ‘a‘) as opened_file:
# Now we log to the specified logfile
opened_file.write(log_string + ‘\n‘)
return wrapped_function
return logging_decorator
@logit()
def myfunc1():
pass
myfunc1()
# Output: myfunc1 was called
# A file called out.log now exists, with the above string
@logit(logfile=‘func2.log‘)
def myfunc2():
pass
myfunc2()
9) 用类作为装饰器
import os
class Logit(object):
def __init__(self, log_file):
self.log_file = log_file
def __call__(self, func):
with open(self.log_file, ‘a‘) as fout:
log_msg = func.__name__ + " was called"
fout.write(log_msg)
fout.write(os.linesep)
# Now, send a notification
self.notify()
def notify(self):
# logit only logs, no more
pass
class EmailLogit(Logit):
‘‘‘
A logit implementation for sending emails to admins
when the function is called.
‘‘‘
def __init__(self, log_file, email=‘[email protected]‘):
self.email = email
super(EmailLogit, self).__init__(log_file)
def notify(self):
# Send an email to self.email
# Will not be implemented here
with open(self.log_file, ‘a‘) as f:
f.write("Do Something....")
f.write(os.linesep)
f.write("Email has send to " + self.email)
f.write(os.linesep)
@Logit("log1.txt")
def myfunc3():
pass
@EmailLogit("log2.txt")
def myfunc4():
pass
用类作为装饰器，我们的代码看上去更简洁， 而且还可以通过继承的方式，实现功能的个性化和复用。
4、可变类型
Python中的可变类型包括列表和字典，这些对象中的元素是可改变的，如
>>> foo = [‘hi‘]
>>> foo += [‘patty‘]
>>> foo
[‘hi‘, ‘patty‘]
>>> foo[0]=‘hello‘
>>> foo
[‘hello‘, ‘patty‘]
>>> fdict = {"name":"patty"}
>>> fdict.update({"age":"23"})
>>> fdict
{‘age‘: ‘23‘, ‘name‘: ‘patty‘}
>>> fdict.update({"age":"25"})
>>> fdict
{‘age‘: ‘25‘, ‘name‘: ‘patty‘}
在方法中，若传入的参数采用可变类型并赋默认值，要注意会出现以下情况：
>>> def add_to(num, target=[]):
... target.append(num)
... return target
...
>>> add_to(1)
[1]
>>> add_to(2)
[1, 2]
>>> add_to(3)
[1, 2, 3]
这是因为， 默认参数在方法被定义时进行计算，而不是每次调用时再计算一次。因此， 为了避免出现上述情况， 当我们期待每次方法被调用时，以一个新的空列表进行计算的时候，可采取如下写法：
>>> def add_to(num, target=None):
... if target is None:
... target = []
... target.append(num)
... return target
...
>>> add_to(1)
[1]
>>> add_to(2)
[2]
5、浅拷贝和深拷贝
Python中，对象的赋值，拷贝（深/浅拷贝）之间是有差异的，如果使用的时候不注意，就可能产生意外的结果。
1) Python中默认是浅拷贝方式
>>> foo = [‘hi‘]
>>> bar = foo
>>> id(foo)
4458211232
>>> id(bar)
4458211232
>>> bar.append("patty")
>>> bar
[‘hi‘, ‘patty‘]
>>> foo
[‘hi‘, ‘patty‘]
注意：id(foo)==id(bar)，说明foo和bar引用的是同一个对象， 当通过bar引用对list进行append操作时， 由于指向的是同一块内存空间，foo的输出与bar是一致的。
2) 深拷贝
>>> foo
[‘hi‘, {‘age‘: 20, ‘name‘: ‘patty‘}]
>>> import copy
>>> slow = copy.deepcopy(foo)
>>> slow
[‘hi‘, {‘age‘: 20, ‘name‘: ‘patty‘}]
>>> slow[0]=‘hello‘
>>> slow
[‘hello‘, {‘age‘: 20, ‘name‘: ‘patty‘}]
>>> foo
[‘hi‘, {‘age‘: 20, ‘name‘: ‘patty‘}]
注意： 由于slow是对foo的深拷贝，实际上是在内存中新开了一片空间，将foo对象所引用的内容复制到新的内存空间中，因此当对slow对像所引用的内容进行update操作后，更改只体现在slow对象的引用上，而foo对象所引用的内容并没有发生改变。
6、集合Collection
1) defaultdict
对于普通的dict,若是获取不存在的key，会引发KeyError错误,如下：
some_dict = {}
some_dict[‘colours‘][‘favourite‘] = "yellow"
# Raises KeyError: ‘colours‘
但是通过defaultdict，我们可以避免这种情况的发生， 如下：
import collections
import json
tree = lambda: collections.defaultdict(tree)
some_dict = tree()
some_dict[‘colours‘][‘favourite‘] = "yellow"
print json.dumps(some_dict)
# Works fine, output: {"colours": {"favourite": "yellow"}}
2) OrderedDict
OrderedDict能够按照我们定义字典时的key顺序打印输出字典，改变value的值不会改变key的顺序， 但是，对key进行删除，重新插入后，key会重新排序到dict的尾部。
from collections import OrderedDict
colours = OrderedDict([("Red", 198), ("Green", 170), ("Blue", 160)])
for key, value in colours.items():
print(key, value)
3）Counter
利用Counter，可以统计特定项的出现次数，如：
from collections import Counter
colours = (
(‘Yasoob‘, ‘Yellow‘),
(‘Ali‘, ‘Blue‘),
(‘Arham‘, ‘Green‘),
(‘Ali‘, ‘Black‘),
(‘Yasoob‘, ‘Red‘),
(‘Ahmed‘, ‘Silver‘),
)
favs = Counter(name for name, colour in colours)
print(favs)
# Counter({‘Yasoob‘: 2, ‘Ali‘: 2, ‘Arham‘: 1, ‘Ahmed‘: 1})
4）deque
deque是一个双端队列，可在头尾分别进行插入，删除操作， 如下：
from collections import deque
queue_d = deque()
queue_d.append(1)
queue_d.append(2)
print queue_d # deque([1, 2])
queue_d.appendleft(3)
print queue_d # deque([3, 1, 2])
queue_d.pop()
print queue_d # deque([3, 1])
queue_d.popleft()
print queue_d # deque([1])
deque可以设置队列的最大长度，当元素数目超过最大长度时，会从当前带插入方向的反方向删除相应数目的元素，如下：
queue_c = deque(maxlen=5, iterable=[2, 4, 6])
queue_c.extend([7, 8])
print queue_c # deque([2, 4, 6, 7, 8], maxlen=5)
queue_c.extend([10, 12])
print(queue_c) # deque([6, 7, 8, 10, 12], maxlen=5)
queue_c.extendleft([18])
print(queue_c) # deque([18, 6, 7, 8, 10], maxlen=5)
5）nametuple
tuple是不可变的列表，不可以对tuple中的元素重新赋值，我们只能通过index去访问tuple中的元素。nametuple可看做不可变的字典，可通过name去访问tuple中的元素。如：
from collections import namedtuple
Animal = namedtuple(‘Animal‘, ‘name age type‘)
perry = Animal(name="perry", age=31, type="cat")
print(perry)
# Output: Animal(name=‘perry‘, age=31, type=‘cat‘)
print(perry.name)
# Output: ‘perry‘
print(perry[0])
# Output: ‘perry‘
print(perry._asdict())
# Output: OrderedDict([(‘name‘, ‘perry‘), (‘age‘, 31), (‘type‘, ‘cat‘)])
7、Object introspection
1) dir: 列举该对象的所有方法
2）type: 返回对象的类型
3）id: 返回对象的id
8、生成器
1）list
>>> squared = [x**2 for x in range(10)]
>>> squared
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
2) dict
{v: k for k, v in some_dict.items()}
3) set
>>> squared = {x**2 for x in range(10)}
>>> squared
set([0, 1, 4, 81, 64, 9, 16, 49, 25, 36])
9、异常处理
try:
print(‘I am sure no exception is going to occur!‘)
except Exception:
print(‘exception‘)
else:
# any code that should only run if no exception occurs in the try,
# but for which exceptions should NOT be caught
print(‘This would only run if no exception occurs. And an error here ‘
‘would NOT be caught.‘)
finally:
print(‘This would be printed in every case.‘)
# Output: I am sure no exception is going to occur!
# This would only run if no exception occurs.
# This would be printed in every case.
else中的语句会在finally之前执行。
10、内置方法
a_list = [[1, 2], [3, 4], [5, 6]]
print(list(itertools.chain.from_iterable(a_list)))
# Output: [1, 2, 3, 4, 5, 6]
# or
print(list(itertools.chain(*a_list)))
# Output: [1, 2, 3, 4, 5, 6]
class A(object):
def __init__(self, a, b, c, d, e, f):
self.__dict__.update({k: v for k, v in locals().items() if k != ‘self‘})
11、for-else语句
for语句的正常结束方式有两种：一是在满足特定条件的情况下break跳出循环，二是所有条件循环结束。 for-else中的else语句只有在所有条件都经过判断然后正常结束for循环的情况下，才被执行，如下：
for x in range(1, 10, 2):
if x % 2 == 0:
print "found even of %d"%x
break
else:
print "not foud even"
# output: not foud even
12、兼容Python 2+和Python 3+
1） 利用 __future__模块在Python 2+的环境中引用Python 3+的模块
2）兼容的模块导入方式
try:
import urllib.request as urllib_request # for Python 3
except ImportError:
import urllib2 as urllib_request # for Python 2
Reference:http://book.pythontips.com/en/latest/index.html