一：改变索引
reindex方法对于Series直接索引，对于DataFrame既可以改变行索引，也可以改变列索引，还可以两个一起改变.
1）对于Series
1 In [2]: seri = pd.Series([4.5,7.2,-5.3,3.6],index = [‘d‘,‘b‘,‘a‘,‘c‘]) 2 3 In [3]: seri 4 Out[3]: 5 d 4.5 6 b 7.2 7 a -5.3 8 c 3.6 9 dtype: float64 10 11 In [4]: seri1 = seri.reindex([‘a‘,‘b‘,‘c‘,‘d‘,‘e‘]) 12 13 In [5]: seri1 14 Out[5]: 15 a -5.3 16 b 7.2 17 c 3.6 18 d 4.5 19 e NaN #没有的即为NaN 20 dtype: float64 21 22 In [6]: seri.reindex([‘a‘,‘b‘,‘c‘,‘d‘,‘e‘], fill_value=0) 23 Out[6]: 24 a -5.3 25 b 7.2 26 c 3.6 27 d 4.5 28 e 0.0 #没有的填充为0 29 dtype: float64 30 31 In [7]: seri 32 Out[7]: 33 d 4.5 34 b 7.2 35 a -5.3 36 c 3.6 37 dtype: float64 38 39 In [8]: seri_2 = pd.Series([‘blue‘,‘purple‘,‘yellow‘], index=[0,2,4]) 40 41 In [9]: seri_2 42 Out[9]: 43 0 blue 44 2 purple 45 4 yellow 46 dtype: object 47 48 #reindex可用的方法：ffill为向前填充，bfill为向后填充 49 50 In [10]: seri_2.reindex(range(6),method=‘ffill‘) 51 Out[10]: 52 0 blue 53 1 blue 54 2 purple 55 3 purple 56 4 yellow 57 5 yellow 58 dtype: object 59 60 In [11]: seri_2.reindex(range(6),method=‘bfill‘) 61 Out[11]: 62 0 blue 63 1 purple 64 2 purple 65 3 yellow 66 4 yellow 67 5 NaN 68 dtype: object
Series的改变索引
2）对于DataFrame
其reindex的函数参数：method="ffill/bfill";fill_value=...[若为NaN时的填充值]；......
1 In [4]: dframe_1 = pd.DataFrame(np.arange(9).reshape((3,3)),index=[‘a‘,‘b‘,‘c‘], 2 columns=[‘Ohio‘,‘Texas‘,‘Cal‘]) 3 In [5]: dframe_1 4 Out[5]: 5 Ohio Texas Cal 6 a 0 1 2 7 b 3 4 5 8 c 6 7 8 9 10 In [6]: dframe_2 = dframe_1.reindex([‘a‘,‘b‘,‘c‘,‘d‘]) 11 12 In [7]: dframe_2 13 Out[7]: 14 Ohio Texas Cal 15 a 0 1 2 16 b 3 4 5 17 c 6 7 8 18 d NaN NaN NaN 19 20 In [16]: dframe_1.reindex(index=[‘a‘,‘b‘,‘c‘,‘d‘],method=‘ffill‘,columns=[‘Ohio‘ 21 ,‘Beijin‘,‘Cal‘]) 22 Out[16]: 23 Ohio Beijin Cal 24 a 0 NaN 2 25 b 3 NaN 5 26 c 6 NaN 8 27 d 6 NaN 8 28 29 In [17]: dframe_1.reindex(index=[‘a‘,‘b‘,‘c‘,‘d‘],fill_value=‘Z‘,columns=[‘Ohio‘ 30 Out[17]: ,‘Cal‘]) 31 Ohio Beijin Cal 32 a 0 Z 2 33 b 3 Z 5 34 c 6 Z 8 35 d Z Z Z 36 37 In [8]: dframe_1.reindex(columns=[‘Chengdu‘,‘Beijin‘,‘Shanghai‘,‘Guangdong‘]) 38 Out[8]: 39 Chengdu Beijin Shanghai Guangdong 40 a NaN NaN NaN NaN 41 b NaN NaN NaN NaN 42 c NaN NaN NaN NaN 43 44 In [9]: dframe_1 45 Out[9]: 46 Ohio Texas Cal 47 a 0 1 2 48 b 3 4 5 49 c 6 7 8 50 51 #用ix关键字同时改变行/列索引 52 In [10]: dframe_1.ix[[‘a‘,‘b‘,‘c‘,‘d‘],[‘Ohio‘,‘Beijing‘,‘Guangdong‘]] 53 Out[10]: 54 Ohio Beijing Guangdong 55 a 0 NaN NaN 56 b 3 NaN NaN 57 c 6 NaN NaN 58 d NaN NaN NaN
DataFrame的改变索引
二：丢弃指定轴的数据
drop方法, 通过索引删除
1）对于Series
1 In [21]: seri = pd.Series(np.arange(5),index=[‘a‘,‘b‘,‘c‘,‘d‘,‘e‘]) 2 3 In [22]: seri 4 Out[22]: 5 a 0 6 b 1 7 c 2 8 d 3 9 e 4 10 dtype: int32 11 12 In [23]: seri.drop(‘b‘) 13 Out[23]: 14 a 0 15 c 2 16 d 3 17 e 4 18 dtype: int32 19 20 In [24]: seri.drop([‘d‘,‘e‘]) 21 Out[24]: 22 a 0 23 b 1 24 c 2 25 dtype: int32
Series的删除数据
2）对于DataFrame
1 In [29]: dframe = pd.DataFrame(np.arange(16).reshape((4,4)),index=[‘Chen‘,‘Bei‘, 2 ‘Shang‘,‘Guang‘],columns=[‘one‘,‘two‘,‘three‘,‘four‘]) 3 4 In [30]: dframe 5 Out[30]: 6 one two three four 7 Chen 0 1 2 3 8 Bei 4 5 6 7 9 Shang 8 9 10 11 10 Guang 12 13 14 15 11 12 #删除行 13 In [31]: dframe.drop([‘Bei‘,‘Shang‘]) 14 Out[31]: 15 one two three four 16 Chen 0 1 2 3 17 Guang 12 13 14 15 18 19 #删除列 20 In [33]: dframe.drop([‘two‘,‘three‘],axis=1) 21 Out[33]: 22 one four 23 Chen 0 3 24 Bei 4 7 25 Shang 8 11 26 Guang 12 15 27 28 #若第一个参数只有一个时可以不要【】
DataFrame的删除数据
三：索引，选取，过滤
1）Series
仍然可以向list那些那样用下标访问，不过我觉得不太还，最好还是选择用索引值来进行访问，并且索引值也可用于切片
In [4]: seri = pd.Series(np.arange(4),index=[‘a‘,‘b‘,‘c‘,‘d‘]) In [5]: seri Out[5]: a 0 b 1 c 2 d 3 dtype: int32 In [6]: seri[‘a‘] Out[6]: 0 In [7]: seri[[‘b‘,‘a‘]] #显示顺序也变了 Out[7]: b 1 a 0 dtype: int32 In [18]: seri[seri<2] #！！元素级别运算！！ Out[18]: a 0 b 1 dtype: int32 In [11]: seri[‘a‘:‘c‘] #索引用于切片 Out[11]: a 0 b 1 c 2 dtype: int32 In [12]: seri[‘a‘:‘c‘]=‘z‘ In [13]: seri Out[13]: a z b z c z d 3 dtype: object
Series选取
2）DataFrame
其实就是获取一个或多个列的问题。需要注意的是，其实DataFrame可以看作多列索引相同的Series组成的，对应DataFrame数据来说，其首行横向的字段才应该看作是他的索引，所以通过dframe【【n个索引值】】可以选出多列Series，而其中的索引值必须是首行横向的字段，否者报错。而想要取列的话可以通过切片完成，如dframe[:2]选出第0和1行。通过ix【参数1（x），参数2（y）】可以在两个方向上进行选取。
1 In [19]: dframe = pd.DataFrame(np.arange(16).reshape((4,4)),index=[‘one‘,‘two‘,‘ 2 three‘,‘four‘],columns=[‘Bei‘,‘Shang‘,‘Guang‘,‘Sheng‘]) 3 4 In [21]: dframe 5 Out[21]: 6 Bei Shang Guang Sheng 7 one 0 1 2 3 8 two 4 5 6 7 9 three 8 9 10 11 10 four 12 13 14 15 11 12 In [22]: dframe[[‘one‘]] #即是开头讲的索引值用的不正确而报错 13 --------------------------------------------------------------------------- 14 KeyError Traceback (most recent call last) 15 <ipython-input-22-c2522043b676> in <module>() 16 ----> 1 dframe[[‘one‘]] 17 18 In [25]: dframe[[‘Bei‘]] 19 Out[25]: 20 Bei 21 one 0 22 two 4 23 three 8 24 four 12 25 26 In [26]: dframe[[‘Bei‘,‘Sheng‘]] 27 Out[26]: 28 Bei Sheng 29 one 0 3 30 two 4 7 31 three 8 11 32 four 12 15 33 34 In [27]: dframe[:2] #取行 35 Out[27]: 36 Bei Shang Guang Sheng 37 one 0 1 2 3 38 two 4 5 6 7 39 40 In [32]: #为了在DataFrame中引入标签索引，用ix字段，其第一个参数是对行的控制，第二个为对列的控制 41 42 In [33]: dframe.ix[[‘one‘,‘two‘],[‘Bei‘,‘Shang‘]] 43 Out[33]: 44 Bei Shang 45 one 0 1 46 two 4 5 47 48 #有此可看出横向的每个字段为dframe实例的属性 49 In [35]: dframe.Bei 50 Out[35]: 51 one 0 52 two 4 53 three 8 54 four 12 55 Name: Bei, dtype: int32 56 57 In [36]: dframe[dframe.Bei<5] 58 Out[36]: 59 Bei Shang Guang Sheng 60 one 0 1 2 3 61 two 4 5 6 7 62 63 In [38]: dframe.ix[dframe.Bei<5,:2] 64 Out[38]: 65 Bei Shang 66 one 0 1 67 two 4 5 68 69 In [43]: dframe.ix[:‘two‘,[‘Shang‘,‘Bei‘]] 70 Out[43]: 71 Shang Bei 72 one 1 0 73 two 5 4
DataFrame选取
四：算术运算
1）Series
在运算时会自动按索引对齐后再运算，且在索引值不重叠时产生的运算结果是NaN值, 用运算函数时可以避免此情况。
1 In [4]: seri_1 = pd.Series([1,2,3,4],index = [‘a‘,‘b‘,‘c‘,‘d‘]) 2 3 In [5]: seri_2 = pd.Series([5,6,7,8,9],index = [‘a‘,‘c‘,‘e‘,‘g‘,‘f‘]) 4 5 In [6]: seri_1 + seri_2 6 Out[6]: 7 a 6 8 b NaN 9 c 9 10 d NaN 11 e NaN 12 f NaN 13 g NaN 14 dtype: float64 15 16 In [8]: seri_1.add(seri_2) 17 Out[8]: 18 a 6 19 b NaN 20 c 9 21 d NaN 22 e NaN 23 f NaN 24 g NaN 25 dtype: float64 26 27 In [7]: seri_1.add(seri_2,fill_value = 0) 28 Out[7]: 29 a 6 30 b 2 31 c 9 32 d 4 33 e 7 34 f 9 35 g 8 36 dtype: float64 37 38 #上面的未重叠区依然有显示值而不是NaN!! 39 #对应的方法是：add：+； mul: X; sub: -; div : /
Series算术运算
2）DataFrame
1 In [10]: df_1 = pd.DataFrame(np.arange(12).reshape((3,4)),columns = list(‘abcd‘) 2 ) 3 In [11]: df_2 = pd.DataFrame(np.arange(20).reshape((4,5)),columns = list(‘abcde‘ 4 )) 5 In [12]: df_1 + df_2 6 Out[12]: 7 a b c d e 8 0 0 2 4 6 NaN 9 1 9 11 13 15 NaN 10 2 18 20 22 24 NaN 11 3 NaN NaN NaN NaN NaN 12 13 In [13]: df_1.add(df_2) 14 Out[13]: 15 a b c d e 16 0 0 2 4 6 NaN 17 1 9 11 13 15 NaN 18 2 18 20 22 24 NaN 19 3 NaN NaN NaN NaN NaN 20 21 In [14]: df_1.add(df_2, fill_value = 0) 22 Out[14]: 23 a b c d e 24 0 0 2 4 6 4 25 1 9 11 13 15 9 26 2 18 20 22 24 14 27 3 15 16 17 18 19
DataFrame算术运算
3）DataFrame与Series之间进行运算
类似：np.array
1 In [15]: arr_1 = np.arange(12).reshape((3,4)) 2 3 In [16]: arr_1 - arr_1[0] 4 Out[16]: 5 array([[0, 0, 0, 0], 6 [4, 4, 4, 4], 7 [8, 8, 8, 8]]) 8 9 In [17]: arr_1 10 Out[17]: 11 array([[ 0, 1, 2, 3], 12 [ 4, 5, 6, 7], 13 [ 8, 9, 10, 11]])
array型
1 In [18]: dframe_1 = pd.DataFrame(np.arange(12).reshape((4,3)),columns=list(‘bde‘ 2 ),index = [‘Chen‘,‘Bei‘,‘Shang‘,‘Sheng‘]) 3 In [19]: dframe_1 4 Out[19]: 5 b d e 6 Chen 0 1 2 7 Bei 3 4 5 8 Shang 6 7 8 9 Sheng 9 10 11 10 11 In [20]: seri = dframe_1.ix[0] 12 13 In [21]: seri 14 Out[21]: 15 b 0 16 d 1 17 e 2 18 Name: Chen, dtype: int32 19 20 In [22]: dframe_1 - seri #每行匹配的进行运算 21 Out[22]: 22 b d e 23 Chen 0 0 0 24 Bei 3 3 3 25 Shang 6 6 6 26 Sheng 9 9 9 27 28 In [23]: seri_2 = pd.Series(range(3),index=[‘b‘,‘e‘,‘f‘]) 29 30 In [24]: dframe_1 - seri_2 31 Out[24]: 32 b d e f 33 Chen 0 NaN 1 NaN 34 Bei 3 NaN 4 NaN 35 Shang 6 NaN 7 NaN 36 Sheng 9 NaN 10 NaN 37 38 In [27]: seri_3 = dframe_1[‘d‘] 39 40 In [28]: seri_3 #注意！Serie_3索引并不与dframe_1的相同，与上面的运算形式不同 41 Out[28]: 42 Chen 1 43 Bei 4 44 Shang 7 45 Sheng 10 46 Name: d, dtype: int32 47 48 In [29]: dframe_1 - seri_3 49 Out[29]: 50 Bei Chen Shang Sheng b d e 51 Chen NaN NaN NaN NaN NaN NaN NaN 52 Bei NaN NaN NaN NaN NaN NaN NaN 53 Shang NaN NaN NaN NaN NaN NaN NaN 54 Sheng NaN NaN NaN NaN NaN NaN NaN 55 #注意dframe的columns已经变成了Series的index和其自己的columns相加了 56 57 #通过运算函数中的axis参数可改变匹配轴以避免上情况 58 #0为列匹配，1为行匹配 59 In [31]: dframe_1.sub(seri_3,axis=0) 60 Out[31]: 61 b d e 62 Chen -1 0 1 63 Bei -1 0 1 64 Shang -1 0 1 65 Sheng -1 0 1 66 67 In [33]: dframe_1.sub(seri_3,axis=1) 68 Out[33]: 69 Bei Chen Shang Sheng b d e 70 Chen NaN NaN NaN NaN NaN NaN NaN 71 Bei NaN NaN NaN NaN NaN NaN NaN 72 Shang NaN NaN NaN NaN NaN NaN NaN 73 Sheng NaN NaN NaN NaN NaN NaN NaN
DataFrame & Series运算
注：axis按轴取可以看成 0：以index为index的Series【竖轴】， 1：以colum为index的Series【横轴】
五：使用函数
使用函数
1 In [6]: dframe=pd.DataFrame(np.random.randn(4,3),columns=list(‘bde‘),index=[‘Che 2 n‘,‘Bei‘,‘Shang‘,‘Sheng‘]) 3 In [7]: dframe 4 Out[7]: 5 b d e 6 Chen 1.838620 1.023421 0.641420 7 Bei 0.920563 -2.037778 -0.853871 8 Shang -0.587332 0.576442 0.596269 9 Sheng 0.366174 -0.689582 -1.064030 10 11 In [8]: np.abs(dframe) #绝对值函数 12 Out[8]: 13 b d e 14 Chen 1.838620 1.023421 0.641420 15 Bei 0.920563 2.037778 0.853871 16 Shang 0.587332 0.576442 0.596269 17 Sheng 0.366174 0.689582 1.064030 18 19 In [9]: func = lambda x: x.max() - x.min() 20 21 In [10]: dframe.apply(func) 22 Out[10]: 23 b 2.425952 24 d 3.061200 25 e 1.705449 26 dtype: float64 27 28 In [11]: dframe.apply(func,axis=1) 29 Out[11]: 30 Chen 1.197200 31 Bei 2.958341 32 Shang 1.183602 33 Sheng 1.430204 34 dtype: float64 35 36 In [12]: dframe.max() #即dframe.max(axis=0) 37 Out[12]: 38 b 1.838620 39 d 1.023421 40 e 0.641420 41 dtype: float64 42 43 In [15]: dframe.max(axis=1) 44 Out[15]: 45 Chen 1.838620 46 Bei 0.920563 47 Shang 0.596269 48 Sheng 0.366174 49 dtype: float64
六：排序
1）按索引排序：sort_index（【axis=0/1,ascending=False/True】）注，其中默认axis为0（index排序），ascending为True（升序）
1 In [16]: seri = pd.Series(range(4),index=[‘d‘,‘a‘,‘d‘,‘c‘]) 2 3 In [17]: seri 4 Out[17]: 5 d 0 6 a 1 7 d 2 8 c 3 9 dtype: int64 10 11 In [18]: seri.sort_index() 12 Out[18]: 13 a 1 14 c 3 15 d 2 16 d 0 17 dtype: int64
Series的索引排序
1 In [22]: dframe 2 Out[22]: 3 c a b 4 Chen 1.838620 1.023421 0.641420 5 Bei 0.920563 -2.037778 -0.853871 6 Shang -0.587332 0.576442 0.596269 7 Sheng 0.366174 -0.689582 -1.064030 8 9 In [23]: dframe.sort_index() 10 Out[23]: 11 c a b 12 Bei 0.920563 -2.037778 -0.853871 13 Chen 1.838620 1.023421 0.641420 14 Shang -0.587332 0.576442 0.596269 15 Sheng 0.366174 -0.689582 -1.064030 16 17 In [24]: dframe.sort_index(axis=1) 18 Out[24]: 19 a b c 20 Chen 1.023421 0.641420 1.838620 21 Bei -2.037778 -0.853871 0.920563 22 Shang 0.576442 0.596269 -0.587332 23 Sheng -0.689582 -1.064030 0.366174
DataFrame的索引排序，用axis制定是按index（默认）还是columns进行排序（1）
2）按值排序sort_values方法【注：order方法已不推荐使用了】
1 In [32]: seri =pd.Series([4,7,np.nan,-1,2,np.nan]) 2 3 In [33]: seri 4 Out[33]: 5 0 4 6 1 7 7 2 NaN 8 3 -1 9 4 2 10 5 NaN 11 dtype: float64 12 13 In [34]: seri.sort_values() 14 Out[34]: 15 3 -1 16 4 2 17 0 4 18 1 7 19 2 NaN 20 5 NaN 21 dtype: float64 22 23 #NaN值会默认排到最后
Series的值排序
1 In [38]: dframe = pd.DataFrame({‘b‘:[4,7,-3,2],‘a‘:[0,1,0,1]})
2
3 In [39]: dframe
4 Out[39]:
5 a b
6 0 0 4
7 1 1 7
8 2 0 -3
9 3 1 2
10
11 In [54]: dframe.sort_values(‘a‘)
12 Out[54]:
13 a b
14 0 0 4
15 2 0 -3
16 1 1 7
17 3 1 2
18
19 In [55]: dframe.sort_values(‘b‘)
20 Out[55]:
21 a b
22 2 0 -3
23 3 1 2
24 0 0 4
25 1 1 7
26
27 In [57]: dframe.sort_values([‘a‘,‘b‘])
28 Out[57]:
29 a b
30 2 0 -3
31 0 0 4
32 3 1 2
33 1 1 7
34
35 In [58]: dframe.sort_values([‘b‘,‘a‘])
36 Out[58]:
37 a b
38 2 0 -3
39 3 1 2
40 0 0 4
41 1 1 7
DataFrame的值排序
七：排名
rank方法
八：统计计算
count：非NaN值 describe：对Series或DataFrame列计算汇总统计 min，max argmin，argmax（整数值）：最值得索引值 idmax，idmin：最值索引值
sum mean：平均数 var：样本方差 std：样本标准差 kurt：峰值 cumsum：累积和 cummin/cummax：累计最值 pct_change：百分数变化
1 In [63]: df = pd.DataFrame([[1.4,np.nan],[7.1,-4.5],[np.nan,np.nan],[0.75,-1.3]] 2 ,index=[‘a‘,‘b‘,‘c‘,‘d‘],columns=[‘one‘,‘two‘]) 3 4 In [64]: df 5 Out[64]: 6 one two 7 a 1.40 NaN 8 b 7.10 -4.5 9 c NaN NaN 10 d 0.75 -1.3 11 12 In [66]: df.sum() 13 Out[66]: 14 one 9.25 15 two -5.80 16 dtype: float64 17 18 In [67]: df.sum(axis=1) 19 Out[67]: 20 a 1.40 21 b 2.60 22 c NaN 23 d -0.55 24 dtype: float64 25 26 #求平均值，skipna：跳过NaN 27 In [68]: df.mean(axis=1,skipna=False) 28 Out[68]: 29 a NaN 30 b 1.300 31 c NaN 32 d -0.275 33 dtype: float64 34 35 36 In [70]: df.idxmax() 37 Out[70]: 38 one b 39 two d 40 dtype: object 41 42 In [71]: df.cumsum() 43 Out[71]: 44 one two 45 a 1.40 NaN 46 b 8.50 -4.5 47 c NaN NaN 48 d 9.25 -5.8 49 50 In [72]: df.describe() 51 Out[72]: 52 one two 53 count 3.000000 2.000000 54 mean 3.083333 -2.900000 55 std 3.493685 2.262742 56 min 0.750000 -4.500000 57 25% 1.075000 -3.700000 58 50% 1.400000 -2.900000 59 75% 4.250000 -2.100000 60 max 7.100000 -1.300000
一些统计计算
九：唯一值，值计数，以及成员资格
unique方法 value_counts：顶级方法 isin方法
1 In [74]: seri = pd.Series([‘c‘,‘a‘,‘d‘,‘a‘,‘a‘,‘b‘,‘b‘,‘c‘,‘c‘]) 2 3 In [75]: seri 4 Out[75]: 5 0 c 6 1 a 7 2 d 8 3 a 9 4 a 10 5 b 11 6 b 12 7 c 13 8 c 14 dtype: object 15 16 In [76]: seri.unique() 17 Out[76]: array([‘c‘, ‘a‘, ‘d‘, ‘b‘], dtype=object) 18 19 In [77]: seri.value_counts() 20 Out[77]: 21 c 3 22 a 3 23 b 2 24 d 1 25 dtype: int64 26 27 In [78]: pd.value_counts(seri.values,sort=False) 28 Out[78]: 29 a 3 30 c 3 31 b 2 32 d 1 33 dtype: int64 34 35 36 In [81]: seri.isin([‘b‘,‘c‘]) 37 Out[81]: 38 0 True 39 1 False 40 2 False 41 3 False 42 4 False 43 5 True 44 6 True 45 7 True 46 8 True 47 dtype: bool
唯一值，值计数，成员资格
十：缺少数据处理
一）删除NaN：dropna方法
1）Series
python中的None即是对应到的Numpy的NaN
1 In [3]: seri = pd.Series([‘aaa‘,‘bbb‘,np.nan,‘ccc‘]) 2 3 In [4]: seri[0]=None 4 5 In [5]: seri 6 Out[5]: 7 0 None 8 1 bbb 9 2 NaN 10 3 ccc 11 dtype: object 12 13 In [7]: seri.isnull() 14 Out[7]: 15 0 True 16 1 False 17 2 True 18 3 False 19 dtype: bool 20 21 In [8]: seri.dropna() #返回非NaN值 22 Out[8]: 23 1 bbb 24 3 ccc 25 dtype: object 26 27 In [9]: seri 28 Out[9]: 29 0 None 30 1 bbb 31 2 NaN 32 3 ccc 33 dtype: object 34 35 In [10]: seri[seri.notnull()] #返回非空值 36 Out[10]: 37 1 bbb 38 3 ccc 39 dtype: object
Series数据处理
2）DataFrame
对于DataFrame事情稍微复杂，有时希望删除全NaN或者含有NaN的行或列。
1 In [15]: df = pd.DataFrame([[1,6.5,3],[1,np.nan,np.nan],[np.nan,np.nan,np.nan],[ 2 np.nan,6.5,3]]) 3 4 In [16]: df 5 Out[16]: 6 0 1 2 7 0 1 6.5 3 8 1 1 NaN NaN 9 2 NaN NaN NaN 10 3 NaN 6.5 3 11 12 In [17]: df.dropna() #默认以行(axis=0)，只要有NaN的就删除 13 Out[17]: 14 0 1 2 15 0 1 6.5 3 16 17 In [19]: df.dropna(how=‘all‘) #只删除全是NaN的行 18 Out[19]: 19 0 1 2 20 0 1 6.5 3 21 1 1 NaN NaN 22 3 NaN 6.5 3 23 24 In [21]: df.dropna(axis=1,how=‘all‘) #以列为标准来丢弃列 25 Out[21]: 26 0 1 2 27 0 1 6.5 3 28 1 1 NaN NaN 29 2 NaN NaN NaN 30 3 NaN 6.5 3 31 32 In [22]: df.dropna(axis=1) 33 Out[22]: 34 Empty DataFrame 35 Columns: [] 36 Index: [0, 1, 2, 3]
DataFrame的数据处理
二）填充NaN：fillna方法
1 In [88]: df 2 Out[88]: 3 one two 4 a 1.40 NaN 5 b 7.10 -4.5 6 c NaN NaN 7 d 0.75 -1.3 8 9 In [90]: df.fillna(0) 10 Out[90]: 11 one two 12 a 1.40 0.0 13 b 7.10 -4.5 14 c 0.00 0.0 15 d 0.75 -1.3
填充NaN
十一：层次化索引
1 In [30]: seri = pd.Series(np.random.randn(10),index=[[‘a‘,‘a‘,‘a‘,‘b‘,‘b‘,‘b‘,‘c 2 ‘,‘c‘,‘d‘,‘d‘],[1,2,3,1,2,3,1,2,2,3]]) 3 In [31]: seri 4 Out[31]: 5 a 1 0.528387 6 2 -0.152286 7 3 -0.776540 8 b 1 0.025425 9 2 -1.412776 10 3 0.969498 11 c 1 0.478260 12 2 0.116301 13 d 2 1.464144 14 3 2.266069 15 dtype: float64 16 17 In [32]: seri[‘a‘] 18 Out[32]: 19 1 0.528387 20 2 -0.152286 21 3 -0.776540 22 dtype: float64 23 24 In [33]: seri.index 25 Out[33]: 26 MultiIndex(levels=[[u‘a‘, u‘b‘, u‘c‘, u‘d‘], [1, 2, 3]], 27 labels=[[0, 0, 0, 1, 1, 1, 2, 2, 3, 3], [0, 1, 2, 0, 1, 2, 0, 1, 1, 2 28 ]]) 29 30 In [35]: seri[‘a‘:‘c‘] 31 Out[35]: 32 a 1 0.528387 33 2 -0.152286 34 3 -0.776540 35 b 1 0.025425 36 2 -1.412776 37 3 0.969498 38 c 1 0.478260 39 2 0.116301 40 dtype: float64 41 42 In [45]: seri.unstack() 43 Out[45]: 44 1 2 3 45 a 0.528387 -0.152286 -0.776540 46 b 0.025425 -1.412776 0.969498 47 c 0.478260 0.116301 NaN 48 d NaN 1.464144 2.266069 49 50 In [46]: seri.unstack().stack() 51 Out[46]: 52 a 1 0.528387 53 2 -0.152286 54 3 -0.776540 55 b 1 0.025425 56 2 -1.412776 57 3 0.969498 58 c 1 0.478260 59 2 0.116301 60 d 2 1.464144 61 3 2.266069 62 dtype: float64
Series层次化索引，利用unstack方法可以转化为DataFrame型数据
1 In [48]: df = pd.DataFrame(np.arange(12).reshape((4,3)),index=[[‘a‘,‘a‘,‘b‘,‘b‘] 2 ,[1,2,1,2]],columns=[[‘Ohio‘,‘Ohio‘,‘Colorado‘],[‘Green‘,‘Red‘,‘Green‘]]) 3 4 In [49]: df 5 Out[49]: 6 Ohio Colorado 7 Green Red Green 8 a 1 0 1 2 9 2 3 4 5 10 b 1 6 7 8 11 2 9 10 11 12 13 In [50]: df.index 14 Out[50]: 15 MultiIndex(levels=[[u‘a‘, u‘b‘], [1, 2]], 16 labels=[[0, 0, 1, 1], [0, 1, 0, 1]]) 17 18 In [51]: df.columns 19 Out[51]: 20 MultiIndex(levels=[[u‘Colorado‘, u‘Ohio‘], [u‘Green‘, u‘Red‘]], 21 labels=[[1, 1, 0], [0, 1, 0]]) 22 23 In [53]: df[‘Ohio‘] 24 Out[53]: 25 Green Red 26 a 1 0 1 27 2 3 4 28 b 1 6 7 29 2 9 10 30 31 In [57]: df.ix[‘a‘,‘Ohio‘] 32 Out[57]: 33 Green Red 34 1 0 1 35 2 3 4 36 37 In [61]: df.ix[‘a‘,‘Ohio‘].ix[1,‘Red‘] 38 Out[61]: 1
DataFrame层次化索引
时间： 2024-10-23 07:01:26