pandas分组聚合案例

美国2012年总统候选人政治献金数据分析

导入包

import numpy as np
import pandas as pd
from pandas import Series,DataFrame

方便操作,将月份和参选人以及所在政党进行定义

months = {'JAN' : 1, 'FEB' : 2, 'MAR' : 3, 'APR' : 4, 'MAY' : 5, 'JUN' : 6,
          'JUL' : 7, 'AUG' : 8, 'SEP' : 9, 'OCT': 10, 'NOV': 11, 'DEC' : 12}
of_interest = ['Obama, Barack', 'Romney, Mitt', 'Santorum, Rick',
               'Paul, Ron', 'Gingrich, Newt']
parties = {
  'Bachmann, Michelle': 'Republican',
  'Romney, Mitt': 'Republican',
  'Obama, Barack': 'Democrat',
  "Roemer, Charles E. 'Buddy' III": 'Reform',
  'Pawlenty, Timothy': 'Republican',
  'Johnson, Gary Earl': 'Libertarian',
  'Paul, Ron': 'Republican',
  'Santorum, Rick': 'Republican',
  'Cain, Herman': 'Republican',
  'Gingrich, Newt': 'Republican',
  'McCotter, Thaddeus G': 'Republican',
  'Huntsman, Jon': 'Republican',
  'Perry, Rick': 'Republican'
 }
df = pd.read_csv('./data/usa_election.txt')
df.head()
C:\anaconda3\lib\site-packages\IPython\core\interactiveshell.py:2728: DtypeWarning: Columns (6) have mixed types. Specify dtype option on import or set low_memory=False.
  interactivity=interactivity, compiler=compiler, result=result)

.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}

.dataframe tbody tr th {
vertical-align: top;
}

.dataframe thead th {
text-align: right;
}

cmte_id cand_id cand_nm contbr_nm contbr_city contbr_st contbr_zip contbr_employer contbr_occupation contb_receipt_amt contb_receipt_dt receipt_desc memo_cd memo_text form_tp file_num
0 C00410118 P20002978 Bachmann, Michelle HARVEY, WILLIAM MOBILE AL 3.6601e+08 RETIRED RETIRED 250.0 20-JUN-11 NaN NaN NaN SA17A 736166
1 C00410118 P20002978 Bachmann, Michelle HARVEY, WILLIAM MOBILE AL 3.6601e+08 RETIRED RETIRED 50.0 23-JUN-11 NaN NaN NaN SA17A 736166
2 C00410118 P20002978 Bachmann, Michelle SMITH, LANIER LANETT AL 3.68633e+08 INFORMATION REQUESTED INFORMATION REQUESTED 250.0 05-JUL-11 NaN NaN NaN SA17A 749073
3 C00410118 P20002978 Bachmann, Michelle BLEVINS, DARONDA PIGGOTT AR 7.24548e+08 NONE RETIRED 250.0 01-AUG-11 NaN NaN NaN SA17A 749073
4 C00410118 P20002978 Bachmann, Michelle WARDENBURG, HAROLD HOT SPRINGS NATION AR 7.19016e+08 NONE RETIRED 300.0 20-JUN-11 NaN NaN NaN SA17A 736166
# 新建一列各个候选人所在党派party
df['party'] = df['cand_nm'].map(parties)
df.head()

.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}

.dataframe tbody tr th {
vertical-align: top;
}

.dataframe thead th {
text-align: right;
}

cmte_id cand_id cand_nm contbr_nm contbr_city contbr_st contbr_zip contbr_employer contbr_occupation contb_receipt_amt contb_receipt_dt receipt_desc memo_cd memo_text form_tp file_num party
0 C00410118 P20002978 Bachmann, Michelle HARVEY, WILLIAM MOBILE AL 3.6601e+08 RETIRED RETIRED 250.0 20-JUN-11 NaN NaN NaN SA17A 736166 Republican
1 C00410118 P20002978 Bachmann, Michelle HARVEY, WILLIAM MOBILE AL 3.6601e+08 RETIRED RETIRED 50.0 23-JUN-11 NaN NaN NaN SA17A 736166 Republican
2 C00410118 P20002978 Bachmann, Michelle SMITH, LANIER LANETT AL 3.68633e+08 INFORMATION REQUESTED INFORMATION REQUESTED 250.0 05-JUL-11 NaN NaN NaN SA17A 749073 Republican
3 C00410118 P20002978 Bachmann, Michelle BLEVINS, DARONDA PIGGOTT AR 7.24548e+08 NONE RETIRED 250.0 01-AUG-11 NaN NaN NaN SA17A 749073 Republican
4 C00410118 P20002978 Bachmann, Michelle WARDENBURG, HAROLD HOT SPRINGS NATION AR 7.19016e+08 NONE RETIRED 300.0 20-JUN-11 NaN NaN NaN SA17A 736166 Republican
# party这一列中有哪些元素
df['party'].unique()
array(['Republican', 'Democrat', 'Reform', 'Libertarian'], dtype=object)
# 统计party列中各个元素出现次数
df['party'].value_counts()
Democrat       292400
Republican     237575
Reform           5364
Libertarian       702
Name: party, dtype: int64
# 查看各个党派收到的政治献金总数contb_receipt_amt
df.groupby(by='party')['contb_receipt_amt'].sum()
party
Democrat       8.105758e+07
Libertarian    4.132769e+05
Reform         3.390338e+05
Republican     1.192255e+08
Name: contb_receipt_amt, dtype: float64
# 查看每天各个党派收到的政治献金总数contb_receipt_amt
df.groupby(by=['contb_receipt_dt','party'])['contb_receipt_amt'].sum()
contb_receipt_dt  party
01-APR-11         Reform              50.00
                  Republican       12635.00
01-AUG-11         Democrat        175281.00
                  Libertarian       1000.00
                  Reform            1847.00
                  Republican      234598.46
01-DEC-11         Democrat        651532.82
                  Libertarian        725.00
                  Reform             875.00
                  Republican      486405.96
01-FEB-11         Republican         250.00
01-JAN-11         Republican        8600.00
01-JAN-12         Democrat         58098.80
                  Reform             515.00
                  Republican       75704.72
01-JUL-11         Democrat        165961.00
                  Libertarian       2000.00
                  Reform             100.00
                  Republican      115848.72
01-JUN-11         Democrat        145459.00
                  Libertarian        500.00
                  Reform              50.00
                  Republican      433109.20
01-MAR-11         Republican        1000.00
01-MAY-11         Democrat         82644.00
                  Reform             480.00
                  Republican       28663.87
01-NOV-11         Democrat        122529.87
                  Libertarian       3000.00
                  Reform            1792.00
                                    ...
30-OCT-11         Reform            3910.00
                  Republican       43913.16
30-SEP-11         Democrat       3373517.24
                  Libertarian        550.00
                  Reform            2050.00
                  Republican     4886331.76
31-AUG-11         Democrat        374387.44
                  Libertarian      10750.00
                  Reform             450.00
                  Republican     1017735.02
31-DEC-11         Democrat       3553072.57
                  Reform             695.00
                  Republican     1094376.72
31-JAN-11         Republican        6000.00
31-JAN-12         Democrat       1418410.31
                  Reform             150.00
                  Republican      869890.41
31-JUL-11         Democrat         20305.00
                  Reform             966.00
                  Republican       12781.02
31-MAR-11         Reform             200.00
                  Republican       62475.00
31-MAY-11         Democrat        351705.66
                  Libertarian        250.00
                  Reform             100.00
                  Republican      301339.80
31-OCT-11         Democrat        204996.87
                  Libertarian       4250.00
                  Reform            3105.00
                  Republican      734601.83
Name: contb_receipt_amt, Length: 1183, dtype: float64
# 将表中日期格式转换为'yyyy-mm-dd'  day-m-y
def transformDate(d):
    day,month,year = d.split('-')
    month = months[month]
    return '20'+year+'-'+str(month)+'-'+day
df['contb_receipt_dt'] = df['contb_receipt_dt'].apply(transformDate)
df.head()

.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}

.dataframe tbody tr th {
vertical-align: top;
}

.dataframe thead th {
text-align: right;
}

cmte_id cand_id cand_nm contbr_nm contbr_city contbr_st contbr_zip contbr_employer contbr_occupation contb_receipt_amt contb_receipt_dt receipt_desc memo_cd memo_text form_tp file_num party
0 C00410118 P20002978 Bachmann, Michelle HARVEY, WILLIAM MOBILE AL 3.6601e+08 RETIRED RETIRED 250.0 2011-6-20 NaN NaN NaN SA17A 736166 Republican
1 C00410118 P20002978 Bachmann, Michelle HARVEY, WILLIAM MOBILE AL 3.6601e+08 RETIRED RETIRED 50.0 2011-6-23 NaN NaN NaN SA17A 736166 Republican
2 C00410118 P20002978 Bachmann, Michelle SMITH, LANIER LANETT AL 3.68633e+08 INFORMATION REQUESTED INFORMATION REQUESTED 250.0 2011-7-05 NaN NaN NaN SA17A 749073 Republican
3 C00410118 P20002978 Bachmann, Michelle BLEVINS, DARONDA PIGGOTT AR 7.24548e+08 NONE RETIRED 250.0 2011-8-01 NaN NaN NaN SA17A 749073 Republican
4 C00410118 P20002978 Bachmann, Michelle WARDENBURG, HAROLD HOT SPRINGS NATION AR 7.19016e+08 NONE RETIRED 300.0 2011-6-20 NaN NaN NaN SA17A 736166 Republican
# 查看老兵(捐献者职业)主要支持谁  :查看老兵们捐赠给谁的钱最多
# 1.将老兵对应的行数据取出
df['contbr_occupation'] == 'DISABLED VETERAN'
old_bing = df.loc[df['contbr_occupation'] == 'DISABLED VETERAN']
# 2.根据候选人分组
old_bing.groupby(by='cand_nm')['contb_receipt_amt'].sum()
cand_nm
Cain, Herman       300.00
Obama, Barack     4205.00
Paul, Ron         2425.49
Santorum, Rick     250.00
Name: contb_receipt_amt, dtype: float64
df['contb_receipt_amt'].max()
1944042.43
#捐赠金额最大的人的职业以及捐献额  .通过query("查询条件来查找捐献人职业")
df.query('contb_receipt_amt == 1944042.43')

.dataframe tbody tr th:only-of-type {
vertical-align: middle;
}

.dataframe tbody tr th {
vertical-align: top;
}

.dataframe thead th {
text-align: right;
}

cmte_id cand_id cand_nm contbr_nm contbr_city contbr_st contbr_zip contbr_employer contbr_occupation contb_receipt_amt contb_receipt_dt receipt_desc memo_cd memo_text form_tp file_num party
176127 C00431445 P80003338 Obama, Barack OBAMA VICTORY FUND 2012 - UNITEMIZED CHICAGO IL 60680 NaN NaN 1944042.43 2011-12-31 NaN X * SA18 763233 Democrat

原文地址:https://www.cnblogs.com/zyyhxbs/p/11708549.html

时间: 2024-10-08 11:46:33

pandas分组聚合案例的相关文章

Pandas Cookbook -- 07 分组聚合、过滤、转换

分组聚合.过滤.转换 简书大神SeanCheney的译作,我作了些格式调整和文章目录结构的变化,更适合自己阅读,以后翻阅是更加方便自己查找吧 import pandas as pd import numpy as np 设定最大列数和最大行数 pd.set_option('max_columns',8 , 'max_rows', 8) 1 聚合 读取flights数据集,查询头部 flights = pd.read_csv('data/flights.csv') flights.head() .

pandas分组与聚合

Pandas分组与聚合 分组 (groupby) 对数据集进行分组,然后对每组进行统计分析 SQL能够对数据进行过滤,分组聚合 pandas能利用groupby进行更加复杂的分组运算 分组运算过程:split->apply->combine 拆分:进行分组的根据 2.应用:每个分组运行的计算规则 3.合并:把每个分组的计算结果合并起来 示例代码: import pandas as pd import numpy as np dict_obj = {'key1' : ['a', 'b', 'a'

Atitit  数据存储的分组聚合 groupby的实现attilax总结

Atitit  数据存储的分组聚合 groupby的实现attilax总结 1. 聚合操作1 1.1. a.标量聚合 流聚合1 1.2. b.哈希聚合2 1.3. 所有的最优计划的选择都是基于现有统计信息来评估3 1.4. 参考资料3 1. 聚合操作 聚合也是我们在写T-SQL语句的时候经常遇到的,我们来分析一下一些常用的聚合操作运算符的特性和可优化项. 1.1. a.标量聚合 流聚合 标量聚合是一种常用的数据聚合方式,比如我们写的语句中利用的以下聚合函数:MAX().MIN().AVG().C

Elasticsearch分组聚合-查询每个A_logtype下有多少数据

Elasticsearch分组聚合 1.查询指定索引下每个A_logtype有多少数据 curl -XPOST 'localhost:19200/ylchou-0-2015-10-07/_search?pretty' -d ' { "size": 0, "aggs": { "group_by_state": { "terms": { "field": "A_logtype" } } }

crm使用FetchXml分组聚合查询

/* 创建者:菜刀居士的博客 * 创建日期:2014年07月09号 */ namespace Net.CRM.FetchXml { using System; using Microsoft.Xrm.Sdk; using Microsoft.Xrm.Sdk.Query; /// <summary> /// 使用FetchXml聚合查询,分组依据 /// </summary> public class FetchXmlExtension { /// <summary> /

mongodb 分组聚合查询

MongoDB,分组,聚合 使用聚合,db.集合名.aggregate- 而不是find 管道在Unix和Linux中一般用于将当前命令的输出结果作为下一个命令的参数.MongoDB的聚合管道将MongoDB文档在一个管道处理完毕后将结果传递给下一个管道处理.管道操作是可以重复的. 每一个操作符(集合)都会接受一连串的文档,对这些文档做一些类型转换,最后将转换后的文档作为结果传递给下一个操作符,对于最后一个操作符,是将结果返回给客户端 //分组(这里制定了分组字段 $+字段名)//这里可以理解为

浅析MySQL使用 GROUP BY 分组聚合与细分聚合

1. 聚合函数(Aggregate Function) MySQL(5.7 ) 官方文档中给出的聚合函数列表(图片)如下: 详情点击https://dev.mysql.com/doc/refman/5.7/en/group-by-functions.html . 除非另有说明,否则聚合函数都会忽略空值(NULL values). 2. 聚合函数的使用 聚合函数通常对 GROUP BY 语句进行分组后的每个分组起作用,即,如果在查询语句中不使用 GROUP BY 对结果集分组,则聚合函数就对结果集

窗口聚合函数与分组聚合函数的异同

窗口聚合函数与分组聚合函数的功能是相同的:唯一不同的是,分组聚合函数通过分组查询来进行,而窗口聚合函数通过OVER子句定义的窗口来进行. --<T-SQL性能调优秘笈---基于SQL Server2012窗口函数>2.1.1窗口聚合函数描述

分页,sql分组聚合

分页 SELECT TOP 页大小 * FROM    (        SELECT ROW_NUMBER() OVER (ORDER BY id) AS RowNumber,* FROM table1    )   as A  WHERE RowNumber > 页大小*(页数-1) 分组聚合 create table tb(id int, value varchar(10))insert into tb values(1, 'aa')insert into tb values(1, 'bb