machine learning in coding(python):根据关键字合并feature,删除无用feature,转化为numpy数组

import pandas as pd
import numpy as np
from sklearn import preprocessing
from keras.models import Sequential
from keras.layers.core import Dense, Activation, Dropout

# load training and test datasets
train = pd.read_csv('../input/train_set.csv', parse_dates=[2,])
test = pd.read_csv('../input/test_set.csv', parse_dates=[3,])

tubes = pd.read_csv('../input/tube.csv')

# create some new features
train['year'] = train.quote_date.dt.year
train['month'] = train.quote_date.dt.month
train['dayofyear'] = train.quote_date.dt.dayofyear
train['dayofweek'] = train.quote_date.dt.dayofweek
train['day'] = train.quote_date.dt.day

test['year'] = test.quote_date.dt.year
test['month'] = test.quote_date.dt.month
test['dayofyear'] = test.quote_date.dt.dayofyear
test['dayofweek'] = test.quote_date.dt.dayofweek
test['day'] = test.quote_date.dt.day

train = pd.merge(train,tubes,on='tube_assembly_id',how='inner')
test = pd.merge(test,tubes,on='tube_assembly_id',how='inner')

train['material_id'].fillna('SP-9999',inplace=True)
test['material_id'].fillna('SP-9999',inplace=True)

# drop useless columns and create labels
idx = test.id.values.astype(int)
test = test.drop(['id', 'tube_assembly_id', 'quote_date'], axis = 1)
labels = train.cost.values
train = train.drop(['quote_date', 'cost', 'tube_assembly_id'], axis = 1)

# convert data to numpy array
train = np.array(train)
test = np.array(test)


from:kaggle

版权声明:本文为博主原创文章,未经博主允许不得转载。

时间: 2024-08-02 10:57:54

machine learning in coding(python):根据关键字合并feature,删除无用feature,转化为numpy数组的相关文章

机器学习系统设计(Building Machine Learning Systems with Python)- Willi Richert Luis Pedro Coelho

机器学习系统设计(Building Machine Learning Systems with Python)- Willi Richert Luis Pedro Coelho 总述 本书是 2014 的,看完以后才发现有第二版的更新,2016.建议阅读最新版,有能力的建议阅读英文版,中文翻译有些地方比较别扭(但英文版的书确实是有些贵). 我读书的目的:泛读主要是想窥视他人思考的方式. 作者写书的目标:面向初学者,但有时间看看也不错.作者说"我希望它能激发你的好奇心,并足以让你保持渴望,不断探索

machine learning in coding(python):拼接原始数据;生成高次特征

拼接原始数据: train_data = pd.read_csv('train.csv') test_data = pd.read_csv('test.csv') all_data = np.vstack((train_data.ix[:,1:-1], test_data.ix[:,1:-1])) numpy下的合并数组vstack和hstack函数: >>> a = np.ones((2,2)) >>> b = np.eye(2) >>> print

machine learning in coding(python):使用xgboost构建预测模型

接上篇:http://blog.csdn.net/mmc2015/article/details/47304591 def xgboost_pred(train,labels,test): params = {} params["objective"] = "reg:linear" params["eta"] = 0.005 params["min_child_weight"] = 6 params["subsamp

machine learning in coding(python):使用贪心搜索【进行特征选择】

print "Performing greedy feature selection..." score_hist = [] N = 10 good_features = set([]) # Greedy feature selection loop while len(score_hist) < 2 or score_hist[-1][0] > score_hist[-2][0]: scores = [] for f in range(len(Xts)): if f no

machine learning in coding(python):使用交叉验证【选择模型超参数】

# Hyperparameter selection loop score_hist = [] Cvals = [0.001, 0.003, 0.006, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.1] for C in Cvals: model.C = C score = cv_loop(Xt, y, model, N) score_hist.append((score,C)) print "C: %f Mean AUC: %f" %(C, scor

学习 Machine Learning Mastery With Python (1)

1 介绍 1.1 机器学习的错误的想法 一定要对python 编程和python语法非常了解 深入学习scikit learn使用的机器学习算法的理论和参数 避免或者不能接触实际项目中的其他部分. 对某些人可能是适用的, 但是对于很多人, 这样会让人觉得望而生畏,经过很长的准备过程学习理论和算法 , 才能开始实际的项目. 很遗憾的是, 这是市面上大部分书籍所采取的办法. 1.2 python 机器学习 这本书主要介绍机器学习的一个子领域叫做预测模型.这个是机器学习在工业界最有用的应用. 本书的组

Building Machine Learning Systems with Python 2

1>监督学习(分类):先让机器学习一下每种花朵的样本数据,然后让他根据这些信息,对未标志出花朵种类的图像进行分类. 2>特征:我们把数据中所有测量的结果都叫特征. 2>交叉验证:极端的叫去一法(leave-one-out)从训练集中拿出一个样本,并在缺少这个样本的数据上训练一个模型,然后看模型是否能够对这个样本正确分类 3>分类模型的组成: 模型结构:采用一个阀值在一个特征上进行划分. 搜素过程:尽可能多的尝试所有特征和阀值的组合. 损失函数:用他来确定哪些可能性不会太差. 4&g

Building.Machine.Learning.Systems.with.Python(2013.7).Willi.Richert.文字版

下载地址 近日网盘管得紧,pdf不能随便分享,下载后请去掉最后的字母 f 即可变成pdf文档了.

Getting started with machine learning in Python

Getting started with machine learning in Python Machine learning is a field that uses algorithms to learn from data and make predictions. Practically, this means that we can feed data into an algorithm, and use it to make predictions about what might