GBDT调参总结

GBDT类库弱学习器参数

一、回归

数据集：已知用户的30个特征，预测用户的信用值

from sklearn.ensemble import GradientBoostingRegressor
from sklearn.grid_search import GridSearchCV

#用平均值填补缺失值
gbdt_train_label = train_data[‘信用分‘]

gbdt_train_data = train_data[columns_]
gbdt_test_data = test_data[columns_]

gbdt_train_data = gbdt_train_data.fillna(gbdt_train_data.mean())
gbdt_test_data = gbdt_test_data.fillna(gbdt_test_data.mean())

#填补6个月平均占比总费用为其他列的平均值
all_rows = list(gbdt_train_data.index)
inf_rows = list(gbdt_train_data.loc[gbdt_train_data[‘6个月平均占比总费用‘] == float(‘inf‘)].index)
inf_mean = gbdt_train_data.ix[list(filter(lambda x: x not in inf_rows,all_rows))].mean() 

gbdt_train_data[‘6个月平均占比总费用‘][gbdt_train_data[‘6个月平均占比总费用‘] == float(‘inf‘)] = inf_mean[‘6个月平均占比总费用‘]

1. 找树的数量最佳值

#找树的数量最佳值
param_gbdt = {‘n_estimators‘:list(range(100,600,50))}
gbdt_search = GridSearchCV(estimator=GradientBoostingRegressor(learning_rate=0.1,min_samples_split=300,min_samples_leaf=20,
                                          max_depth=8,max_features=‘sqrt‘,subsample=0.8,random_state=75),
                          param_grid=param_gbdt,scoring=‘neg_mean_squared_error‘,iid=False,cv=3)
gbdt_search.fit(gbdt_train_data,gbdt_train_label)
print(gbdt_search.grid_scores_)
print(gbdt_search.best_params_)
print(gbdt_search.best_score_)

2.  找树的深度最佳值

#找树的深度最佳值
param_gbdt1 = {‘max_depth‘:[6,7,8,9,10]}
gbdt_search1 = GridSearchCV(estimator=GradientBoostingRegressor(learning_rate=0.1,n_estimators = 200,min_samples_split=300,
                            min_samples_leaf=20,max_features=‘sqrt‘,subsample=0.8,random_state=75),
                            param_grid=param_gbdt1,scoring=‘neg_mean_squared_error‘,iid=False,cv=5)
gbdt_search1.fit(gbdt_train_data,gbdt_train_label)
print(gbdt_search1.grid_scores_)
print(gbdt_search1.best_params_)
print(gbdt_search1.best_score_)

3. 找min_samples_split，min_samples_leaf最佳值

#找min_samples_split，min_samples_leaf最佳值
param_gbdt2 = {‘min_samples_split‘:[500,700,900,1100],
               ‘min_samples_leaf‘:[30,50,70,90]}
gbdt_search2 = GridSearchCV(estimator=GradientBoostingRegressor(learning_rate=0.1,n_estimators = 100,max_depth=6,
                                max_features=‘sqrt‘,subsample=0.8,random_state=75),n_jobs=3,
                            param_grid=param_gbdt2,scoring=‘neg_mean_squared_error‘,iid=False,cv=5)
gbdt_search2.fit(gbdt_train_data,gbdt_train_label)
print(gbdt_search2.grid_scores_)
print(gbdt_search2.best_params_)
print(gbdt_search2.best_score_)

4. 粗调完成,得到了上述参数的最佳大致范围，接下来，可以进行细调，就是在上述最佳参数附近范围进行搜索,这里就只对n_estimators和学习率进行调整

param_gbdt3 = {‘learning_rate‘:[0.06,0.08,0.1],
               ‘n_estimators‘:[100,150,200,250]}
gbdt_search3 = GridSearchCV(estimator=GradientBoostingRegressor(min_samples_split=700,min_samples_leaf=70,
                                max_depth=9,max_features=‘sqrt‘,subsample=0.8,random_state=75),n_jobs=3,
                            param_grid=param_gbdt3,scoring=‘neg_mean_squared_error‘,iid=False,cv=5)
gbdt_search3.fit(gbdt_train_data,gbdt_train_label)
print(gbdt_search3.grid_scores_)
print(gbdt_search3.best_params_)
print(gbdt_search3.best_score_)

可以看到，最佳学习率为 0.08，提升树数量为 150；此时auc提升到0.8551 。　　

二、分类

参考文献：

【1】scikit-learn 梯度提升树(GBDT)调参小结

原文地址：https://www.cnblogs.com/nxf-rabbit75/p/10593524.html