本文共 4437 字，大约阅读时间需要 14 分钟。

Hyperparameter Grid Search

GridSearchCV的名字其实可以拆分为两部分，GridSearch和CV，即网格搜索和交叉验证。

 

常用参数

estimator：所使用的分类器，如estimator=RandomForestClassifier(min_samples_split=100,min_samples_leaf=20,max_depth=8,max_features='sqrt',random_state=10), 并且传入除需要确定最佳的参数之外的其他参数。每一个分类器都需要一个scoring参数，或者score方法。

param_grid：值为字典或者列表，即需要最优化的参数的取值，param_grid =param_test1，param_test1 = {'n_estimators':range(10,71,10)}。

scoring :准确度评价标准，默认None,这时需要使用score函数；或者如scoring='roc_auc'，根据所选模型不同，评价准则不同。字符串（函数名），或是可调用对象，需要其函数签名形如：scorer(estimator, X, y)；如果是None，则使用estimator的误差估计函数。

 

就是param_grid中的每一个{}都会去试

import pandas as pdfrom sklearn.preprocessing import LabelEncoderfrom sklearn.model_selection import train_test_splitfrom sklearn.model_selection import GridSearchCVfrom sklearn.svm import SVCfrom sklearn.pipeline import make_pipelinefrom sklearn.preprocessing import StandardScalerdf = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/wdbc.data', header=None)X = df.loc[:, 2:].valuesy = df.loc[:, 1].valuesle = LabelEncoder()y = le.fit_transform(y)X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, stratify=y, random_state=1)pipe_svc = make_pipeline(StandardScaler(), SVC(random_state=1))param_range = [0.0001, 0.001, 0.01, 0.1, 1.0, 10.0, 100.0, 1000.0]param_grid = [              {'svc__C': param_range, 'svc__kernel': ['linear']},               {'svc__C': param_range, 'svc__gamma': param_range, 'svc__kernel': ['rbf']}             ]gs = GridSearchCV(estimator=pipe_svc, param_grid=param_grid, scoring='accuracy', cv=10, n_jobs=1)#gs是
    
     gs.fit(X_train, y_train)print(gs.best_score_)print(gs.best_params_)clf = gs.best_estimator_clf.fit(X_train, y_train)print('Test accuracy: %.3f' % clf.score(X_test, y_test))
    

Nested Cross-Validation

import pandas as pdimport numpy as npfrom sklearn.preprocessing import LabelEncoderfrom sklearn.model_selection import train_test_splitfrom sklearn.model_selection import GridSearchCVfrom sklearn.svm import SVCfrom sklearn.pipeline import make_pipelinefrom sklearn.preprocessing import StandardScalerfrom sklearn.model_selection import cross_val_scoredf = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/wdbc.data', header=None)X = df.loc[:, 2:].valuesy = df.loc[:, 1].valuesle = LabelEncoder()y = le.fit_transform(y)X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, stratify=y, random_state=1)pipe_svc = make_pipeline(StandardScaler(), SVC(random_state=1))param_range = [0.0001, 0.001, 0.01, 0.1, 1.0, 10.0, 100.0, 1000.0]param_grid = [{'svc__C': param_range, 'svc__kernel': ['linear']},               {'svc__C': param_range, 'svc__gamma': param_range, 'svc__kernel': ['rbf']}]gs = GridSearchCV(estimator=pipe_svc, param_grid=param_grid, scoring='accuracy', cv=2)scores = cross_val_score(gs, X_train, y_train, scoring='accuracy', cv=5)print('CV accuracy: %.3f +/- %.3f' % (np.mean(scores), np.std(scores)))

 

 

We could use the nested cross-validation approach to compare an SVM model to a simple decision tree classifier; for simplicity, we will only tune its depth parameter

import pandas as pdimport numpy as npfrom sklearn.preprocessing import LabelEncoderfrom sklearn.model_selection import train_test_splitfrom sklearn.model_selection import GridSearchCVfrom sklearn.svm import SVCfrom sklearn.pipeline import make_pipelinefrom sklearn.preprocessing import StandardScalerfrom sklearn.model_selection import cross_val_scorefrom sklearn.tree import DecisionTreeClassifierdf = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/breast-cancer-wisconsin/wdbc.data', header=None)X = df.loc[:, 2:].valuesy = df.loc[:, 1].valuesle = LabelEncoder()y = le.fit_transform(y)X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, stratify=y, random_state=1)pipe_svc = make_pipeline(StandardScaler(), SVC(random_state=1))param_range = [0.0001, 0.001, 0.01, 0.1, 1.0, 10.0, 100.0, 1000.0]param_grid = [{'svc__C': param_range, 'svc__kernel': ['linear']},               {'svc__C': param_range, 'svc__gamma': param_range, 'svc__kernel': ['rbf']}]gs = GridSearchCV(estimator=DecisionTreeClassifier(random_state=0),                   param_grid=[{'max_depth': [1, 2, 3, 4, 5, 6, 7, None]}], scoring='accuracy', cv=2)scores = cross_val_score(gs, X_train, y_train, scoring='accuracy', cv=5)print('CV accuracy: %.3f +/- %.3f' % (np.mean(scores), np.std(scores)))

 

 

 

 

转载地址：http://hmygf.baihongyu.com/