参考:http://scikit-learn.org/stable/modules/model_persistence.html
训练了模型之后,我们希望可以保存下来,遇到新样本时直接使用已经训练好的保存了的模型,而不用重新再训练模型。本节介绍pickle在保存模型方面的应用。(After
training a scikit-learn model, it is desirable to have a way to persist the model for future use without having to retrain. The following section gives you an example of how to persist a model with pickle. We’ll also review a few security and maintainability
issues when working with pickle serialization.)
1、persistence example
It
is possible to save a model in the scikit by using Python’s built-in persistence model, namely pickle:
>>> from sklearn import svm >>> from sklearn import datasets >>> clf = svm.SVC() >>> iris = datasets.load_iris() >>> X, y = iris.data, iris.target >>> clf.fit(X, y) SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0, degree=3, gamma=0.0, kernel=‘rbf‘, max_iter=-1, probability=False, random_state=None, shrinking=True, tol=0.001, verbose=False) >>> import pickle >>> s = pickle.dumps(clf) >>> clf2 = pickle.loads(s) >>> clf2.predict(X[0]) array([0]) >>> y[0] 0
有些情况下(more
efficient on objects that carry large numpy arrays internally)使用joblib’s 代替pickle (joblib.dump & joblib.load)。之后我们甚至可以在另一个pathon程序中load保存好的模型(pickle也可以。。。):
>>> from sklearn.externals import joblib >>> <strong>joblib.dump(clf, 'filename.pkl') >>> clf = joblib.load('filename.pkl') </strong>
Note
joblib.dump returns a list of filenames. Each individual numpy array contained in the clf object
is serialized as a separate file on the filesystem. All files are required in the same folder when reloading the model with joblib.load.
2、security & maintainability limitations
pickle
(and joblib by extension)在maintainability and security方面有些问题,因为:
- Never unpickle untrusted data
- Models saved in one version of scikit-learn might not load in another version.
为了能够在scikit-learn未来的版本中重构已保存好的模型,需要pickled时添加一些metadata:
- The training data, e.g. a reference to a immutable snapshot
- The python source code used to generate the model
- The versions of scikit-learn and its dependencies
- The cross validation score obtained on the training data
further discussion,refer this talk
by Alex Gaynor.
版权声明:本文为博主原创文章,未经博主允许不得转载。