Grid search model isn't recognized as fitted for Graphviz

Question

I find this really weird, and the code is really straight forward. What am I doing wrong ?

from sklearn.model_selection import GridSearchCV
scoring_type="accuracy"
preprocess_data(X,y,0)
p_grid = {'min_samples_split':np.arange(2,10),'min_samples_leaf': np.arange(1,10)}
Tree_opt = GridSearchCV(estimator=DecisionTreeClassifier(random_state=42), param_grid=p_grid, scoring=scoring_type, cv=10)
Tree_opt.fit(X_train,y_train)

print("Best training params: {}".format(Tree_opt.best_params_))
print("Best training Score: {}".format(Tree_opt.best_score_))
dot_data = tree.export_graphviz(Tree_opt, out_file=None,feature_names=labels,class_names=class_names,filled=True, rounded=True,special_characters=True)
graph = graphviz.Source(dot_data)

graph
print(Tree_opt.score(X_test, y_test))

And I'm getting the following error:

NotFittedError                            Traceback (most recent call last)
<ipython-input-14-1c7fa906f99b> in <module>
      8 print("Best training params: {}".format(Tree_opt.best_params_))
      9 print("Best training Score: {}".format(Tree_opt.best_score_))
---> 10 dot_data = tree.export_graphviz(Tree_opt, out_file=None,feature_names=labels,class_names=class_names,filled=True, rounded=True,special_characters=True)
     11 graph = graphviz.Source(dot_data)
     12 

~\Anaconda3\lib\site-packages\sklearn\tree\export.py in export_graphviz(decision_tree, out_file, max_depth, feature_names, class_names, label, filled, leaves_parallel, impurity, node_ids, proportion, rotate, rounded, special_characters, precision)
    394                 out_file.write('%d -> %d ;\n' % (parent, node_id))
    395 
--> 396     check_is_fitted(decision_tree, 'tree_')
    397     own_file = False
    398     return_string = False

~\Anaconda3\lib\site-packages\sklearn\utils\validation.py in check_is_fitted(estimator, attributes, msg, all_or_any)
    949 
    950     if not all_or_any([hasattr(estimator, attr) for attr in attributes]):
--> 951         raise NotFittedError(msg % {'name': type(estimator).__name__})
    952 
    953 

NotFittedError: This GridSearchCV instance is not fitted yet. Call 'fit' with appropriate arguments before using this method.

I also have this warning I don't think it's important:

C:\Users\Flow\Anaconda3\lib\site-packages\sklearn\model_selection\_search.py:841: DeprecationWarning: The default of the `iid` parameter will change from True to False in version 0.22 and will be removed in 0.24. This will change numeric results when test-set sizes are unequal.
  DeprecationWarning)

I think the fit works because I can get the first prints as follows:

Best training params: {'min_samples_leaf': 3, 'min_samples_split': 2}
Best training Score: 0.8809523809523809

Answer 1

Try using Tree_opt.best_estimator_. The grid search object passes fit and score functions through to the best estimator when refit=True, but other (esp. more specific) methods need to be called directly from best estimator object.

Answer 2

Here is the modified correct code :

from sklearn.model_selection import GridSearchCV
scoring_type="accuracy"
preprocess_data(X,y,0)
p_grid = {'min_samples_split':np.arange(2,10),'min_samples_leaf': np.arange(1,10)}
Tree_opt = GridSearchCV(estimator=DecisionTreeClassifier(random_state=42), param_grid=p_grid, scoring=scoring_type, cv=10, refit=True)
Tree_opt.fit(X_train,y_train)


print("Best training params: {}".format(Tree_opt.best_params_))
print("Best training Score: {}".format(Tree_opt.best_score_))
print("Best training Score: {}".format(Tree_opt.best_estimator_))
Tree_opt.best_estimator_.fit(X_train, y_train)
dot_data = tree.export_graphviz(Tree_opt.best_estimator_, out_file=None,feature_names=labels,class_names=class_names,filled=True, rounded=True,special_characters=True)
graph = graphviz.Source(dot_data)

graph

Indeed we need to call on .best_estimator_ instance