Using Scorer Object for Classifier Score Method for scikit-learn

Question

I have written my custom scorer object which is necessary for my problem and which I've called "p_value_scoring_object".

For the function sklearn.cross_validation.cross_val_score one of the parameters is "scoring", which allows to use this scorer object.

However, this option is not available for the score method of a classifier. Is sklearn just lacking that feature, or is there a way around it?

from sklearn.datasets import load_iris
from sklearn.cross_validation import cross_val_score
from sklearn.tree import DecisionTreeClassifier
clf = DecisionTreeClassifier(random_state=0)
iris = load_iris()
cross_val_score(clf, iris.data, iris.target, cv=10,scoring=p_value_scoring_object)

This works. However, this doesn't:

clf.fit(iris.data,iris.target)
clf.score(iris.data,iris.target,scoring=p_value_scoring_object)

Answer 1

Why should it work? scoring is an argument of cross_val_score, not score. The documentation says score "returns the mean accuracy on the given test data and labels." This behavior is not specific to DecisionTreeClassifier, but common to classifiers in sklearn.

Perhaps you want sklearn.metrics?