I have written my custom scorer object which is necessary for my problem and which I've called "p_value_scoring_object".
For the function sklearn.cross_validation.cross_val_score one of the parameters is "scoring", which allows to use this scorer object.
However, this option is not available for the score method of a classifier. Is sklearn just lacking that feature, or is there a way around it?
from sklearn.datasets import load_iris
from sklearn.cross_validation import cross_val_score
from sklearn.tree import DecisionTreeClassifier
clf = DecisionTreeClassifier(random_state=0)
iris = load_iris()
cross_val_score(clf, iris.data, iris.target, cv=10,scoring=p_value_scoring_object)
This works. However, this doesn't:
clf.fit(iris.data,iris.target)
clf.score(iris.data,iris.target,scoring=p_value_scoring_object)