1
$\begingroup$

pretty simple question here but just can't seem to find the answer in the normally great documentation for sklearn.

I am working with binary classifiers, but we can just assume i am using LogisticRegression, and I was wondering if there is a general way to have the classifier select, say only 10 best (most sure) data points?

For example, say I train a set with 500K data points, and my test set has 10K lines, and out of the 10K, I just want to choose 10 that have the highest chance of being true positives. Does this make sense?

I have read about, and have been playing with the class_weights attribute, which works well for giving more/less weight to each of the binary outcome classes, but its not quite working for what I want in that it always give different number of position predictions, and I can't really tell how sure the classifier is about each one of those.

$\endgroup$
5
$\begingroup$

class_weight is used in the process of model-training to train(=fit) a better model (let us call it clf).

Your question is is about choosing the most sure predictions. You just need to predict the probability (for binary classification this will be the probability of positive class)

y_test_predicted_probability = clf.predict_proba(X_test)

and then choose 10 points with the highest y_test_predicted_probability

#some code to do this
top_picks_indexes = y_test_predicted_probability[:,1].argsort()[-10:] # chose top 10 probabilities for class = 1

# create a vector, Y_top_picks, with all zeros except ones for the selected top probabilities
Y_top_picks = np.zeros(len(X_test))
Y_top_picks[top_picks_indexes] = 1
|improve this answer|||||
$\endgroup$
  • $\begingroup$ Yes thank you, after asking I looked harder at each of the methods of the classifiers and came across the predict_proba and predict_log_proba methods. I was thinking that is what I was looking for, I will try it out. $\endgroup$ – jeffery_the_wind May 7 '17 at 8:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.