1
$\begingroup$

In order to build a predict model with two categories (buy or not buy),I want to use RandomForest and predict with type='prob', so I can have a prob of someone buy or not buy. So, with this outcome I can clusterize and make groups, like this:

group A: costumer who has [100 to 80]% of buy. group B: costumer who has [81 to 60]% of buy. ...

But I don't know the appropriate evaluation metric to measure the accuracy of this model. I guess that I can't use a confusion matrix.

Maybe I can use a ROC curve, and or measure KS between the buy group with the not buy group. But I'm not sure about this metrics.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

You should select some threshold, let's say 0.5 and treat customers with probability below threshold as not buy and above as buy. Based on this you can compute accuracy of your model. You can also check the ROC curve of the model. You can check various metric for binary classifier here

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you! Is a good ideia use a cutoff prob and define a class and then compute accuracy.! $\endgroup$
    – Michael Elma
    Commented Dec 10, 2015 at 17:23
0
$\begingroup$

Group the entires by the probability score you assigned them (e.g. 100-95%, 95-90%,....) and then measure the likelihood of their purchases from ground truth. You can then perform a Chi-squared calculation between your predicted and measured outcomes.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.