1
$\begingroup$

Once the Model is built we want to check its performance, i did the following

  1. Predicted it on training set.
  2. Compute confusion matrix and ROC curve on training set.
  3. Predicted on test set
  4. Computed Confusion Matrix and ROC Curve on test set.

I want to know is it a right way and what more precise can be done?

I am using Decision Tress Random Forest SVM Both Linear Radial Naive Bayeian

Logistic regression and Neural Network. Do all of them have different

Performance Measure

Improve this question
$\endgroup$
1
  • $\begingroup$ Usually you want a specific measure to use a specific measure based on the confusion matrix. The measure to choose should be according to your goals. For details see datascience.stackexchange.com/questions/806/… $\endgroup$
    – DaL
    Commented Apr 24, 2016 at 6:28

1 Answer 1

Reset to default
2
$\begingroup$

You should provide more information about your problem. How large is your data set? Do you have hyper-parameters? What is your classifier?

Normally, you should split your entire data set into three parts: training, validation and testing sets. The testing set you should never use before the model is finally tuned. If you data set is not large enough, you should you use the cross-validation method to measure the error of your classifier.

There are several metrics to assess a performance of your classifier: accuracy, precision and recall, Cohen's kappa, F1-score, etc.

If you are working with python and scikit-learn, I would recommend to check this book. It is nicely written and explains basics of model assessments.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.