0
$\begingroup$

I tried to predict on training set but i got accuracy of 100%. However on the testset, i got an accuracy of 62%. Should i be worried of getting high accuacies on training dataset using caret? why accuracy on training dataset is 100% using random forest caret , does it indicate overfitting?

enter image description here

$\endgroup$
2
$\begingroup$

Yes, those accuracies dont say anything.

You have overfitted to 310 data points which is pretty easy using RF. The fact is that you said you "predicted" on training data. Well, when you learned on training data and you ask prediction for points you already gave the model it simply knows them.

You need to use a cross validation technique (here i propose Leave-One-Out technique as you dont have much data points). RF is more robust than DT towards overfitting, but that is not applied to this setting. Here you getting an exam in which you ask a question you already gave to the student. He just simply knows the answer.

$\endgroup$
0
$\begingroup$

In my opinion it does indicate overfitting in the given case. Commonly overfitting occurs when one gets very good accuracy (or other score) on a training set, and a lousy accuracy on a test set. Moreover tree-like models are very easy to overfit.

However not always getting 100% accuracy means that the model is overfitted. If the data does not contain noise or the chosen features allow for absolute separation of classes, then 100% accuracy does not indicate anything bad. However such situation should occur for a training and test set as well.

In the given case, I would suggest cross validation procedure, or using validation set if you have enough data points.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.