0
$\begingroup$

enter image description here

The graph above shows how accuracy stops increasing after reaching a certain number of features. There are also sudden drops in accuracy at some points. Can this be attrrubuted to overfitting? I am training a decision tree by the way.

$\endgroup$
  • $\begingroup$ Overfitting needs a validation set to be measured. $\endgroup$ – David Masip Apr 20 '18 at 9:03
0
$\begingroup$

I can tell from your screenshot that you are plotting the validation accuracy. When you overfit your training accuracy should be very high, but your validation accuracy should get lower and lower. Or if you think in terms of error rather than accuracy you should see the following plot in case of overfitting. In the figure below the x-axis contains the training progress, i.e. the number of training iterations. The training error (blue) keeps decreasing, while the validation error (red) starts increasing at the point where you start overfitting.

enter image description here

This picture is from the wikipedia article on overfitting by the way: https://en.wikipedia.org/wiki/Overfitting Have a look.

So to answer your question: No, I don't think you are overfitting. If increasing the number of features would make the overfitting more and more significant the validation accuracy should be falling, not stay constant. In your case it seems that more features are simply no longer adding additional benefit for the classification.

$\endgroup$
  • $\begingroup$ " In your case it seems that more features are simply no longer adding additional benefit for the classification" so what is the reason for this if not overfitting? Thanks! $\endgroup$ – mc8 Feb 15 '17 at 12:45
  • $\begingroup$ oh I get it. I set the maximum number of splits to 4 that is why overfitting is not possible. So is it useless to plot the accuracy in terms of the number of features? $\endgroup$ – mc8 Feb 15 '17 at 12:58
  • $\begingroup$ Not sure what you mean by setting the maximum number of splits? Do you mean the tree depth? Regarding, my statement about no additional benefit. Suppose you want to classify fruit. Let's say you have access to the following features: colour, weight, length, width. Using only the colour is not so good. Additionally using the weight and the length should help you distinguish most fruit almost perfectly. Then also adding the width may not be so useful anymore since you can already tell apart the fruit based on the first three. $\endgroup$ – Chrigi Feb 15 '17 at 14:19
  • $\begingroup$ Actually, for your plot above, how do you choose which features to use? If for some reason you think there is a need not to use all of them, then you should use some feature selection or dimensionality reduction technique to reduce your feature space. $\endgroup$ – Chrigi Feb 15 '17 at 14:21
  • $\begingroup$ i just picked the features ramdomly i used time and frequency domain features then used the ReliefF algorithm to rank the features. $\endgroup$ – mc8 Feb 15 '17 at 14:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.