0
$\begingroup$

I have a dataset with 16 features and 32 class labels, which shows the following behavior:

  1. Neural network classification: high accuracy on train 100%, but low accuracy on the test set 3% (almost like random classification). If I make the network less flexible (reduce the number of neurons or hidden layers), then the train and test accuracy become about 10%.

  2. Gradient boosting tree classification: exactly same behavior. Flexible model results in 100% accuracy over train, but random accuracy on the test. If I reduce the flexibility, the train and test accuracy will be very low around 10%.

What could be the reason? How can I fix this? Any other algorithm I can try?

Here is the distribution of the target data:

enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

This seems like a perfect candidate of Overfitting in training data. You need to use simpler model or reduce complexity of existing model.

Also to better undertsand the problem please share number of observations for each class. i presume you dont have enough data to create a good model.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for your answer. As I explained, when I use a simpler model, then the train and test accuracy become about 10%. $\endgroup$
    – Albert
    Feb 18, 2022 at 17:59
  • $\begingroup$ Could you please share distribution of target data $\endgroup$
    – Ashwiniku918
    Feb 18, 2022 at 19:19
  • $\begingroup$ @Ashwinku918 Thanks. I added it. $\endgroup$
    – Albert
    Feb 22, 2022 at 20:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.