1
$\begingroup$

I try to train convolutional neural net for a classification problem. However, my neural net is not learning anything. It guesses only two labels and ignores the rest. Even though I try to train to overfit my neural net, the loss function is not decreasing at all. My I try to make my network go as deep as 12 layers of the convolutional neural net in order to overfit the subsampling data. However, it is not working. Do you have any suggestion what to look?

$\endgroup$
  • $\begingroup$ Could you please provide the distribution in terms of proportion, of the target categorical variable: Could it be because that the target classes are imbalanced? $\endgroup$ – jkyh Dec 10 '16 at 11:02
  • $\begingroup$ Does it need to be balanced if I want to overfit it? Those two labels were contributed more than 50 percent of the total label in the training data. $\endgroup$ – MakaraPr Dec 10 '16 at 12:13
  • $\begingroup$ Add more details to your question along with the snippet of your code $\endgroup$ – Nain Dec 10 '16 at 14:07
  • $\begingroup$ There are lots of ways that a neural network can go wrong. To get anything more than a guess on what the problem is, you will need to show some of your data preparation, training and testing code, plus explain your data and show a few samples. $\endgroup$ – Neil Slater Dec 10 '16 at 14:48
  • $\begingroup$ Start with your loss function, then confirm your Xs and labels are in the correct order, then post some code here $\endgroup$ – bold Dec 11 '16 at 8:58
2
$\begingroup$

Make sure that while training, you are passing the data in a random order. It looks like you are passing all images of one class at a time then the next class etc. Shuffle the images and pass it in a random manner.

$\endgroup$
1
$\begingroup$

There are a bazillion things that can go wrong when training a DNN, and it's often helpful to heavily simplify your starter net/data before you move onto your actual problem.

Without any more information on the specifics of your setup, etc, the first thing I would recommend for you to try is to overfit your net to random noise. Don't make it deep just yet! You want to make sure that everything else if your setup is kosher first.

So: Create a random-noise data set with labels, OR extract a small (< 50) or so images for you to use as your dev set, and try to overfit on those in the very beginning, with a vastly simplified net. This should always be your first step.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.