I coded a 38 layer CNN and 8 layer CNN but there's something wrong in my 38 layer CNN, which doesn't learn anything. Not able to fugure out what's wrong. They were trained on CIFAR. NN

  • $\begingroup$ What nonlinearity are you using? The traces in your plot say "VGG". Is your VGG 8 there the same architecture as described in the original paper? What changes (if any) have you made? $\endgroup$ – bogovicj Nov 7 '19 at 12:56
  • $\begingroup$ It's the same as in the paper. I am wondering.. can it be because of not batch normalizing? Because I am not doing it for both the networks as of now? $\endgroup$ – datarocker Nov 7 '19 at 15:21
  • $\begingroup$ That's possible. My first guess was that the deeper network suffered more from the vanishing gradient problem, but if you're using ReLU, then vanishing gradients are less likely to be the cause of what you're seeing. No matter what, it'll be hard to tell exactly what the issue is without digging into detail. $\endgroup$ – bogovicj Nov 7 '19 at 22:26
  • $\begingroup$ I am using Leaky ReLU for the implementation. But still I am wondering how can the 8 layer perform better than 38 layer without batch normalization? $\endgroup$ – datarocker Nov 7 '19 at 23:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.