0
$\begingroup$

I've read the ResNet paper (https://arxiv.org/pdf/1512.03385.pdf)

They determined the layer degradation as that model with less layers learn quicker than with more. It can be visiable on plots below: enter image description here

It looks like they say like overfitting is cause of it in subsection ("Exploring Over 1000 layers.")

The testing result of this 1202-layer network is worse than that of our 110-layer network. We argue that this is because of overfitting

I think those are related like degradation causes overfitting, because it is like fitting too complex functions and is same as fitting too many layers. If that is correct than I do know know what type of overfitting is meant when the 1000+ layers are trained.

How to understand difference between those 2 concepts?

$\endgroup$
3
  • $\begingroup$ Difference between what two concepts? What are your thoughts on that? What's your current level of understanding and what specifically are you unsure about? Rather than asking about the difference between two concepts, it's usually better to try to understand each concept on its own. If there's some aspect of a concept you don't understand, ask about that. Usually once you understand two concepts you'll be able to identify the difference yourself. $\endgroup$
    – D.W.
    Jan 23 '20 at 17:41
  • $\begingroup$ I think those are related like degradation causes overfitting, because it is like fitting too complex functions and is same as fitting too many layers. If that is correct than I do know know what type of overfitting is meant when the 1000+ layers are trained. Can you tell me if I think correctly? $\endgroup$
    – Hyphen
    Jan 24 '20 at 16:20
  • $\begingroup$ Degradation: Adding more layers increases the training error. Overfitting: Adding more layers increases the difference between training and testing errors, training error goes lower but testing doesn't (or it can go higher). $\endgroup$
    – hafiz031
    Nov 23 '20 at 21:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.