Why are the weights of my first layer and last layer in the CNN change while the middle layers don't?

Question

The weights of my first and last convolution layers do change in a noticeable way. However, the rest of my convolution layers, in the middle, do not.

I should add that all convolution layers' biases change noticeably, however the kernel weights of the middle layers (as opposed to the first and last layers) do not change significantly.

Has anyone else encountered this?

Can anyone suggest why this is might be happening?

Can I rule out vanishing gradients...?

Any ideas for resolution (if this is at all a problem)?

Thanks in advance.

I'm happy to add details based on comments.

Answer 1

Its ofcourse dataspecific, but this behavior is mostly likely not harmful. What could indicate dangerous behaviours is wild behaviour of cv-metric, learning rates etc...

You can think of every cnn layer task to learn specific features/pecularities of a photo. In the first layers they will catch the most general patterns, edges etc... Middle layers are more specific and they look at some finer details but not all too specific. And final layers are there for the small details that can really discriminate the classes and make the right prediction.

So in your case weights are not updated significantly because information in middle layers is not that significant.