Could someone please explain to me in details (possibly from mathematical point of view) what is the role of Equivariance and Invariance in Convolutional Neural Networks, and how are they actually achieved? I found a similar question here but I did not get the full picture.
Several answers have been provided to the role of equivariance or invariance question "What is the difference between “equivariant to translation” and “invariant to translation”.
Depending on how local they are, scalar and Convolution operators tend to be equivariant, max/min or range are more invariant, and subsampling/pooling can somewhat link those behavior.
Your question is discussed in detail in a video about generalized CNNs.