I understand the mathematical formalism behind how neural networks work as a classifier or perform regression analysis. But I face difficulty to realize how they are such a great denoising instrument. Even a simple fully connected single hidden layer network can denoise appreciable amount of noise from a relatively simple enough signal (with proper training of course). Do neural networks mimic one or more mathematical operations of filter (such as convolution) and that's why they are so effective in denoising application?
in general CNN can generate images, so we can feed them noised images and images without noise at output and it can learn how to figure out pixel color based on his neighbours, actually you can't reconstruct perfectly because noise cause removal of some information from image and all you can do it to find "as best as possible" values to restore original image
for more info this clip is very informative https://www.youtube.com/watch?v=z4vl3Z6NFW8
Denoising an image is typically done using Encoder-Decoder models called Denoising Autoencoders, and there's a formal reason for it. Let's take a look at these architectures:
We have an Encoder that generates a compressed representation of your input data, and a Decoder that restores it to normal. The reason why this architecture is so useful is that in order to be able to reconstruct the input signal, the Network must learn efficient ways to represent the same information with less nodes at the center. This operation consists of noise reduction. Noise is by definition something that cannot be learnt, and the model must focus only on the most relevant bits of information in order to work effectively.
Denoising Autoencoders are trained specifically for that. These model learn to map a noisy input with a cleaned version of the same piece of data. The model, that is made especially for noise reduction, learns to extract useful information from an image, and discards anything that seems noisy.
PS: In more advanced architectures, Denoising Autoencoders are paired with a Discriminator network: while the Disciminator tries to distinguish denoised versus real images, the Denoising Autoencoder tries to fool the Discriminator generating denoised images that look more and more realistic. In that case you'd have a GAN, but that's not strictly necessary to understand image denoising.