I have been reading into autoencoders for the purpose of denoising data. In the examples i found (eg. [1, 2, 3], which are the first few google results) they have the following input/output:

  • Input layer: The original data with some artificial noise added
  • Output layer: The original data (noiseless)

I fail to see the practical application of this setup as it seems to require noiseless data to begin with. If we already have noisy data, then add even more noise to the input, wont the the network learn to remove the artificial noise, but not the original noise?

Is a better approach to not add noise, but to just use a standard autoencoder where input/output are the same? It likely still will have some denoising properties as noise is random and therefore hard/impossible to represent in a lower dimensionality, while the patterns in the data are not. Therefore the noise might get lost in the embedding layer.

  • 1
    $\begingroup$ They're useful for tasks that are not even about removing noise. Diffusion models are usually trained to denoise image data. These are a vital part of newest image generation methods like DALL-E, Stable-diffusion, and midjourney. $\endgroup$
    – bogovicj
    Commented Aug 23, 2022 at 15:29
  • $\begingroup$ @bogovicj a) Isnt the main purpose of diffusion models to generate new samples? Also, to my understanding they seem to share same problems as VAEs for the purpose of denoising as they too seem to require noiseless data for training to begin with. b) I understand that adding noise to a VAE can e.g. be useful to prevent overfitting, but its seems weird to me that all the results for "denoising autoencoder" show examples which are actually about denoising data, but in a fashion that does not seem to have a practical application. $\endgroup$ Commented Aug 24, 2022 at 8:48


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.