Training a convolutional neural network for image denoising in Matlab

Question

I am currently trying to train CNNs to remove Poisson noise from images. The software I am using is Matlab 2018b, however the results I am getting are poor.

I have followed the steps provided in the following link. Here it is stated that we need to make a denoisingImageDatastore, which holds patches of our training images, and applies Gaussian noise to them. This isn't what I am looking for, but for the sake of practice I decided to try it out on my of PASCAL VOC dataset. I will now roughly outline the steps in my matlab code for this.

1. I first created an imageDatastore holding 45 of our images used for training.
2. Fifteen of those images will be used for validation
3. Now we create denoisingImagedatastores for both the training and validation set. There will be 60 patches per image, with the patch size 50x50
4. Then we specify the training options
5. I then specified the network layers using dnCNNLayers function from matlab
6. The next step is to train the network using the trainNetwork function

Here is a screen shot from the end of the training process.

I have done only 100 iterations, but the results are not too bad. Here we can se the pristine image, the noisy image and the denoised image using this network.

On the following link is the code I used for this. If you have any suggestions how the RMSE could be further lowered, it would be appreciated.

Please note that I have previously trained the same network on a much larger dataset and with nearly 10000 iterations, and the best rmse it could achieve was about 2.3. So obviously some parameters need to be changed.

Answer 1

So, I can point a few things that may cause a sub-optimal result:

• Your model may be too simple. Although using CNNs, which are pretty much the state of the art in Neural Networks based image processing, you need to pay attention to the archietcture you are using. For instance, you can take a look into that paper, where the authors have used CNNs + residual learning to denoise images
• Your dataset isn't complex enough/isn't big enough. As far as you said, you have pre-trained the network on a much larger dataset, however, the notion of "large dataset" isn't precise enough. Maybe you don't have enough data to train the model. In that sense, it may be useful use a well documented existing architecture where you can compare your results with those reported, and see if you indeed have the data needed to train it.
• Another concern is about the noise on train/test data. If your model is to complex, and your data too poor, it may be memorizing the mechanics of denoising gaussian noise (which seems to be the noise component in your training data), and hence does not generalize good on Poisson noise. To solve this, you may consider change the noise component on the train data (if it was generated artificially), or getting a different dataset.
• Change losses, take a look on other metrics. Loss choice have a great impact on the final model, since they determine how the gradient is calculated. For instance, following Goodfellow, Bengio and Aaron reasoning on the book "Deep Learning",

  For example, mean squared error applied to the pixels of an image implicitly specifies that an underlying cause is only salient if it significantly changes the brightness of a large number of pixels. This can be problematic if the task we wish to solve involves interacting with small objects. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.

Hence, despite looking into RMSE, I may recommend Peak to Signal Noise Ratio (PSNR) or Structural Similarity (SSIM).