There exists a mapping from input image to out image. Say input image is a piece of paper with a square hole in the center, and output image is the shadow of the input image when light shines on the paper, which also looks like a square, but with some blurs and different in size. (The real pattern is far more complicated than this square hole) I'm trying to implement a neural network that can learning this kind of mapping, when given the input image, the network can predict the "shadow" of the input image. I've tried to use a convolutional autoencoder-like structure (like this https://blog.keras.io/building-autoencoders-in-keras.html) by replace the target image from the input image it self to the output image training set. But it didn't looks work well. I would like to know what kind of neural network is suitable for this task? And what kind of loss function is suitable?
-
$\begingroup$ Could you elaborate a bit on your network structure and size? Also to understand how it did not work well, please also provide information about the size of your training data set and how you trained it. $\endgroup$– mjulApr 26, 2018 at 8:26
1 Answer
I would like to know what kind of neural network is suitable for this task?
A convolutional autoencoder is a good thing to try, but there is a minor issue with it for this type of problem: the latent representation has often lost spatial awareness (especially if FC layers were used at the end of the encoder). For a task like this, a pixel $p_{ij}$ in the input is likely to be far more related (and useful) to the corresponding (same position) pixel $\widehat{p}_{ij}$ in the output, compared to some other, far away pixel $p_{\ell k}$ in the input.
For this reason, I'd suggest using a U-net architecture, as this preserves spatial localities throughout the network.
References:
Long et al, Fully Convolutional Networks for Semantic Segmentation
Ronneberger et al, U-Net: Convolutional Networks for Biomedical Image Segmentation
And what kind of loss function is suitable?
A good starting point is the standard mean $L_1$ or $L_2$ loss over the output image. Since your images are not natural images, I hesitate to recommend perceptual losses, but they could work as well.
Without more details about your architecture and data, I'm afraid I cannot say much more than these though.
