During the training phase, a GAN model essentially makes two neural networks compete against one another; the generator and the discrimator. As always we start with a training dataset, e.g. of images. Then:
- The discrimator network is given images and must deicde whether or not they are real images
- The generator learns how to generate new images, with the aim of tricking the discriminator into thinking the image is real
Here is a simple sketch from this article:
Because both networks are fighting over the same metric (the loss of the discrimator network), this idea is also described as a min-max problem. The discriminator is minimising it's own error, while the generator is maximising the same error.
To see where GANs all started, have a skim of the seiminal paper by Ian Goodfellow. For a nice video introduction:
2. Inference using GANs
Using the process outlined above, we have trained two neural networks. Performing inference could actually mean using either one of those (or a combination of both, I suppose). In your case, it would seem you only require the generator part:
I am using DCGAN ( Deep Convolution GAN ) to generate images
This means you would take your trained generator model and use it to generate random samples by providing different random noise states as input (as is shown in the diagram above). Hopefully the learned distribution of your generator during training will be diverse enough to produce different samples. If it ends up always producing the same image, given different random noise samples as input, you are seeing an example of mode collapse (see linked video above).
3. GANs on embedded devices
To end up with a generator that is really of any practical use, you will need to do a lot of training, with many examples. State-of-the-art models use databases of images such as ImageNet, which contains 1 million images over 1000 categories.
To perform this kind of training from scratch, I really don't think you will have any luck on an embedded or micro device, such as a Jetson.
To perform inference, however, you might be able to do it. This would simply require loading the pre-trained generator model onto the device (along with the deep learning framework required e.g. Tensorflow). This might work, and I have seen people manage to do this - there were frame rates of around 1 images per second on a conservative convolutional network.
Memory constraints are just a problem as computation itself. On an embedded device, this will likely be a painfully slow process, but could work out if you are able to optimise many parts of the network and training process. You could e.g. compress images to smaller scales, use a final output layer which has fewer neurons, or even go down the path of using less floating point precision in your calculations! See this paper on 8-bit training (as opposed to standard 32-bit). There are also smaller versions of well-known models such as tiny-YOLO.
Another approach to edge-training (online training) you might take would be to start with transfer learning. You could take a model that was already trained on a task similar to your own, which you will have to search for in open-source projects. This model could then be loaded onto the embedded device and made to work as was done for inference, however we want to train. We do this by keeping the weights from the pre-trained model, freezing the majority of the layers and just allowing the final one or two layers to actually be fine-tuned with your new edge data.