I am using DCGAN ( Deep Convolution GAN ) to generate images. However, I want to run it on embedded devices, such as Intel Movidiusor Jetson.

But, while going through several internet articles, I came to know that embedded devices are only used for inference in case of deep learning.

I have a few questions :

  1. GAN is basically a generative network. How can it have inference ? Will the inference and training be same thing for GAN ?

  2. Can it (GAN) be run on embedded devices , where memory is really an issue?

  • $\begingroup$ The newest Movidius and Jetson has 4GB+ memory, which should be enough for a very wide range of deep learning models. $\endgroup$ – Jon Nordby Aug 28 '18 at 18:12
  • $\begingroup$ Is your use case for a disconnected device? If not, this seems like something where a SOA/API solution is much more appropriate. $\endgroup$ – I_Play_With_Data Sep 26 '18 at 18:42

1. Mini introduction to Generative Adversarial Networks (GANs)

During the training phase, a GAN model essentially makes two neural networks compete against one another; the generator and the discrimator. As always we start with a training dataset, e.g. of images. Then:

  • The discrimator network is given images and must deicde whether or not they are real images
  • The generator learns how to generate new images, with the aim of tricking the discriminator into thinking the image is real

Here is a simple sketch from this article:

basic idea of a GAN

Because both networks are fighting over the same metric (the loss of the discrimator network), this idea is also described as a min-max problem. The discriminator is minimising it's own error, while the generator is maximising the same error.

More resources:

To see where GANs all started, have a skim of the seiminal paper by Ian Goodfellow. For a nice video introduction:

2. Inference using GANs

Using the process outlined above, we have trained two neural networks. Performing inference could actually mean using either one of those (or a combination of both, I suppose). In your case, it would seem you only require the generator part:

I am using DCGAN ( Deep Convolution GAN ) to generate images

This means you would take your trained generator model and use it to generate random samples by providing different random noise states as input (as is shown in the diagram above). Hopefully the learned distribution of your generator during training will be diverse enough to produce different samples. If it ends up always producing the same image, given different random noise samples as input, you are seeing an example of mode collapse (see linked video above).

3. GANs on embedded devices

To end up with a generator that is really of any practical use, you will need to do a lot of training, with many examples. State-of-the-art models use databases of images such as ImageNet, which contains 1 million images over 1000 categories.


To perform this kind of training from scratch, I really don't think you will have any luck on an embedded or micro device, such as a Jetson. To perform inference, however, you might be able to do it. This would simply require loading the pre-trained generator model onto the device (along with the deep learning framework required e.g. Tensorflow). This might work, and I have seen people manage to do this - there were frame rates of around 1 images per second on a conservative convolutional network.

Memory constraints are just a problem as computation itself. On an embedded device, this will likely be a painfully slow process, but could work out if you are able to optimise many parts of the network and training process. You could e.g. compress images to smaller scales, use a final output layer which has fewer neurons, or even go down the path of using less floating point precision in your calculations! See this paper on 8-bit training (as opposed to standard 32-bit). There are also smaller versions of well-known models such as tiny-YOLO.


Another approach to edge-training (online training) you might take would be to start with transfer learning. You could take a model that was already trained on a task similar to your own, which you will have to search for in open-source projects. This model could then be loaded onto the embedded device and made to work as was done for inference, however we want to train. We do this by keeping the weights from the pre-trained model, freezing the majority of the layers and just allowing the final one or two layers to actually be fine-tuned with your new edge data.


GAN is basically a generative network. How can it have inference ? Will the inference and training be same thing for GAN ?

Well, GANs are essentially two neural networks in series. One is called the Generator and the other is called the Discriminator. You need to train these two networks sequentially, i.e. you train the Generator and then you train the Discriminator (not at the same time). The inference of a GAN is performed by the Discriminator network, which labels the output of the Generator. I found this post very helpful, I hope you find it too!

Can it (GAN) be run on embedded devices , where memory is really an issue?

Well, if your embedded resources can afford two neural networks running at the same time, then it should be ok.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.