0
$\begingroup$

Most generative adversarial networks learn the distribution of the dataset and then generate a sample of 10's to 100's of images with similar distributions. I am curious if there is any research regenerating a single image.

I have looked into Super Resolution GAN's and they somewhat model this. They try to estimate a highly resolved image from its lower resolution counterpart. Could that be used to recreate an image?

Thanks for the help.

$\endgroup$
5
  • $\begingroup$ What do you mean with "recreate"? $\endgroup$ – noe Feb 13 '18 at 22:32
  • $\begingroup$ @ncasas i mean feeding the image to the gan and seeing it being recreated $\endgroup$ – paws Feb 15 '18 at 21:07
  • $\begingroup$ This is precisely what an autoencoder does. Are you targetting GANs for a specific reason? $\endgroup$ – noe Feb 16 '18 at 8:52
  • $\begingroup$ @ncasas i've read GAN's have had more success at doing this $\endgroup$ – paws Feb 17 '18 at 3:54
  • $\begingroup$ @ncasas also my requirement is something very specific, it is a single image application $\endgroup$ – paws Feb 17 '18 at 3:56
2
$\begingroup$

A generative adversarial network is comprised of two parts which are necessary for the training, the generator and the discriminator. They are playing a game against each other, the discriminator tried to learn what is real and what is fake, and the generator tries to create fake instances which will fool the discriminator.

At the input of the discriminator you will use real images, for example the MNIST numbers. At the input of the generator you will put just random noise vectors. These vectors will then be shaped by the layers of the generator to create some image, hopefully resembling the MNIST dataset. If it fools the discriminator, it wins that round. Otherwise, it will tune its weights using backpropagation in order to produce better quality results in the next round.

Once training is complete, you will detach the discriminator and use only the generator. Then for every single random noise vector fed to the input of the generator you will receive an associated image.

Here is a GaN implementation. You will need to put a path "./gan/images/" i the directory where you are running this GaN.

import numpy as np
import time
from tensorflow.examples.tutorials.mnist import input_data

from keras.layers import Input
from keras.models import Model
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Dense, Activation, Flatten, Reshape
from keras.layers import Conv2D, Conv2DTranspose, UpSampling2D
from keras.layers import LeakyReLU, Dropout
from keras.layers import BatchNormalization
from keras.optimizers import Adam, RMSprop
from keras.datasets import mnist

import matplotlib.pyplot as plt

class GAN():
    def __init__(self):
        self.img_rows = 28 
        self.img_cols = 28
        self.channels = 1
        self.img_shape = (self.img_rows, self.img_cols, self.channels)

        optimizer = Adam(0.0002, 0.5)

        # Build and compile the discriminator
        self.discriminator = self.build_discriminator()
        self.discriminator.compile(loss='binary_crossentropy', 
            optimizer=optimizer,
            metrics=['accuracy'])

        # Build and compile the generator
        self.generator = self.build_generator()
        self.generator.compile(loss='binary_crossentropy', optimizer=optimizer)

        # The generator takes noise as input and generated imgs
        z = Input(shape=(28,28,))
        img = self.generator(z)

        # For the combined model we will only train the generator
        self.discriminator.trainable = False

        # The valid takes generated images as input and determines validity
        valid = self.discriminator(img)

        # The combined model  (stacked generator and discriminator) takes
        # noise as input => generates images => determines validity 
        self.combined = Model(z, valid)
        self.combined.compile(loss='binary_crossentropy', optimizer=optimizer)

    def build_generator(self):

        noise_shape = (28,28,)

        model = Sequential()

        model.add(Reshape((28*28,), input_shape=noise_shape))
        model.add(Dense(256))
        model.add(LeakyReLU(alpha=0.2))
        model.add(BatchNormalization(momentum=0.8))
        model.add(Dense(512))
        model.add(LeakyReLU(alpha=0.2))
        model.add(BatchNormalization(momentum=0.8))
        model.add(Dense(1024))
        model.add(LeakyReLU(alpha=0.2))
        model.add(BatchNormalization(momentum=0.8))
        model.add(Dense(np.prod(self.img_shape), activation='tanh'))
        model.add(Reshape(self.img_shape))   

        model.summary()

        noise = Input(shape=noise_shape)
        img = model(noise)

        return Model(noise, img)

    def build_discriminator(self):

        img_shape = (self.img_rows, self.img_cols, self.channels)

        model = Sequential()

        model.add(Conv2D(32, kernel_size=(3, 3),
                 input_shape=img_shape))
        model.add(MaxPooling2D(pool_size=(2, 2)))
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))

        model.add(Flatten())
        model.add(Dense(64))
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dense(1, activation='sigmoid'))
        model.summary()

        img = Input(shape=img_shape)
        validity = model(img)

        return Model(img, validity)

    def train(self, epochs, batch_size=128, save_interval=50):

        # Load the dataset
        (X_train, _), (_, _) = mnist.load_data()

        # Rescale -1 to 1
        X_train = (X_train.astype(np.float32) - 127.5) / 127.5
        X_train = np.expand_dims(X_train, axis=3)

        half_batch = int(batch_size / 2)

        for epoch in range(epochs):

            # ---------------------
            #  Train Discriminator
            # ---------------------

            # Select a random half batch of images
            idx = np.random.randint(0, X_train.shape[0], half_batch)
            imgs = X_train[idx]

            noise = np.random.normal(0, 1, (half_batch,28,28,))

            # Generate a half batch of new images
            gen_imgs = self.generator.predict(noise)

            # Train the discriminator
            d_loss_real = self.discriminator.train_on_batch(imgs, np.ones((half_batch, 1)))
            d_loss_fake = self.discriminator.train_on_batch(gen_imgs, np.zeros((half_batch, 1)))
            d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)


            # ---------------------
            #  Train Generator
            # ---------------------

            noise = np.random.normal(0, 1, (batch_size,28,28,))

            # The generator wants the discriminator to label the generated samples
            # as valid (ones)
            valid_y = np.array([1] * batch_size)

            # Train the generator
            g_loss = self.combined.train_on_batch(noise, valid_y)

            # Plot the progress
            print ("%d [D loss: %f, acc.: %.2f%%] [G loss: %f]" % (epoch, d_loss[0], 100*d_loss[1], g_loss))

            # If at save interval => save generated image samples
            if epoch % save_interval == 0:
                self.save_imgs(epoch)

    def save_imgs(self, epoch):
        r, c = 5, 5
        noise = np.random.normal(0, 1, (r * c, 28,28,))
        gen_imgs = self.generator.predict(noise)

        # Rescale images 0 - 1
        gen_imgs = 0.5 * gen_imgs + 0.5

        fig, axs = plt.subplots(r, c)
        cnt = 0
        for i in range(r):
            for j in range(c):
                axs[i,j].imshow(gen_imgs[cnt, :,:,0], cmap='gray')
                axs[i,j].axis('off')
                cnt += 1
        fig.savefig("gan/images/mnist_%d.png" % epoch)
        plt.close()


if __name__ == '__main__':
    gan = GAN()
    gan.train(epochs=30000, batch_size=64, save_interval=200)
$\endgroup$
3
  • $\begingroup$ can you point to me good implementations of GAN's? $\endgroup$ – paws Feb 15 '18 at 21:09
  • $\begingroup$ github.com/osh/KerasGAN, using Keras. $\endgroup$ – JahKnows Feb 16 '18 at 3:29
  • $\begingroup$ I added a better GaN implementation. $\endgroup$ – JahKnows Mar 2 '18 at 2:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.