1
$\begingroup$

Objective

Seeking for suggestions and advice why the DCGAN training is not working.


Task

Train DCGAN to learn to generate CIFAR10-like images. Each CIFAR10 image has the shape (32,32,3) where (32x32) is the image size and 3 are channels (RGB).

enter image description here

Used the Keras dataset and the data is scaled to [-1, 1].

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()
x_train = tf.concat((x_train, x_test), axis=0)
X = tf.cast(x_train, dtype='float32')
X = (X - 127.5) / 127.5    # scale to [-1,1]

References

Problem

Apparently the training is not working generating images below (as the epoch progresses).

Generated images

enter image description here

Training loss log

0 [D loss: 0.000550, acc.: 100.00%] [G loss: 0.028351]
1 [D loss: 0.000183, acc.: 100.00%] [G loss: 0.013826]
2 [D loss: 0.000071, acc.: 100.00%] [G loss: 0.009705]
3 [D loss: 0.000055, acc.: 100.00%] [G loss: 0.007897]
4 [D loss: 0.000028, acc.: 100.00%] [G loss: 0.006519]
5 [D loss: 0.000023, acc.: 100.00%] [G loss: 0.004263]
6 [D loss: 0.000019, acc.: 100.00%] [G loss: 0.002820]
7 [D loss: 0.000016, acc.: 100.00%] [G loss: 0.002480]
8 [D loss: 0.000014, acc.: 100.00%] [G loss: 0.002264]
9 [D loss: 0.000019, acc.: 100.00%] [G loss: 0.102190]
10 [D loss: 0.000011, acc.: 100.00%] [G loss: 0.004024]
11 [D loss: 0.000003, acc.: 100.00%] [G loss: 0.002417]
12 [D loss: 0.000002, acc.: 100.00%] [G loss: 0.002455]
13 [D loss: 0.000002, acc.: 100.00%] [G loss: 0.001967]
14 [D loss: 0.000002, acc.: 100.00%] [G loss: 0.002482]
15 [D loss: 0.000003, acc.: 100.00%] [G loss: 0.005420]
16 [D loss: 0.000001, acc.: 100.00%] [G loss: 0.004363]
17 [D loss: 0.002413, acc.: 100.00%] [G loss: 0.025292]
18 [D loss: 0.000302, acc.: 100.00%] [G loss: 0.001496]
19 [D loss: 0.000112, acc.: 100.00%] [G loss: 0.000931]
20 [D loss: 0.000127, acc.: 100.00%] [G loss: 0.001128]
21 [D loss: 0.000054, acc.: 100.00%] [G loss: 0.000581]
22 [D loss: 0.000049, acc.: 100.00%] [G loss: 0.000507]
23 [D loss: 0.000027, acc.: 100.00%] [G loss: 0.000414]
24 [D loss: 0.000046, acc.: 100.00%] [G loss: 0.000371]
25 [D loss: 0.000019, acc.: 100.00%] [G loss: 0.000319]

Neural network

Generator

Expand a vector of random values of shape (100,) into a (32,32,3) image.

Layer (type)                 Output Shape              Param #   
=================================================================
dense_1 (Dense)              (None, 4096)              413696    
_________________________________________________________________
reshape (Reshape)            (None, 4, 4, 256)         0         
_________________________________________________________________
up_sampling2d (UpSampling2D) (None, 8, 8, 256)         0         
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 8, 8, 128)         295040    
_________________________________________________________________
batch_normalization_3 (Batch (None, 8, 8, 128)         512       
_________________________________________________________________
activation (Activation)      (None, 8, 8, 128)         0         
_________________________________________________________________
up_sampling2d_1 (UpSampling2 (None, 16, 16, 128)       0         
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 16, 16, 128)       147584    
_________________________________________________________________
batch_normalization_4 (Batch (None, 16, 16, 128)       512       
_________________________________________________________________
activation_1 (Activation)    (None, 16, 16, 128)       0         
_________________________________________________________________
up_sampling2d_2 (UpSampling2 (None, 32, 32, 128)       0         
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 32, 32, 128)       147584    
_________________________________________________________________
batch_normalization_5 (Batch (None, 32, 32, 128)       512       
_________________________________________________________________
activation_2 (Activation)    (None, 32, 32, 128)       0         
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 32, 32, 3)         3459      
_________________________________________________________________
activation_3 (Activation)    (None, 32, 32, 3)         0         
=================================================================
Total params: 1,008,899
Trainable params: 1,008,131
Non-trainable params: 768

Discriminator

Reduce a image of size (32, 32, 3) down to a probability between (0,1).

Layer (type)                 Output Shape              Param #   
=================================================================
conv2d (Conv2D)              (None, 16, 16, 64)        1792      
_________________________________________________________________
leaky_re_lu (LeakyReLU)      (None, 16, 16, 64)        0         
_________________________________________________________________
dropout (Dropout)            (None, 16, 16, 64)        0         
_________________________________________________________________
conv2d_1 (Conv2D)            (None, 8, 8, 128)         73856     
_________________________________________________________________
batch_normalization (BatchNo (None, 8, 8, 128)         512       
_________________________________________________________________
leaky_re_lu_1 (LeakyReLU)    (None, 8, 8, 128)         0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 8, 8, 128)         0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 4, 4, 128)         147584    
_________________________________________________________________
batch_normalization_1 (Batch (None, 4, 4, 128)         512       
_________________________________________________________________
leaky_re_lu_2 (LeakyReLU)    (None, 4, 4, 128)         0         
_________________________________________________________________
dropout_2 (Dropout)          (None, 4, 4, 128)         0         
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 4, 4, 256)         295168    
_________________________________________________________________
batch_normalization_2 (Batch (None, 4, 4, 256)         1024      
_________________________________________________________________
leaky_re_lu_3 (LeakyReLU)    (None, 4, 4, 256)         0         
_________________________________________________________________
dropout_3 (Dropout)          (None, 4, 4, 256)         0         
_________________________________________________________________
flatten (Flatten)            (None, 4096)              0         
_________________________________________________________________
dense (Dense)                (None, 1)                 4097      
=================================================================
Total params: 524,545
Trainable params: 523,521
Non-trainable params: 1,024

Code

dcgan = DCGAN(
    dataset=X, 
    rows=32,
    cols=32,
    channels=3,    
    latent_dim=100,
    batch_size=256,
)
class DCGAN():
    def __init__(self, dataset, rows, cols, channels, latent_dim = 100, batch_size=256):
        # data
        self.dataset = dataset
        self.batch_size = batch_size
        
        # Input shape
        self.img_rows = rows
        self.img_cols = cols
        self.channels = channels
        self.img_shape = (self.img_rows, self.img_cols, self.channels)
        self.latent_dim = latent_dim

        optimizer = Adam(0.0002, 0.5)

        # Build and compile the discriminator
        self.discriminator = self.build_discriminator()
        self.discriminator.compile(
            loss='binary_crossentropy',
            optimizer=optimizer,
            metrics=['accuracy']
        )

        # Build the generator
        self.generator = self.build_generator()
        self.generator.trainable = True

        # The generator takes noise as input and generates imgs
        z = Input(shape=(self.latent_dim,))
        img = self.generator(z)

        # For the gan model we will only train the generator
        self.discriminator.trainable = False

        # The discriminator takes generated images as input and determines validity
        valid = self.discriminator(img)

        # The gan model  (stacked generator and discriminator)
        # Trains the generator to fool the discriminator
        self.gan = Model(z, valid)
        self.gan.compile(loss='binary_crossentropy', optimizer=optimizer)

    # select real samples
    def generate_real_samples(self, n_samples):
        # choose random instances
        indices = tf.random.uniform(
            shape=[n_samples, 1], 
            minval=0,
            maxval=self.dataset.shape[0], 
            dtype=tf.int32
        )
        X = tf.gather_nd(self.dataset, indices)
        y = tf.ones([n_samples,1])    # label 1
        return X, y        

    def generate_random_noise(self, n_samples):
        Z = tf.random.normal(
            shape=(n_samples, self.latent_dim),
            mean=0.0, 
            stddev=1.0, 
            dtype=tf.dtypes.float32, 
            seed=None
        )
        return Z

    # generate fake samples
    def generate_fake_samples(self, n_samples):
        # noise = np.random.normal(0, 1, (self.batch_size, self.latent_dim))
        Z = self.generate_random_noise(n_samples)              
        X = self.generator.predict(Z)
        y = tf.zeros([n_samples,1])    # label 0

        return X, y

    def build_generator(self):
        model = Sequential()

        model.add(Dense(4 * 4 * self.batch_size, activation="relu", input_dim=self.latent_dim))
        model.add(Reshape((4, 4, self.batch_size)))
        model.add(UpSampling2D())
        model.add(Conv2D(128, kernel_size=3, padding="same"))
        model.add(BatchNormalization(momentum=0.9))
        model.add(Activation("relu"))
        model.add(UpSampling2D())
        model.add(Conv2D(128, kernel_size=3, padding="same"))
        model.add(BatchNormalization(momentum=0.9))
        model.add(Activation("relu"))
        model.add(UpSampling2D())
        model.add(Conv2D(128, kernel_size=3, padding="same"))
        model.add(BatchNormalization(momentum=0.9))
        model.add(Activation("relu"))
        model.add(Conv2D(self.channels, kernel_size=3, padding="same"))
        model.add(Activation("tanh"))
        model.summary()

        noise = Input(shape=(self.latent_dim,))
        img = model(noise)
        return Model(noise, img)

    def build_discriminator(self):
        model = Sequential()

        model.add(Conv2D(64, kernel_size=3, strides=2, input_shape=self.img_shape, padding="same"))
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))
        model.add(Conv2D(128, kernel_size=3, strides=2, padding="same"))
        model.add(BatchNormalization(momentum=0.8))
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))
        model.add(Conv2D(128, kernel_size=3, strides=2, padding="same"))
        model.add(BatchNormalization(momentum=0.8))
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))
        model.add(Conv2D(256, kernel_size=3, strides=1, padding="same"))
        model.add(BatchNormalization(momentum=0.8))
        model.add(LeakyReLU(alpha=0.2))
        model.add(Dropout(0.25))
        model.add(Flatten())
        model.add(Dense(1, activation='sigmoid'))

        model.summary()

        img = Input(shape=self.img_shape)
        validity = model(img)

        return Model(img, validity)

    def train(self, epochs, save_interval=50):
        for epoch in range(epochs):
             # If at save interval => save generated image samples
            for batch in range(self.dataset.shape[0] // self.batch_size + 1):
                # ---------------------
                #  Train Discriminator
                # ---------------------
                # Select a random half of true images
                reals, true_labels = self.generate_real_samples(self.batch_size)
                fakes, false_labels = self.generate_fake_samples(self.batch_size)

                # Sample noise and generate a batch of fake images

                # Train the discriminator (real as 1 and generated as 0)'
                self.discriminator.trainable = True
                d_loss_real = self.discriminator.train_on_batch(reals, true_labels)
                d_loss_fake = self.discriminator.train_on_batch(fakes, false_labels)
                d_loss = tf.math.divide(tf.add(d_loss_real, d_loss_fake), 2.0)

                # ---------------------
                #  Train Generator
                # ---------------------
                # Train the generator (for discriminator to mistake fake as real)
                self.discriminator.trainable = False
                self.generator.trainable = True
                Z = self.generate_random_noise(self.batch_size) 
                g_loss = self.gan.train_on_batch(Z, true_labels)

            # Plot the progress
            print(
                "%d [D loss: %f, acc.: %.2f%%] [G loss: %f]" % 
                (epoch, d_loss[0], 100*d_loss[1], g_loss)
            )
            if epoch % save_interval == 0:
                self.save_imgs(epoch)

    def save_imgs(self, epoch):
        r, c = 5, 5
        Z = self.generate_random_noise(r*c) 
        images = self.generator.predict(Z)
        save_images(images, r, c, epoch)
def save_images(images, r, c, epoch):
    # scale from [-1,1] to [0,1]
    images = (images + 1.0) / 2.0

    fig, axs = plt.subplots(r, c)
    cnt = 0
    for i in range(r):
        for j in range(c):
            #axs[i,j].imshow(gen_imgs[cnt], cmap=mpl.cm.binary)
            axs[i,j].imshow(images[cnt])
            axs[i,j].axis('off')
            cnt += 1
    fig.savefig("/content/drive/MyDrive/dcgan_cifar10_%d.png" % epoch)
    plt.close()
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.