Conditional variational autoencoder: Feeding labeled MNIST to encoder with Keras

Question

I am looking for a code implementation of a CVAE using MNIST in Keras.

I found this Youtube video: https://youtu.be/8wrLjnQ7EWQ that does VAE, but I am not sure how do I convert this and make encoder to take in labels as well.

I have:

• ont-hot encoded the lables
• normalized images
• reshaped them

Now I want to feed it to the encoder.

I have this following code:

input_img = Input(shape=[input_shape], name='encoder_input')
x = Conv2D(32, 3, padding='same', activation='relu')(input_img)
x = Conv2D(64, 3, padding='same', activation='relu', strides=(2, 2))(x)
x = Conv2D(64, 3, padding='same', activation='relu')(x)
x = Conv2D(64, 3, padding='same', activation='relu')(x)

conv_shape = K.int_shape(x)  # Shape of conv to be provided to decoder

How do I modify input to pass labels with the image data?

PS: This code only works with keras 1x compatibility. Would be interested to know how to convert it to so it works in keras 2x as well. I am fairly new so help will be appreciated.

Answer 1

Nevermind. You just flatten the cnn layers and then use concatenate function to join the labels.

input_img = Input(shape=[input_shape], name='encoder_input')
x = Conv2D(32, 3, padding='same', activation='relu')(input_img)
x = Conv2D(64, 3, padding='same', activation='relu', strides=(2, 2))(x)
x = Conv2D(64, 3, padding='same', activation='relu')(x)
x = Conv2D(64, 3, padding='same', activation='relu')(x)
flat = Flatten()(x)

from tensorflow.keras.layers import concatenate
inputs = concatenate([flat, l])

where l is one-hot encoded labels