Autoencoder give wrong results

Question

• I have studied Autoencoders and tried to implement a simple one.
• I have built a model with one hidden layer.
• I ran it with the MNIST digits dataset and plotted the digits before the Autoencoder and after it.
• I saw some examples that used a hidden layer of size 32 or 64, I tried it and it didn't give the same (or something close to) the source images.
• I tried to change the hidden layer to a size of 784 (same as the input size, just to test the model) but got the same results.

What am I missing? Why the examples on the web show good results and when I test them, I am getting different results?

import tensorflow as tf
from tensorflow.python.keras.layers import Input, Dense
from tensorflow.python.keras.models import Model, Sequential
from tensorflow.python.keras.datasets import mnist
import numpy as np
import matplotlib.pyplot as plt

#   Build models
hiden_size = 784 # After It didn't work for 32 , I have tried 784 which didn't improve results
input_layer = Input(shape=(784,))
decoder_input_layer = Input(shape=(hiden_size,))
hidden_layer = Dense(hiden_size, activation="relu", name="hidden1")
autoencoder_output_layer = Dense(784, activation="sigmoid", name="output")

autoencoder = Sequential()
autoencoder.add(input_layer)
autoencoder.add(hidden_layer)
autoencoder.add(autoencoder_output_layer)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

encoder = Sequential()
encoder.add(input_layer)
encoder.add(hidden_layer)

decoder = Sequential()
decoder.add(decoder_input_layer)
decoder.add(autoencoder_output_layer)

#
#   Prepare Input
(x_train, _), (x_test, _) = mnist.load_data()
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:])))
x_test = x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))

#
# Fit & Predict
autoencoder.fit(x_train, x_train,
                epochs=50,
                batch_size=256,
                validation_data=(x_test, x_test),
                verbose=1)

encoded_imgs = encoder.predict(x_test)
decoded_imgs = decoder.predict(encoded_imgs)

#
# Show results
n = 10  # how many digits we will display
plt.figure(figsize=(20, 4))
for i in range(n):
    # display original
    ax = plt.subplot(2, n, i + 1)
    plt.imshow(x_test[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    # display reconstruction
    ax = plt.subplot(2, n, i + 1 + n)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()

Answer 1

I believe you are training the autoencoder and creating the new image using two other models i.e. encoder, decoder.

For that,
First create the encoder, decoder and then use them in Model API to create the Encoder-Decoder.

Should also change the Loss and Activation to MSE, ReLU. Though current is working fine.

# This change will fix the issue
decoded_imgs = autoencoder.predict(x_test)

Output for hidden_size = 16