# Autoencoder general questions and poor loss

I'm trying to get a simple autoencoder working on the iris dataset to explore autoencoders at a basic level. I'm running into an issue where the loss of the model is extremely high (>20).

Can someone help me understand if this model looks normal to them to begin with?

Some questions I'd love some help on understanding:

• There are 3 possible outputs for y - thus I used Softmax in the final layer - if I was to OHE the output, would using something like Sigmoid be more appropriate as the values are bound between 0 and 1?

• Altering the smallest change in the layers (encoding layer going to 6 instead of 3) --> causes a major shift in the loss -- is this normal?

• Each run of the autoencoder produces a different result - is this normal that it is not deterministic?

• Why does the last layer have to be the same size (4) as the input dimension - are we able to force this to allow for an output of 3 for example? I know I can read from a latent layer, but then I can't fit the model based on that layer.

  import pandas as pd
import numpy as np

from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
from sklearn.pipeline import Pipeline
from sklearn import datasets
from tensorflow.keras.layers import Input, Dense, BatchNormalization, LeakyReLU
from tensorflow.keras import backend, layers, models, metrics, utils
from tensorflow.keras import regularizers, Input, Model, optimizers

x = iris.data
y = iris.target.reshape(-1, 1)
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.20)
input_dim = Input(shape=(X_train.shape[1],))

encoded = layers.Dense(6, input_dim='input_dim')(input_dim)
encoded = BatchNormalization()(encoded)
encoded = LeakyReLU()(encoded)
encoded = layers.Dense(3)(encoded)

decoded = layers.Dense(4, activation='softmax')(encoded)
autoencoder = Model(inputs=input_dim, outputs=decoded)

autoencoder.compile(optimizer=opt
, loss='categorical_crossentropy'
, metrics=[metrics.CategoricalAccuracy()])

history = autoencoder.fit([X_train]
, [X_train]
, epochs=16
, batch_size=2
, verbose=2
, validation_data=((X_test),(X_test))
)


Thank you for any help!

• OHE means?..... one hot?... Jun 26 at 12:49
• Yes, using OHE as one hot encoding - thanks for reminding me to clarify Jun 27 at 1:11

There is some confusion in your question, since you are talking about an AE, but you are using softmax as output... which makes almost no sense

First of all, the loss by itself, has no meaning:
If I give you two models with two different losses (in your case, two AE), you cannot, in any way, guess which is the best one (therefore a loss that is 20 means nothing)

4. an autoencoder aims to learn the identity function ($$decoder(encoder(x)) = x$$) therefore obviously no, the input has to have the same shape as the output