# Keras: Very high loss for Autoencoder

I am trying to implement an autoencoder for prediction of multiple labels using Keras. This is a snippet:

input = Input(shape=(768,))
hidden1 = Dense(512, activation='relu')(input)
compressed = Dense(256, activation='relu', activity_regularizer=l1(10e-6))(hidden1)
hidden2 = Dense(512, activation='relu')(compressed)
output = Dense(768, activation='sigmoid')(hidden2) # sigmoid is used because output of autoencoder is a set of probabilities

model = Model(input, output)
model.compile(optimizer='adam', loss='categorical_crossentropy') # categorical_crossentropy is used because it's prediction of multiple labels
history = model.fit(x_train, x_train, epochs=100, batch_size=50, validation_split=0.2)


I ran this in Jupyter Notebook (CPU) and I am getting loss and validation loss as: loss: 193.8085 - val_loss: 439.7132
but when I ran it in Google Colab (GPU), I am getting very high loss and validation loss: loss: 28383285849773932.0000 - val_loss: 26927464965996544.0000.

What could be the reason for this behavior?

• You say that the output is a set of probabilities and that it's a prediction of multiple labels. Are these probabilities part of a single categorical probability distribution or are they independent?
– noe
May 12, 2021 at 6:58
• @noe, Thank you for getting back. I didn't quite understand what you meant. What I am trying to achieve is: The train and test set contains groups of labels. When the model is given the test data, it predicts probability values. Labels in the train set that has probability value higher than threshold probability value have higher chance of occurring close to the labels in the test data. May 12, 2021 at 7:21