Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I'm trying to implement an autoencoder for text. But I don't know which loss function I should use ? I tried using the mse but I get a huge loss 1063442. I'm trying to build a very simple autoencoder using only LSTM layers in Keras. On the keras-blog they use binary-crossentropy but I think the reason for this is because they use black and white images.
Thanks
The cause of your huge loss is probably a lack of normalization. As an example, in the Keras-blog post you linked, they conduct such normalization by dividing the input by 255, which is the maximum value an element in the input can be of.
As for which loss function to use, if you have a look at this keras-blog post, they construct a LSTM-based auto encoder for text which uses categorical crossentropy as a loss function.
From my experience, the cosine similarity loss (tf.keras.losses.CosineSimilarity) works best for text autoencoders with Keras. You can find the documentation at: https://www.tensorflow.org/api_docs/python/tf/keras/losses/CosineSimilarity
