I would like to construct an LSTM autoencoder model for sequence anomaly detection where the sequences can be varying in length. I understand based on this answer that padding and masking can be used to handle variable-length inputs, but how can I require that the output of the autoencoder is the same length as the input? I am using Keras.
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The sequence length is usually marked by a special token called "EOS" (end-of-sequence) appearing just after the last token in the sentence and before any padding tokens.
In your autoencoder, the loss (presumably the categorical cross-entropy) will make the model learn to generate the same output as the input, including the EOS token.
This way, the model learns to generate EOS tokens like any other token in the sentence. At inference time, you stop decoding once the model generates an EOS token.
