I have numeric signals from two sensors, and I would like to create a mapping using sequence-to-sequence autoencoder. I used the transformer architecture, and it seems to be learning - the loss is getting lower both for the training and validation sets over time, and when using the decoder on both inputs - memory (that is, $encoder(source)$) and $target$, the decoder would return a value, very similar to $target$ (and also low $MSE$ values).
Nevertheless, in the inference stage, results look very much regardless of the input I add.
My current approach for inference, when we don't see $target$, is:

  1. Encode $source$ to get $memory$
  2. Feed the decoder
    • $momory$ (which is supposedly encodes the entire $source$ and projects it into some latent space).
    • $sos$ - start of sequence token. Even though I work with numeric signals, I borrowed $sos$ from the NLP domain to tell my decoder it is time to start decoding.
  3. Do 2 iteratively, take the last predicted value, append it to my $result$ sequence, and stop after $T$ times.

Both my $source$ signals and $target$ signals are normalized to $[-1,1]$.

Am I doing the inference wrong?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.