0
$\begingroup$

I want to ask for time series anomaly detection we can apply tnn on multiple features or not?

I used transformer for sentiment analysis where I have to provide a sentence and it predicts its output as positive or negative. In another case I provided a single word and model predicts its language. This is how it works that it takes single column input where as in time series dataset there are more than one columns as input.

I implemented transformer neural network i am confused how can i add more layers in transformer like other neural network architecture ?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

You want to know if you can use anomaly detection with transformer in multiple features, right? The answer is yes and the most straighforward solution is to keep one detection system per feature if there is no dependencies between them. But if there are anomalies depending on a group of features, it would require a more complex transformer architecture, including a multi-head attention function or something similar.

Improve this answer
$\endgroup$
9
  • $\begingroup$ I just updated the code in the question . could you verify its correct way or not .. besides i am confused how can i add more layers in transformer like other neural network architecture ? I mean forward propagation then backward propagation and all layered architecture $\endgroup$
    – user12
    Aug 26, 2021 at 17:53
  • $\begingroup$ I don't know torch very well but it seems correct. What result do you have? $\endgroup$
    – Nicolas Martin
    Aug 27, 2021 at 13:35
  • $\begingroup$ You can use the transformer's parameters: num_encoder_layers=?, num_decoder_layers=? You don't have to set the same quantity of layers. Results might be better with more layers in the encoder (ex:5) than the decoder (ex:3). $\endgroup$
    – Nicolas Martin
    Aug 30, 2021 at 9:17
  • 1
    $\begingroup$ Quote from your website: "What makes the transformer architecture different from other encoder-decoder architectures is the fact that it uses no variant of recurrent network (such as long-short-term-memory) and instead captures dependencies between different time instants by a preprocessing technique called positional encoding together with an architecture pattern called attention. " $\endgroup$
    – Nicolas Martin
    Aug 30, 2021 at 14:55
  • 1
    $\begingroup$ There is no neural network indeed, but there are layers of attention mechanisms (see the last link I've sent), which are a set of functions (multihead attention, add&norm, feed forward,...) depending if they are at the encoder or at the decoder. Notice that decoding requires more calculation. $\endgroup$
    – Nicolas Martin
    Aug 30, 2021 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.