Skip to main content
A.B's user avatar
A.B's user avatar
A.B's user avatar
A.B
  • Member for 9 years
  • Last seen more than a month ago
  • London, Uk
comment
What to do with Transformer Encoder output?
@noe in the picture you shared, shouldnt it be the last token instead? how could we predict based on first token, while we havent seen all tokens (not even the first one)
awarded
comment
Predicting parallel time series with multiple features
Hello Adam, Thankyou for the answer (I didn't got notification, just visited by chance. Agreed that LSTM will have more data. But will model be trained sequentially? I mean obtaining embeddings for store A, and then using same LSTM learning / updating (fine-tuning would be more appropriate) them for store B to obtain store B's embedding.? (for each batch ofcourse)
awarded
awarded
comment
Why we shift target(output) by one offset in language modelling
Oh okay, thanks. Lets see if we can get a theoretically reason for this as ana snwer from someone :)
comment
Why we shift target(output) by one offset in language modelling
Nikos, Okay, I meant i didnt get "next series" part of thing. Do you mean feeding it back for the next prediction in an autoregressive manner?
comment
Why we shift target(output) by one offset in language modelling
Also, for taking one offset shifted, wouldnt network give good accuracy as it will simple learn to predict a shifted value? (but that would not be the case actually)
comment
Why we shift target(output) by one offset in language modelling
Nikos, Thankyou for the comment. Your second comment (taking last index from predcition) makes sense. Can you please elaborate more on first comment ( "given a series of predictions may better predict a single value or even the next series ")
comment
Why we shift target(output) by one offset in language modelling
Kindly let me know if this question is more suited for other statckexhange platform
Loading…
awarded
Loading…
awarded
comment
Predicting parallel time series with multiple features
@AdamOudad the second option suppose that the features arent dependent on each other (atleast in timesteps features) as they have separate LSTMs, as LSTM for feature one wont take into account anything from other time series(2nd feature). Which might not be good if these 2 different time series(feature's timesteps) are dependent or follow dependent pattern, right?
awarded
awarded
accepted
Loading…