Hot answers tagged

2

The choice is mostly about your specific task: what do you need/want to do? Many-to-one (single values) models have lower error, on average, since the quality of outputs decreases the more further in time you're trying to predict. Many-to-one (multiple values) sometimes is required by the task though. An alternative could be to employ a Many-to-one (single ...


1

You can still use 0 as Start Of the Sequence token. Shift the input data by adding a constant to all values, for example adding 10. Then prepend a 0 to the input. A linear transformation of input will not affect the ability of machine learning models to learn. Make sure to apply the same transformation to both training and prediction stages.


1

As you have seen, normally you need a "special token" to be given to the decoder as the first element in its input to start the autoregressive generation. However, given that your output are real (floating point) numbers, it is a bit trickier, as you are not dealing with a discrete token vocabulary where you could simply reserve a token for that. I would ...


1

Simple RNN Cells follow this pattern: Given the following data: input data: X weights: wx recursive weights: wRec Initialize initial hidden state to 0 For each state, one by one: Update new hidden state as: (Input data * weights) + (Hidden state + recursive weights) In Python code: def compute_states(X, wx, wRec): ...


1

The general formulation of attention with queries, keys and values corresponds to a re-retrieval view on attention: you have some queries that you use to retrieve some values based on keys that correspond to them. With RNNs, attention is used for sequence-to-sequence models like machine translation. (Time series forecasting is usually formulated as sequence ...


1

The logits have the distribution according to the len of vocabulary and the model training. So, you can use np.argmax(logits) to get the prediction, but normally to the application of generating script is more interesting to take into account a factor of aleatory which in this case is the function "random_categorical" which is used to get the value according ...


Only top voted, non community-wiki answers of a minimum length are eligible