New answers tagged lstm
1
The issue was resolved once I used Tensor View to reshape the mini-batches for the features in the training and in the validation set. As a side note, view() enable fast and memory-efficient reshaping, slicing, and element-wise operations, by avoiding an explicit data copy.
It turned out that in the earlier implementation torch.unsqueeze() did not reshape ...
4
That layer isn't required indeed as it also encodes the sequence, albeit in a different way than BERT.
What I assume is that in a BERT-BiLSTM-CRF, setup, the BERT layer is either frozen or difficult to fine-tune due to its sheer size. Which is likely why the BiLSTM layer has been added there.
0
It could be a number of different reasons but when I had that problem in the past it was usually due to too high of a learning rate or the optimizer. I recommended either dropping the initial learning rate or going with vanilla SGD. Occasionally I saw problems with Adam particularly if you have no warmup. You might want to try more general hyperparameter ...
0
I am also fairly new to this, but am working on the same type of problem where I have to predict the sentence based on previous data. Like you want giving X value predict y value.
So, basically you need to create a dataset as per your requirement and once your dataset is ready you can train your model with encoder-decoder or LSTM layers.
Please find below ...
0
A dense layer will output a fixed-sized vector. This will be what you want for a classification task for example, say sentiment classification.
A TimeDistributedDense will apply a dense layer to each output of the sequence. So let's say you have a text input, represented as a sequence of word embeddings, you would apply an LSTM cell and then the same dense ...
1
What you would typically do in your case is to apply a sentence alignment tool. Some popular options for that are:
hunalign: a classical tool that relies on a bilingual dictionary.
bleualign: it aligns based on the BLEU score similarity
vecalign: it is based on sentence embeddings, like LASER's.
I suggest you take a look at the preprocessing applied for ...
1
The tough part about this problem is evaluating what moves are "correct" per se. In a fighting game sequence, there may be 2 or more moves which both work in theory in the current frame. If you are optimizing for knocking the other character out of frame, it would be useful to build a reward set which optimizes for this.
It may be worthwhile to ...
0
As I tried to predict the position without normalizing... the error was in the data. After normalizing the positions everything worked as expected.
1
You answered yourself [sequence length, 1] is correct assuming you work with a single sentence. (Or actually, the 1 dimension depends on implementation.)
In practice, the data is typically batched, so it will be [batch, sequence length 1]. This can be element-wise multiplied with the encoder states of dimension [batch, sequence length, hidden size] and ...
1
I'm not particularly expert in this but I'm quite sure that the variations in the price of anything depend mostly on external factors: news of the day, economic indicators, stockmarket movements, etc.
As a rule of thumb, if a human with a lot of time can't do it, usually a ML model can't do it either. In this case if an expert in finance is given the history ...
0
A "projection" is a simple linear/dense layer, that is, a matrix multiplication and a bias vector addition.
It is called projection because you "project" a representation of dimensionality $M$ into a representation space of dimensionality $N$.
Sometimes, especially for sequences or 2D data, these projections are implemented as a ...
Top 50 recent answers are included
