New answers tagged

1

The issue was resolved once I used Tensor View to reshape the mini-batches for the features in the training and in the validation set. As a side note, view() enable fast and memory-efficient reshaping, slicing, and element-wise operations, by avoiding an explicit data copy. It turned out that in the earlier implementation torch.unsqueeze() did not reshape ...


4

That layer isn't required indeed as it also encodes the sequence, albeit in a different way than BERT. What I assume is that in a BERT-BiLSTM-CRF, setup, the BERT layer is either frozen or difficult to fine-tune due to its sheer size. Which is likely why the BiLSTM layer has been added there.


0

It could be a number of different reasons but when I had that problem in the past it was usually due to too high of a learning rate or the optimizer. I recommended either dropping the initial learning rate or going with vanilla SGD. Occasionally I saw problems with Adam particularly if you have no warmup. You might want to try more general hyperparameter ...


0

I am also fairly new to this, but am working on the same type of problem where I have to predict the sentence based on previous data. Like you want giving X value predict y value. So, basically you need to create a dataset as per your requirement and once your dataset is ready you can train your model with encoder-decoder or LSTM layers. Please find below ...


0

A dense layer will output a fixed-sized vector. This will be what you want for a classification task for example, say sentiment classification. A TimeDistributedDense will apply a dense layer to each output of the sequence. So let's say you have a text input, represented as a sequence of word embeddings, you would apply an LSTM cell and then the same dense ...


1

What you would typically do in your case is to apply a sentence alignment tool. Some popular options for that are: hunalign: a classical tool that relies on a bilingual dictionary. bleualign: it aligns based on the BLEU score similarity vecalign: it is based on sentence embeddings, like LASER's. I suggest you take a look at the preprocessing applied for ...


1

The tough part about this problem is evaluating what moves are "correct" per se. In a fighting game sequence, there may be 2 or more moves which both work in theory in the current frame. If you are optimizing for knocking the other character out of frame, it would be useful to build a reward set which optimizes for this. It may be worthwhile to ...


0

As I tried to predict the position without normalizing... the error was in the data. After normalizing the positions everything worked as expected.


1

You answered yourself [sequence length, 1] is correct assuming you work with a single sentence. (Or actually, the 1 dimension depends on implementation.) In practice, the data is typically batched, so it will be [batch, sequence length 1]. This can be element-wise multiplied with the encoder states of dimension [batch, sequence length, hidden size] and ...


1

I'm not particularly expert in this but I'm quite sure that the variations in the price of anything depend mostly on external factors: news of the day, economic indicators, stockmarket movements, etc. As a rule of thumb, if a human with a lot of time can't do it, usually a ML model can't do it either. In this case if an expert in finance is given the history ...


0

A "projection" is a simple linear/dense layer, that is, a matrix multiplication and a bias vector addition. It is called projection because you "project" a representation of dimensionality $M$ into a representation space of dimensionality $N$. Sometimes, especially for sequences or 2D data, these projections are implemented as a ...


Top 50 recent answers are included