User Darren Cook

8 votes

Transformer model: Why are word embeddings scaled before adding positional encodings?

answered Jan 18, 2021 at 20:14

8 votes

Accepted

Why deep learning models still use RELU instead of SELU, as their activation function?

answered Oct 5, 2021 at 9:52

4 votes

Accepted

Does multi-head attention remove the need for self-attention?

answered Apr 12, 2021 at 18:40

3 votes

Can we fine-tune a model on the same dataset which it is pretrained on?

answered Apr 19, 2021 at 19:42

3 votes

Choosing best model produced from different algorithms. Metric produced by cross-validation on the train set or metric produced on the test set?

answered Feb 15, 2021 at 21:05

3 votes

Accepted

What reccent alternatives to LSTM are there for regression problems?

answered Aug 17, 2021 at 9:38

2 votes

Accepted

Approximation of long sequence of layers by one layer

answered Feb 8, 2021 at 19:02

2 votes

How to make sure that the features learned by a neural network are not correlated?

answered Feb 15, 2021 at 21:17

2 votes

Accepted

Question on embedding similarity / nearest neighbor methods [SCANN Paper]

answered Jan 13, 2022 at 10:36

2 votes

Accepted

Is roc auc graph better than roc auc score? If yes why?

answered Dec 13, 2021 at 19:27

2 votes

Identifying competition climbing styles from World Cup data

answered Jan 4 at 9:21

2 votes

Why cant we use normalise position encodings instead of the cos and sine encodings used in the Transformer paper?

answered Aug 7 at 18:21

1 vote

Recommended way to embed a text thousands of tokens long?

answered Aug 7 at 18:38

1 vote

Transformer model is very slow and doesn't predict well

answered Sep 27, 2021 at 8:22

1 vote

What does it mean order of input sequence does not matter for transformer self-attention head?

answered Jul 25 at 10:00

1 vote

Accepted

Why is T5 often used in text-to-data for text prompt encoders?

answered Oct 2 at 19:52

1 vote

torch cuda not able to identify gpu on aws g4dn.xlarge

answered Oct 9 at 20:07

1 vote

Accepted

How to control a decision tree?

answered Dec 20, 2021 at 18:35

1 vote

How to classify text and predict if it belongs to the group or not?

answered Jan 23 at 18:57

1 vote

Accepted

What ML techniques could be used for biometric feature extraction and ID generation?

answered Feb 21 at 20:27

1 vote

Accepted

Im looking for good neurons silmilarity metric

answered Mar 7 at 17:47

1 vote

Accepted

Transformers for time series - what is the role of the encoder?

answered Jul 18 at 7:17

1 vote

Accepted

Feature engineering of timestamp for time series analysis

answered Jan 24, 2022 at 17:31

1 vote

Which ML approach is the best for huge state spaces?

answered Jan 24, 2022 at 17:51

1 vote

Find smooth global maximum from noisy points

answered Feb 2, 2022 at 16:23

1 vote

Mixed language OCR

answered Jul 26, 2022 at 15:18

1 vote

Is big data a fallacy if most phenomena can be mostly described by few variables?

answered Oct 31, 2022 at 19:34

1 vote

Why do the performance of DL models increase with the volume of data while that of ML models will flat out or even decrease?

answered Jul 20, 2021 at 11:06

1 vote

Accepted

Organising the preprocessing of a dataset that will be consumed by multiple models

answered Oct 11, 2021 at 20:16

1 vote

How BERT model differentiate words with different meanings?

answered Sep 13, 2021 at 18:18

Stack Exchange Network

42 Answers

Transformer model: Why are word embeddings scaled before adding positional encodings?

Why deep learning models still use RELU instead of SELU, as their activation function?

Does multi-head attention remove the need for self-attention?

Can we fine-tune a model on the same dataset which it is pretrained on?

Choosing best model produced from different algorithms. Metric produced by cross-validation on the train set or metric produced on the test set?

What reccent alternatives to LSTM are there for regression problems?

Approximation of long sequence of layers by one layer

How to make sure that the features learned by a neural network are not correlated?

Question on embedding similarity / nearest neighbor methods [SCANN Paper]

Is roc auc graph better than roc auc score? If yes why?

Identifying competition climbing styles from World Cup data

Why cant we use normalise position encodings instead of the cos and sine encodings used in the Transformer paper?

Recommended way to embed a text thousands of tokens long?

Transformer model is very slow and doesn't predict well

What does it mean order of input sequence does not matter for transformer self-attention head?

Why is T5 often used in text-to-data for text prompt encoders?

torch cuda not able to identify gpu on aws g4dn.xlarge

How to control a decision tree?

How to classify text and predict if it belongs to the group or not?

What ML techniques could be used for biometric feature extraction and ID generation?

Im looking for good neurons silmilarity metric

Transformers for time series - what is the role of the encoder?

Feature engineering of timestamp for time series analysis

Which ML approach is the best for huge state spaces?

Find smooth global maximum from noisy points

Mixed language OCR

Is big data a fallacy if most phenomena can be mostly described by few variables?

Why do the performance of DL models increase with the volume of data while that of ML models will flat out or even decrease?

Organising the preprocessing of a dataset that will be consumed by multiple models

How BERT model differentiate words with different meanings?