# Tag Info

### BPE vs WordPiece Tokenization - when to use / which?

In contrast to BPE, WordPiece does not choose the most frequent symbol pair, but the one that maximizes the likelihood of the training data once added to the vocabulary. So what does this mean exactly?...
• 1

### Creating class labels for custom DataSets efficiently (HuggingFace)

you can combine data of test and train in a single data frame. Then you should split data frame using scalar test train split
• 141
Accepted

### Creating class labels for custom DataSets efficiently (HuggingFace)

This is a coding style issue, so people may well have different opinions! But I don't see any problem with the way you've coded it. If you really want to reduce the number of lines of code you could ...
• 439
1 vote

### What Preprocessing is Needed for Semantic Search Using Pre-trained Hugging Face Transformers?

Resumes are quite different from classic text because there are many proper nouns (names, companies, places, etc.) and other data difficult to classify (phone numbers, marks, age, etc.). That's why ...
• 2,793
1 vote

### Do I need training data in multiple languages for a multilingual transformer?

As far as I know, few multilingual models study their representation space to see if the representations of different languages occupy overlapping regions. The ones that do, usually find that the ...
• 16k
1 vote
Accepted

### Transformers vs RNN basic doubt

There are multiple concepts mixed in your question. Contextual vs. non-contextual word embeddings: word2vec is a non-contextual approach to obtaining token embeddings. This means that a specific word ...
• 16k
Accepted

### Smaller embedding size causes lower loss

New Answer The loss of a text generation task like question generation is normally the average categorical cross-entropy of the output at every time step. Drastically reducing the number of tokens ...
• 16k
1 vote

### Transformers - Why Self Attention calculate dot product of q and k from of same word?

It is not necessarily the case. The matrics $K$ and $Q$ can be very different. The intuition is that these two projections allow the model to search for a particular piece of information in the hidden ...
• 1,406