Skip to main content

Questions tagged [huggingface]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
2 answers
557 views

How can you get a Huggingface fine-tuning model with the Trainer class from your own text where you can set the arguments for truncation and padding?

I want to find out the role of truncation and padding in Huggingface Transformers pretrained models and any fine-tuning model on top of that. Therefore I played around with these parameters, but I ...
questionto42's user avatar
7 votes
1 answer
10k views

Minimal working example or tutorial showing how to use Pytorch's nn.TransformerDecoder for batch text generation in training and inference modes?

I want to solve a sequence-to-sequence text generation task (e.g. question answering, language translation, etc.). For the purposes of this question, you may assume that I already have the input part ...
Pablo Messina's user avatar
3 votes
1 answer
1k views

How to i get word embeddings for out of vocabulary words using a transformer model?

When i tried to get word embeddings of a sentence using bio_clinical bert, for a sentence of 8 words i am getting 11 token ids(+start and end) because "embeddings" is an out of vocabulary ...
cerofrais's user avatar
  • 131
2 votes
1 answer
1k views

Fine-tuning a pre-trained LLM for question-answering

Objective My goal is to fine-tune a pre-trained LLM on a dataset about Manchester United's (MU's) 2021/22 season (they had a poor season). I want to be able to prompt the fine-tuned model with ...
Tom Bomer's user avatar
1 vote
1 answer
1k views

How to prepare texts to BERT/RoBERTa models?

I have an artificial corpus I've built (not a real language) where each document is composed of multiple sentences which again aren't really natural language sentences. I want to train a language ...
IsaacLevon's user avatar
0 votes
1 answer
166 views

Outdated Transformers TextDataset class drops last block when text overlaps. Replace by datasets Dataset class as input of Trainer train_dataset?

Why I try to replace the transformers TextDataset class with datasets Dataset class I stumbled upon this when I tried to make ...
questionto42's user avatar