5 votes
Accepted

Should you care about truncation and padding in an LLM even if it has a very large tokenizer.max_length so that truncation will never happen?

The large number you are seeing is not the maximum length, but the maximum representable integer at that precision. It's there because no maximum length has been set. The original GPT-2 has a maximum ...
noe's user avatar
  • 26.6k
2 votes
Accepted

Understanding processors in huggingface tokenizer library

The answer is in your screenshot: For each of these (special tokens and sentences), we also specify the corresponding token type ID after a colon Therefore, the :0...
noe's user avatar
  • 26.6k
2 votes

Open-Source Large Language Models (LLM): Your experience and recommendation

You should check the Open LLM Leaderboard from Huggingface. They maintain a ranking of open LLMs in different tasks. The ones finetuned to follow instructions are marked. When clicking on each model, ...
noe's user avatar
  • 26.6k
2 votes
Accepted

Can I run falcon-7b on a free google colab?

Yes there are a couple things you can do to fit the model into google colab's disk. Reduce the batch size for train and test. This will reduce the GPU memory used for each epoch. Default size set in ...
spectre's user avatar
  • 2,065
2 votes

How can you get a Huggingface fine-tuning model with the Trainer class from your own text where you can set the arguments for truncation and padding?

If you use transformers example scripts, for example, summarization, you can control padding and truncation with command line arguments --pad_to_max_length, ...
Valentas's user avatar
  • 1,209
2 votes
Accepted

How to use is_split_into_words with Huggingface NER pipeline

evalute.evaluator.token_classification.py has this line: data = data.map(lambda x: {input_column: join_by.join(x[input_column])}) So even though the ...
Trail Map's user avatar
2 votes

Public Email Classification Dataset but not Spam vs Ham

If I am understanding correctly, you want to create a model that takes an email body and assigns some probability to a pre-specified set of classes (feedback, complaint, lost and found, etc.) ...
AndrewJaeyoung's user avatar
2 votes
Accepted

Generate VTT file from speech to text

Have you looked at using the return_timestamps keyword? It seems you can also get the timestamps on either the word on text segment level (for the Whisper model at ...
Oxbowerce's user avatar
  • 7,507
2 votes
Accepted

Getting rid of the warning "The following columns ... have been ignored" and "ValueError: batch_size should be a positive integer value ..."

I had an outdated version of transformers. I upgraded thetransformers package from: ...
questionto42's user avatar
2 votes
Accepted

Falcon-7B llm giving random output

You are using the GPT-2 tokenizer instead of the model's own tokenizer. Try getting the tokenizer from the model instead: ...
noe's user avatar
  • 26.6k
1 vote
Accepted

How can you get a Huggingface fine-tuning model with the Trainer class from your own text where you can set the arguments for truncation and padding?

Bad workaround: TextDataset classes No control of truncation/padding in Transformer TextDataset classes At least for TextDataset and LineByLineTextDataset, there just might be no such attribute: ...
questionto42's user avatar
1 vote
Accepted

Outdated Transformers TextDataset class drops last block when text overlaps. Replace by datasets Dataset class as input of Trainer train_dataset?

You need to from datasets import load_dataset, and even though the warning tells you so, you do not seem to need to import the Dataset class if you just want to run ...
questionto42's user avatar
1 vote
Accepted

Multilingual sentence generation with Hugging Face

There are a few options there: Different models on HuggingFace GPT2 should fit your use case. If performance is really critical, I'd give distilGPT2 a try. They only generate English sentences but ...
Valentin Calomme's user avatar
1 vote

RuntimeError: mat1 and mat2 shapes cannot be multiplied (25x7 and 1x512)

The error occurs at decoder_output = self.decoder_input_layer(tgt) self.decoder_input_layer is defined as: ...
Karl's user avatar
  • 636
1 vote

How can someone evaluate llama index model?

Is there any other way to evaluate model ? Assuming you have a test set with gold answers, you can use a text generation metric. See Evaluation of Text Generation: A Survey. Typical metrics: TF-IDF ...
Franck Dernoncourt's user avatar
1 vote
Accepted

LMM Fine Tuning - Supervised Fine Tuning Trainer (SFTTrainer) vs transformers Trainer

The short answer is that a Supervised Fine Tuning Trainer (SFTTrainer) is used for Instruct Fine Tuning. The HuggingFace library SFTTrainer has also support for training with QLoRA (4-bit Quantised ...
Alex Punnen's user avatar
1 vote

Fine-tuning a pre-trained LLM for question-answering

Check the steps at the Huggingface beginner's guide Question Answering with SQuAD 2.0 to begin with a normal question answering model. Have some look at the Fine-tuning guide at OpenAI as well (which ...
questionto42's user avatar
1 vote

Dynamic batching and padding batches for NLP in deep learning libraries

Great questions! When training a model with batches of data with padded inputs, the padded tokens will take up GPU memory. This is because the padding zeros are still being stored in memory, even if ...
Pluviophile's user avatar
  • 3,848

Only top scored, non community-wiki answers of a minimum length are eligible