Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [huggingface]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
7 votes
1 answer
10k views

Minimal working example or tutorial showing how to use Pytorch's nn.TransformerDecoder for batch text generation in training and inference modes?

I want to solve a sequence-to-sequence text generation task (e.g. question answering, language translation, etc.). For the purposes of this question, you may assume that I already have the input part ...
Pablo Messina's user avatar
4 votes
1 answer
1k views

Bert for QuestionAnswering input exceeds 512

I'm training Bert on question answering (in Spanish) and i have a large context, only the context exceeds 512, the total question + context is 10k, i found that longformer is bert like for long ...
Simone's user avatar
  • 242
4 votes
2 answers
2k views

How to use is_split_into_words with Huggingface NER pipeline

I am using Huggingface transformers for NER, following this excellent guide: https://huggingface.co/blog/how-to-train. My incoming text has already been split into words. When tokenizing during ...
Alan Buxton's user avatar
4 votes
1 answer
404 views

How to measure the accuracy of an NLP paraphrasing model?

I using the HuggingFace library to do sentence paraphrasing (given an input sentence, the model outputs a paraphrase). How am I supposed to compare the results of two separate models (one trained with ...
carrot_142's user avatar
3 votes
1 answer
2k views

LMM Fine Tuning - Supervised Fine Tuning Trainer (SFTTrainer) vs transformers Trainer

When should one opt for the Supervised Fine Tuning Trainer (SFTTrainer) instead of the regular Transformers Trainer when it comes to instruction fine-tuning for Language Models (LLMs)? From what I ...
Marvin Martin's user avatar
3 votes
1 answer
1k views

How to i get word embeddings for out of vocabulary words using a transformer model?

When i tried to get word embeddings of a sentence using bio_clinical bert, for a sentence of 8 words i am getting 11 token ids(+start and end) because "embeddings" is an out of vocabulary ...
cerofrais's user avatar
  • 131
2 votes
2 answers
245 views

How to improve language model ex: BERT on unseen text in training?

I am using pre-trained language model for binary classification. I fine-tune the model by training on data my downstream task. The results are good almost 98% F-measure. However, when I remove a ...
IS92's user avatar
  • 123
2 votes
2 answers
919 views

Should you care about truncation and padding in an LLM even if it has a very large tokenizer.max_length so that truncation will never happen?

I want to find out the role of truncation and padding in Huggingface Transformers pretrained models and/or any fine-tuning models on top. Taking a large language model like the German GPT2 shows that ...
questionto42's user avatar
2 votes
1 answer
33 views

Since LoRA parameters are randomly initialized, shouldn't that mean that initially breaks a models output?

I have just tried using LoRA on Llama 3 8B and I found without doing any fine tuning it performed pretty well on my dataset. But then I realized that surely the LoRA parameters are randomly ...
Ameen Izhac's user avatar
2 votes
2 answers
2k views

Dynamic batching and padding batches for NLP in deep learning libraries

This is the usual way we train modern deep learning models for NLP, e.g. with Huggingface libraries where we have a fix length for the input no. of tokens/subwoords unit. https://huggingface.co/docs/...
alvas's user avatar
  • 2,410
2 votes
1 answer
377 views

What Preprocessing is Needed for Semantic Search Using Pre-trained Hugging Face Transformers?

I am building a project for my bachelor thesis and am wondering how to prepare my raw data. The goal is to program some kind of semantic search for job postings. My data set consists of stored web ...
node_env's user avatar
2 votes
1 answer
161 views

How can someone evaluate llama index model?

I have built a openAI llama index based model which takes multiple pdf and able to give chatbot based response. I want to evaluate the llm for accuracy. I already know method such as Rouge and Bleu. ...
Prashant Pandya's user avatar
2 votes
1 answer
1k views

Fine-tuning a pre-trained LLM for question-answering

Objective My goal is to fine-tune a pre-trained LLM on a dataset about Manchester United's (MU's) 2021/22 season (they had a poor season). I want to be able to prompt the fine-tuned model with ...
Tom Bomer's user avatar
2 votes
0 answers
284 views

How to train a Task Specific Knowledge Distillation model using Hugging face model

I was referring to this code: https://github.com/philschmid/knowledge-distillation-transformers-pytorch-sagemaker/blob/master/knowledge-distillation.ipynb From @philschmid I could follow most of the ...
MAC's user avatar
  • 277
2 votes
1 answer
3k views

HuggingFace Transformers is giving loss: nan - accuracy: 0.0000e+00

I am a HuggingFace Newbie and I am fine-tuning a BERT model (distilbert-base-cased) using the Transformers library but the training loss is not going down, instead ...
JasonExcel's user avatar
1 vote
1 answer
32 views

How do I get model.generate() to omit the input sequence from the generation?

I'm using Huggingface to do inference on llama-3-B. Here is my model: ...
Ameen Izhac's user avatar
1 vote
1 answer
20 views

Public Email Classification Dataset but not Spam vs Ham

Context Working to deliver a POC on automated email classification (in customer service context) to tag emails as related to feedback, complain, lost and found etc. The tags are not entirely exclusive,...
Della's user avatar
  • 335
1 vote
1 answer
42 views

Falcon-7B llm giving random output

I am using a falcon 7B model for a chatbot without any finetuning with the following code ...
Saket Vempaty's user avatar
1 vote
1 answer
40 views

Understanding processors in huggingface tokenizer library

tl;dr What are the :0 and :1 in the following huggingface processors reference usage given on their page: ...
figs_and_nuts's user avatar
1 vote
1 answer
1k views

How to prepare texts to BERT/RoBERTa models?

I have an artificial corpus I've built (not a real language) where each document is composed of multiple sentences which again aren't really natural language sentences. I want to train a language ...
IsaacLevon's user avatar
1 vote
2 answers
4k views

Adding a new token to a transformer model without breaking tokenization of subwords

I'm running an experiment investigating the internal structure of large pre-trained models (BERT and RoBERTa, to be specific). Part of this experiment involves fine-tuning the models on a made-up new ...
Jigsaw's user avatar
  • 121
1 vote
1 answer
427 views

Masked Language Modeling on Domain-specific Data

My goal is to have a language model that understands the relationships between words and can fill the masks in a sentence related to a specific domain. At first, I thought about pretraining or even ...
mitra mirshafiee's user avatar
1 vote
1 answer
280 views

dealing with HuggingFace's model's tokens

I have a few questions regarding tokenizing word/characters/emojis for different huggingface models. From my understanding, a model would only perform best during inference if the token of the input ...
user113789's user avatar
1 vote
1 answer
2k views

How to do NER predictions with Huggingface BERT transformer

I am trying to do a prediction on a test data set without any labels for an NER problem. Here is some background. I am doing named entity recognition using tensorflow and Keras. I am using huggingface ...
Khachatur Mirijanyan's user avatar
1 vote
0 answers
60 views

Could someone help with fine-tuning dolphin-2.2.1?

Could someone help with fine-tuning dolphin-2.2.1? I have a problem with training: my train\loss - 0 and validation\loss - 0.000... after 800-1000 steps and this is overfitting ...
kabba62's user avatar
  • 11
1 vote
0 answers
111 views

Fine-tuning Hugging Face’s Llama Model with Unlabelled Data from PDFs from niche domain

I’m unsure about the next steps. Specifically, I have the following questions: How can I prepare my unlabelled data for the fine-tuning process? What’s the best way to fine-tune the Llama model with ...
Divya Patel's user avatar
1 vote
1 answer
441 views

How to Use Multiple Adapters with a Pretrained Model in Hugging Face Transformers for Inference?

I have a pretrained Llama-2 model in the models_hf directory and two fine-tuned adapters: a summarization adapter in ...
Aun Zaidi's user avatar
1 vote
0 answers
68 views

Deploying a model with GPU and pay-per-inference

I may have the wrong stack exchange. If that's the case, could someone point me to a stack that could help with this. Anyways... My backend employs a sentence transformer model from HuggingFace. Since ...
AlwaysLearning's user avatar
1 vote
0 answers
210 views

How to load Hugginface model on CPU?

My system runs out of memory on GPU. But I want just to test if it works and want to load it on CPU. How do I do that? Adding device = torch.device(‘cpu’) before ...
Myron's user avatar
  • 123
1 vote
0 answers
231 views

How to monitor training of text generation models?

I'm finetuning a pretrained Huggingface model based on Transformers for a downstream Text Generation task, but I have doubts on how the fine-tuning process should be monitored: In classification, I ...
Ciodar's user avatar
  • 161
1 vote
0 answers
733 views

Creating LLM chatbot using llama-index + langchain

As the title suggests: I'm trying to build a chatbot which his goal should be sort of like "chatgpt". The chatbot will be installed on Slack workspace and I'm struggling with which scope I ...
Ben's user avatar
  • 209
1 vote
0 answers
84 views

trainable weights in automodel and comparison with lora

if i use RobertaForSequenceClassification or AutoModelForSequenceClassification, are all the weights trained or only the new classification head trained also , i am also noticing for "roberta-...
prajwal rao's user avatar
1 vote
2 answers
1k views

Fine-tuned MLM based RoBERTa not improving performance

We have lots of domain-specific data (200M+ data points, each document having ~100 to ~500 words) and we wanted to have a domain-specific LM. We took some sample data points (2M+) & fine-tuned ...
Kalsi's user avatar
  • 11
1 vote
2 answers
84 views

Fine-tune GPT on sketch data (stroke-3)

These past days I have started a personal project where I would like to build a model that, given an uncompleted sketch, it can finish it. I was planning on using some pretrained models that are ...
ilved17's user avatar
  • 41
1 vote
1 answer
77 views

What could cause pre-trained Opus-MT models have wildly varying inference time when being used with transformers library?

I have been testing pre-trained Opus-MT models ported to transformers library for python implementation. Specifically, I am using opus-mt-en-fr for English to French translation. And the tokenizer and ...
Shan Dou's user avatar
  • 131
1 vote
0 answers
370 views

Dataset Format for fine tuning deepset/roberta-base-squad2 hugging face transformer model

I have been trying to fine tune the roberta model for QnA to my specific domain (healthcare). I am unable to find the correct way to provide the dataset format to the tokenizer in order to fine tune ...
Tushar Sethi's user avatar
1 vote
0 answers
188 views

Could Attention_mask in T5 be a float in [0,1]?

I was inspecting T5 model from hf https://huggingface.co/docs/transformers/model_doc/t5 . attention_mask is presented as ...
Dave's user avatar
  • 13
1 vote
0 answers
1k views

How to save hugging face fine tuned model using pytorch and distributed training

I am fine tuning masked language model from XLM Roberta large on google machine specs. When I copy the model using gsutil and subprocess from container to GCP ...
MAC's user avatar
  • 277
1 vote
0 answers
371 views

How to Fine Tune a BERT model for sentiment analysis to get the best f1 score

I am building a multi-class sentiment analysis BERT model that's optimized to give the best f1 score. More specifically, I train each epoch by optimizing binary cross entropy per class, taking the ...
vgoklani's user avatar
  • 238
1 vote
1 answer
52 views

HuggingFace hate detection model

I am trying to train and evaluate a hate detection model using the HuggingFace Transformers library and this dataset. Model performance is secondary, just trying to get it going. I have preprocessed ...
Mughees Asif's user avatar
1 vote
0 answers
2k views

Huggingface - TypeError: 'TensorSliceDataset' object is not subscriptable

I'm trying to make my own model for translate a language to another with T5ForConditionalGeneration and Huggingface using no pretrained model (I need to use my own dataset and tokenizer because no ...
Tristan Bilot's user avatar
1 vote
0 answers
34 views

Can a reformer model really handle long-range dependency?

I read this article about new attention model called Reformer. Here is the main strength of this model: The Reformer pushes the limit of longe sequence modeling by its ability to process up to half a ...
Kenenbek Arzymatov's user avatar
1 vote
0 answers
366 views

Is it possible to fine-tune a (Spanish RoBERTa) model for a different task?

I'm doing sentiment analysis of Spanish tweets. After reviewing some of the recent literature, I've seen that there's been a most recent effort to train a RoBERTa model exclusively on Spanish text. It ...
LeLuc's user avatar
  • 131
1 vote
0 answers
28 views

Personal Project Classifying Bank Account Data - NLP Noob [closed]

Background: I'm looking to get up to speed with some of the newer ML classification techniques and keep my ML python fresh in my spare time/learn some new skills, so as a challenge I'm trying to see ...
user2065472's user avatar
1 vote
0 answers
267 views

How to do batch inference on Hugging face pretrained models?

I want to do batch inference on MarianMT model. Here's the code: ...
sai_varshittha's user avatar
1 vote
0 answers
28 views

Train and validation sets splits using load_data

I'm using the package "datasets". The code I have: ...
Adolf Miszka's user avatar
1 vote
0 answers
16 views

Is it correct to load weights from task Masked Language Modeling to train Causal Language Modeling

I intend to use 2 tasks of modelling including (a) Causal language modelling & (b) Mask language modelling for training my new added tokens My pseudo-code is below ...
Lắc Lê's user avatar
1 vote
0 answers
114 views

Top-K vs AUC - communicating results and next steps [closed]

I have a bi-LSTM multi-label text classification model which when training on a highly imbalanced dataset with 1000 possible labels gives a top-k (k=5) categorical accuracy of 86% and a focal loss of ...
ML_Engine's user avatar
  • 111
1 vote
0 answers
269 views

Huggingface Library - Multi-document summarization

Can BART, PEGASUS ... etc. API in huggingface library be used to directly perform multi document summarization? (e.g. here: https://huggingface.co/transformers/model_doc/bart.html)
asahi kibou's user avatar
1 vote
0 answers
159 views

How to apply pruning on a BERT model?

I have trained a BERT model using ktrain (tensorflow wrapper) to recognize emotion on text, it works but it suffers from really slow inference. That makes my model not suitable for a production ...
Stamatis Tiniakos's user avatar