Skip to main content

Questions tagged [huggingface]

The tag has no usage guidance.

62 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
1 answer
1k views

How to i get word embeddings for out of vocabulary words using a transformer model?

When i tried to get word embeddings of a sentence using bio_clinical bert, for a sentence of 8 words i am getting 11 token ids(+start and end) because "embeddings" is an out of vocabulary ...
cerofrais's user avatar
  • 131
2 votes
0 answers
288 views

How to train a Task Specific Knowledge Distillation model using Hugging face model

I was referring to this code: https://github.com/philschmid/knowledge-distillation-transformers-pytorch-sagemaker/blob/master/knowledge-distillation.ipynb From @philschmid I could follow most of the ...
MAC's user avatar
  • 277
2 votes
1 answer
3k views

HuggingFace Transformers is giving loss: nan - accuracy: 0.0000e+00

I am a HuggingFace Newbie and I am fine-tuning a BERT model (distilbert-base-cased) using the Transformers library but the training loss is not going down, instead ...
JasonExcel's user avatar
1 vote
0 answers
72 views

Could someone help with fine-tuning dolphin-2.2.1?

Could someone help with fine-tuning dolphin-2.2.1? I have a problem with training: my train\loss - 0 and validation\loss - 0.000... after 800-1000 steps and this is overfitting ...
kabba62's user avatar
  • 11
1 vote
0 answers
122 views

Fine-tuning Hugging Face’s Llama Model with Unlabelled Data from PDFs from niche domain

I’m unsure about the next steps. Specifically, I have the following questions: How can I prepare my unlabelled data for the fine-tuning process? What’s the best way to fine-tune the Llama model with ...
Divya Patel's user avatar
1 vote
1 answer
540 views

How to Use Multiple Adapters with a Pretrained Model in Hugging Face Transformers for Inference?

I have a pretrained Llama-2 model in the models_hf directory and two fine-tuned adapters: a summarization adapter in ...
Aun Zaidi's user avatar
1 vote
0 answers
71 views

Deploying a model with GPU and pay-per-inference

I may have the wrong stack exchange. If that's the case, could someone point me to a stack that could help with this. Anyways... My backend employs a sentence transformer model from HuggingFace. Since ...
AlwaysLearning's user avatar
1 vote
0 answers
237 views

How to load Hugginface model on CPU?

My system runs out of memory on GPU. But I want just to test if it works and want to load it on CPU. How do I do that? Adding device = torch.device(‘cpu’) before ...
Myron's user avatar
  • 123
1 vote
0 answers
235 views

How to monitor training of text generation models?

I'm finetuning a pretrained Huggingface model based on Transformers for a downstream Text Generation task, but I have doubts on how the fine-tuning process should be monitored: In classification, I ...
Ciodar's user avatar
  • 161
1 vote
0 answers
742 views

Creating LLM chatbot using llama-index + langchain

As the title suggests: I'm trying to build a chatbot which his goal should be sort of like "chatgpt". The chatbot will be installed on Slack workspace and I'm struggling with which scope I ...
Ben's user avatar
  • 209
1 vote
0 answers
85 views

trainable weights in automodel and comparison with lora

if i use RobertaForSequenceClassification or AutoModelForSequenceClassification, are all the weights trained or only the new classification head trained also , i am also noticing for "roberta-...
prajwal rao's user avatar
1 vote
2 answers
1k views

Fine-tuned MLM based RoBERTa not improving performance

We have lots of domain-specific data (200M+ data points, each document having ~100 to ~500 words) and we wanted to have a domain-specific LM. We took some sample data points (2M+) & fine-tuned ...
Kalsi's user avatar
  • 11
1 vote
2 answers
87 views

Fine-tune GPT on sketch data (stroke-3)

These past days I have started a personal project where I would like to build a model that, given an uncompleted sketch, it can finish it. I was planning on using some pretrained models that are ...
ilved17's user avatar
  • 41
1 vote
1 answer
80 views

What could cause pre-trained Opus-MT models have wildly varying inference time when being used with transformers library?

I have been testing pre-trained Opus-MT models ported to transformers library for python implementation. Specifically, I am using opus-mt-en-fr for English to French translation. And the tokenizer and ...
Shan Dou's user avatar
  • 131
1 vote
0 answers
376 views

Dataset Format for fine tuning deepset/roberta-base-squad2 hugging face transformer model

I have been trying to fine tune the roberta model for QnA to my specific domain (healthcare). I am unable to find the correct way to provide the dataset format to the tokenizer in order to fine tune ...
Tushar Sethi's user avatar
1 vote
0 answers
191 views

Could Attention_mask in T5 be a float in [0,1]?

I was inspecting T5 model from hf https://huggingface.co/docs/transformers/model_doc/t5 . attention_mask is presented as ...
Dave's user avatar
  • 13
1 vote
0 answers
1k views

How to save hugging face fine tuned model using pytorch and distributed training

I am fine tuning masked language model from XLM Roberta large on google machine specs. When I copy the model using gsutil and subprocess from container to GCP ...
MAC's user avatar
  • 277
1 vote
0 answers
373 views

How to Fine Tune a BERT model for sentiment analysis to get the best f1 score

I am building a multi-class sentiment analysis BERT model that's optimized to give the best f1 score. More specifically, I train each epoch by optimizing binary cross entropy per class, taking the ...
vgoklani's user avatar
  • 238
1 vote
0 answers
2k views

Huggingface - TypeError: 'TensorSliceDataset' object is not subscriptable

I'm trying to make my own model for translate a language to another with T5ForConditionalGeneration and Huggingface using no pretrained model (I need to use my own dataset and tokenizer because no ...
Tristan Bilot's user avatar
1 vote
0 answers
37 views

Can a reformer model really handle long-range dependency?

I read this article about new attention model called Reformer. Here is the main strength of this model: The Reformer pushes the limit of longe sequence modeling by its ability to process up to half a ...
Kenenbek Arzymatov's user avatar
1 vote
0 answers
369 views

Is it possible to fine-tune a (Spanish RoBERTa) model for a different task?

I'm doing sentiment analysis of Spanish tweets. After reviewing some of the recent literature, I've seen that there's been a most recent effort to train a RoBERTa model exclusively on Spanish text. It ...
LeLuc's user avatar
  • 131
1 vote
0 answers
278 views

How to do batch inference on Hugging face pretrained models?

I want to do batch inference on MarianMT model. Here's the code: ...
sai_varshittha's user avatar
1 vote
0 answers
28 views

Train and validation sets splits using load_data

I'm using the package "datasets". The code I have: ...
Adolf Miszka's user avatar
1 vote
0 answers
16 views

Is it correct to load weights from task Masked Language Modeling to train Causal Language Modeling

I intend to use 2 tasks of modelling including (a) Causal language modelling & (b) Mask language modelling for training my new added tokens My pseudo-code is below ...
Lắc Lê's user avatar
1 vote
0 answers
270 views

Huggingface Library - Multi-document summarization

Can BART, PEGASUS ... etc. API in huggingface library be used to directly perform multi document summarization? (e.g. here: https://huggingface.co/transformers/model_doc/bart.html)
asahi kibou's user avatar
1 vote
0 answers
161 views

How to apply pruning on a BERT model?

I have trained a BERT model using ktrain (tensorflow wrapper) to recognize emotion on text, it works but it suffers from really slow inference. That makes my model not suitable for a production ...
Stamatis Tiniakos's user avatar
1 vote
1 answer
3k views

Overfitting in Huggingface's TFBertForSequenceClassification

I'm using Huggingface's TFBertForSequenceClassification for multilabel tweets classification. During training the model archives good accuracy, but the validation accuracy is poor. I've tried to solve ...
Shahad Mahmud's user avatar
1 vote
1 answer
279 views

Should weight distribution change more when fine-tuning transformers-based classifier?

I'm using pre-trained DistilBERT model from Huggingface with custom classification head, which is almost the same as in the reference implementation: ...
Marcin Zablocki's user avatar
0 votes
0 answers
14 views

Hugging Face Real Time Object Detection Deployment

I'm developing a live object detection app using Streamlit and the YOLOv8 model. The app runs smoothly with real-time inference on my local machine. However, when I deploy it to Hugging Face Spaces, ...
Shah Zeb's user avatar
0 votes
0 answers
13 views

How can I make my Hugging Face fine-tuned model's config.json file reference a specific revision/commit from the original pretrained model?

I uploaded this model: https://huggingface.co/pamessina/CXRFE, which is a fine-tuned version of this model: https://huggingface.co/microsoft/BiomedVLP-CXR-BERT-specialized Unfortunately, CXR-BERT-...
Pablo Messina's user avatar
0 votes
0 answers
14 views

Does it common for LM (hundreds million parameters) beat LLM (billion parameters) for binary classification task?

Preface I am trying to fine-tune the transformer-based model (LM and LLM). The LM that I used is DEBERTA, and the LLM is LLaMA 3. The task is to classify whether a text contains condescending language ...
sempraEdic's user avatar
0 votes
0 answers
20 views

NLP: how to handle bad tokenization

I get nonsense when trying to translate the following german sentence to swedish using google/madlad400-3b-mt: a. Natürliche Personen: BundID mit ELSTER-Zertifikat oder nPA/eID/eAT-Authentifizierung ...
Mathermind's user avatar
0 votes
0 answers
30 views

Instruction LLM - extract data from text wrongly continues

I'm trying to fine-tune open sourced LLMs, for now let's stick with Mistral-7b-instruct model. My task is a follow: I have emails, that represents "price requests" for shipments sends by our ...
sagi's user avatar
  • 101
0 votes
0 answers
383 views

"No sentence-transformers model found with name" on huggingface even though it exists

I am trying to use infgrad/stella-base-en-v2 on hugging to generate embeddings using langchain The model exists on the huggingface hub The model is listed on the MTEB leaderboard The model has ...
figs_and_nuts's user avatar
0 votes
0 answers
26 views

Not able to use huggingface inference API to get text embeddings

From the tutorials I am using the example that is provided ...
figs_and_nuts's user avatar
0 votes
0 answers
24 views

Does Fine Tuning with Custom Label Build Upon the Capability of Zero Shot Classification or Does It Train from Scratch?

The task is to classify email text bodies into exclusive categories like feedback, complaint etc. I have a labelled dataset available having about 350 samples. I have tried the ...
Della's user avatar
  • 335
0 votes
0 answers
13 views

Training split generation - Extremely Slow

I am fine-tuning mbert on wikipedia dataset, loaded with Datasets (Hugging face) ...
lingvenvist's user avatar
0 votes
0 answers
9 views

Will hypermeters tuned on sampled dataset work for the whole dataset?

I'm doing multi-label classification on text data using BERT model. Since the dataset is huge, around 50 thousand rows, I was thinking to use stratify sampling on dataset to reduce it to around 2-4 ...
Shaurya Uniyal's user avatar
0 votes
0 answers
7 views

dimensions conflict when trying to use a pretrained wav2vec2-xls-r-300m model

i am trying to use a pretrained model facebook/wav2vec2-xls-r-300m and fine tune it for audio classification and more specifically emotion recognition. i am using an audio labeled dataset ( 12 labels )...
stanley101's user avatar
0 votes
0 answers
38 views

How to find LLM that is best at STS task?

I'm trying to find large language models that maps an embedding vector in proximity if they are semantically similar, in Korean. I tried looking at bunch of leaderboard such as MTEB_ko-ko STS, AI Hub ...
haneulkim's user avatar
  • 469
0 votes
0 answers
9 views

Reducing language bias for text classification, transformer model

I am working on a text classification model predicting classes for text. We have languages from many parts of the world and some of our classes are dominated by specific languages. The model we are ...
Carl Rynegardh's user avatar
0 votes
0 answers
127 views

Safetensors are slower than torch.load. Why?

Lately, I have been testing safetensors. It claims that the safetensors are faster and safer than pickle. So I did my own comparison with the code below: ...
mark's user avatar
  • 103
0 votes
0 answers
17 views

How to perform inference on a finetuned falcon 7b model fine tuned on open assistant dataset

I finetuned a falcon 7b model on the open assistant dataset using the official colab notebook provided by huggingface at https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=...
Saket Vempaty's user avatar
0 votes
0 answers
75 views

What happens when I set is_decoder to True in the bert API from huggingface?

Please help me understand the implications of initialising the bert model from huggingface with is_decoder parameter set to True ...
figs_and_nuts's user avatar
0 votes
0 answers
47 views

F1 and Exact-Match (EM) Score in Extractive QA NLP

I have a question as to how the F1 should be calculated in NLP and whether the text normalization is optional or not. So I have been working on a project where we created a closed-domain extractive QA ...
tt40kiwi's user avatar
  • 101
0 votes
0 answers
68 views

How to use Bertweet model for topic modeling

The problem is implementation of Bertweet in a topic-modeling project with understandable output like BERTopic, i want to use it on a relatively large (20k tweets) unlabelled dataset to segment it ...
Rossin's user avatar
  • 1
0 votes
0 answers
313 views

How to get Llama-2 Rotary Embeddings?

I want to get the Llama-2 rotary embeddings. I do print(model) and get the following output: In the picture I highlight the rotary embeddings. How can get the ...
Christian01's user avatar
0 votes
0 answers
29 views

Errors in results after saving model vs using directly from memory

I am trying to save a Fine Tuned model using trainer.save_model() but after I load the saved_model it just responds with the input back again and does not give any ...
Nikhil Verma's user avatar
0 votes
0 answers
249 views

Specifying arguments of HuggingFaceHub

In this tutorial, when specifying llm = HuggingFaceHub( repo_id=repo_id, model_kwargs={"temperature": 0.5, "max_length": 64}) only ...
Karl 17302's user avatar
0 votes
0 answers
27 views

How can BERT/Transformer models accept input batches of different sizes?

I understand that all inputs in a batch need to be of the same size. However, it seems BERT/Transformers models can accept batches with different sizes as input. How is that possible? I thought we ...
PS1's user avatar
  • 101