Skip to main content

Questions tagged [huggingface]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
3 votes
1 answer
1k views

How to i get word embeddings for out of vocabulary words using a transformer model?

When i tried to get word embeddings of a sentence using bio_clinical bert, for a sentence of 8 words i am getting 11 token ids(+start and end) because "embeddings" is an out of vocabulary ...
cerofrais's user avatar
  • 131
0 votes
0 answers
152 views

List of Google T5 possible operations

I am trying to use the huggingface.co pre-trained model of Google T5 (https://huggingface.co/t5-base) for a variety of tasks. But I can`t find a list of many tasks it really supports and how to ...
tschomacker's user avatar
0 votes
1 answer
205 views

Which script can be used to finetune BERT for SQuAD question answering in Hugging Face library?

I have gone through lot of blogs which talk about run_squad.py script from Hugging Face, but I could not find it in the latest repo. So, which script has to be used ...
user2478236's user avatar
0 votes
0 answers
166 views

BERT for classification model degenerates into all-positive predictions

As a learning project, I'm training a BERT model with the CoLA dataset to detect sentence acceptability. Unfortunately my model is learning to classify every instance as "acceptable", and I'...
jdferreira's user avatar
1 vote
0 answers
270 views

Huggingface Library - Multi-document summarization

Can BART, PEGASUS ... etc. API in huggingface library be used to directly perform multi document summarization? (e.g. here: https://huggingface.co/transformers/model_doc/bart.html)
asahi kibou's user avatar
1 vote
0 answers
161 views

How to apply pruning on a BERT model?

I have trained a BERT model using ktrain (tensorflow wrapper) to recognize emotion on text, it works but it suffers from really slow inference. That makes my model not suitable for a production ...
Stamatis Tiniakos's user avatar
4 votes
1 answer
1k views

Bert for QuestionAnswering input exceeds 512

I'm training Bert on question answering (in Spanish) and i have a large context, only the context exceeds 512, the total question + context is 10k, i found that longformer is bert like for long ...
Simone's user avatar
  • 242
1 vote
1 answer
3k views

Overfitting in Huggingface's TFBertForSequenceClassification

I'm using Huggingface's TFBertForSequenceClassification for multilabel tweets classification. During training the model archives good accuracy, but the validation accuracy is poor. I've tried to solve ...
Shahad Mahmud's user avatar
1 vote
1 answer
279 views

Should weight distribution change more when fine-tuning transformers-based classifier?

I'm using pre-trained DistilBERT model from Huggingface with custom classification head, which is almost the same as in the reference implementation: ...
Marcin Zablocki's user avatar
0 votes
1 answer
1k views

BERT - How Question answering is different than classification

Basically I am trying to understand how question answering works in case of BERT. Code for both classes QuestionAnswering and Classification is pasted below for reference. My understanding is: <...
Sandeep Bhutani's user avatar

1 2
3