Questions tagged [huggingface]
The huggingface tag has no usage guidance.
17
questions
1
vote
0answers
23 views
Top-K vs AUC - communicating results and next steps [closed]
I have a bi-LSTM multi-label text classification model which when training on a highly imbalanced dataset with 1000 possible labels gives a top-k (k=5) categorical accuracy of 86% and a focal loss of ...
1
vote
0answers
36 views
How to i get word embeddings for out of vocabulary words using a transformer model?
When i tried to get word embeddings of a sentence using bio_clinical bert, for a sentence of 8 words i am getting 11 token ids(+start and end) because "embeddings" is an out of vocabulary ...
0
votes
0answers
17 views
List of Google T5 possible operations
I am trying to use the huggingface.co pre-trained model of Google T5 (https://huggingface.co/t5-base) for a variety of tasks. But I can`t find a list of many tasks it really supports and how to ...
0
votes
1answer
25 views
which script can be used to finetune BERT fro squad question answering in hugging face library
I have gone through lot of blogs which talk about run_squad.py script from hugging face, but I could not find it in the latest repo. So, which script has to be used ...
0
votes
0answers
64 views
BERT for classification model degenerates into all-positive predictions
As a learning project, I'm training a BERT model with the CoLA dataset to detect sentence acceptability. Unfortunately my model is learning to classify every instance as "acceptable", and I'...
1
vote
0answers
6 views
Does transformer learn from context to context
We are fine-tuning a transformer-based question answering model to answer the questions about the novels with the chronological plot. We are breaking up the novel into chapters and then using them as ...
0
votes
0answers
23 views
Huggingface Library - Multi-document summarization
Can BART, PEGASUS ... etc. API in huggingface library be used to directly perform multi document summarization?
(e.g. here: https://huggingface.co/transformers/model_doc/bart.html)
1
vote
0answers
72 views
How to apply pruning on a BERT model?
I have trained a BERT model using ktrain (tensorflow wrapper) to recognize emotion on text, it works but it suffers from really slow inference. That makes my model not suitable for a production ...
4
votes
1answer
198 views
Bert for QuestionAnswering input exceeds 512
I'm training Bert on question answering (in Spanish) and i have a large context, only the context exceeds 512, the total question + context is 10k, i found that longformer is bert like for long ...
0
votes
0answers
52 views
PyTorch Hugging Face - Language generation with torchscript model
Have a fine-tuned a summarization model following the Hugging Face seq2seq guide (starting from sshleifer/distilbart-xsum-12-6).
We are interested in using AWS elastic inference for deployment for ...
0
votes
0answers
81 views
huggingface bert - need help on working of code
Looking for some explanation of understanding of the BERT implementation by huggingface. I would explain my understanding below and then ask question:
Below is code for question answering ...
0
votes
1answer
361 views
Overfitting in Huggingface's TFBertForSequenceClassification
I'm using Huggingface's TFBertForSequenceClassification for multilabel tweets classification. During training the model archives good accuracy, but the validation accuracy is poor. I've tried to solve ...
0
votes
0answers
219 views
BERT Implementaion for Sequence Classification
I am trying to implement BERT using HuggingFace - transformers implementation.
I am following two links: by analytics-vidhya and by HuggingFace
Below is the code:
...
0
votes
0answers
67 views
HuggingFace/Transformers Implementation for Classification
I am trying to implement BERT using HuggingFace - transformers implementation.
I am following two links:
by analytics-vidhya and by HuggingFace
If we consider inputs for both the implementations:
1) ...
0
votes
0answers
50 views
How can I make a whitespace tokenizer and use it to build a language model from scratch using transformers
I have raised a ticket in github, whose link is here
Which has a problem statement as mentioned below. If you have any solution to this, do answer.
I am trying to make a language model using a ...
1
vote
1answer
42 views
Should weight distribution change more when fine-tuning transformers-based classifier?
I'm using pre-trained DistilBERT model from Huggingface with custom classification head, which is almost the same as in the reference implementation:
...
0
votes
1answer
433 views
BERT - How Question answering is different than classification
Basically I am trying to understand how question answering works in case of BERT. Code for both classes QuestionAnswering and Classification is pasted below for reference. My understanding is:
<...