6
votes
Accepted
Which is the fastest image pretrained model?
The answer will depend on some things such as your hardware and the image you process. Additional, we should distinguish if you are talking about a single run through the network in training mode or ...
- 14.5k
5
votes
What are the good parameter ranges for BERT hyperparameters while finetuning it on a very small dataset?
Data augumentation
If the number of text data is small, text data argumentations may be applicable e.g. nlpaug. Applying text summarization, removing stopwords or punctuations would be a simple way to ...
- 555
3
votes
How to combine different models in Keras?
You can get the output of your models with model.output or get_layer
and combine them with ...
- 352
2
votes
Is it possible to add new vocabulary to BERT's tokenizer when fine-tuning?
To my understanding words unknown to the tokenizer will be masked with [UNKNOWN].
Your understanding is not correct.
BERT's vocabulary is defined not at word level, but at subword level. This means ...
- 20.5k
2
votes
What are the good parameter ranges for BERT hyperparameters while finetuning it on a very small dataset?
How many classes do you have? Bert can handle a high-quality 12k dataset for binary classification. I recommend duplicating your positive test case 4x and sampling a 5k test cases from your negative ...
- 151
2
votes
Accepted
CNNs - Hyperparameter tuning with different training sizes of the same data set
First suggestion: you should first find a CNN architecture that satisfies you, and then stick with it.
Second suggestion: be careful with cross validation. CNNs are extremely "heavy" models, they can ...
- 5,957
2
votes
Fine tuning Convolutional Neural Network with a learnable first layer
It's impossible to fine-tune ResNet with grayscale images directly because it was trained with color images from ImageNet.
There are 5 solutions:
Convert grayscale images to RGB images staying gray, ...
- 4,454
2
votes
Training a CNN in production on new data
This is a GOOD question. Say you have taken a pre-trained CNN model and then fine-tuned it on your data. The model is productionized and works well for a few weeks but now you observe a drift. The ...
- 818
1
vote
How to limit a GPT chatbot in specific domain?
You can just include a clause in the prompt, before providing the user question, to make it kindly refuse to answer questions not related to the target domain.
- 20.5k
1
vote
About improving the classifier when using a pre-trained model
Both of your models (with and with no extra layer) have high train accuracy but much lower scores on test data which indicates that both models overfit, i.e. they suffer from high variance (using the ...
- 5,290
1
vote
Accepted
Pretrained vs. finetuned model
Even if both expressions are often considered the same in practice, it is crucial to draw a line between "reuse" and "fine-tune".
We reuse a model to keep some of its inner ...
- 4,454
1
vote
Is it okay to fine-tuning bert with large context for sequence classification?
There is a limiting factor here, which is the positional embeddings.
In BERT, positional embeddings are trainable (not sinusoidal) and support a maximum of 512 positions. To exceed such a sequence ...
- 20.5k
1
vote
Transformer similarity fine-tuned way too often predicts pairs as similar
For NLP related tasks the transformer tries it's best to match your output distribution but as with all ml tasks it will fail on some parts of your data. Your task is somewhat similar to bert's next ...
- 73
1
vote
Is it possible to fine-tuning BERT by training it on multiple datasets? (Each dataset having it's own purpose)
This is possible but the BERT model will lose its purpose. Each NLP task will have its optimal loss value. If many tasks are fine-tuned on the same model, the optimal loss function for all the tasks ...
1
vote
Accepted
Combining textual and numeric features into pre-trained Transformer BERT
When they are talking about aggregating their outputs, they mean the final embeddings (just before the classification layer) not the output of the network itself.
You take the embeddings from both of ...
- 672
1
vote
Fine tune the RetinaNet model in PyTorch
I am also trying to do a similar thing. The code below should work. After loading the pretrained weights on COCO dataset, we need to replace the classifier layer with our own.
...
- 133
1
vote
Accepted
Is it a good idea to combine fine tuning and feature extraction techniques?
The concept of multimodal learning is relevant: in this case, combining data from two modalities: 1) image signal using ResNet50 and 2) genomic features extracted from genes. Multimodal learning for ...
- 131
1
vote
Training Accuracy is getting higher, but Valid Loss and Accuracy is same every epoch
As you can see from the Train and Validation loss (and also accuracy). While your model is able to learn, your validation results do not improve. This means underfitting, or in this specific case, ...
- 1,188
1
vote
Does finetuning BERT involving updating all of the parameters or just the final classification layer?
Both approaches are reasonable. Updating the BERT weights will train for longer period of time, but should give more accurate results.
- 884
1
vote
How to improve a CNN without changing the architecture?
Well there are a number of ways you can improve a CNN network without changing the architecture.
So I will try to explain each of them as much as I can also since your Validation Loss is diverging ...
- 11
1
vote
Difference between using BERT as a 'feature extractor' and fine tuning BERT with its layers fixed
No, approaches 1 and 2 are not the same:
In approach 1 (feature extraction), you not only take BERT's output, but normally take the internal representation of all or some of BERT's layers.
In ...
- 20.5k
Only top scored, non community-wiki answers of a minimum length are eligible