Questions tagged [finetuning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
1answer
20 views

How to freeze certain layers in models obtained from keras.applications

I am currrently trainning to use transfer learning on ResNet152 obtained from Keras Applications: ...
0
votes
0answers
11 views

How to fine-tune GPT-J with small dataset

Firstly, thank you so much for looking at this post. I could really use some help. I have followed this guide as closely as possible: https://github.com/kingoflolz/mesh-transformer-jax I'm trying to ...
0
votes
0answers
12 views

Question and answers datasets

I am looking for a dataset similar to XQuAD. I would need it in German, but it is not tragic if it is in another language since it can be translated. It would also be okay if the format is not the ...
0
votes
0answers
11 views

How can I build my voice speech-to-text model?

I found an instruction to build such kind of custom model on Azure. Prepare data for Custom Speech However, I would like to either fine-tune or transfer learning on Google Colaboratory or docker. In ...
0
votes
1answer
40 views

Is it possible to fine-tuning BERT by training it on multiple datasets? (Each dataset having it's own purpose)

BERT can be fine-tuned on a dataset for a specific task. Is it possible to fine-tune it on all these datasets for different tasks and then be utilized for these tasks instead of fine-tuning a BERT ...
0
votes
0answers
15 views

How long does it take to fine-tune XLNet?

XLNet takes a lot more time than BERT during pre-training. This results in XLNet performing better than BERT in over 20 NLP tasks. How long does XLNet take for fine-tuning (let's assume this is ...
0
votes
0answers
31 views

NeMo Conformer-CTC Predicts Same Word Repeatedly When Fine-Tuning

I'm using the NeMo Conformer-CTC small on the LibriSpeech dataset (the clean subset, around 29K inputs, using 90% for training and 10% for testing). I use Pytorch Lightning. When I try to train, the ...
0
votes
0answers
4 views

Transfer learning on images with higher dynamic range

Is it possible to fine-tune a CNN-based model previously trained on images with 8 bits depth [0 ~ 2^8] to fit a 16 bits depth [0 ~ 2^16] images? if there is any research paper that confirm that, it ...
0
votes
0answers
19 views

pytorch lightning produces no checkpoint when learning rate fine tuning ison

My problem is concerning with using the automatic learning rate finder of pytorch lightning. In case I use this feature there isn't any checkpoint output produced at any time during the training of ...
0
votes
0answers
14 views

End-to-end machine learning project processes

I've read a book chapter that walks you through all the steps involved in an end-to-end machine learning project. After doing all the practical exercises I'm still not quite sure that my way of ...
0
votes
1answer
186 views

Is it possible to add new vocabulary to BERT's tokenizer when fine-tuning?

I want to fine-tune BERT by training it on a domain dataset of my own. The domain is specific and includes many terms that probably weren't included in the original dataset BERT was trained on. I know ...
0
votes
1answer
22 views

How to use new dataset on a pretrained neural network model?

I have built a dataset that I would like to pass to a pretrained model in oder to perform some predictions. I am looking for some steps/processes to guide me in this. Should I fine tune?If so what ...
0
votes
1answer
95 views

Incompatible shapes (None, 1) and (None, 5) with Keras VGGFace Finetuning

Categories to learn and predict: df.race.unique() array(['0', '1', '3', '2', '4'], dtype=object) Data: ...
1
vote
1answer
481 views

Combining textual and numeric features into pre-trained Transformer BERT

I have a dataset with 3 columns: Text Meta-data (intending to extract features from it, then use those i.e., numerical features) Target label Question 1: How can I use a pre-trained BERT instance on ...
0
votes
1answer
561 views

Fine tune the RetinaNet model in PyTorch

I would like to fine the pre-trained RetinaNet model available in torchvision in order to create my own object detection. I'm trying to replicate what is done for the FastRCNN at this link: https://...
0
votes
1answer
33 views

Is it a good idea to combine fine tuning and feature extraction techniques?

I have a normal/tumor medical images dataset and, for the same patients, also the relative genomics, and my goal is to predict if a patient has a tumor by combining all the information. To achieve ...
0
votes
1answer
87 views

Why not using linear regression for finetuning the last layer of a neural network?

In transfer learning, often only the last layer of the network is retrained using gradient descent. However, the last layer of a common neural network performs only a linear transformation, so why do ...
0
votes
1answer
23 views

Which part should be frozen during transfer learning?

I want to use transfer learning and fine tuning and I need to decide which part of the original model will be used and which part will be frozen. I'm thinking about four possilbe cases: small/large ...
0
votes
1answer
36 views

Training Accuracy is getting higher, but Valid Loss and Accuracy is same every epoch

I'm doing a transfer learning with ResNet50. My dataset is clothes(224x224x3), and 49 category(classes) -> training data 1000 per 1 category, total 49000. And valid data 200 per 1 category, total ...
1
vote
2answers
461 views

Does finetuning BERT involving updating all of the parameters or just the final classification layer?

Currently learning and reading about transformer models, I get that during the pretraining stage the BERT model is trained on a large corpus via MLM and NSP. But during finetuning, for example trying ...
0
votes
1answer
79 views

How to improve a CNN without changing the architecture?

I'm currently using an autoencoder CNN that's built upon the VGG-16 architecture that was designed by someone else. I want to replicate their results using their dataset first but I'm finding that: -...
1
vote
2answers
2k views

Difference between using BERT as a 'feature extractor' and fine tuning BERT with its layers fixed

I understand that there are two ways of leveraging BERT for some NLP classification task: BERT might perform ‘feature extraction’ and its output is input further to another (classification) model ...
1
vote
1answer
28 views

Train on multi-domains, then fine-tune on specific domain

Would it make sense to first train a model on images from multiple domains, and then do "fine-tuning" on one specific domain to improve its performance on it? For instance, one could train an object ...
3
votes
1answer
2k views

How to combine different models in Keras?

I have a pre-trained network, consist of two parts, the feature extraction, and the similarity learning. The network takes two inputs and predicts the images are same or not. The feature extraction ...
1
vote
0answers
34 views

Post-classification after inference in deep learning models

I designed a fire detection using Deep Learning binary classification in Keras (fire vs none). It's a simple model with a few layers. In my training dataset, I ...
1
vote
0answers
145 views

fine tune BERT in a small GPU

I want to to fine tune the BERT base model but the only accelerated HW I have access to is a couple of Quadro 600 GPUs which only pack 1GB RAM and 96 CUDA cores each. My question: is it even possible ...
9
votes
2answers
2k views

What are the good parameter ranges for BERT hyperparameters while finetuning it on a very small dataset?

I need to finetune BERT model (from the huggingface repository) on a sentence classification task. However, my dataset is really small.I have 12K sentences and only 10% of them are from positive ...
0
votes
2answers
130 views

Tuning a classifier for high precision, with no regard for recall

I understand this falls under the decision making aspect, rather than the probabilistic, but for the purposes of some work I am doing, I need the classifier to have very high precision, as I can't ...
1
vote
1answer
414 views

CNNs - Hyperparameter tuning with different training sizes of the same data set

I would like to compare how much the classification performance (test accuracy) of CNNs changes depending on the size of the data set. For this I would like to use a data set like MNIST or Fashion ...
3
votes
1answer
2k views

Which is the fastest image pretrained model?

I had been working with pre-trained models and was just curious to know the fastest forward propagating model of all the computer vision pre-trained models. I have been trying to achieve faster ...