Questions tagged [finetuning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
18 views

Fine tune the RetinaNet model in PyTorch

I would like to fine the pre-trained RetinaNet model available in torchvision in order to create my own object detection. I'm trying to replicate what is done for the FastRCNN at this link: https://...
0
votes
0answers
9 views

How fine-tune a pretrained model from a github repository on my own dataset pytorch [closed]

I would like to finetune a model in a github repository on my own dataset, how can I do that?
0
votes
1answer
10 views

Is it a good idea to combine fine tuning and feature extraction techniques?

I have a normal/tumor medical images dataset and, for the same patients, also the relative genomics, and my goal is to predict if a patient has a tumor by combining all the information. To achieve ...
0
votes
1answer
21 views

Why not using linear regression for finetuning the last layer of a neural network?

In transfer learning, often only the last layer of the network is retrained using gradient descent. However, the last layer of a common neural network performs only a linear transformation, so why do ...
0
votes
1answer
21 views

Which part should be frozen during transfer learning?

I want to use transfer learning and fine tuning and I need to decide which part of the original model will be used and which part will be frozen. I'm thinking about four possilbe cases: small/large ...
0
votes
1answer
28 views

Training Accuracy is getting higher, but Valid Loss and Accuracy is same every epoch

I'm doing a transfer learning with ResNet50. My dataset is clothes(224x224x3), and 49 category(classes) -> training data 1000 per 1 category, total 49000. And valid data 200 per 1 category, total ...
1
vote
2answers
172 views

Does finetuning BERT involving updating all of the parameters or just the final classification layer?

Currently learning and reading about transformer models, I get that during the pretraining stage the BERT model is trained on a large corpus via MLM and NSP. But during finetuning, for example trying ...
0
votes
0answers
15 views

Trouble with loading pre-trained CNN weights without classification layers for different input dimension

I am trying to load pre-trained CNN weights but without classification (i.e. top) layers. Basically, I want to do exactly what tf.keras.applications.ResNet50 class ...
0
votes
0answers
104 views

weight decay in ResNet50

Can someone please guide for implementing weight decay in transfer learning approach? I want to regularize the pre-trained model ResNet50, where I'm fine-tuning the model for an image classification ...
0
votes
0answers
423 views

How to fine-tune BERT for Question Answering?

I wish to train two domain-specific models: Domain 1: Constitution and related Legal Documents Domain 2: Technical and related documents. For Domain 1, I've access to a text-corpus with texts from ...
0
votes
1answer
51 views

How to improve a CNN without changing the architecture?

I'm currently using an autoencoder CNN that's built upon the VGG-16 architecture that was designed by someone else. I want to replicate their results using their dataset first but I'm finding that: -...
1
vote
2answers
1k views

Difference between using BERT as a 'feature extractor' and fine tuning BERT with its layers fixed

I understand that there are two ways of leveraging BERT for some NLP classification task: BERT might perform ‘feature extraction’ and its output is input further to another (classification) model ...
1
vote
0answers
17 views

Train on multi-domains, then fine-tune on specific domain

Would it make sense to first train a model on images from multiple domains, and then do "fine-tuning" on one specific domain to improve its performance on it? For instance, one could train an object ...
3
votes
1answer
948 views

How to combine different models in Keras?

I have a pre-trained network, consist of two parts, the feature extraction, and the similarity learning. The network takes two inputs and predicts the images are same or not. The feature extraction ...
1
vote
0answers
32 views

Post-classification after inference in deep learning models

I designed a fire detection using Deep Learning binary classification in Keras (fire vs none). It's a simple model with a few layers. In my training dataset, I ...
1
vote
0answers
113 views

fine tune BERT in a small GPU

I want to to fine tune the BERT base model but the only accelerated HW I have access to is a couple of Quadro 600 GPUs which only pack 1GB RAM and 96 CUDA cores each. My question: is it even possible ...
8
votes
1answer
716 views

What are the good parameter ranges for BERT hyperparameters while finetuning it on a very small dataset?

I need to finetune BERT model (from the huggingface repository) on a sentence classification task. However, my dataset is really small.I have 12K sentences and only 10% of them are from positive ...
0
votes
2answers
102 views

Tuning a classifier for high precision, with no regard for recall

I understand this falls under the decision making aspect, rather than the probabilistic, but for the purposes of some work I am doing, I need the classifier to have very high precision, as I can't ...
1
vote
1answer
389 views

CNNs - Hyperparameter tuning with different training sizes of the same data set

I would like to compare how much the classification performance (test accuracy) of CNNs changes depending on the size of the data set. For this I would like to use a data set like MNIST or Fashion ...
3
votes
1answer
1k views

Which is the fastest image pretrained model?

I had been working with pre-trained models and was just curious to know the fastest forward propagating model of all the computer vision pre-trained models. I have been trying to achieve faster ...