Questions tagged [finetuning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
1 answer
24 views

About improving the classifier when using a pre-trained model

I have tried adding a layer in the Resnet Model as shown: ...
2 votes
1 answer
26 views

How to further fine-tune a transformer NLP model on domain specific dataset, after general fine-tuning

I would like to fine-tune a pre-trained BERT-like model for a semantic similarity analysis task in the fashion of the SNLI/MNLI task (i.e. classify sentence pairs to "entailment" or "...
  • 41
1 vote
1 answer
30 views

Fine tuning Convolutional Neural Network with a learnable first layer

I have a classification task using grayscale images and I want to leverage from pretrained networks. There are a lot of resources out there presenting how to fine tune large neural nets like resnet, ...
0 votes
0 answers
21 views

Parameters for training a sentence-similarity model using Bert?

I have a list of sentences : sentences = ["Missing Plate", "Plate not found"] I am trying to find the most similar sentences in the list by ...
0 votes
0 answers
20 views

Transfer learning on the same samples for progressively more specific classification

I have a classification task in which classes exist within a directed graph. That is a class may have subclasses which share an is-a relationship with their parent class. Now, I have a relatively ...
2 votes
1 answer
27 views

How can I fine tune a model to detect digits, used to detect denominations of currency notes

So the task at hand is to detect the denomination of any currency banknote. The dataset I have is about 2k images of each denomination (12 in total). An example banknote (after noise removal, erosion ,...
0 votes
0 answers
107 views

Fine Tuning BERT for text summarization

I was trying to follow this notebook to fine-tune BERT for the text summarization task. Everything was good till I come to this instruction in section Evaluation to evaluate my model: ...
1 vote
0 answers
40 views

Is it possible to "fine-tune" a pre-trained logistic regression model?

Fine tuning is a concept commonly used in deep learning. We may have a pre-trained model and then fine-tune it to our specific task. Does that apply to simple models, such as logistic regression? For ...
0 votes
0 answers
165 views

ValueError: Mixed precision training with AMP or APEX (`--fp16` or `--bf16`) and half precision evaluation (`--fp16) can only be used on CUDA devices

i’m fine tuning the wav2vec-xlsr model. i’ve created a virtual env for that and i’ve installed cuda 11.0 and tensorflow-gpu==2.5.0 but it gives the following error : ValueError: Mixed precision ...
1 vote
1 answer
55 views

Pretrained vs. finetuned model

I have a doubt regarding terminology. When dealing with huggingface transformer models, I often read about "using pretrained models for classification" vs. "fine-tuning a pretrained ...
  • 289
1 vote
1 answer
132 views

Is it okay to fine-tuning bert with large context for sequence classification?

I want to create sequence classification bert model. The input of model will be 2 sentence. But i want to fine tuning the model with large context data which consists of multiple sentences(which ...
  • 11
1 vote
1 answer
20 views

Tuning model by my metric

My project is using a metric to evaluate the performance of regression model, it is not belong to basic metric in Machine learning (MSE,MAE,...). So, how can I tuning model base on my metric ?
  • 21
0 votes
1 answer
15 views

Are most deep learning models online learning models?

I'm online learning starter. from my perspective, online learning model is the model which can update its paramater with data flows(I've seen a article pointing out that incremental model is ...
  • 1
0 votes
1 answer
19 views

Transformer similarity fine-tuned way too often predicts pairs as similar

I fine-tuned a transformer for classification to compute similarity between names. This is a toy example for the training data: ...
  • 101
0 votes
0 answers
28 views

Value accuracy remains the same

I have used my own build model and also fine-tuned other two model ResNeT50 and VGG16, but val_acc remains the same for them all. ...
0 votes
0 answers
29 views

Where to download the weights for PyTorch Efficientnet-b6

I would like to know how to download the weights for PyTorch Efficientnet-b6 architecture. Only the weights, not the entire architecture.
  • 663
0 votes
1 answer
572 views

How to freeze certain layers in models obtained from keras.applications

I am currrently trainning to use transfer learning on ResNet152 obtained from Keras Applications: ...
  • 7
0 votes
0 answers
166 views

How to fine-tune GPT-J with small dataset

Firstly, thank you so much for looking at this post. I could really use some help. I have followed this guide as closely as possible: https://github.com/kingoflolz/mesh-transformer-jax I'm trying to ...
0 votes
0 answers
61 views

How can I build my voice speech-to-text model?

I found an instruction to build such kind of custom model on Azure. Prepare data for Custom Speech However, I would like to either fine-tune or transfer learning on Google Colaboratory or docker. In ...
  • 119
1 vote
1 answer
315 views

Is it possible to fine-tuning BERT by training it on multiple datasets? (Each dataset having it's own purpose)

BERT can be fine-tuned on a dataset for a specific task. Is it possible to fine-tune it on all these datasets for different tasks and then be utilized for these tasks instead of fine-tuning a BERT ...
0 votes
0 answers
60 views

How long does it take to fine-tune XLNet?

XLNet takes a lot more time than BERT during pre-training. This results in XLNet performing better than BERT in over 20 NLP tasks. How long does XLNet take for fine-tuning (let's assume this is ...
1 vote
1 answer
1k views

Is it possible to add new vocabulary to BERT's tokenizer when fine-tuning?

I want to fine-tune BERT by training it on a domain dataset of my own. The domain is specific and includes many terms that probably weren't included in the original dataset BERT was trained on. I know ...
0 votes
1 answer
33 views

How to use new dataset on a pretrained neural network model?

I have built a dataset that I would like to pass to a pretrained model in oder to perform some predictions. I am looking for some steps/processes to guide me in this. Should I fine tune?If so what ...
  • 1
0 votes
1 answer
879 views

Incompatible shapes (None, 1) and (None, 5) with Keras VGGFace Finetuning

Categories to learn and predict: df.race.unique() array(['0', '1', '3', '2', '4'], dtype=object) Data: ...
2 votes
1 answer
1k views

Combining textual and numeric features into pre-trained Transformer BERT

I have a dataset with 3 columns: Text Meta-data (intending to extract features from it, then use those i.e., numerical features) Target label Question 1: How can I use a pre-trained BERT instance on ...
1 vote
1 answer
1k views

Fine tune the RetinaNet model in PyTorch

I would like to fine the pre-trained RetinaNet model available in torchvision in order to create my own object detection. I'm trying to replicate what is done for the FastRCNN at this link: https://...
  • 193
0 votes
1 answer
74 views

Is it a good idea to combine fine tuning and feature extraction techniques?

I have a normal/tumor medical images dataset and, for the same patients, also the relative genomics, and my goal is to predict if a patient has a tumor by combining all the information. To achieve ...
1 vote
1 answer
270 views

Why not using linear regression for finetuning the last layer of a neural network?

In transfer learning, often only the last layer of the network is retrained using gradient descent. However, the last layer of a common neural network performs only a linear transformation, so why do ...
0 votes
1 answer
35 views

Which part should be frozen during transfer learning?

I want to use transfer learning and fine tuning and I need to decide which part of the original model will be used and which part will be frozen. I'm thinking about four possilbe cases: small/large ...
  • 3
0 votes
1 answer
40 views

Training Accuracy is getting higher, but Valid Loss and Accuracy is same every epoch

I'm doing a transfer learning with ResNet50. My dataset is clothes(224x224x3), and 49 category(classes) -> training data 1000 per 1 category, total 49000. And valid data 200 per 1 category, total ...
1 vote
2 answers
823 views

Does finetuning BERT involving updating all of the parameters or just the final classification layer?

Currently learning and reading about transformer models, I get that during the pretraining stage the BERT model is trained on a large corpus via MLM and NSP. But during finetuning, for example trying ...
  • 11
0 votes
1 answer
99 views

How to improve a CNN without changing the architecture?

I'm currently using an autoencoder CNN that's built upon the VGG-16 architecture that was designed by someone else. I want to replicate their results using their dataset first but I'm finding that: -...
  • 1
1 vote
2 answers
2k views

Difference between using BERT as a 'feature extractor' and fine tuning BERT with its layers fixed

I understand that there are two ways of leveraging BERT for some NLP classification task: BERT might perform ‘feature extraction’ and its output is input further to another (classification) model ...
1 vote
1 answer
49 views

Train on multi-domains, then fine-tune on specific domain

Would it make sense to first train a model on images from multiple domains, and then do "fine-tuning" on one specific domain to improve its performance on it? For instance, one could train an object ...
3 votes
1 answer
3k views

How to combine different models in Keras?

I have a pre-trained network, consist of two parts, the feature extraction, and the similarity learning. The network takes two inputs and predicts the images are same or not. The feature extraction ...
1 vote
0 answers
37 views

Post-classification after inference in deep learning models

I designed a fire detection using Deep Learning binary classification in Keras (fire vs none). It's a simple model with a few layers. In my training dataset, I ...
  • 177
1 vote
0 answers
178 views

fine tune BERT in a small GPU

I want to to fine tune the BERT base model but the only accelerated HW I have access to is a couple of Quadro 600 GPUs which only pack 1GB RAM and 96 CUDA cores each. My question: is it even possible ...
  • 23
10 votes
2 answers
5k views

What are the good parameter ranges for BERT hyperparameters while finetuning it on a very small dataset?

I need to finetune BERT model (from the huggingface repository) on a sentence classification task. However, my dataset is really small.I have 12K sentences and only 10% of them are from positive ...
  • 219
0 votes
2 answers
152 views

Tuning a classifier for high precision, with no regard for recall

I understand this falls under the decision making aspect, rather than the probabilistic, but for the purposes of some work I am doing, I need the classifier to have very high precision, as I can't ...
1 vote
1 answer
426 views

CNNs - Hyperparameter tuning with different training sizes of the same data set

I would like to compare how much the classification performance (test accuracy) of CNNs changes depending on the size of the data set. For this I would like to use a data set like MNIST or Fashion ...
4 votes
1 answer
2k views

Which is the fastest image pretrained model?

I had been working with pre-trained models and was just curious to know the fastest forward propagating model of all the computer vision pre-trained models. I have been trying to achieve faster ...
  • 2,255