Questions tagged [finetuning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
14 views

Is it possible to "fine-tune" a pre-trained logistic regression model?

Fine tuning is a concept commonly used in deep learning. We may have a pre-trained model and then fine-tune it to our specific task. Does that apply to simple models, such as logistic regression? For ...
user avatar
0 votes
0 answers
16 views

ValueError: Mixed precision training with AMP or APEX (`--fp16` or `--bf16`) and half precision evaluation (`--fp16) can only be used on CUDA devices

i’m fine tuning the wav2vec-xlsr model. i’ve created a virtual env for that and i’ve installed cuda 11.0 and tensorflow-gpu==2.5.0 but it gives the following error : ValueError: Mixed precision ...
user avatar
1 vote
1 answer
32 views

Pretrained vs. finetuned model

I have a doubt regarding terminology. When dealing with huggingface transformer models, I often read about "using pretrained models for classification" vs. "fine-tuning a pretrained ...
user avatar
  • 257
1 vote
1 answer
26 views

Is it okay to fine-tuning bert with large context for sequence classification?

I want to create sequence classification bert model. The input of model will be 2 sentence. But i want to fine tuning the model with large context data which consists of multiple sentences(which ...
user avatar
  • 11
1 vote
1 answer
19 views

Tuning model by my metric

My project is using a metric to evaluate the performance of regression model, it is not belong to basic metric in Machine learning (MSE,MAE,...). So, how can I tuning model base on my metric ?
user avatar
  • 21
0 votes
1 answer
13 views

Are most deep learning models online learning models?

I'm online learning starter. from my perspective, online learning model is the model which can update its paramater with data flows(I've seen a article pointing out that incremental model is ...
user avatar
  • 1
0 votes
1 answer
18 views

Transformer similarity fine-tuned way too often predicts pairs as similar

I fine-tuned a transformer for classification to compute similarity between names. This is a toy example for the training data: ...
user avatar
  • 101
0 votes
0 answers
16 views

Value accuracy remains the same

I have used my own build model and also fine-tuned other two model ResNeT50 and VGG16, but val_acc remains the same for them all. ...
user avatar
0 votes
0 answers
20 views

Where to download the weights for PyTorch Efficientnet-b6

I would like to know how to download the weights for PyTorch Efficientnet-b6 architecture. Only the weights, not the entire architecture.
user avatar
  • 663
0 votes
1 answer
149 views

How to freeze certain layers in models obtained from keras.applications

I am currrently trainning to use transfer learning on ResNet152 obtained from Keras Applications: ...
user avatar
  • 7
0 votes
0 answers
65 views

How to fine-tune GPT-J with small dataset

Firstly, thank you so much for looking at this post. I could really use some help. I have followed this guide as closely as possible: https://github.com/kingoflolz/mesh-transformer-jax I'm trying to ...
user avatar
0 votes
0 answers
18 views

How can I build my voice speech-to-text model?

I found an instruction to build such kind of custom model on Azure. Prepare data for Custom Speech However, I would like to either fine-tune or transfer learning on Google Colaboratory or docker. In ...
user avatar
  • 119
0 votes
1 answer
186 views

Is it possible to fine-tuning BERT by training it on multiple datasets? (Each dataset having it's own purpose)

BERT can be fine-tuned on a dataset for a specific task. Is it possible to fine-tune it on all these datasets for different tasks and then be utilized for these tasks instead of fine-tuning a BERT ...
user avatar
0 votes
0 answers
31 views

How long does it take to fine-tune XLNet?

XLNet takes a lot more time than BERT during pre-training. This results in XLNet performing better than BERT in over 20 NLP tasks. How long does XLNet take for fine-tuning (let's assume this is ...
user avatar
0 votes
0 answers
58 views

NeMo Conformer-CTC Predicts Same Word Repeatedly When Fine-Tuning

I'm using the NeMo Conformer-CTC small on the LibriSpeech dataset (the clean subset, around 29K inputs, using 90% for training and 10% for testing). I use Pytorch Lightning. When I try to train, the ...
user avatar
0 votes
0 answers
5 views

Transfer learning on images with higher dynamic range

Is it possible to fine-tune a CNN-based model previously trained on images with 8 bits depth [0 ~ 2^8] to fit a 16 bits depth [0 ~ 2^16] images? if there is any research paper that confirm that, it ...
user avatar
0 votes
0 answers
59 views

pytorch lightning produces no checkpoint when learning rate fine tuning ison

My problem is concerning with using the automatic learning rate finder of pytorch lightning. In case I use this feature there isn't any checkpoint output produced at any time during the training of ...
user avatar
0 votes
0 answers
18 views

End-to-end machine learning project processes

I've read a book chapter that walks you through all the steps involved in an end-to-end machine learning project. After doing all the practical exercises I'm still not quite sure that my way of ...
user avatar
0 votes
1 answer
799 views

Is it possible to add new vocabulary to BERT's tokenizer when fine-tuning?

I want to fine-tune BERT by training it on a domain dataset of my own. The domain is specific and includes many terms that probably weren't included in the original dataset BERT was trained on. I know ...
user avatar
0 votes
1 answer
26 views

How to use new dataset on a pretrained neural network model?

I have built a dataset that I would like to pass to a pretrained model in oder to perform some predictions. I am looking for some steps/processes to guide me in this. Should I fine tune?If so what ...
user avatar
  • 1
0 votes
1 answer
528 views

Incompatible shapes (None, 1) and (None, 5) with Keras VGGFace Finetuning

Categories to learn and predict: df.race.unique() array(['0', '1', '3', '2', '4'], dtype=object) Data: ...
user avatar
1 vote
1 answer
831 views

Combining textual and numeric features into pre-trained Transformer BERT

I have a dataset with 3 columns: Text Meta-data (intending to extract features from it, then use those i.e., numerical features) Target label Question 1: How can I use a pre-trained BERT instance on ...
user avatar
1 vote
1 answer
1k views

Fine tune the RetinaNet model in PyTorch

I would like to fine the pre-trained RetinaNet model available in torchvision in order to create my own object detection. I'm trying to replicate what is done for the FastRCNN at this link: https://...
user avatar
  • 193
0 votes
1 answer
57 views

Is it a good idea to combine fine tuning and feature extraction techniques?

I have a normal/tumor medical images dataset and, for the same patients, also the relative genomics, and my goal is to predict if a patient has a tumor by combining all the information. To achieve ...
user avatar
1 vote
1 answer
198 views

Why not using linear regression for finetuning the last layer of a neural network?

In transfer learning, often only the last layer of the network is retrained using gradient descent. However, the last layer of a common neural network performs only a linear transformation, so why do ...
user avatar
0 votes
1 answer
28 views

Which part should be frozen during transfer learning?

I want to use transfer learning and fine tuning and I need to decide which part of the original model will be used and which part will be frozen. I'm thinking about four possilbe cases: small/large ...
user avatar
  • 3
0 votes
1 answer
40 views

Training Accuracy is getting higher, but Valid Loss and Accuracy is same every epoch

I'm doing a transfer learning with ResNet50. My dataset is clothes(224x224x3), and 49 category(classes) -> training data 1000 per 1 category, total 49000. And valid data 200 per 1 category, total ...
user avatar
1 vote
2 answers
637 views

Does finetuning BERT involving updating all of the parameters or just the final classification layer?

Currently learning and reading about transformer models, I get that during the pretraining stage the BERT model is trained on a large corpus via MLM and NSP. But during finetuning, for example trying ...
user avatar
  • 11
0 votes
1 answer
91 views

How to improve a CNN without changing the architecture?

I'm currently using an autoencoder CNN that's built upon the VGG-16 architecture that was designed by someone else. I want to replicate their results using their dataset first but I'm finding that: -...
user avatar
  • 1
1 vote
2 answers
2k views

Difference between using BERT as a 'feature extractor' and fine tuning BERT with its layers fixed

I understand that there are two ways of leveraging BERT for some NLP classification task: BERT might perform ‘feature extraction’ and its output is input further to another (classification) model ...
user avatar
1 vote
1 answer
35 views

Train on multi-domains, then fine-tune on specific domain

Would it make sense to first train a model on images from multiple domains, and then do "fine-tuning" on one specific domain to improve its performance on it? For instance, one could train an object ...
user avatar
3 votes
1 answer
2k views

How to combine different models in Keras?

I have a pre-trained network, consist of two parts, the feature extraction, and the similarity learning. The network takes two inputs and predicts the images are same or not. The feature extraction ...
user avatar
1 vote
0 answers
37 views

Post-classification after inference in deep learning models

I designed a fire detection using Deep Learning binary classification in Keras (fire vs none). It's a simple model with a few layers. In my training dataset, I ...
user avatar
  • 177
1 vote
0 answers
161 views

fine tune BERT in a small GPU

I want to to fine tune the BERT base model but the only accelerated HW I have access to is a couple of Quadro 600 GPUs which only pack 1GB RAM and 96 CUDA cores each. My question: is it even possible ...
user avatar
  • 23
9 votes
2 answers
3k views

What are the good parameter ranges for BERT hyperparameters while finetuning it on a very small dataset?

I need to finetune BERT model (from the huggingface repository) on a sentence classification task. However, my dataset is really small.I have 12K sentences and only 10% of them are from positive ...
user avatar
  • 199
0 votes
2 answers
143 views

Tuning a classifier for high precision, with no regard for recall

I understand this falls under the decision making aspect, rather than the probabilistic, but for the purposes of some work I am doing, I need the classifier to have very high precision, as I can't ...
user avatar
1 vote
1 answer
422 views

CNNs - Hyperparameter tuning with different training sizes of the same data set

I would like to compare how much the classification performance (test accuracy) of CNNs changes depending on the size of the data set. For this I would like to use a data set like MNIST or Fashion ...
user avatar
4 votes
1 answer
2k views

Which is the fastest image pretrained model?

I had been working with pre-trained models and was just curious to know the fastest forward propagating model of all the computer vision pre-trained models. I have been trying to achieve faster ...
user avatar
  • 2,225