Questions tagged [transfer-learning]

Transfer learning is the process of learning a set of characteristics from one data and applying this "knowledge" to another similar dataset (i.e. using the same model across datasets).

Filter by
Sorted by
Tagged with
1 vote
0 answers
11 views

Dealing with little available data: transfer learning

Suppose I seek to predict a certain numerical value, whereby the data set which contains the predetermined correct labels is only very small. However, I'm also provided a large data set with a label ...
user avatar
  • 11
1 vote
1 answer
29 views

Pretrained vs. finetuned model

I have a doubt regarding terminology. When dealing with huggingface transformer models, I often read about "using pretrained models for classification" vs. "fine-tuning a pretrained ...
user avatar
  • 257
0 votes
0 answers
5 views

Model transfer with limit to none label information

I have this problem I hope to get some help here. Say I have a type of product A whose measurements are X_A and an outcome property is ...
user avatar
  • 21
1 vote
1 answer
28 views

Cannot achieve good result while Transfer Learning CIFAR-10 on ResNet50 - Keras

I'm trying to Transfer Learn ResNet50 for image classification of the CIFAR-10 dataset. It's stated in the original paper and also ResNet50 documentation on keras.io that the ResNet should have a ...
user avatar
  • 11
0 votes
0 answers
26 views

Train model using Transfer Learning, the validation accuracy not learning

I am new to transfer learning. I am doing face mask detection in 4 classes(no facemask wearing, incorrect facemask wearing, correct facemask wearing, double mask wearing). My objective is to compare ...
user avatar
0 votes
1 answer
65 views

model.fit vs model.evaluate gives different results?

The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or ...
user avatar
  • 101
0 votes
0 answers
21 views

Finetune XLM-RoBERTa on Tensorflow

I want to finetune pre-trained XLM-RoBERTa from HuggingFace for Text classification. I have categorical data in English. I want to finetune model on Tensorflow-keras. Can anyone let me know how can I ...
user avatar
0 votes
0 answers
122 views

Finetune XLM-RoBERTa on TF-keras for text classification

I am trying to finetune pre-trained XLM-RoBERTa on Tensorflow-keras. I am using dataset in English for text classification. I have used xlm-roberta-base tokenizer to tokenize the sentences. I am using ...
user avatar
0 votes
0 answers
18 views

Baseline model and transfer learning

I've tried to find any guidance on using transfer learning when building baseline models for ML projects (CNN in my case) but found no clues on good practices in the matter. My logic says that no ...
user avatar
0 votes
0 answers
15 views

How to stack Transfer Learning models in a Sequential

To make a nice architecture, I wanted to stack Transfer Learning models one over the other. The three models I wanted to stack were : VGG16 InceptionV3 Resnet50 So, I defined the three models as ...
user avatar
0 votes
1 answer
959 views

No module named 'model'

I am trying to use the CoAtNet class in the following link CoAtNet Class from Github but I always have error while I am running the following lines: ...
user avatar
  • 27
0 votes
0 answers
15 views

Update deep learning model with a new class and data for that class

I wanted to create a deep learning model with Keras capable of continually updating, but I do not want to retrain the model on the whole dataset, only on the new data. That is: if for example ...
user avatar
0 votes
0 answers
23 views

Using Validation Set in Transfer Learning for Feature Extractor Preprocessor

I have a set of images of products. I am using transfer learning for images feature extraction in this way : I load a model (res-net, vgg) I add 2 dense layers, first one will be my features and the ...
user avatar
  • 101
-1 votes
3 answers
630 views

Transfer learning on YOLOv5 for character and shape detection

The task is to detect rotated alphanumeric characters embedded on colored shapes. We will have an aerial view of the object (from a UAS: Unarmed Aerial System), something of this sort: (One Uppercase ...
user avatar
0 votes
0 answers
13 views

What should I visualize for humor detection model to gain some useful insight?

I was going through bunch (1,2,3) of humor detection paper. But most papers don't include any visualizations, say some graph related to model being trained. I was thinking to train some language ...
user avatar
  • 137
0 votes
0 answers
48 views

Transfer Learning and Batch Normalization layers

I am trying to use ResNet152 ( pretrained on imagenet) to solve a certain classification task. According to Keras, one strategy for transfer learning is that we do not use the Resnet classifier and ...
user avatar
  • 7
0 votes
0 answers
28 views

Image size for training transfer learning model for object detection purposes

I am trying to build transfer learning model to detect objects from video streams. There will be at least two or three different objects (classes) which are quite different from each other. The ...
user avatar
  • 1
0 votes
1 answer
140 views

How to freeze certain layers in models obtained from keras.applications

I am currrently trainning to use transfer learning on ResNet152 obtained from Keras Applications: ...
user avatar
  • 7
2 votes
0 answers
22 views

Transfer learning + Selective Fine Tuning

This is a two part question: Background: I have a rather small set (thousands) of medical images. I am using pretrained models off of kersa.applications without ...
user avatar
  • 121
0 votes
0 answers
18 views

How can I build my voice speech-to-text model?

I found an instruction to build such kind of custom model on Azure. Prepare data for Custom Speech However, I would like to either fine-tune or transfer learning on Google Colaboratory or docker. In ...
user avatar
  • 119
1 vote
2 answers
59 views

Distribution Shift vs Transfer Learning

Transfer learning (TL) is a research problem in machine learning (ML) that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem [1] ...
user avatar
0 votes
1 answer
54 views

How does T5 model work on input and target data while transfer learning?

I am working on a project where I want the model to generate job description based on Role, Industry, Skills. I have trained my data and got the resultant output. I ...
user avatar
  • 33
0 votes
0 answers
7 views

Preprocessing for Transfer Learning Model with an Inception Network

I am trying to build an image classification model using an Inception Network as the base. This is a simple binary classification model. My images are available in many smaller directories within one ...
user avatar
0 votes
1 answer
783 views

What is the difference between batch_encode_plus() and encode_plus()

I am doing a project using T5 Transformer. I have read documentations related to T5 Transformer model. While using T5Tokenizer I am kind of confused with tokenizing my sentences. Can someone please ...
user avatar
  • 33
0 votes
0 answers
22 views

Understanding how transfer learning happens in named entity recognition task

I was going through word embedding video in Andrew Ng's coursera course Sequence modeling. In this video, he gives following two examples: Sally Johnson is an orange farmer. Robert Lin is a durian ...
user avatar
  • 165
0 votes
1 answer
28 views

How many layers of a pretrained model shoud be frozen?

I'm following an example of transfer learning where the blogger has frozen the first 20 layers of MobileNet. My question is , that is there any rule of thumb for how many layers should be frozen? ...
user avatar
0 votes
0 answers
7 views

Does transfer learning works with more precise data?

I have a problem when I have 2 types of datasets for a classification task: a large one with which I intend to teach general rules of the field a smaller one, where quite similar instances occur but ...
user avatar
  • 123
0 votes
1 answer
181 views

Is it possible to fine-tuning BERT by training it on multiple datasets? (Each dataset having it's own purpose)

BERT can be fine-tuned on a dataset for a specific task. Is it possible to fine-tune it on all these datasets for different tasks and then be utilized for these tasks instead of fine-tuning a BERT ...
user avatar
0 votes
0 answers
6 views

How do pretrained models using SQUAD dataset work on an any other dataset?

I see in some Kaggle contests people have used models pretrained in SQUAD dataset for building QA systems for the dataset given in the contest. How does this work? How can a pretrained model in a ...
user avatar
0 votes
0 answers
5 views

Transfer learning on images with higher dynamic range

Is it possible to fine-tune a CNN-based model previously trained on images with 8 bits depth [0 ~ 2^8] to fit a 16 bits depth [0 ~ 2^16] images? if there is any research paper that confirm that, it ...
user avatar
0 votes
0 answers
33 views

How Pretraining part actually work in Wav2vec models? Which data is qualify to be the adequat for fine-tuning part the model of speech2text

Pretraining and fine-tuning the algorithm of wav2vec2.0, the new one using in FAcebookAI to do speech to text for low-resource language. I didn't actually get how the model does the pretraining part ...
user avatar
0 votes
0 answers
24 views

General question about transfer learning in time series classification

This paper (https://arxiv.org/abs/1811.01533) investigated the extent to which transfer learning improves the results of time series classifications. It turned out that it is better to use a source ...
user avatar
0 votes
0 answers
34 views

Accuracy and Val accuracy not equal

I am working on Covid-19 detection with pre-trained models called Resnet50 and VGG19. However, during training the accuracy increases and the validation accuracy decreases? What is the issue in my ...
user avatar
0 votes
1 answer
402 views

To freeze or not, batch normalisation in ResNet when transfer learning

I'm using a ResNet50 model pretrained on ImageNet, to do transfer learning, fitting an image classification task. The easy way of doing this is simply freezing the conv layers (or really all layers ...
user avatar
1 vote
1 answer
97 views

What is the difference between Multi task learning and domain generalization

I was wondering about the differences between "multi-task learning" and "domain generalization". It seems to me that both of them are types of inductive transfer learning but I'm ...
user avatar
0 votes
1 answer
130 views

Can I use a different image input size for transfer learning?

Most pre-trained CNN models accept a $224x224$ input size when they were trained. Can I use $256x256$ to get a higher accuracy?
user avatar
  • 3
0 votes
0 answers
18 views

Empty model when using ResNet50 for transfer learning

I am working on transfer learning using ResNet50. I have a code which was working four months before in Google Colab but when I checked it now, it is no longer working. I am not sure whether this is ...
user avatar
2 votes
1 answer
599 views

Fine-tuning pre-trained Word2Vec model with Gensim 4.0

With Gensim < 4.0, we can retrain a word2vec model using the following code: ...
user avatar
  • 51
0 votes
0 answers
33 views

Q. Why is my testing accuracy varying slightly with batch size (97.7% - 100%)?

As you will see, when batch size is set to 1, I'm consistently getting 97.7% testing accuracy for all 10 iterations. However, when batch size is set to 64, I'm getting a testing accuracy of 100% 7 out ...
user avatar
0 votes
0 answers
11 views

Volatile training loss chart on transfer learning mobile_netV1

Above is a graph of my latest attempt at transfer learning from a mobile_netV1 model already trained to 1 million steps, doing transfer learning with 50000 additional steps to my new dataset. The ...
user avatar
0 votes
0 answers
22 views

Transfer learning on siamese network with limited data

This may be a silly example, but it should be similar enough to my true research question without giving specifics. Let's say I have a pretrained Siamese neural network that tells you similarity ...
user avatar
0 votes
0 answers
14 views

Determine Transfer Learning Strategy for NER task

I worked on a Transfer Learning project in which I created a training dataset (labeled) and I used a pre-trained BERT model and fine-tuned it. The project was an NLP project in which I performed ...
user avatar
1 vote
2 answers
218 views

Why does my InceptionV3 model give a high training accuracy (99%), a high validation accuracy (95%+) but a very low testing accuracy (55%)?

Note: Please go through this in its entirety. My objective here is not just to get a high testing accuracy but to explain why it is so low in spite of validation accuracy being so high. I am a ...
user avatar
0 votes
0 answers
25 views

How to handle out of vocabulary word efficiently

I have a corpus which consists of some scientific keywords for example MKLS , SDEV all these are abbreviation of some methodology and they can be custom also based on business convenience. One way to ...
user avatar
0 votes
0 answers
26 views

Are imagenet weights same for all pretrained models?

I am new to transfer learning and I have one image classification problem, and I was thinking about training two pretrained model on TensorFlow: Inception and ...
user avatar
0 votes
1 answer
33 views

Transfer Learning or Custom Network?

I am learning Computer Vision and I was wondering if it's usually worth it to build a custom convolutional network from scratch (through trials and errors) or if using transfer learning with a popular ...
user avatar
1 vote
1 answer
44 views

Does transfer learning make sense for small neural networks with only one or two hidden layers?

I am testing transfer learning on rather small neural networks with only two hidden layers of 20 neurons on tabular data. None of my experiments yields any improvement over a basic neural network. Is ...
user avatar
0 votes
0 answers
74 views

Data To Text NLG for financial reports

I am working on a project where I want to replace a template-based approach for financial reporting with an end-to-end approach with NLG. The template-based approach takes as input some financial data ...
user avatar
0 votes
2 answers
74 views

Is there wights of voice or audio for VGG or Inception?

I want to use VGG16 (or VGG19) for voice clustering task. I read some articles which suggest to use ...
user avatar
  • 267
1 vote
0 answers
30 views

ImageNet vs CIFAR pre-trained models for biomedical image classification

I'm doing research on transfer learning for biomedical image classification, mainly skin lesion classification. From what I know, both CIFAR and ImageNet are pre-trained on natural images and not ...
user avatar