Questions tagged [transfer-learning]

Transfer learning is the process of learning a set of characteristics from one data and applying this "knowledge" to another similar dataset (i.e. using the same model across datasets).

Filter by
Sorted by
Tagged with
0 votes
1 answer
21 views

How to increase FER2013 dataset validation_accuracy for only 3 classes i.e, happy,sad,neutral?

I am building a face emotion detection model using vgg16. Using FER2013 dataset for 7 classes i am getting= train_accuracy=97%, validation_accuracy=90%. but when i tried with 3 classes i.e, happy,sad,...
0 votes
1 answer
22 views

Hello guys, is dimension reduction required for tensorflow? [closed]

I am working on face emotion detection using FER2013 dataset using tensorflow and vgg16 model. I am applying t-sne to my training dataset for dimensionality reduction. My question is that "is ...
0 votes
0 answers
10 views

Switch between the heads of a model during inference

I have 200 neural networks which I trained using transfer learning on text. They all share the same weights except for their heads which are trained on different tasks. Is it possible to merge those ...
0 votes
2 answers
15 views

Autoencoder vs Pre-trained network for feature extraction

I wanted to know if anyone has any sort of guidance on what is better for image classification on a lot of classes (about 400) with a small amount of samples per class (around 20), for relatively big ...
0 votes
0 answers
31 views

Weird consequence of not freezing layers in Neural Network

I was researching about "why are we freezing layers" and I came across the answer says "to not lose the information of pre-trained model" But; we are just freezing early layers (I ...
  • 85
0 votes
0 answers
19 views

How do you measure the degree of influence of pre-training on a fine-tuning task?

Context I have a randomly initialized (deep learning) model A, and a pre-trained model Ap. The two are trained (or fine-tuned, more specifically for Ap) on the same target task* and I call them A' and ...
0 votes
0 answers
13 views

How do I train an instance segmentation model with an image classification dataset?

Say I have a clean dataset of different types of utensils. By clean, I mean the images are already cropped, augmented, and ready to use for image classification models. If I want to detect multiple ...
  • 11
0 votes
0 answers
11 views

Which Keras Application to choose for transfer learning?

I am working on using transfer learning in Keras to analyze Chest X-rays. I am going to choose one of the high performing Keras applications. https://keras.io/api/applications/ How should I choose ...
0 votes
0 answers
7 views

Different Kernel Initializers in my prediction layer with Transfer Learning could affect performance?

So I have this model right here and the task is to classify 3 labels.: ...
0 votes
0 answers
24 views

How to do transfer learning from word2vec to time series prediction?

The problem goes as follows. There are economic data consisting of output level for each industry at time $t$. But the data are multiple-period data, so this is time series data as well. I want to ...
  • 101
0 votes
0 answers
38 views

Marginal Probability Distribution of Feature space - meaning

I'm reading some literature on Transfer Learning in NLP, and this is one of the definitions that I came across in Pan & Yang (2010) Here is another definition from Sebastian Ruder which is a ...
0 votes
0 answers
36 views

Transfer learning: As simple as running trained models on new data?

So there's a domain of interest where the machine learning models are all specific to one entity. Let's call it a building. So there's a model made for every building. The literature in the domain all ...
  • 121
0 votes
0 answers
32 views

How to get attention weights from BERT

I am trying to make transfer learning in the encoder and decoder parts of the transformer. The encoder is a whole feature extraction part and there is the feature extraction part in the decoder too(I ...
  • 85
0 votes
0 answers
20 views

Transfer Learning transformer architecture

I would like to make transfer learning for transformer architecture. The input of the encoder and decoder must be word embeddings. So I wanted to use a pre-trained BERT model as a word embedder and ...
  • 85
0 votes
0 answers
20 views

Transfer learning on the same samples for progressively more specific classification

I have a classification task in which classes exist within a directed graph. That is a class may have subclasses which share an is-a relationship with their parent class. Now, I have a relatively ...
1 vote
1 answer
39 views

How to summarize very large neural networks?

I am doing a lot of work with transfer learning at the moment (using keras and tensorflow if that is relevant). I am having a lot of issues in sufficiently summarizing the very large models. This post:...
  • 11
1 vote
0 answers
13 views

Dealing with little available data: transfer learning

Suppose I seek to predict a certain numerical value, whereby the data set which contains the predetermined correct labels is only very small. However, I'm also provided a large data set with a label ...
  • 111
1 vote
1 answer
55 views

Pretrained vs. finetuned model

I have a doubt regarding terminology. When dealing with huggingface transformer models, I often read about "using pretrained models for classification" vs. "fine-tuning a pretrained ...
  • 289
0 votes
0 answers
6 views

Model transfer with limit to none label information

I have this problem I hope to get some help here. Say I have a type of product A whose measurements are X_A and an outcome property is ...
  • 21
1 vote
1 answer
128 views

Cannot achieve good result while Transfer Learning CIFAR-10 on ResNet50 - Keras

I'm trying to Transfer Learn ResNet50 for image classification of the CIFAR-10 dataset. It's stated in the original paper and also ResNet50 documentation on keras.io that the ResNet should have a ...
  • 11
0 votes
0 answers
33 views

Train model using Transfer Learning, the validation accuracy not learning

I am new to transfer learning. I am doing face mask detection in 4 classes(no facemask wearing, incorrect facemask wearing, correct facemask wearing, double mask wearing). My objective is to compare ...
0 votes
1 answer
462 views

model.fit vs model.evaluate gives different results?

The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or ...
  • 101
0 votes
0 answers
38 views

Finetune XLM-RoBERTa on Tensorflow

I want to finetune pre-trained XLM-RoBERTa from HuggingFace for Text classification. I have categorical data in English. I want to finetune model on Tensorflow-keras. Can anyone let me know how can I ...
0 votes
0 answers
276 views

Finetune XLM-RoBERTa on TF-keras for text classification

I am trying to finetune pre-trained XLM-RoBERTa on Tensorflow-keras. I am using dataset in English for text classification. I have used xlm-roberta-base tokenizer to tokenize the sentences. I am using ...
0 votes
0 answers
21 views

Baseline model and transfer learning

I've tried to find any guidance on using transfer learning when building baseline models for ML projects (CNN in my case) but found no clues on good practices in the matter. My logic says that no ...
0 votes
0 answers
30 views

How to stack Transfer Learning models in a Sequential

To make a nice architecture, I wanted to stack Transfer Learning models one over the other. The three models I wanted to stack were : VGG16 InceptionV3 Resnet50 So, I defined the three models as ...
0 votes
1 answer
2k views

No module named 'model'

I am trying to use the CoAtNet class in the following link CoAtNet Class from Github but I always have error while I am running the following lines: ...
  • 27
0 votes
0 answers
16 views

Update deep learning model with a new class and data for that class

I wanted to create a deep learning model with Keras capable of continually updating, but I do not want to retrain the model on the whole dataset, only on the new data. That is: if for example ...
0 votes
0 answers
25 views

Using Validation Set in Transfer Learning for Feature Extractor Preprocessor

I have a set of images of products. I am using transfer learning for images feature extraction in this way : I load a model (res-net, vgg) I add 2 dense layers, first one will be my features and the ...
  • 101
-1 votes
3 answers
1k views

Transfer learning on YOLOv5 for character and shape detection

The task is to detect rotated alphanumeric characters embedded on colored shapes. We will have an aerial view of the object (from a UAS: Unarmed Aerial System), something of this sort: (One Uppercase ...
  • 103
0 votes
0 answers
13 views

What should I visualize for humor detection model to gain some useful insight?

I was going through bunch (1,2,3) of humor detection paper. But most papers don't include any visualizations, say some graph related to model being trained. I was thinking to train some language ...
  • 137
0 votes
0 answers
64 views

Transfer Learning and Batch Normalization layers

I am trying to use ResNet152 ( pretrained on imagenet) to solve a certain classification task. According to Keras, one strategy for transfer learning is that we do not use the Resnet classifier and ...
  • 7
0 votes
0 answers
85 views

Image size for training transfer learning model for object detection purposes

I am trying to build transfer learning model to detect objects from video streams. There will be at least two or three different objects (classes) which are quite different from each other. The ...
  • 1
0 votes
1 answer
572 views

How to freeze certain layers in models obtained from keras.applications

I am currrently trainning to use transfer learning on ResNet152 obtained from Keras Applications: ...
  • 7
2 votes
0 answers
26 views

Transfer learning + Selective Fine Tuning

This is a two part question: Background: I have a rather small set (thousands) of medical images. I am using pretrained models off of kersa.applications without ...
  • 121
0 votes
0 answers
61 views

How can I build my voice speech-to-text model?

I found an instruction to build such kind of custom model on Azure. Prepare data for Custom Speech However, I would like to either fine-tune or transfer learning on Google Colaboratory or docker. In ...
  • 119
1 vote
2 answers
105 views

Distribution Shift vs Transfer Learning

Transfer learning (TL) is a research problem in machine learning (ML) that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem [1] ...
0 votes
1 answer
201 views

How does T5 model work on input and target data while transfer learning?

I am working on a project where I want the model to generate job description based on Role, Industry, Skills. I have trained my data and got the resultant output. I ...
  • 33
0 votes
0 answers
8 views

Preprocessing for Transfer Learning Model with an Inception Network

I am trying to build an image classification model using an Inception Network as the base. This is a simple binary classification model. My images are available in many smaller directories within one ...
0 votes
1 answer
2k views

What is the difference between batch_encode_plus() and encode_plus()

I am doing a project using T5 Transformer. I have read documentations related to T5 Transformer model. While using T5Tokenizer I am kind of confused with tokenizing my sentences. Can someone please ...
  • 33
0 votes
0 answers
22 views

Understanding how transfer learning happens in named entity recognition task

I was going through word embedding video in Andrew Ng's coursera course Sequence modeling. In this video, he gives following two examples: Sally Johnson is an orange farmer. Robert Lin is a durian ...
  • 185
0 votes
1 answer
120 views

How many layers of a pretrained model shoud be frozen?

I'm following an example of transfer learning where the blogger has frozen the first 20 layers of MobileNet. My question is , that is there any rule of thumb for how many layers should be frozen? ...
0 votes
0 answers
7 views

Does transfer learning works with more precise data?

I have a problem when I have 2 types of datasets for a classification task: a large one with which I intend to teach general rules of the field a smaller one, where quite similar instances occur but ...
  • 123
1 vote
1 answer
315 views

Is it possible to fine-tuning BERT by training it on multiple datasets? (Each dataset having it's own purpose)

BERT can be fine-tuned on a dataset for a specific task. Is it possible to fine-tune it on all these datasets for different tasks and then be utilized for these tasks instead of fine-tuning a BERT ...
0 votes
0 answers
58 views

Accuracy and Val accuracy not equal

I am working on Covid-19 detection with pre-trained models called Resnet50 and VGG19. However, during training the accuracy increases and the validation accuracy decreases? What is the issue in my ...
0 votes
1 answer
640 views

To freeze or not, batch normalisation in ResNet when transfer learning

I'm using a ResNet50 model pretrained on ImageNet, to do transfer learning, fitting an image classification task. The easy way of doing this is simply freezing the conv layers (or really all layers ...
1 vote
1 answer
119 views

What is the difference between Multi task learning and domain generalization

I was wondering about the differences between "multi-task learning" and "domain generalization". It seems to me that both of them are types of inductive transfer learning but I'm ...
0 votes
1 answer
346 views

Can I use a different image input size for transfer learning?

Most pre-trained CNN models accept a $224x224$ input size when they were trained. Can I use $256x256$ to get a higher accuracy?
  • 3
2 votes
1 answer
939 views

Fine-tuning pre-trained Word2Vec model with Gensim 4.0

With Gensim < 4.0, we can retrain a word2vec model using the following code: ...
  • 51
1 vote
2 answers
331 views

Why does my InceptionV3 model give a high training accuracy (99%), a high validation accuracy (95%+) but a very low testing accuracy (55%)?

Note: Please go through this in its entirety. My objective here is not just to get a high testing accuracy but to explain why it is so low in spite of validation accuracy being so high. I am a ...