Questions tagged [transfer-learning]

Transfer learning is the process of learning a set of characteristics from one data and applying this "knowledge" to another similar dataset (i.e. using the same model across datasets).

Filter by
Sorted by
Tagged with
11 votes
3 answers
14k views

What are the consequences of not freezing layers in transfer learning?

I am trying to fine tune some code from a Kaggle kernel. The model uses pretrained VGG16 weights (via 'imagenet') for transfer learning. However, I notice there is no layer freezing of layers as is ...
user avatar
  • 327
9 votes
1 answer
819 views

How to arrange the dataset/images for CNN+LSTM

I am working on an image classification problem using Transfer Learning with Resnet50 as base model (in Keras) (For example Class A and Class B). There is a time factor involved in this ...
user avatar
  • 1,401
7 votes
3 answers
5k views

Is there any proven disadvantage of transfer learning for CNNs?

Suppose I know that I want to use a ResNet-101 architecture for my specific problem. There are ReseNet-101 models trained on ImageNet. Is there any disadvantage of using those pre-trained models and ...
user avatar
  • 17.5k
6 votes
2 answers
1k views

What is the purpose of untrainable weights in Keras

In this page it is mentioned that when trainable=false, the weight won't be updated and is used for optimization, too. But I still do not understand how it can be ...
user avatar
6 votes
2 answers
14k views

Default value of learning rate in adam optimizer - Keras

I am working on a image classification problem using Transfer Learning. Parameters used given below: Adam optimizer with learning rate - 0.0001 ...
user avatar
  • 1,401
6 votes
2 answers
1k views

Effect of Stop-Word Removal on Transformers for Text Classification

The domain here is essentially topic classification, so not necessarily a problem where stop-words have an impact on the analysis (as opposed to, say, sentiment analysis where structure can affect ...
user avatar
  • 113
6 votes
3 answers
2k views

meaning of fine-tuning in nlp task

There are two types of transfer learning model. One is feature extraction, where the weights of the pre-trained model are not changed while training on the actual task and other is the weights of the ...
user avatar
  • 501
6 votes
2 answers
13k views

Why is input preprocessing in VGG16 in Keras not 1/255.0

I am just trying to use pre-trained vgg16 to make prediction in Keras like this. ...
user avatar
5 votes
2 answers
5k views

Is Flatten() layer in keras necessary?

In CNN transfer learning, after applying convolution and pooling,is Flatten() layer necessary? I have seen an example where after removing top layer of a vgg16 ,first applied layer was ...
user avatar
  • 669
5 votes
2 answers
658 views

Feeding 3 consecutive video frames to a CNN to track a tennis ball

I want to use CNN transfer learning to track a tennis ball from TV broadcasts of tennis matches. I used VGG annotating tool annotation tool link (use version 1 of the tool for compatibility with ...
user avatar
5 votes
2 answers
147 views

Can neural networks be adapted without recreating them completely?

If I have, for example, a classification network which can tell if there is a dog or a cat in a picture, is it possible to adapt the network so it can also learn to detect a mouse? Without making a ...
user avatar
5 votes
2 answers
7k views

How to add a CNN layer on top of BERT?

I am just playing with bert (Bidirectional Encoder Representation from Transformer) Research Paper Suppose I want to add any other model or layers like Convolutional Neural Network layers (CNN), Non ...
user avatar
  • 51
5 votes
1 answer
448 views

How to use MNIST dataset to make predictions on similar images (colorblindness charts)?

I am trying to use the MNIST dataset to train a convolutional neural network to classify digits written in colorblindness charts. As some people have suggested, I have tried playing with the ...
user avatar
  • 151
4 votes
1 answer
2k views

Which is the fastest image pretrained model?

I had been working with pre-trained models and was just curious to know the fastest forward propagating model of all the computer vision pre-trained models. I have been trying to achieve faster ...
user avatar
  • 2,235
4 votes
2 answers
884 views

Why are results without Transfer Learning better than with Transfer Learning?

I developed a neural network for license plate recognition and used the EfficientNet architecture (https://keras.io/api/applications/efficientnet/#efficientnetb0-function) with and without pretrained ...
user avatar
  • 93
4 votes
1 answer
363 views

Make the CNN to say "I don't know"

I am currently working on an image classification problem. To ease the implementation I used transfer learning in Keras with ...
user avatar
  • 1,401
4 votes
1 answer
2k views

How to properly resize input images for transfer learning

I have to resize some images of different size to 224x224 before they can be passed as input for VGG19, and then apply ...
user avatar
  • 663
4 votes
1 answer
355 views

Transfer learning by concatenating the last classification layer

Before going into an obvious XY problem, I will explain you what I'm trying to do. I'm training a simple MobileNet pre-trained with Imagenet for multiclass classification. What I do is freeze all the ...
user avatar
  • 153
3 votes
3 answers
4k views

Overfitting while fine-tuning pre-trained transformer

Pretrained transformers (GPT2, Bert, XLNET) are popular and useful because of their transfer learning capabilities. Just as a reminder: The goal of Transfer learning is is to transfer knowledge gained ...
user avatar
3 votes
3 answers
3k views

Neural Network Model using Transfer Learning not learning

I am a beginner in Deep Learning and working on Road Crack detection using transfer learning. I am working on binary classification with two classes , crack and no crack. My distribution of two ...
user avatar
  • 31
3 votes
2 answers
1k views

PyTorch: How to use pytorch pretrained for single channel image

If I have to create a model in pytorch for images having only single channel. How can I transform my model to adopt to this new architecture without having the need to compromise the pre-trained ...
user avatar
  • 2,235
3 votes
1 answer
1k views

incremental learning vs transfer learning

Can anyone explain me how incremental learning differs from transfer learning with example? Also does Transfer learning limited to neural networks?
user avatar
3 votes
1 answer
4k views

Existing pre-trained NLP models to detect if a text input is a question

I would like to quickly filter text data into question and non-questions. Using the presence of question mark in the text is too crude. Are there any existing models I can use to aid me with my task?
user avatar
3 votes
2 answers
2k views

How does BERT deal with catastrophic forgetting?

In the ULMFit paper authors propose a strategy of gradual unfreezing in order to deal with catastrophic forgetting. That is, when the model starts be fine-tuned according to a downstream task, there ...
user avatar
3 votes
3 answers
251 views

Can CNNs detect features of different images?

In lecture, we talked about “parameter sharing” as a benefit of using convolutional networks. Which of the following statements about parameter sharing in ConvNets are true? (Check all that apply.) ...
user avatar
  • 31
3 votes
1 answer
45 views

What's the definition of retraining?

In transfer learning, we always use new data to retrain the pre-trained model. But, what is the specific and official definition of retraining? Or what papers mentioned this definition, in transfer ...
user avatar
  • 31
3 votes
1 answer
788 views

Error when trying Transfer Learning

I'm trying to train a model which is an extension of Google's Inception-V3 for the purpose of recognizing and classifying whether there is any pneumonia using x-ray images. I've used Tensorflow-Hub ...
user avatar
3 votes
1 answer
198 views

Transfer Learning Question: Extending the Functionality of a Multipose-Estimation Machine Learning Model?

I have experimented with a number of different machine learning models used for pose estimation. Most of them output a heatmap and offsets for the detected person(s) in the image. I really like the ...
user avatar
3 votes
2 answers
196 views

How to choose layer from which to unfreeze image classification model

I'm wondering what steps do you take to decide on the part of the model to unfreeze. Do you do multiple experiments? Since the use of GPU is expensive, you must have some guidelines. Note: I know ...
user avatar
  • 51
3 votes
0 answers
59 views

How to help neuronal network with an other model

I am working on an image classification problem, the input data normally is images to classify, but I thought latitude and longitude would play something on these satellite images. I sorted by ...
user avatar
  • 443
3 votes
1 answer
1k views

Keras bug NasNetlarge no top

I am trying to use NasNetlarge in Keras without the top but I cant get rid of the top: ...
user avatar
2 votes
3 answers
9k views

Over fitting in Transfer Learning with small dataset

I am using Transfer Learning to perform image classification. Base model used : Resnet50 using ImageNet dataset ...
user avatar
  • 1,401
2 votes
3 answers
2k views

Imbalanced Dataset (Transformers): How to Decide on Class Weights?

I'm using SimpleTranformers to train and evaluate a model. Since the dataset I am using is severely imbalanced, it is recommended that I assign weights to each ...
user avatar
  • 183
2 votes
2 answers
716 views

How to add a new category to a existing trained deep learning model?

i have trained my deep learning model initially with 5 classes now i want to add another class without training the whole model over again for those 5 classes. How can I do that?
user avatar
2 votes
2 answers
938 views

Why do I need pre-trained weights in transfer learning?

I am using a Mask-RCNN. I first chose the resnet50 backbone then downloaded COCO pre-trained weights. Why do I need pre-trained weights for transfer learning? The transfer learning approach is to ...
user avatar
  • 141
2 votes
2 answers
61 views

Does partial transfer learning require a lot of computer power?

I want to be sure my understanding of the problem is correct. I want to do image classification and current state of the art in my field is achieved by transfer learning with VGG16. Since image on ...
user avatar
  • 155
2 votes
3 answers
7k views

Keras load pre-trained weights. Shape mismatch

I have some trouble loading pre-trained weights with Keras. Let's say I have a keras model model and that my weights are stored at ...
user avatar
  • 238
2 votes
2 answers
1k views

Why is performance worse when my time-series data is not shuffled prior to a train/test split vs. when it is shuffled prior to the split?

We are running RandomForest model on a time-series data. The model is run in real time and is refit every time a new row is added. Since it is a timeseries data, we set shuffle to false while ...
user avatar
2 votes
1 answer
108 views

What tasks you train with one set of features and predict with another?

The most common scenario in supervised learning is to have data points with a set of features and train a model to make classification predictions afterward. Usually, for predictions to make sense ...
user avatar
2 votes
1 answer
459 views

Fine-tuning NLP models

In computer vision, if we don't have a large training set, a common method is to start with a pre-trained model for some related task (e.g., ImageNet) and fine-tune that model to solve our problem. ...
user avatar
  • 2,988
2 votes
1 answer
774 views

Transfer learning (on pre-trained inception net model) for multi label classification is giving similar probability for all labels

Number of labels: 1000, Dataset size: 200000 images Final probability for 1000 labels is in the range of 0.3 to 0.34. I was expecting large variation in probabilities. Can someone tell me what I am ...
user avatar
  • 205
2 votes
1 answer
103 views

Recursive Transfer Learning

Is there any methodology called Recursive Transfer Learning? For example, let's consider a situation that we have a lack of data while training a convolution neural network (CNN) for object detection ...
user avatar
2 votes
2 answers
1k views

Model not learning when using transfer learning

I am working on a personal project on image classification (two classes) and am trying to see how the MobileNet v2 structure would perform. While training the training accuracy is already quite high ...
user avatar
  • 5,809
2 votes
4 answers
97 views

Image Classification using Single Class Dataset using Transfer Learning [closed]

I only have around 1000 images of vehicle. I need to train a model that can identify if the image is vehicle or not-vehicle. I do not have a dataset for not-vehicle, as it could be anything besides ...
user avatar
2 votes
1 answer
76 views

Transfer learning for a regression problem

if my understanding is correct, in case of image classification and NLP, if I have a pre-trained model, to train on new data, I can reshape the data according to the pre-trained model. So there is no ...
user avatar
  • 515
2 votes
1 answer
149 views

Resource and useful tips on Transfer Learning in NLP

I have a few label data for training and testing a DNN. Main purpose of my work is to train a model which can do a binary classification of text. And for this purpose, I have around 3000 label data ...
user avatar
  • 21
2 votes
0 answers
24 views

Transfer learning + Selective Fine Tuning

This is a two part question: Background: I have a rather small set (thousands) of medical images. I am using pretrained models off of kersa.applications without ...
user avatar
  • 121
2 votes
1 answer
750 views

Fine-tuning pre-trained Word2Vec model with Gensim 4.0

With Gensim < 4.0, we can retrain a word2vec model using the following code: ...
user avatar
  • 51
2 votes
0 answers
211 views

TensorFlow - TFRecords load and transform images with bounding boxes

I'm trying to build a 'Car Classifier' using TensorFlow. I have 1000 labelled JPG images, 800x800, complete with bounding boxes and associated annotations.coco.json; split into train/validate/test ...
user avatar
2 votes
1 answer
161 views

Any useful tips on transfer learning for a text classification task

I am doing a supervised binary text classification task. I want to classify the texts from site A, site B, and site C. The in-domain performance looks OK for texts of each site. (92%-94% accuracy). ...
user avatar
  • 145