Questions tagged [transfer-learning]

Transfer learning is the process of learning a set of characteristics from one data and applying this "knowledge" to another similar dataset (i.e. using the same model across datasets).

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Can we fine-tune a model on the same dataset which it is pretrained on?

So I was reading this paper (about a use case of pretraining then self-training) which got me thinking - suppose I pre-train a model on a particular dataset, then fine-tune it again on the same ...
0
votes
1answer
23 views

How to find out what portions of an image is helping CNN to classify it

I am working on an image classification problem using Transfer learning. Right now, I am getting an accuracy of 75% on train data and 67% in test data. Now I want to understand what portions/parts of ...
0
votes
1answer
21 views

Why not using linear regression for finetuning the last layer of a neural network?

In transfer learning, often only the last layer of the network is retrained using gradient descent. However, the last layer of a common neural network performs only a linear transformation, so why do ...
0
votes
3answers
47 views

When can it be called transfer learning?

A common definition of transfer learning is: "Transfer learning is the improvement of learning in a new task through the transfer of knowledge from a related task that has already been learned.&...
0
votes
0answers
16 views

Pretrained model for spectrogram images

I'm working on a sound classification problem. For that, I'm converting the audio signals into spectrogram images and using transfer learning to classify them. Currently I'm using pretrained models ...
1
vote
1answer
32 views

Validation loss diverging away from the training loss

I used the XLNET for a sentiment classifier in determining whether a comment is positive or negative. I was able to get good results But when I plotted the validation and training losses I saw this ...
0
votes
0answers
15 views
1
vote
0answers
9 views

Latent space for cross domain numerical features

I would like to find the shared latent space between two set of features. I have source and target domain features already extracted from images. I have 4 set of feature vectors for normal and ...
0
votes
1answer
39 views

How to use BERT in seq2seq model? [closed]

I would like to use pretrained BERT as encoder of transformer model. The decoder has the same vocabulary as encoder and I am going to use shared embeddings. But I need ...
1
vote
1answer
37 views

Transfer Learning on Resnets/VGGs — Validation accuracy can never be over 75%

I am trying to classify skin cancer images into two categories -- malignant and benign. Literatures suggest that using pre-trained resnet/vgg network achieves more than 90% accuracy. However, with my ...
1
vote
2answers
51 views

Would there be any reason to pretrain BERT on specific texts?

So the official BERT English model is trained on Wikipedia and BookCurpos (source). Now, for example, let's say I want to use BERT for Movies tag recommendation. Is there any reason for me to pretrain ...
0
votes
1answer
21 views

Which part should be frozen during transfer learning?

I want to use transfer learning and fine tuning and I need to decide which part of the original model will be used and which part will be frozen. I'm thinking about four possilbe cases: small/large ...
0
votes
0answers
23 views

Is it possible to use unlabeled text articles for summarization when fine tuning BERT?

I know that unlabeled data could be used in pre-training but if I want to do a fine tuning of unlabeled articles for summarization, is it mandatory that the articles are labeled with existing ...
0
votes
0answers
12 views

Any tips on transfer learning for a regression problem using 4D images as input?

I developed a CNN based on EfficientNet in order to predict the weight of piles of some materials in an image (the labels are the weights in kg and the input is RGBD tensors of the object). I have two ...
0
votes
0answers
26 views

List of Google T5 possible operations

I am trying to use the huggingface.co pre-trained model of Google T5 (https://huggingface.co/t5-base) for a variety of tasks. But I can`t find a list of many tasks it really supports and how to ...
0
votes
0answers
9 views

Differential Learning Rates To Train Parts of A Network Faster

So I've had a rather "out there" idea. I want to train a dense network on a regression problem based on tabular data but I'd also like it to incorporate image data. My idea was to use a CNN ...
2
votes
0answers
42 views

TensorFlow - TFRecords load and transform images with bounding boxes

I'm trying to build a 'Car Classifier' using TensorFlow. I have 1000 labelled JPG images, 800x800, complete with bounding boxes and associated annotations.coco.json; split into train/validate/test ...
0
votes
0answers
13 views

Transfer learning from great labelled time series data to one with low quality labelling

I have a source dataset containing outputs from a sensor per minute and have made extra effort to label them correctly for approx. 3 weeks. I trained CNN-BLSTM network on that dataset which classifies ...
0
votes
1answer
221 views

BERT uses WordPiece, RoBERTa uses BPE

In the original BERT paper, section 'A.2 Pre-training Procedure', it is mentioned: The LM masking is applied after WordPiece tokenization with a uniform masking rate of 15%, and no special ...
3
votes
2answers
244 views

Effect of Stop-Word Removal on Transformers for Text Classification

The domain here is essentially topic classification, so not necessarily a problem where stop-words have an impact on the analysis (as opposed to, say, sentiment analysis where structure can affect ...
1
vote
1answer
24 views

Is it possible to use a pretrained scikit learn model to make predictions on a dataset with different features (than those used during training)

Say we have a model trained on dataset A, which has a number of features, as usual. We then persist that model to disk and use it when we need to run inference (make predictions). Usually we run ...
0
votes
2answers
30 views

Logic behind pre-trained weights and transfer learning

I am not sure about the logic behind, how pre-trained weights actually make sense and translate into a new problem. To be more specific; for example in a object detection network, how would a model's ...
0
votes
1answer
36 views

Is applying pre-trained model on a different type of corpus called transfer learning?

I trained my classification model on corpus A and evaluated it on corpus B. I do it, because for corpus A I have a lot more labeled sentences than for B. Nature of sentences used in A is different ...
1
vote
1answer
171 views

model.predict() accuracy extremely low on training dataset

This question is similar to this. I'm new to ML, and I am trying to classify breast cancer histology images using EfficientNets with Transfer Learning. The dataset is small (400 images in total - ...
1
vote
1answer
205 views

What's the difference between transfer learning and feature extraction in CNN?

So from what i understand, transfer learning is the fact of training a model on a dataset where you have a lot of data, then keeping most of trained coefficients, and only re-training the last layer ...
1
vote
1answer
32 views

Transfer learning with many small datasets

Context I am working on a NLP-model that can classify documents into one of N categories. I have document data from a number of different customers. The document topics are similar across customers ...
2
votes
2answers
112 views

Can CNNs detect features of different images?

In lecture, we talked about “parameter sharing” as a benefit of using convolutional networks. Which of the following statements about parameter sharing in ConvNets are true? (Check all that apply.) ...
1
vote
2answers
40 views

Why does Transfer Learning works better on smaller datasets than on larger ones?

This question is not about the utility of Tranfer Learning compared with regular supervised learning. 1. Context I'm studying Health-Monitoring techniques, and I practice on the C-MAPSS dataset. The ...
1
vote
1answer
1k views

Overfitting while fine-tuning pre-trained transformer

Pretrained transformers (GPT2, Bert, XLNET) is popular and useful because of their transfer learning capabilities. Just to remind: The goal of Transfer learning is is to transfer knowledge gained from ...
1
vote
2answers
2k views

Is Flatten() layer in keras necessary?

In CNN transfer learning, after applying convolution and pooling,is Flatten() layer necessary? I have seen an example where after removing top layer of a vgg16 ,first applied layer was ...
0
votes
0answers
15 views

Trouble with loading pre-trained CNN weights without classification layers for different input dimension

I am trying to load pre-trained CNN weights but without classification (i.e. top) layers. Basically, I want to do exactly what tf.keras.applications.ResNet50 class ...
4
votes
2answers
457 views

Why are results without Transfer Learning better than with Transfer Learning?

I developed a neural network for license plate recognition and used the EfficientNet architecture (https://keras.io/api/applications/efficientnet/#efficientnetb0-function) with and without pretrained ...
0
votes
1answer
97 views

How to combine the features extracted from different CNN architectures? [closed]

I want to combine the features extracted from different CNN architectures into my fully connected layer. How to proceed?
1
vote
1answer
203 views

Understanding Transfer Learning of Word Embeddings

I can't quite visualize how transfer learning of pre-trained word embeddings is useful in an NLP task( say named entity recognition ) . I'm studying Andrew NG's Sequence Models course and he seems to ...
1
vote
2answers
544 views

Imbalanced Dataset (Transformers): How to Decide on Class Weights?

I'm using SimpleTranformers to train and evaluate a model. Since the dataset I am using is severely imbalanced, it is recommended that I assign weights to each ...
2
votes
1answer
93 views

Any useful tips on transfer learning for a text classification task

I am doing a supervised binary text classification task. I want to classify the texts from site A, site B, and site C. The in-domain performance looks OK for texts of each site. (92%-94% accuracy). ...
0
votes
0answers
106 views

weight decay in ResNet50

Can someone please guide for implementing weight decay in transfer learning approach? I want to regularize the pre-trained model ResNet50, where I'm fine-tuning the model for an image classification ...
0
votes
1answer
35 views

Pre-trained CNN model makes Poor Predictions on Test Images Dataset

I have tried using several a pretrained models (MobileNet) for multiclass predictions. There are 42 classes and the distributions of the images are even across the 42 classes. This is my code: ...
0
votes
0answers
23 views

Transfer learning by using vgg in pytorch

I am using vgg16 for image classification. I want to test my transfered model with the following code: ...
1
vote
2answers
215 views

How many layers should I replace in transfer learning CNN

I am designing a convolutional neural network that I believe requires transfer learning to function in practice. The network will be a character level CNN for text classification, more specifically, ...
0
votes
1answer
67 views

ELMo - How does the model transfer its learning/weights on new sentences

Word2vec and Glove embeddings have the same vector representation for every word in the corpus and does not take context into consideration. For eg: The dog does bark at people The bark of the tree ...
1
vote
0answers
38 views

Transfer learning with Keras for medical image classification

Good afternoon; I'm trying to do Transfer Learning from pre-trained model on imagenet to solve a classification task with Lung CT slices. These slices are stored in ...
1
vote
1answer
60 views

What is the theoretical differences of Multitask learning vs Fine tuning based transfer learning?

Suppose, I have the following scenarios. I have a bunch of fruits, i.e., apple, orange, and banana. I simply made a Multitask model, where my network first tell me which fruit it is, and then telling ...
0
votes
0answers
94 views

Why siamese network using ResNet50 architecture has worse results than network trained from beginning?

I am trying to build product recognition tool based on ResNet50 architecture as below ...
1
vote
1answer
47 views

Daily new data for my neural network, and I want transfer(?) learning

I made my neural network, it is pre-trained for 180 days of data. ...
0
votes
1answer
703 views

How to use fine tuning of BERT when i have unlabelled dataset of text documents?

I have gained a basic understanding of using BERT for various NLP/text mining tasks. When it comes to fine-tuning of BERT, I always see that fine-tuning is performed using some classification tasks. ...
0
votes
0answers
24 views

Transfer learning with different number of features

I have a newly collected dataset with 18 features, but since the study is just started, there only over 200 samples in this dataset. And for an old version of this study, we have over 8000 samples but ...
2
votes
1answer
745 views

Existing pre-trained NLP models to detect if a text input is a question

I would like to quickly filter text data into question and non-questions. Using the presence of question mark in the text is too crude. Are there any existing models I can use to aid me with my task?
1
vote
0answers
13 views

From Patch-based Classifier to Full Image classifier

I was wondering if it is feasible to train patch-based image classifier, due to small amount of data, and then use it in order to initialize training for full image classification, but this time on ...
0
votes
0answers
11 views

Are neural networks modular? An example

BACKGROUND Consider a supervised problem which is based on two scalar features (1) and (2) as well as a third, "time-dependent", feature consisting of a sequence of five values (3)-(7). For ...