Questions tagged [transfer-learning]

Transfer learning is the process of learning a set of characteristics from one data and applying this "knowledge" to another similar dataset (i.e. using the same model across datasets).

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

Teachable Machine audio Project

I am working on an audio use case where I already trained a basic AI-ML model on Teachable Machine. I would like to further utilized this trained model and perform transfer learning on my own dataset. ...
0
votes
0answers
5 views

How do pretrained models using SQUAD dataset work on an any other dataset?

I see in some Kaggle contests people have used models pretrained in SQUAD dataset for building QA systems for the dataset given in the contest. How does this work? How can a pretrained model in a ...
0
votes
0answers
4 views

Transfer learning on images with higher dynamic range

Is it possible to fine-tune a CNN-based model previously trained on images with 8 bits depth [0 ~ 2^8] to fit a 16 bits depth [0 ~ 2^16] images? if there is any research paper that confirm that, it ...
0
votes
0answers
9 views

How Pretraining part actually work in Wav2vec models? Which data is qualify to be the adequat for fine-tuning part the model of speech2text

Pretraining and fine-tuning the algorithm of wav2vec2.0, the new one using in FAcebookAI to do speech to text for low-resource language. I didn't actually get how the model does the pretraining part ...
0
votes
0answers
21 views

General question about transfer learning in time series classification

This paper (https://arxiv.org/abs/1811.01533) investigated the extent to which transfer learning improves the results of time series classifications. It turned out that it is better to use a source ...
0
votes
0answers
29 views

Accuracy and Val accuracy not equal

I am working on Covid-19 detection with pre-trained models called Resnet50 and VGG19. However, during training the accuracy increases and the validation accuracy decreases? What is the issue in my ...
0
votes
0answers
62 views

To freeze or not, batch normalisation in ResNet when transfer learning

I'm using a ResNet50 model pretrained on ImageNet, to do transfer learning, fitting an image classification task. The easy way of doing this is simply freezing the conv layers (or really all layers ...
1
vote
1answer
35 views

What is the difference between Multi task learning and domain generalization

I was wondering about the differences between "multi-task learning" and "domain generalization". It seems to me that both of them are types of inductive transfer learning but I'm ...
0
votes
1answer
44 views

Can I use a different image input size for transfer learning?

Most pre-trained CNN models accept a $224x224$ input size when they were trained. Can I use $256x256$ to get a higher accuracy?
0
votes
0answers
13 views

Empty model when using ResNet50 for transfer learning

I am working on transfer learning using ResNet50. I have a code which was working four months before in Google Colab but when I checked it now, it is no longer working. I am not sure whether this is ...
2
votes
0answers
124 views

Fine-tuning pre-trained Word2Vec model with Gensim 4.0

With Gensim < 4.0, we can retrain a word2vec model using the following code: ...
1
vote
0answers
29 views

Q. Why is my testing accuracy varying slightly with batch size (97.7% - 100%)?

As you will see, when batch size is set to 1, I'm consistently getting 97.7% testing accuracy for all 10 iterations. However, when batch size is set to 64, I'm getting a testing accuracy of 100% 7 out ...
1
vote
0answers
8 views

Volatile training loss chart on transfer learning mobile_netV1

Above is a graph of my latest attempt at transfer learning from a mobile_netV1 model already trained to 1 million steps, doing transfer learning with 50000 additional steps to my new dataset. The ...
0
votes
0answers
11 views

Transfer learning on siamese network with limited data

This may be a silly example, but it should be similar enough to my true research question without giving specifics. Let's say I have a pretrained Siamese neural network that tells you similarity ...
0
votes
0answers
13 views

Determine Transfer Learning Strategy for NER task

I worked on a Transfer Learning project in which I created a training dataset (labeled) and I used a pre-trained BERT model and fine-tuned it. The project was an NLP project in which I performed ...
1
vote
2answers
55 views

Why does my InceptionV3 model give a high training accuracy (99%), a high validation accuracy (95%+) but a very low testing accuracy (55%)?

Note: Please go through this in its entirety. My objective here is not just to get a high testing accuracy but to explain why it is so low in spite of validation accuracy being so high. I am a ...
0
votes
0answers
18 views

How to handle out of vocabulary word efficiently

I have a corpus which consists of some scientific keywords for example MKLS , SDEV all these are abbreviation of some methodology and they can be custom also based on business convenience. One way to ...
0
votes
0answers
16 views

Are imagenet weights same for all pretrained models?

I am new to transfer learning and I have one image classification problem, and I was thinking about training two pretrained model on TensorFlow: Inception and ...
0
votes
1answer
33 views

Transfer Learning or Custom Network?

I am learning Computer Vision and I was wondering if it's usually worth it to build a custom convolutional network from scratch (through trials and errors) or if using transfer learning with a popular ...
1
vote
1answer
22 views

Does transfer learning make sense for small neural networks with only one or two hidden layers?

I am testing transfer learning on rather small neural networks with only two hidden layers of 20 neurons on tabular data. None of my experiments yields any improvement over a basic neural network. Is ...
0
votes
0answers
50 views

Data To Text NLG for financial reports

I am working on a project where I want to replace a template-based approach for financial reporting with an end-to-end approach with NLG. The template-based approach takes as input some financial data ...
0
votes
2answers
18 views

Is there wights of voice or audio for VGG or Inception?

I want to use VGG16 (or VGG19) for voice clustering task. I read some articles which suggest to use ...
1
vote
0answers
19 views

ImageNet vs CIFAR pre-trained models for biomedical image classification

I'm doing research on transfer learning for biomedical image classification, mainly skin lesion classification. From what I know, both CIFAR and ImageNet are pre-trained on natural images and not ...
1
vote
1answer
24 views

Pytorch: Starting with a high loss value, but the loss converged at the end. I dont know if the model could start with a loss > 100. Help!

I have been trying to attempt plant disease detection using transfer learning methods. I chose ResNet50 first. I also performed a baseline model which is a CNN model. In resnet50, I used cross entropy ...
0
votes
1answer
15 views

Using vgg16 or inception with wights equals to None

When using pre-trained models like vgg16 or inception, it seems that one of the benfits of using pre-trained model, is to save ...
0
votes
0answers
26 views

Is this a task of meta-learning or transfer learning?

I have a task that I am not able to identify if it is of transfer or meta learning. I want to know this, in order to ask help in solving it, because there are some parts that I have not understood. ...
0
votes
0answers
24 views

How can I use transfer learning to predict height given age in Japan, using a model developed with USA data?

Suppose I have a (training) set of $n$ observation $\{(Y_i^{(U)},X_i^{(U)})\}_{i=1}^n$ of age $X_i^{(U)}$ and height $Y_i^{(U)}$ from people in the USA. Now suppose I also have a (test) set of $m$ ...
0
votes
1answer
17 views

Storing and loading bottleneck features for transfer learning on large data sets (Keras)

I would like to apply transfer learning on a pretty large image data set in order to solve a classification problem. Currently I load a pre-trained net without the top layers, add my own top layers, ...
0
votes
0answers
12 views

Classification vs Regression Training

I'm working on a problem where I have images(600) and corresponding scores. I have used VGG19 pretrained architecture where I connected the last CNN layer of VGG19 with 3 new fully dense layer. As a ...
0
votes
0answers
26 views

Pre-trained models for classifying images with more than 3 input channels

I am working on an image classification problem. My input images have 19 channels. I tried building a CNN model that can handle such inputs. Now I want to learn transfer learning along with this. But ...
0
votes
0answers
30 views

is padding input images better than resizing image?

I am working with images and was thinking of pre-training VGG19 and EffecientNETB0 model on my dataset. However, a query:- Is ...
0
votes
1answer
24 views

Can the performance of a CNN be dependent on the train-test-val split random seed?

I am doing multi-class classification and comparing the effects of 2 image enhancement techniques (IET). IET 1 performs better than IET 2 at random seed x (for train-test-val split) IET 2 performs ...
1
vote
1answer
137 views

Can we fine-tune a model on the same dataset which it is pretrained on?

So I was reading this paper (about a use case of pretraining then self-training) which got me thinking - suppose I pre-train a model on a particular dataset, then fine-tune it again on the same ...
1
vote
1answer
32 views

How to find out what portions of an image is helping CNN to classify it

I am working on an image classification problem using Transfer learning. Right now, I am getting an accuracy of 75% on train data and 67% in test data. Now I want to understand what portions/parts of ...
0
votes
1answer
51 views

Why not using linear regression for finetuning the last layer of a neural network?

In transfer learning, often only the last layer of the network is retrained using gradient descent. However, the last layer of a common neural network performs only a linear transformation, so why do ...
0
votes
3answers
60 views

When can it be called transfer learning?

A common definition of transfer learning is: "Transfer learning is the improvement of learning in a new task through the transfer of knowledge from a related task that has already been learned.&...
1
vote
0answers
60 views

Pretrained model for spectrogram images

I'm working on a sound classification problem. For that, I'm converting the audio signals into spectrogram images and using transfer learning to classify them. Currently I'm using pretrained models ...
1
vote
1answer
154 views

Validation loss diverging away from the training loss

I used the XLNET for a sentiment classifier in determining whether a comment is positive or negative. I was able to get good results But when I plotted the validation and training losses I saw this ...
0
votes
0answers
16 views
1
vote
0answers
11 views

Latent space for cross domain numerical features

I would like to find the shared latent space between two set of features. I have source and target domain features already extracted from images. I have 4 set of feature vectors for normal and ...
0
votes
1answer
301 views

How to use BERT in seq2seq model? [closed]

I would like to use pretrained BERT as encoder of transformer model. The decoder has the same vocabulary as encoder and I am going to use shared embeddings. But I need ...
1
vote
1answer
51 views

Transfer Learning on Resnets/VGGs -- Validation accuracy can never be over 75%

I am trying to classify skin cancer images into two categories -- malignant and benign. Literatures suggest that using pre-trained resnet/vgg network achieves more than 90% accuracy. However, with my ...
1
vote
2answers
151 views

Would there be any reason to pretrain BERT on specific texts?

So the official BERT English model is trained on Wikipedia and BookCurpos (source). Now, for example, let's say I want to use BERT for Movies tag recommendation. Is there any reason for me to pretrain ...
0
votes
1answer
22 views

Which part should be frozen during transfer learning?

I want to use transfer learning and fine tuning and I need to decide which part of the original model will be used and which part will be frozen. I'm thinking about four possilbe cases: small/large ...
0
votes
0answers
33 views

Is it possible to use unlabeled text articles for summarization when fine tuning BERT?

I know that unlabeled data could be used in pre-training but if I want to do a fine tuning of unlabeled articles for summarization, is it mandatory that the articles are labeled with existing ...
0
votes
0answers
24 views

Any tips on transfer learning for a regression problem using 4D images as input?

I developed a CNN based on EfficientNet in order to predict the weight of piles of some materials in an image (the labels are the weights in kg and the input is RGBD tensors of the object). I have two ...
0
votes
0answers
36 views

List of Google T5 possible operations

I am trying to use the huggingface.co pre-trained model of Google T5 (https://huggingface.co/t5-base) for a variety of tasks. But I can`t find a list of many tasks it really supports and how to ...
0
votes
0answers
9 views

Differential Learning Rates To Train Parts of A Network Faster

So I've had a rather "out there" idea. I want to train a dense network on a regression problem based on tabular data but I'd also like it to incorporate image data. My idea was to use a CNN ...
2
votes
0answers
134 views

TensorFlow - TFRecords load and transform images with bounding boxes

I'm trying to build a 'Car Classifier' using TensorFlow. I have 1000 labelled JPG images, 800x800, complete with bounding boxes and associated annotations.coco.json; split into train/validate/test ...
0
votes
0answers
13 views

Transfer learning from great labelled time series data to one with low quality labelling

I have a source dataset containing outputs from a sensor per minute and have made extra effort to label them correctly for approx. 3 weeks. I trained CNN-BLSTM network on that dataset which classifies ...