Questions tagged [transfer-learning]
Transfer learning is the process of learning a set of characteristics from one data and applying this "knowledge" to another similar dataset (i.e. using the same model across datasets).
196
questions
1
vote
0
answers
12
views
Transfer learning for tabular data
I wonder if transfer learning can be used in tabular data similarly to how it's used in neural networks for image recognition. My idea would be to train a "general" model and then "...
- 125
0
votes
0
answers
13
views
How to build a model where each data point has different levels of information?
Let’s say I want to predict the weight of a person given information about them; height & sex.
Now, let’s say that that I have additional information about roughly 50% of the individuals included ...
- 131
4
votes
0
answers
51
views
+50
Why did I got opposite results of the original "How transferable are features in deep neural networks" paper?
I got tasked with reproducing the results of the influential "How transferable are features in deep neural networks?" paper in a DL class I'm taking (Full code).
I got the exact opposite ...
- 91
0
votes
0
answers
13
views
How can I change my input shape in the architecture for the cnn(transfer learning)?
I have already made a model and trained it, and then saved the model along with its weights. The input shape in that model is [900,300,1] which is [height,width,channel].
I want to use the same model ...
0
votes
0
answers
8
views
Does Keras cache activations if lower layers are frozen?
I've been experimenting a bit with gradually adding layers to a model, while freezing N layers and only leaving the N+1th layer trainable. In my mind, the time to train any layer should be roughly ...
- 101
1
vote
0
answers
51
views
How can I use Time-GPT for pretraining my model
I am mentioning Time-GPT here as a placeholder example. It can be any pretrained model.
Suppose I have a dataset that requires some time series prediction. How can I leverage a well-trained model and ...
0
votes
0
answers
10
views
Help with a MaNet finetuning (binary semantic segmentation task)
Introduction: I am currently working on a computer vision problem, I have satellite images and I have to detect a particular archeological structure (Tell). I have access to the previously made ...
0
votes
2
answers
37
views
Is a Random Forest Capable of Learning and Predicting Numerical Trends in Panel Data?
In a panel data set consisting of exponential functions, each indexed by an integer i ranging from 0 to 100. The exponential function is defined as f(i, t) = A(i) * e^(-r(i) * t), where A(i) is the ...
0
votes
0
answers
17
views
How to perform inference on a finetuned falcon 7b model fine tuned on open assistant dataset
I finetuned a falcon 7b model on the open assistant dataset using the official colab notebook provided by huggingface at https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=...
0
votes
0
answers
24
views
Application of transformers to tabular data
Is anyone using transformer based models for tabular data in real data science jobs as of 2024
I mean models like
Tabnet
Tabtransformer
ARM-net
SAINT
FT-Transformer
Non-Parametric Transformers
I got ...
- 216
1
vote
1
answer
59
views
How to choose (mean, std) for normalization in transfer learning?
I'm working on transfer learning based on ResNet50 pretrained model. Basically, I remove the last layer of ResNet50 and add new head layer. Then I train the model with my image dataset.
Obviously, my ...
- 111
0
votes
1
answer
67
views
VGG16 Transfer Learning for image binary classification - suspected overfitting
I'm using VGG16 for transfer learning on a binary image classification task about human posture. The sample totaled about 2,000 images, with about 900 and 1,000 images in each category, respectively. ...
- 3
0
votes
1
answer
164
views
What is the "Extract" token and how is the final Linear layer applied in GPT?
In the manuscript of GPT, the authors have given the following image:
Questions:
What is the final "Extract" (token?)? Is it the "END" token?
How is the final linear layer ...
- 833
1
vote
1
answer
791
views
issue loading the ckpt file PytorchStreamReader failed reading zip archive: failed finding central directory
I am trying to load the ckpt file and getting error
PytorchStreamReader failed reading zip archive: failed finding central directory
Here is the code
...
- 35
0
votes
0
answers
84
views
Different generated patches from original image using vision transformer (ViT)
I am using ViT for image classification, I scaled images in range of [-1,1], and I also padded images. Then, I used the following code to see the original image and generated patches, but the output ...
- 1
0
votes
0
answers
28
views
Why use sliding window input features in deep learning?
I was reading through the DNABERT paper and found that their input features were k-mers. This is equivalent to using rolling/sliding window features in the other common family of sequential problem, ...
- 131
0
votes
1
answer
152
views
Combine two separate models created via Transfer Learning?
Suppose have two 'image classification' models created by transfer learning on the same base model[1], each producing a different set of labels/classes. Trained at different times, with different ...
- 171
0
votes
0
answers
14
views
How to analyze social media data to see its impact on a game's sales
I work for a console gaming giant. We forecasted the sales for a RPG game that was to be released few months back. But the actual sales was twice the forecast. This compelled the developers to ...
- 113
0
votes
0
answers
44
views
Prevent Overfitting in Transfer Learning with small data
I have built a feed forward neural network to predict heat pumps energy consumption. Now, i want to use this model as a domain for other heat pumps via transfer learning.
I want to simulate the case ...
- 1
0
votes
0
answers
26
views
Fine-tuning Pretrained Models for Web DOM Interaction Prediction Task
I am currently working on a side project that involves predicting changes to a webpage's DOM based on user interactions. The idea is to input the initial DOM state and a user interaction, and predict ...
0
votes
1
answer
141
views
Better results when adding a dropout layer before a single layer classifier - counter intuitive result
I am working on an multi-class image classification problem (with 9 classes), i am using a pretrained DenseNet121 (on ImageNet), i'm using Keras.
i am using densenet as a feature extractor, with a ...
- 3
6
votes
3
answers
4k
views
Further Training a pre-trained LLM
My goal is to use the general knowledge and language understanding of a pre-trained LLM and to continue training on a smaller domain specific corpus to improve the model's knowledge on the domain. ...
- 91
0
votes
0
answers
193
views
Fine-Tuning / Transfer learning results in worse performance
My task is creating a model for QA-purposes.
I have only ~200 samples on a specific domain of questions.
Using a pretrained like DeBERTa without any further changes results in f1 scores of ~35%.
To ...
0
votes
1
answer
63
views
Can I add a new output class to a decoder and train only the final layer?
I am wondering how to approach a project, where I would like to increase the number of output classes of an already trained network. I have very good reason to believe that the model has already ...
- 123
1
vote
1
answer
48
views
Speech to Text for Unsupported Language
I'm working on a project to plug the good old speech recognition in my app. However, I wish to do it in my country's dialect which is not supported by the major APIs like Azure, AWS, etc.
My country's ...
0
votes
0
answers
490
views
Model.train( ) returns automatically in YOLO v8
I'm trying to find the best sets of hyperparameters for my YOLO V8 model on my custom dataset with RayTune. I wanted to train the model with model.train() and return some of the evaluation metrics, ...
0
votes
0
answers
58
views
The embedding output of bert
I want to get the embedding matrix of the Bert model (the input before the first block layers) to feed it into another architecture. I really appreciate it if you help me with that. Thanks
- 3
0
votes
1
answer
82
views
Single model or multiple models for predicting at each level in a multi-level classification problem
Given a flat structured data with features that can be considered hierarchical, where each feature is at a different level (e.g., Brand at the top level, Product, Color, and Size at different levels), ...
0
votes
1
answer
102
views
Not able to understand Transfer Learning with Vgg16
So, I have to work with Vgg16 in my semester group project, and was following this to do transfer learning.
I don't understand CNN much, but am learning currently.
The very first problem was that ...
1
vote
1
answer
120
views
Training a CNN in production on new data
How should I approach training a convolutional neural network in production on new data when I detect model performance degradation due to data or concept drift? Resources like this one and this one ...
1
vote
1
answer
378
views
CNN good results on train and test, bad results on real world data
I'm trying to build a neural network for an age detection task.
Here some details :
Dataset: I am using the "facial age" Kaggle dataset and the "UTKFace" dataset for a total of ...
0
votes
0
answers
1k
views
Invalid argument error: logits and labels must be broadcastable
Whenever I try to execute this vgg16 code I get an error like this:
...
0
votes
1
answer
322
views
How to increase FER2013 dataset validation_accuracy for only 3 classes i.e, happy,sad,neutral?
I am building a face emotion detection model using vgg16.
Using FER2013 dataset
for 7 classes i am getting=
train_accuracy=97%, validation_accuracy=90%.
but when i tried with 3 classes i.e, happy,sad,...
0
votes
1
answer
57
views
Hello guys, is dimension reduction required for tensorflow? [closed]
I am working on face emotion detection using FER2013 dataset using tensorflow and vgg16 model.
I am applying t-sne to my training dataset for dimensionality reduction.
My question is that "is ...
0
votes
2
answers
483
views
Autoencoder vs Pre-trained network for feature extraction
I wanted to know if anyone has any sort of guidance on what is better for image classification on a lot of classes (about 400) with a small amount of samples per class (around 20), for relatively big ...
0
votes
1
answer
246
views
Weird consequence of not freezing layers in Neural Network
I was researching about "why are we freezing layers" and I came across the answer says "to not lose the information of pre-trained model" But; we are just freezing early layers (I ...
- 101
0
votes
0
answers
41
views
Transfer learning: As simple as running trained models on new data?
So there's a domain of interest where the machine learning models are all specific to one entity. Let's call it a building. So there's a model made for every building. The literature in the domain all ...
- 121
1
vote
1
answer
67
views
How to summarize very large neural networks?
I am doing a lot of work with transfer learning at the moment (using keras and tensorflow if that is relevant). I am having a lot of issues in sufficiently summarizing the very large models.
This post:...
- 11
1
vote
0
answers
13
views
Dealing with little available data: transfer learning
Suppose I seek to predict a certain numerical value, whereby the data set which contains the predetermined correct labels is only very small. However, I'm also provided a large data set with a label ...
- 111
1
vote
1
answer
224
views
Pretrained vs. finetuned model
I have a doubt regarding terminology. When dealing with huggingface transformer models, I often read about "using pretrained models for classification" vs. "fine-tuning a pretrained ...
- 289
1
vote
1
answer
387
views
Cannot achieve good result while Transfer Learning CIFAR-10 on ResNet50 - Keras
I'm trying to Transfer Learn ResNet50 for image classification of the CIFAR-10 dataset.
It's stated in the original paper and also ResNet50 documentation on keras.io that the ResNet should have a ...
- 11
1
vote
2
answers
2k
views
model.fit vs model.evaluate gives different results?
The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or ...
- 111
0
votes
1
answer
4k
views
No module named 'model'
I am trying to use the CoAtNet class in the following link
CoAtNet Class from Github
but I always have error while I am running the following lines:
...
- 37
0
votes
0
answers
67
views
Using Validation Set in Transfer Learning for Feature Extractor Preprocessor
I have a set of images of products. I am using transfer learning for images feature extraction in this way :
I load a model (res-net, vgg)
I add 2 dense layers, first one will be my features and the ...
- 101
-1
votes
3
answers
2k
views
Transfer learning on YOLOv5 for character and shape detection
The task is to detect rotated alphanumeric characters embedded on colored shapes. We will have an aerial view of the object (from a UAS: Unarmed Aerial System), something of this sort:
(One Uppercase ...
- 103
0
votes
1
answer
3k
views
How to freeze certain layers in models obtained from keras.applications
I am currrently trainning to use transfer learning on ResNet152 obtained from Keras Applications:
...
- 7
2
votes
0
answers
42
views
Transfer learning + Selective Fine Tuning
This is a two part question:
Background:
I have a rather small set (thousands) of medical images. I am using pretrained models off of kersa.applications without ...
- 121
1
vote
2
answers
525
views
Distribution Shift vs Transfer Learning
Transfer learning (TL) is a research problem in machine learning (ML) that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem [1]
...
- 6,252
0
votes
1
answer
649
views
How does T5 model work on input and target data while transfer learning?
I am working on a project where I want the model to generate job description based on Role, Industry, Skills. I have trained my data and got the resultant output.
I ...
- 43
1
vote
1
answer
8k
views
What is the difference between batch_encode_plus() and encode_plus()
I am doing a project using T5 Transformer. I have read documentations related to T5 Transformer model. While using T5Tokenizer I am kind of confused with tokenizing my sentences.
Can someone please ...
- 43