Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [transfer-learning]

Transfer learning is the process of learning a set of characteristics from one data and applying this "knowledge" to another similar dataset (i.e. using the same model across datasets).

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Can transfer learning on VGG better align with Across the Spider-Verse styles for improved style transfer?

Our project is working on implementing style transfer using styles from Across the Spider-verse. We've gotten some good outputs so far, but were wondering if we could better align the VGG model to the ...
Geo C's user avatar
  • 1
1 vote
0 answers
13 views

Transfer Learning - GoogLeNet - Training Times || Loss not converging || Pytorch

Hi Community and thanks in advance for the help. I am working on transfer learning - specifically GoogLeNet model with the Food101 Dataset. Code is below. I think everything is in order from data ...
James's user avatar
  • 11
2 votes
0 answers
50 views

Transfer learning for tabular data

I wonder if transfer learning can be used in tabular data similarly to how it's used in neural networks for image recognition. My idea would be to train a "general" model and then "...
Dudelstein's user avatar
0 votes
0 answers
13 views

How to build a model where each data point has different levels of information?

Let’s say I want to predict the weight of a person given information about them; height & sex. Now, let’s say that that I have additional information about roughly 50% of the individuals included ...
the man's user avatar
  • 139
4 votes
0 answers
62 views

Why did I got opposite results of the original "How transferable are features in deep neural networks" paper?

I got tasked with reproducing the results of the influential "How transferable are features in deep neural networks?" paper in a DL class I'm taking (Full code). I got the exact opposite ...
OfirD's user avatar
  • 91
0 votes
0 answers
15 views

How can I change my input shape in the architecture for the cnn(transfer learning)?

I have already made a model and trained it, and then saved the model along with its weights. The input shape in that model is [900,300,1] which is [height,width,channel]. I want to use the same model ...
beschichtung346's user avatar
0 votes
0 answers
10 views

Does Keras cache activations if lower layers are frozen?

I've been experimenting a bit with gradually adding layers to a model, while freezing N layers and only leaving the N+1th layer trainable. In my mind, the time to train any layer should be roughly ...
John Smith's user avatar
1 vote
0 answers
59 views

How can I use Time-GPT for pretraining my model

I am mentioning Time-GPT here as a placeholder example. It can be any pretrained model. Suppose I have a dataset that requires some time series prediction. How can I leverage a well-trained model and ...
Mohammad Mosiur's user avatar
0 votes
0 answers
10 views

Help with a MaNet finetuning (binary semantic segmentation task)

Introduction: I am currently working on a computer vision problem, I have satellite images and I have to detect a particular archeological structure (Tell). I have access to the previously made ...
Alessandro Pistola's user avatar
0 votes
2 answers
44 views

Is a Random Forest Capable of Learning and Predicting Numerical Trends in Panel Data?

In a panel data set consisting of exponential functions, each indexed by an integer i ranging from 0 to 100. The exponential function is defined as f(i, t) = A(i) * e^(-r(i) * t), where A(i) is the ...
Emad Ezzeldin's user avatar
0 votes
0 answers
17 views

How to perform inference on a finetuned falcon 7b model fine tuned on open assistant dataset

I finetuned a falcon 7b model on the open assistant dataset using the official colab notebook provided by huggingface at https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=...
Saket Vempaty's user avatar
0 votes
0 answers
28 views

Application of transformers to tabular data

Is anyone using transformer based models for tabular data in real data science jobs as of 2024 I mean models like Tabnet Tabtransformer ARM-net SAINT FT-Transformer Non-Parametric Transformers I got ...
Ggjj11's user avatar
  • 216
1 vote
1 answer
79 views

How to choose (mean, std) for normalization in transfer learning?

I'm working on transfer learning based on ResNet50 pretrained model. Basically, I remove the last layer of ResNet50 and add new head layer. Then I train the model with my image dataset. Obviously, my ...
Morgan Cheng's user avatar
0 votes
1 answer
80 views

VGG16 Transfer Learning for image binary classification - suspected overfitting

I'm using VGG16 for transfer learning on a binary image classification task about human posture. The sample totaled about 2,000 images, with about 900 and 1,000 images in each category, respectively. ...
MaxHo's user avatar
  • 3
0 votes
1 answer
193 views

What is the "Extract" token and how is the final Linear layer applied in GPT?

In the manuscript of GPT, the authors have given the following image: Questions: What is the final "Extract" (token?)? Is it the "END" token? How is the final linear layer ...
figs_and_nuts's user avatar
1 vote
1 answer
955 views

issue loading the ckpt file PytorchStreamReader failed reading zip archive: failed finding central directory

I am trying to load the ckpt file and getting error PytorchStreamReader failed reading zip archive: failed finding central directory Here is the code ...
Shruti's user avatar
  • 35
0 votes
0 answers
84 views

Different generated patches from original image using vision transformer (ViT)

I am using ViT for image classification, I scaled images in range of [-1,1], and I also padded images. Then, I used the following code to see the original image and generated patches, but the output ...
Zara Nz's user avatar
0 votes
0 answers
28 views

Why use sliding window input features in deep learning?

I was reading through the DNABERT paper and found that their input features were k-mers. This is equivalent to using rolling/sliding window features in the other common family of sequential problem, ...
Avatrin's user avatar
  • 131
0 votes
1 answer
179 views

Combine two separate models created via Transfer Learning?

Suppose have two 'image classification' models created by transfer learning on the same base model[1], each producing a different set of labels/classes. Trained at different times, with different ...
barryhunter's user avatar
0 votes
0 answers
14 views

How to analyze social media data to see its impact on a game's sales

I work for a console gaming giant. We forecasted the sales for a RPG game that was to be released few months back. But the actual sales was twice the forecast. This compelled the developers to ...
Ritik P. Nayak's user avatar
0 votes
0 answers
45 views

Prevent Overfitting in Transfer Learning with small data

I have built a feed forward neural network to predict heat pumps energy consumption. Now, i want to use this model as a domain for other heat pumps via transfer learning. I want to simulate the case ...
MBC_222's user avatar
0 votes
0 answers
27 views

Fine-tuning Pretrained Models for Web DOM Interaction Prediction Task

I am currently working on a side project that involves predicting changes to a webpage's DOM based on user interactions. The idea is to input the initial DOM state and a user interaction, and predict ...
DonnyDato's user avatar
0 votes
1 answer
220 views

Better results when adding a dropout layer before a single layer classifier - counter intuitive result

I am working on an multi-class image classification problem (with 9 classes), i am using a pretrained DenseNet121 (on ImageNet), i'm using Keras. i am using densenet as a feature extractor, with a ...
user062's user avatar
7 votes
3 answers
4k views

Further Training a pre-trained LLM

My goal is to use the general knowledge and language understanding of a pre-trained LLM and to continue training on a smaller domain specific corpus to improve the model's knowledge on the domain. ...
Arthuro's user avatar
  • 101
0 votes
0 answers
223 views

Fine-Tuning / Transfer learning results in worse performance

My task is creating a model for QA-purposes. I have only ~200 samples on a specific domain of questions. Using a pretrained like DeBERTa without any further changes results in f1 scores of ~35%. To ...
max245905's user avatar
0 votes
1 answer
68 views

Can I add a new output class to a decoder and train only the final layer?

I am wondering how to approach a project, where I would like to increase the number of output classes of an already trained network. I have very good reason to believe that the model has already ...
aqua's user avatar
  • 123
1 vote
1 answer
51 views

Speech to Text for Unsupported Language

I'm working on a project to plug the good old speech recognition in my app. However, I wish to do it in my country's dialect which is not supported by the major APIs like Azure, AWS, etc. My country's ...
GargantuanChad's user avatar
0 votes
0 answers
514 views

Model.train( ) returns automatically in YOLO v8

I'm trying to find the best sets of hyperparameters for my YOLO V8 model on my custom dataset with RayTune. I wanted to train the model with model.train() and return some of the evaluation metrics, ...
abdus samad's user avatar
0 votes
1 answer
82 views

Single model or multiple models for predicting at each level in a multi-level classification problem

Given a flat structured data with features that can be considered hierarchical, where each feature is at a different level (e.g., Brand at the top level, Product, Color, and Size at different levels), ...
Kedharnath Kb's user avatar
0 votes
1 answer
108 views

Not able to understand Transfer Learning with Vgg16

So, I have to work with Vgg16 in my semester group project, and was following this to do transfer learning. I don't understand CNN much, but am learning currently. The very first problem was that ...
royal_awake's user avatar
1 vote
1 answer
150 views

Training a CNN in production on new data

How should I approach training a convolutional neural network in production on new data when I detect model performance degradation due to data or concept drift? Resources like this one and this one ...
Fijoy Vadakkumpadan's user avatar
1 vote
1 answer
450 views

CNN good results on train and test, bad results on real world data

I'm trying to build a neural network for an age detection task. Here some details : Dataset: I am using the "facial age" Kaggle dataset and the "UTKFace" dataset for a total of ...
Daniel_Fortesque's user avatar
0 votes
0 answers
1k views

Invalid argument error: logits and labels must be broadcastable

Whenever I try to execute this vgg16 code I get an error like this: ...
Pratik Pradhan's user avatar
0 votes
1 answer
334 views

How to increase FER2013 dataset validation_accuracy for only 3 classes i.e, happy,sad,neutral?

I am building a face emotion detection model using vgg16. Using FER2013 dataset for 7 classes i am getting= train_accuracy=97%, validation_accuracy=90%. but when i tried with 3 classes i.e, happy,sad,...
Pratik Pradhan's user avatar
0 votes
1 answer
57 views

Hello guys, is dimension reduction required for tensorflow? [closed]

I am working on face emotion detection using FER2013 dataset using tensorflow and vgg16 model. I am applying t-sne to my training dataset for dimensionality reduction. My question is that "is ...
Pratik Pradhan's user avatar
0 votes
2 answers
544 views

Autoencoder vs Pre-trained network for feature extraction

I wanted to know if anyone has any sort of guidance on what is better for image classification on a lot of classes (about 400) with a small amount of samples per class (around 20), for relatively big ...
MrStealYourFrog's user avatar
0 votes
1 answer
261 views

Weird consequence of not freezing layers in Neural Network

I was researching about "why are we freezing layers" and I came across the answer says "to not lose the information of pre-trained model" But; we are just freezing early layers (I ...
canP's user avatar
  • 101
0 votes
0 answers
41 views

Transfer learning: As simple as running trained models on new data?

So there's a domain of interest where the machine learning models are all specific to one entity. Let's call it a building. So there's a model made for every building. The literature in the domain all ...
There's user avatar
  • 121
1 vote
1 answer
67 views

How to summarize very large neural networks?

I am doing a lot of work with transfer learning at the moment (using keras and tensorflow if that is relevant). I am having a lot of issues in sufficiently summarizing the very large models. This post:...
Oskar's user avatar
  • 11
1 vote
0 answers
13 views

Dealing with little available data: transfer learning

Suppose I seek to predict a certain numerical value, whereby the data set which contains the predetermined correct labels is only very small. However, I'm also provided a large data set with a label ...
Richard's user avatar
  • 111
1 vote
1 answer
234 views

Pretrained vs. finetuned model

I have a doubt regarding terminology. When dealing with huggingface transformer models, I often read about "using pretrained models for classification" vs. "fine-tuning a pretrained ...
lazarea's user avatar
  • 299
1 vote
1 answer
392 views

Cannot achieve good result while Transfer Learning CIFAR-10 on ResNet50 - Keras

I'm trying to Transfer Learn ResNet50 for image classification of the CIFAR-10 dataset. It's stated in the original paper and also ResNet50 documentation on keras.io that the ResNet should have a ...
Mahdi's user avatar
  • 11
1 vote
2 answers
2k views

model.fit vs model.evaluate gives different results?

The following is a small snippet of the code, but I'm trying to understand the results of model.fit with train and test dataset vs the model.evaluate results. I'm not sure if they do not match up or ...
ptn77's user avatar
  • 111
0 votes
1 answer
4k views

No module named 'model'

I am trying to use the CoAtNet class in the following link CoAtNet Class from Github but I always have error while I am running the following lines: ...
Beba.S's user avatar
  • 37
0 votes
0 answers
68 views

Using Validation Set in Transfer Learning for Feature Extractor Preprocessor

I have a set of images of products. I am using transfer learning for images feature extraction in this way : I load a model (res-net, vgg) I add 2 dense layers, first one will be my features and the ...
W.314's user avatar
  • 101
-1 votes
3 answers
2k views

Transfer learning on YOLOv5 for character and shape detection

The task is to detect rotated alphanumeric characters embedded on colored shapes. We will have an aerial view of the object (from a UAS: Unarmed Aerial System), something of this sort: (One Uppercase ...
satan 29's user avatar
  • 103
0 votes
1 answer
3k views

How to freeze certain layers in models obtained from keras.applications

I am currrently trainning to use transfer learning on ResNet152 obtained from Keras Applications: ...
AAA's user avatar
  • 7
2 votes
0 answers
42 views

Transfer learning + Selective Fine Tuning

This is a two part question: Background: I have a rather small set (thousands) of medical images. I am using pretrained models off of kersa.applications without ...
Ali's user avatar
  • 121
1 vote
2 answers
555 views

Distribution Shift vs Transfer Learning

Transfer learning (TL) is a research problem in machine learning (ML) that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem [1] ...
Carlos Mougan's user avatar
0 votes
1 answer
667 views

How does T5 model work on input and target data while transfer learning?

I am working on a project where I want the model to generate job description based on Role, Industry, Skills. I have trained my data and got the resultant output. I ...
10sha25's user avatar
  • 43