2
$\begingroup$

if my understanding is correct, in case of image classification and NLP, if I have a pre-trained model, to train on new data, I can reshape the data according to the pre-trained model. So there is no problem even if the new data is slightly different from the previous data. I am trying to use transfer learning for a regression problem. Consider I train a base model with 15 parameters and 1 million rows. I train a model. Now if I want to use this model for a similar problem case but I have only 14 parameters, one parameter is missing. Will the pre-trained model be of any use. Is there a way I can use transfer learning in such cases?

Thank you.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The question is if you can provide the trained model with the data it expects to provide a good output.

In your example, for the model to provide an output, you will have to provide 15 inputs. If you have only 14, can you put a value for input 15 that will make sense to the model? If feature 15 is a person's age, can you make an estimate? If you trained a model on 16x16 images (256 features) but now you have images that are 4x4 (16 features), you may be able to just provide 256 - 16 = 240 black pixels and the model may work fine. Or you could reformat your data to multiply each pixel into 4x4 patch, and get a 16x16 image that way.

What you cannot do, is simply reshape the input so that it's 15 columns wide. If you do that, the first example you provide to your model will contain 14 features from the first real example, plus feature number 1 from the next case. That way, the model is using data from two different examples to make a prediction and it's getting a value for feature 1 on the input node where it expects a value feature 15, and feature 1 and feature 15 can have very different meaning.

Improve this answer
$\endgroup$
3
  • $\begingroup$ okay, so I will have to estimate the missing parameter to use the trained model, is my understanding correct ? $\endgroup$
    – chink
    May 25, 2019 at 14:35
  • $\begingroup$ Probably. Is your data structured in the sense that you have 14 features that measure something about each example, such as the age, the height, and the number of times per year the person eats pizza? Or is it unstructured, like the pixels of an image? Assuming structured data, you need to fill the column you're missing somehow. Is it feasible to make an estimation? Does the model permit to leave this column blank? $\endgroup$
    – Paul
    May 25, 2019 at 14:40
  • 1
    $\begingroup$ its structured data, measuring different features like age, height etc. So I will have to make an estimation of the missing column. $\endgroup$
    – chink
    May 25, 2019 at 15:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.