1
$\begingroup$

Transfer learning (TL) is a research problem in machine learning (ML) that focuses on storing knowledge gained while solving one problem and applying it to a different but related problem [1]

Distribution Shift The conditions under which the system was developed will differ from those in which we use the system. [2]

I consider there is no difference between distribution shift and dataset shift. But between transfer learning and distribution shift? What are the differences?

Can we say that transfer learning is an intended distribution shift?

Improve this question
$\endgroup$
3
  • $\begingroup$ Well, in transfer learning the model itself changes, too (although only partially), so the formal answer would be "no"; I can see your rationale though, and there are indeed some (superficial?) similarities. $\endgroup$
    – desertnaut
    Nov 3, 2021 at 17:19
  • $\begingroup$ Simply saying Transfer Learning relates to the shift of Ys, while Distribution (or Domain) shift relates to the shift of Xs. $\endgroup$
    – Anvar Kurmukov
    May 6, 2022 at 17:07
  • $\begingroup$ @AnvarKurmukov you are referring to the type of distribution shift, i don't see how it relates to the question $\endgroup$
    – Carlos Mougan
    May 7, 2022 at 11:49

2 Answers 2

Reset to default
1
$\begingroup$

Yes - One difference is between transfer learning and distribution shift is intention and knowledge of a different dataset.

There are many types of transfer learning. Sometimes models are trained on one dataset and applied to another dataset without additional training. This has to be the case when there are no labels on the second dataset. Other times models are trained on one dataset and then fined tuned on another dataset. This can be the case when the second data has labels.

Improve this answer
$\endgroup$
0
0
$\begingroup$

Actually, Transfer Learning has a broader definition. It includes distribution shift (covariate shift, sample bias etc..) as well. You could apply a TL method using a model which has been trained on a dataset having different distribution from the test set. Transfer learning includes also shift in the distribution of the labels or the fine tuning of a model trained in a different task.

Please check the following survey https://ieeexplore.ieee.org/document/5288526

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.