When can it be called transfer learning?

Question

A common definition of transfer learning is:

"Transfer learning is the improvement of learning in a new task through the transfer of knowledge from a related task that has already been learned."

— Chapter 11: Transfer Learning, Handbook of Research on Machine Learning Applications, 2009.

This raises the question, when a task can be termed "related". Let's assume a neural networks is trained to estimate house prices for american houses. Could it be called transfer learning, if I retrain/finetune the model to estimate european house prices? Is it still the same task or can it be considered a separate task?

Answer 1

Good question but rather besides the point.

What I mean is that even though estimating european house prices can be considered the same task (as estimating american house prices), nevertheless in this example the NN was re-purposed using transfer learning methods from another task.

Whether one would like to call the european house prices as same or related task, is not so much the point here (which can become pedantic), but rather if transfer learning methods were used and how succesfull they were.

IMHO

Answer 2

Generally, the ANNs, as you probably already know, generate representations in a hierarchical manner. Having said that, the network captures very basic patterns of information at early layers which are then combined in a more abstract way at later stages. Build upon this, and although I am not a house market specialist, I would assume that the relations captured at the early layers are generic and hold to some extent to all house markets.

To conclude, I would assume that even if two tasks are seemingly unrelated, using transfer learning might be beneficial at least at extracting early level representations. For example if I had a trained fruit detector, I would use that model to initialize the weights of a car detection model in hope that the fruit detector is capable of generating some low level features at its early stages such as edges, circles etc.

Answer 3

When you are using a different base model for use in your own model. Generally, taken model is trained on a huge dataset.