Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Usually the neural network training has at least 2 steps:
What is the same of 2-nd step, is it Fine-tuning or Transfer-learning? And what is the different between Fine-tuning and Transfer-learning?
Generally, I would refer to this as transfer learning or network adaptation. That is, taking a network that has learned useful features from one domain and adapting that network and its developed features to another domain.
That said, there appear to be many sources that closely conflate fine tuning with transfer learning. Therefore, I would say the difference in terminology is primarily opinion-based and suggest closure of this question on those grounds.
Fine-tuning is the process in which the parameters of a trained model must be adjusted very precisely while we are trying to validate that model taking into account a small data set that does not belong to the train set.
That small validation data set comes from the same distribution as the data set used for the training of the model. The split of the available data to train and validation set is random.
Transfer Learning or Domain Adaptation is related to the difference in the distribution of the train and test set.
So it is something broader than Fine tuning, which means that we know a priori that the train and test come from different distribution and we are trying to tackle this problem with several techniques depending on the kind of difference, instead of just trying to adjust some parameters (usually we are doing this for reasons as preventing overfitting etc.)
