I can't quite visualize how transfer learning of pre-trained word embeddings is useful in an NLP task( say named entity recognition ) . I'm studying Andrew NG's Sequence Models course and he seems to say if the training set for the target task is very less, then transfer learning of word embeddings is helpful in a way that unknown words in the training set can be handled in the application .
Let's consider the task of Named Entity recognition ,
My question is , what does the very small training set for the target task contain ? Are they word embeddings or sentences labeled with entities ?
Does he seem to suggest that if the training set is of just labeled sentences whose words have embeddings in the pre-trained model , then words which are not present in the training set but are closer to those already in the training set also get captured effectively in the application ?
Eg : Consider 'Orange' is in training set . But , 'Apple' is not .
So , in the sentences , ' I like Orange Juice ' and ' I like Apple Juice ' , Apple gets recognized as a fruit , even though it's not in the training set since it is closer to Orange .
Am I right in my assumption ? Or can someone please correct and explain to me if I am not ?