I have a few label data for training and testing a DNN. Main purpose of my work is to train a model which can do a binary classification of text. And for this purpose, I have around 3000 label data and 60000 unlabeled data available to me. My data type is related to instructions (like- open the door[label-1], give me a cup of water[label-1], give me money[label-0] etc.) In this case, I heard that Transferring knowledge from other models will help me a lot. Can anyone give me some useful resource for transfer learning in NLP domain?
I already did a few experiments. I used GLoVE as a pretrained embeddings. Then test it with my label data. But got around 70% accuracy. Also tried with embedding built using my own data (63k) and then train the model. Got 75% accuracy on the test data. My model architecture is given below-
Q1: I have a quick question will it be referred to as Transfer learning if I use GLOVE embeddings in model?
Any kind of help is welcomed. Even someone has other ideas for building a model without using transfer learning is welcomed.