I am given a dataset that has free-form text and a category associated with it. There are 100 different categories and 3000 records for each category. The goal is build a multiclass classification model. I have created a simple neural network with 10,000 input features/words and the results were fairly good (~88%).
The issue I am facing is with an unlabeled dataset that I have, that is missing the category label. This dataset is very large, and has much more than 100 categories. I am only interested in being able to classify unlabeled data for the 100 categories that I have, but I am not sure how to approach this.
One thought I had was to build a Word Embedding model for the labeled data. This model could be used to calculate a document vector for the unlabeled data and find a similar document from the labeled dataset. This would allow me to assign labels to some of the data in the unlabeled dataset. Is there a better way to approaching this problem?