How can I find contextually related words and classify into custom tags/labels?

Question

PROBLEM:

Suppose if I have a small dataset containing some words and their tags/labels. The main task is to provide tags to other words(which are not in the dataset) based on their contextual relationship with the words already in the dataset.

Let's say, for example, my custom dataset includes

              Soap --> label__(cleaning_agent)
              pencil--> label__(stationary_item)
              mobile--> label__(electronics)
              washingmachine--> label(electronics)
              and so on.

I want my program to be able to correctly predict the label of an unknown word, e.g.

         washing powder to its correct category
         label__(cleaning_agent)
         radio to label__(electronics) etc.

ACTIONS:

Now the main problem is to find the relation between two words based on context, but I am not able to decide what can be the parameters for finding that.

I have tried a naive approach using datamuse API and fastText library.

    Naive approach is as follows->
        step 1-> find all the related words of the given word(let's say W) e.g. pencil using datamuse API.
        step 2-> combine them into a string(let's say S) with spaces in between them
        step 3-> use the label name, W, S as the training dataset for fastText.

NOTE: fastText requires label name, word, sentence(can be from news articles, blogs, Wikipedia etc ) as a context for that word.

RESULTS: fastText is not providing any reliable results. I am thinking of building a neural network kind of thing for this purpose. But I am not able to decide what can be the input parameters for our data.

The main problem is about custom word tagging. Our program should be able to tag unknown(not in training dataset) words to their most probable classes based on some score.

As I am new to NLP, I want to know what can be the next move forward.

Answer 1

You are working with word classification so you really don't have any contextual information that you can leverage. So the best thing would be to use freely available contextual information called as word2vec. I would suggest using the pretrained GLOVE Word2Vec as they are much cleaner than the Google ones. I am going to suggest two approaches for doing this:

Naive Approach

1. Load the pretrained word vectors.
2. For every new word calculate cosine similarity with every item in your training list.
3. Predict the label for the word which is closest to your new word.
4. Alternatively you can calculate cosine similarity with the labels and just predict the label which comes the closest.

Better Approach

1. Load the pretrained word vectors.
2. Extract the vectors for every item in the training set.
3. One hot encode the labels.
4. Train a neural net or any multiclass classification algorithm using the vectors as features and save the model.
5. For every new word extract the vector and run the model on this vector.
6. The predicted label is your output.

You can use the first approach to run quick tests and check the efficacy of the approach and then move on to the second one if you found it fit to your use.