How to calculate which word fits the best given a context and possible words?

Question

I have this task for research purposes and searched a while for a framework or a paper which already took care of this problem.

Unfortunately I don't find anything which helps me with my problem.

I have a sentence like

if the age of the applicant is **higher** than 18, then ...

and a list of words like

higher, bigger, greater, wider ...

which are all a

Now I want to find find out, which of the given words approximately fits the best at the predefined position in the sentence.

The best fitting word in this example would be 'greater', but for example 'higher' would be also fine. In my specific case, I want to show an error message if someone would write 'wider', because this doesn't make sense in this semantic context.

I hope that I explained my problem good enough.

Answer 1

There are two options :

1. CBOW . Modify Word2Vec CBOW code to save the whole trained model (current implementations only persist embedding layer)

CBOW Model: This method takes the context of each word as the input and tries to predict the word corresponding to the context.

Intro : https://towardsdatascience.com/introduction-to-word-embedding-and-word2vec-652d0c2060fa Example : https://www.tensorflow.org/tutorials/representation/word2vec

2. Train an LSTM / GRU to predict next word (given previous N words)

Karpathy's article is probably the best introduction to text generation with RNN (this works at character level, you will have to modify it to work at word level [Word-Vector level])

http://karpathy.github.io/2015/05/21/rnn-effectiveness/

Example :

https://medium.com/phrasee/neural-text-generation-generating-text-using-conditional-language-models-a37b69c7cd4b