how can i leverage NLP features like SRL, LSA, POS, NER, entity type , relation type with deep learning to find semantic similarity of texts?

Question

I have texts similar to the ones below, and I want to find semantic similarity between these texts and the intent.

1) what are the different steps to resign ?

1) procedure to resign ?

1) what are the process do i have to follow to resign ?

1) what are the common schemes followed for resignation by any company !!

2) to whom i need to contact after i resign ?

3) i did resign today.

4) if you misbehave again , i'll resign !!

4) your rude behavior made me to resign.

5) different company have different policy on resignation !!

To tackle above problem i found " Quoras Model to handle Duplicate Questions " interesting . so i thought let's try this but with small changes. and changes is instead doing binary classification , let's do multi class classification. so i was trying to do multi class classification on a "Quora Data Set". my intention was to classify each question with their duplicate questions into one class on a vector space using LSTM, CNN models. LSTM worked if sequence remain unchanged. but as you know questions structure can changes keeping intent same (or we can call them as duplicate questions.). so to handle such variation of question i tried with CNN. CNN worked for some small data set but for large data set it become very sensitive or overlapping.

So i am thinking " can i leverage NLP with deep learning to find intent and semantic relation ?"

I have POS tags, NER, SRL, LST, entity type, relation type, etc features. How can i leverage these features of NLP in deep learning to achieve a state-of-the-art result?

There is a paper, "When Are Tree Structures Necessary for Deep Learning of Representations?", that uses parse tree as input for recursive neural network model. Is there any similar work? Can any one give me data sets used in this paper?

Answer 1

TL;DR:

Represent words as word vectors. Then add extra dimensions to the word vectors. In these extra dimensions, include POS, NER, etc. features in a numeric form.

Longer Version:

Say you have a word2vec/Glove model with each word represented by a 100 dimensional vector. Additionally, you have features like POS, NER, etc. for each word. Instead of representing each word with just 100 dimensions, represent it with 100 + n dimensions and fill these n slots with the nlp features.

This will give the vectors an additional meaning which you have defined manually.

Suggestions

You could benefit from using other models like Doc2Vec/Sent2Vec which can represent a whole sentence as vector. With these vectors you could query similarity of sentences.

So say you've trained it on your data, just query :

model.most_similar_cosmul(SENT_42)

where SENT_42 represents "to whom i need to contact after i resign ?", you will get a list of sentences similar to it.

I suggest you use gensim's doc2vec model in python.

Answer 2

One promising algorithm is "Word Mover's Distance". It treats document similarity as an optimization problem between word embedding vectors. The goal is to minimize the cost of "travel" between sets of word vectors. Given the semantic encoding inherent in word vectors, it can model aspects of intention and meaning.

Word Mover's Distance (WMD) is relatively simple to implement. However, it is computationally expensive because of the number of comparisons it makes.

WMD completely ignores POS, NER, parse trees, and all other related NLP techniques. Those restrictions make it easier to apply, however, limits the properties of language it can model.