Text similarity using RNN

Question

Data set contains records of short text, typically a sentence. The goal is to find duplicated records and similar records. Currently, I have tried R package 'text2vec', the glove word vectors and the similarity APIs provided by the package.

There is a smaller subset of this data which is already tagged as duplicated. Currently, I have not factored in this as part of model training. Also, using the text2vec package, the results are not great on this test set. So now I am considering RNNs which are known to perform well in text similarity.

Now, I need help in feature engineering and preparing my input layer. Sentences S1 and S2 that need to be compared differ in length (the word representations differ in their dimensions). How to normalize this difference? Should I consider bag-of-words or glove word vectors (word vectors being much superior representations)? Any inputs in this regard will help.

Answer 1

Doc2Vec, Mikolov's paper will solve your problem. Here is the paper. You can find a gensim implementationhere. While using RNN, using GLOVE or Googl Word2Vec will be always useful even if your sequences are of the same length. To solve the variable sequence problem you can pad the sentences and use bucketing or just pad to one uniform length, truncating longer sentences and padding shorter sentences.

Answer 2

Unless you have a lot of data, I have my doubts whether training RNNs for similarity will give you significant improvements.

As a baseline, I would go the traditional way first and engineer some features for each pair of text, e.g.:

• Number of common words in both texts (words should be stemmed)
• Number of words in each sentence
• Cosine similarity on TF-IDF vectors (unigrams, word n-grams, character n-grams
• Glove similarity (you can try the pretrained vectors from wikipedia)
• Longest common subsequence (LCS)

Now you can train a binary classifier on your labeled pairs dataset. For example, start with Logistic Regression or Random Forests. Maybe you can already reach the performance you are looking for.

For further reference, search relevance classification is a very similar problem. You can check out the solutions from the Kaggle competitions Homedepot and Crowdflower to get more inspiration for feature engineering.