Text understanding and mapping

Question

I am looking at a problem where I have two documents, both are textual documents. The first one is a long (few pages long) textual document, while the second one contains about 10 short texts, each being a one liner. The problem is to automatically understand the 1st document and "map" key ideas from here to one or more short texts in the second document. I am not being able to get a good grip on the problem as to how to proceed. I tried the following approach:

first collected as much similar documents as possible which are relevant; I have obtained 136 such documents, each few pages long. Then build a document level embedding using each sentence as a document (doc2vec). Then for each short sentence in the second document, inferred an embedding based on the document model I built. Then tried to find the most similar sentences for each of these short texts from the 2nd document. However the results are not good. Only about 30% match is what I am getting and the corresponding sentences are not much related to the target sentence.

I am wondering if this is the right approach or are there any other approach available which I am not aware of.

Your advice would be greatly appreciated. It may be the case that my explanation above is not adequate. Please let me know and I will try to improve it.

Your input would be greatly appreciated.

Thanks

Answer 1

Instead of using embeddings and doc2vec, maybe you should start by an easier and more straightforward information retrieval approach. Start by building a bag-of-words representation of your long document. Given its vocabulary $V$ of size $|V|$, you represent it as well as the short texts for the second document in a space $\mathbb{R}^{|V|}$. This will give a vector for the first document and several vectors, one for each of the lines, for the second document.

Proceed by calculating the similarity (euclidean distance) of the vectors of the lines of the second document with the vector of the first document.

There are many ways in this line of though that would enable you to use word embeddings if you want, but I would suggest starting with the simplest approach first.