Text Mining of Research Paper Abstracts

Question

I'm trying to build a system which tackles the following problem. Given a list of research papers L, I would like to extract the following information from each paper:

• The problem addressed by the paper
• Solution to the problem proposed by the paper
• (Optional) Results, i.e. how good is the solution

I have dealt with the easy part, i.e. setting up the blocks that interact with web search engines (e.g. Google scholar) and gather the data, given a set of search keys.

However, I don't have the knowledge to set up the 'intelligent' part, i.e. the one which extracts the aforementioned information from the text data. I would prefer to extract information based on the Abstract, but analysis the complete text wouldn't be a dealbreaker for me. I'm mostly interested in Computer Networking papers, e.g. published on SIGCOMM, CoNEXT, etc.

My questions are:

• Are there tools - preferably open source - which can extract this information from research papers?
• If not, what techniques do you recommend me to start looking at (e.g. research papers, tutorials, etc.) to accomplish this?

Answer 1

In order to train some supervised learning algorithm to identify 'Problem' and 'Solution', you need to somehow generate some data that has labels of these things, which may be your best bet. So you would need pieces of article text that you know are 'Problems' and 'Solutions'. Then the trained algorithm could identify new articles based on the training set. But I imagine this is a lot of work to generate if you don't have this training data to begin with.

You can also try some unsupervised method, for example you may split each article into multiple overlapping parts, and compute TFIDF vectors of each part. Do this for many articles, then cluster the TFIDF vectors. You can then look at your clusters and hope one of them is generally capturing the 'Problem', and another the 'Solution'. It will probably be very messy, and you'll have to clean it up with some post-hoc heuristic, depending on what the result looks like. I think this approach would be hard to actually get to work well, but it would avoid having to obtain (or create) a labeled data set.

Finally, if you look at articles you may be able to devise generally good heuristics (e.g. the solution might be the first paragraph in a section titled 'Discussion' 70% of the time) to bolster both of these approaches, and if the article patterns are repeatable enough you may be able to just use heuristics to do a generally decent job. Since articles are often very similarly structured, this is where I would start to see how far I can get, and how much I can simplify the problem using heuristics before passing the simplified problem to the other methods.