Parse documents to obtain subjective sentiment

Question

I'm working on a project which deals with MRC (Machine Reading Comprehension).
I would like a machine to read an article and give me the sentiments based on a provided token.

For Instance given the input: "Jackson Heights should see an increase in real estate, While other areas of brookln should see a steep decline".

Given the token: "Jackson Heights" Should return a score indicating positive sentiment: e.g .72

Given the token: "Brooklyn" Should return a score indicating negative sentiment: e.g -.72

I've tried to solve this problem in several different ways. I've been following along with a tutorial for sentiment analysis. I've linked to the corresponding jupyter notebook.

The issue is that this "Sentiment Analysis" doesn't really do much for multiple reasons.

1. Sentiment analysis removes stop words such as weren't and were. So the statement "I would buy google stock" and the statement "I wouldn't buy google stock" would evaluate to the same sentiment
2. Sentiment analysis is only per text so understanding what subject applies to sentiment is important. e.g "I love chicken and I hate fish" may return a negative sentiment because neural nets have a tendency to put a stronger weight on statements closer to the end. However the sentiments to be tokenized based on the subject.

I've investigate a few open source technologies for this such as AllenNLP However, this didn't seem promising because the model assumes the question can be answered. Given the text: "The price of GOOG should increase by 3% next week" and a question "How much will Yahoo increase" the output still highlights "3%" instead of returning nothing.

After I investigated a simple Question answering bot using Kaggle.
This seems a bit more robust but it seems a bit overkill, I don't want to investigate a complicated AI model if there is a simpler solution.

To put the question in its simplest form.

I would like to parse a document into a list of tokens with their sentiment score. However the solutions i've tried, remove important information, and full NLP seems to be a bit overkill.

could someone outline an simple efficient process for obtaining tokenized sentiment analysis?

Answer 1

As far as I understand, the task you try to do would require a system to correctly analyze the sentence at a semantic level. In general this is Natural Language Understanding and there's no solution to this problem.

Given your examples it looks like one problem is separating the different parts of the sentence, so that you can analyze the correct part of the sentence for each entity independently. This is doable: you can use shallow or deep parsing methods to extract the chunks or subtrees, and then you could apply sentiment analysis to just to the part which contains the target entity.

In general don't expect perfect results with sentiment analysis or any semantic task. Even though DL has brought great progress in the recent years, these problems are active research questions (and might still be researched for a long time).