1
$\begingroup$

I have used the stanford movie review dataset for creating a experimentation of sentiment analysis.

Managed to create a basic application on top of Spark using the Naive bayes classification algorithm.

Steps that I did for pre-processing from the spark ML pipeline

  • Tokenization
  • Bigrams

The provided dataset above also has a testing dataset with itself which is separate of the training set. After training it I got around 97% accuracy which I believe is pretty good for Naive bayes.

Now can I use this ML model to predict for other texts such as email/chat etc., My guess is that this dataset has a large enough collection of words to perform good predictions and certain english words regardless of the business context like "I dont like this","This does not look good" is the same across different domains such as Movies/Emails/Chats etc.

I have not done the experiment since the data that I need to get hold of belongs to the customer and due to privacy restrictions we cannot access the data.

Any help/guidance would be much appreciated.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

It depends.

You're basically asking if your sample (training data) is representative of the population (all written words).

  1. Are you doing sentiment analysis on movie reviews? It'll work great.
  2. Are you doing sentiment analysis on TV reviews? It'll probably work great.
  3. Are you doing sentiment analysis on book reviews? I would give better than 50-50 odds it'll work.
  4. Are you doing sentiment analysis on Twitter posts? Now we're getting shaky. People tend to write much less, use less formal language, and use more emojis which your movie review model wouldn't have seen.

That being said, there are definitely "generic" sentiment analysis services like here. Try out your model against Algorithmia on what you would consider a generic set of data (e.g. a bunch of tweets) and see how it does.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks for the info. I tried algorithmia and it seems only very negative messages show up as negative. For example sentence like "Needs improvement" shows up as positive . Yes, for twitter it wont work since there are is a lot of things involved, but just considering business speak would it work? $\endgroup$
    – Greedy Coder
    Mar 28, 2017 at 10:52
  • $\begingroup$ It would depend on what kind of business speak. Do they use lots of jargon or lingo? For example, movie reviews aren't going to help if they talk like this: managementconsulted.com/about/dictionary $\endgroup$
    – CalZ
    Mar 28, 2017 at 12:23
  • $\begingroup$ Thanks for the help. the accuracy was indeed different on a different domain. With that said, I just came across the predictive sentiment service by metamind which was later acquired by salesforce. How do they achieve accuracy? Do they get custom tagged datasets from customers to solve them? Because they cannot obviously read them manually due to privacy issues. $\endgroup$
    – Greedy Coder
    Mar 30, 2017 at 12:58
  • $\begingroup$ It is unlikely they will say exactly what data set they used for the same reason Coca Cola doesn't share their formula :) You can read them discussing research here though: metamind.io/research.html $\endgroup$
    – CalZ
    Mar 30, 2017 at 15:09
  • 1
    $\begingroup$ @MahdiAmrollahi - ideally, you want the model trained on the same domain (or as close as possible). Movies vs TV is not very different and reviewers look for many of the same things. What makes someone positive about a movie (e.g., good story) is not relevant to a car so that would hurt performance in my opinion. $\endgroup$
    – CalZ
    Jan 14, 2022 at 16:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.