I have used the stanford movie review dataset for creating a experimentation of sentiment analysis.
Managed to create a basic application on top of Spark using the Naive bayes classification algorithm.
Steps that I did for pre-processing from the spark ML pipeline
The provided dataset above also has a testing dataset with itself which is separate of the training set. After training it I got around 97% accuracy which I believe is pretty good for Naive bayes.
Now can I use this ML model to predict for other texts such as email/chat etc., My guess is that this dataset has a large enough collection of words to perform good predictions and certain english words regardless of the business context like "I dont like this","This does not look good" is the same across different domains such as Movies/Emails/Chats etc.
I have not done the experiment since the data that I need to get hold of belongs to the customer and due to privacy restrictions we cannot access the data.
Any help/guidance would be much appreciated.