8
$\begingroup$

I am currently working on sentiment analysis using Python. I wanted to find whether reviews given for a movie is positive or negative based on sentiment analysis. I have found a training dataset as provided in this link.

This dataset have reviews and a score with 1 indicating review is positive and 0 indicating movie review is negative, but it has less number of records. I have a test dataset which I will predicting based on training set. My test dataset has complex and long words for which my python ML model sometimes gives positive result for a negative reviews (returning result as 1 for negative review). I am looking for better dataset to train my model, so that my model can predict well. Could you please advise me any good/large and effective training dataset to use for this scenario? If you could share any links that would be great. The training data can be in format as 1 for positive reviews and 0 for negative reviews or even polarity like pos for positive reviews or neg for negative reviews.

Improve this question
$\endgroup$
1
  • $\begingroup$ how different are the sizes of your training and testing datasets? After building your model, are you using cross validation? $\endgroup$
    – Bach
    Commented Apr 15, 2016 at 17:05

5 Answers 5

Reset to default
9
$\begingroup$

You can use the SAR14 dataset of 234K IMDb movie reviews. The construction of the SAR14 dataset is detailed in the paper "Sentiment Classification on Polarity Reviews: An Empirical Study Using Rating-based Features".

Improve this answer
$\endgroup$
3
$\begingroup$

There are many datasets available.

Improve this answer
$\endgroup$
0
0
$\begingroup$

The Stanford Sentiment Analysis dataset is based on Rotten Tomatoes reviews, has parses and sentiment annotation down to the syntactic component level.

Improve this answer
$\endgroup$
0
$\begingroup$

~7000 sample data entries is definitely not enough considering that a more-or-less reliable dictionary of sentiment-loaded words consists of several thousand words each for both positive and negative sentiments. Basically, when you train the model you build such a dictionary in some sense.

There is, however, an existing training/test dataset consisting of 50000 reviews which is a bit better than what you have.

At the same time, while the amount of training sample data contributes to the quality of the classifier, it's also important that the style and dictionary used by authors of texts in the training set was similar to your test texts. In addition, text processing tricks like stemming may increase the training efficiency.

For more information, you might want to look at this blog posts several colleagues and I wrote about creating training and test data sets.

Improve this answer
$\endgroup$
0
$\begingroup$

According to your requirements, this dataset will be a good choice for you to work on. It is available on Kaggle: https://www.kaggle.com/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv

IMDB dataset has 50K movie reviews for natural language processing or Text analytics. This is a dataset for binary sentiment classification containing substantially more data than previous benchmark datasets. We provide a set of 25,000 highly polar movie reviews for training and 25,000 for testing. So, predict the number of positive and negative reviews using either classification or deep learning algorithms.

Extra:

  1. You can also check out my Sentiment Analysis on Movie Reviews using this link: https://github.com/sweta-kesur-nlp-playground/nlp-play1-movie-reviews/blob/master/Sweta-1-NLP-play-Movies.ipynb
  2. I have also deployed it on Heroku: https://swetakesurnlp-playground.herokuapp.com/
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.