1
$\begingroup$

I would like to extract topics from a set of movie subtitles, and possibly see if there is any relation with the viewer's rating. I have thought about creating a DocumentTermMatrix where each document is one movie, and than applying LDA in order to find the topics. However, I have never classified documents, and I have no idea about how to find out if one topic is more likely to have good reviews than others. I would like to create something graphical that shows both the clusters of topics and their relation to the rating... Any advice would be very useful!

$\endgroup$

1 Answer 1

0
$\begingroup$

I think that this experiment makes some sense, keeping in mind that:

  • The dialogues (subtitles) are probably a decent indicator for the main topic of the movie, but not a perfect one.
  • Similarly, there might be some correlation between dialogues/topics and ratings, but obviously the ratings don't only depend on the topic (or dialogues).

Note: in case the goal is to see the relation between dialogue and rating, you could also consider training a supervised regression model which predicts ratings directly from the subtitles text.

Topic modelling calculates a distribution of topics for every movie, so in general you can have a movie which belongs to different topics at varying degrees. Of course, it's common to assign a single topic to an instance (here a movie) by taking the maximum probability topic. If you do this, you can easily obtain the set of movies for every topic, and then calculate any statistic related to rating for the topic, for example the mean rating for this topic.

In terms of visualization, a common way to represent a topic is with the set of N words which are most strongly associated with it. You could do a word clouds with these. You can also represent the histograms of the ratings distributions by cluster.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.