Approaching a multi-class classification problem but without labels

Question

I am working on a business problem where I have a movie description dataset. In this dataset I've columns as - Movie title, Movie plot summary, Date of Release. Now based on this information and using machine learning I want to predict which category the movie falls into. For example The Conjuring should fall into Horror and Thriller i.e a multiclass classification problem. Now the problem is I don't have a label column besides the movie description and other info. Now I want my model to predict which categories a movie(unseen to model) should fall into. I have decided 5 labels that I want to consider - Horror, Thriller, Comedy, Romantic and Emotional. So, I want the dataset to look like this -

---

Conjuring| Description | Title | Horror,Thriller

---

The notebook| Description| Title | Romantic,Emotional

---

I believe if I want to proceed this problem as a classification problem then I have to think of some way to create labels to existing dataset by some script and logic. If not supervised then maybe if I can do clustering first and then based on where the data point lies I can do classification later on.

What I have tried ?

Once I decided what my 5 labels should be, I made 50 synonyms for each and then iterated the description of the movies and based on the number of occurrence of words I made frequency and based on majority of the occurrence I decided which category a movie should fall into. Very bad results from this approach.

I used K means clusters from the data and tried to extract information from the clusters. Could not get very meaningful information though.

To be very honest I am pretty clueless and just want a direction how to approach this problem.

Answer 1

Your specific problem can be solved by Googling.

Here is a solution that

1. Searches the "imdb [year] [movie name]" in Google,
2. Finds its IMDb address and fetches the IMDb page, and then
3. Searches for the genres inside the IMDb page.

I have changed "romantic" to "romance", and "emotional" to "drama" to match the IMDb vocabulary.

from requests import get
import re

titles=["2013+Conjuring", "2004+The+notebook"]
genres = ['horror', 'thriller', 'comedy', 'romance', 'drama']
matched_genres = {}
for title in titles:
  query = "https://www.google.com/search?q=imdb+" + title
  print(query)
  search_result = get(query).text.lower()
  imdb_id = re.findall("https://www.imdb.com/title/(tt\d+)/", search_result)[0]
  imdb_address = "https://www.imdb.com/title/%s/" % imdb_id
  print(imdb_address)
  imdb_result = get(imdb_address).text.lower()
  matched_genres[title] = []
  for genre in genres:
    # find ">genre<" inside tags
    if imdb_result.find(">%s<" % genre) > -1:  
      matched_genres[title].append(genre)

print(matched_genres)

Output

https://www.google.com/search?q=imdb+2013+Conjuring
https://www.imdb.com/title/tt1457767/
https://www.google.com/search?q=imdb+2004+The+notebook
https://www.imdb.com/title/tt0332280/
{'2013+Conjuring': ['horror', 'thriller'], '2004+The+notebook': ['romance', 'drama']}

This solution could be improved by

1. Querying movie titles in parallel,
2. Directly querying IMDb API,
3. Handling edge cases (for example when the first IMDb url is irrelevant or no IMDb page is found, etc.),

etc.

Answer 2

Since your problem is not precisely what you ask (seeing the comments you did), I might suggest that your problem has nothing to do with multi label classification, but more with summary extraction (summaries can be single keywords such as "science fiction". You could look for topic modelling algorithms to start. In any case, you'll need some task specific heuristics to achieve what you want, since your problem is not well formalized as a machine learning task.