3
$\begingroup$

I have extracted ~550 video scripts (subtitles) from 11 free courses on the Coursera platform. I have pre-processed them in terms of punctuation removal, stop words removal, tokenization, stemming and lemmatization. Now, I've been advised that for my task I can attempt to use a simple Bag of Words. However I am not sure how exactly would that help me towards classifying my text into one out of six categories. The categories are related to the intent a video material was created with and more precisely, which part explains a concept, which part discusses an example, which part gives practical advice etc. Below are my categories:

ConceptDescription-> Explanation of the Main concept(s)
ConceptMention-> Mentioning of a concept, related to the main concept
Methodology / Technique-> To achieve something, what should one do
Summary-> Summary of the discussed material or of the whole course
Application-> Practical advise for the concept
Example-> Concept example

By manually reading several files from 3 of the courses, I created a dictionary, containing spoken language words, that may help me identify which class a specific sentence/paragraph falls into. However I do NOT have a train dataset for a classifier. So my idea was to use that dictionary to label my data, e.g. sentence 1 as Summary, sentence 4 as ConceptDescription and sentence 12 as Example and then marking sentences 2 and 3 the same as 1, sentence 5-11 like sentence 4 etc.

My question is, is this idea too lame? And is there a way to create at least an average quality training dataset in a way that is not manual? Or if manual check is the only option, is there an option where I would need to do manual labeling on only a small fraction of the files, say 50 out of 550 and classification would still produce average to good result? I don't aim at perfect result, but I aim at something less time-consuming due to limited time.

I also played with tf-idf which outputs terms, but of course, not really what I need, so that was a bit random.

Thanks in advance for your help. Any specific ideas and algorithms would be very welcome.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ It would help to do some analysis of the scripts to identify aspects that distinguish the various categories. Once you do this manually for some examples, you could consider writing some rules based on the observations. The rest of the examples can be labeled using the rules. For a model-based approach, if you label a small set of examples (~50), then a simple model (Naive Bayes, etc.) can potentially be trained on these. $\endgroup$
    – raghu
    May 12, 2018 at 13:28
  • $\begingroup$ what kind of rules do you mean? $\endgroup$
    – A.D.
    May 12, 2018 at 20:15
  • 1
    $\begingroup$ The rules can target specific patterns that occur in the scripts. These can be individual words, phrases, or multiple words (not necessarily consecutive) that indicate a specific category. This can be used if the analysis shows common patterns across scripts in a category. $\endgroup$
    – raghu
    May 13, 2018 at 4:36
  • $\begingroup$ @raghu can you please write the first comment as an answer, so I can mark it as such? :) And thanks for your help :) $\endgroup$
    – A.D.
    May 14, 2018 at 11:02

2 Answers 2

Reset to default
1
$\begingroup$

It would help to do some analysis of the scripts to identify aspects that distinguish the various categories. Once you do this manually for some examples, you could consider writing some rules based on the observations. The rest of the examples can be labeled using the rules. For a model-based approach, if you label a small set of examples (~50), then a simple model (Naive Bayes, etc.) can potentially be trained on these.

Improve this answer
$\endgroup$
1
$\begingroup$

Since you know the quantity of labels (6) you can use k-means algorithm to cluster your data into 6 groups. I recommend you to represent each using the tfidf method. You can implement your code using sklearn functions.

Improve this answer
$\endgroup$
5
  • $\begingroup$ How do you mean to use tf-idf? if we say I have labeled sentences, that would mean to get tf-idf of all the sentences labeled with the same label? Is this what you mean? And then apply k-means? Also, how do I assess the quality of my result with precision, recall etc? $\endgroup$
    – A.D.
    May 15, 2018 at 19:20
  • 1
    $\begingroup$ k-means is an unsupervised learning method, so you don't need to provide the labels. To obtain the learned labels, what you can do is to extract the main features of the centroid of each cluster. Let's say that cluster 1 has a centroid [1,0,0] and cluster 2 has a centroid [0,1,0], so the main feature of cluster 1 could be "example" (a word represented by the first component of the vectorizatoin) and so on. After that you should analyze if these labels have correspondence with the correct topics you suppose each cluster must have. And, in this case, compute recall and precision. $\endgroup$
    – Federico Caccia
    May 15, 2018 at 19:32
  • 1
    $\begingroup$ The tf-idf representation is a previous step you should compute, without taking into account the labels. You can check an example I have coded to check how to fit the model and get the labels. $\endgroup$
    – Federico Caccia
    May 15, 2018 at 19:35
  • $\begingroup$ oh gosh.. ok i will look into that. Although it sounds like quite a bit of work. Thanks a lot for the info! I don't have questions now, because it's not totally clear to know what to ask, but will definitely check it out. Thanks! P.S. Checking the code it doesn't look tooo complicated $\endgroup$
    – A.D.
    May 17, 2018 at 20:24
  • $\begingroup$ Ok I try to make the example very simple. Anyway, if you have more questions you can ask again, good look! $\endgroup$
    – Federico Caccia
    May 18, 2018 at 0:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.