# Build train data set for natural language text classification?

I have extracted ~550 video scripts (subtitles) from 11 free courses on the Coursera platform. I have pre-processed them in terms of punctuation removal, stop words removal, tokenization, stemming and lemmatization. Now, I've been advised that for my task I can attempt to use a simple Bag of Words. However I am not sure how exactly would that help me towards classifying my text into one out of six categories. The categories are related to the intent a video material was created with and more precisely, which part explains a concept, which part discusses an example, which part gives practical advice etc. Below are my categories:

ConceptDescription-> Explanation of the Main concept(s)
ConceptMention-> Mentioning of a concept, related to the main concept
Methodology / Technique-> To achieve something, what should one do
Summary-> Summary of the discussed material or of the whole course
Application-> Practical advise for the concept
Example-> Concept example

By manually reading several files from 3 of the courses, I created a dictionary, containing spoken language words, that may help me identify which class a specific sentence/paragraph falls into. However I do NOT have a train dataset for a classifier. So my idea was to use that dictionary to label my data, e.g. sentence 1 as Summary, sentence 4 as ConceptDescription and sentence 12 as Example and then marking sentences 2 and 3 the same as 1, sentence 5-11 like sentence 4 etc.

My question is, is this idea too lame? And is there a way to create at least an average quality training dataset in a way that is not manual? Or if manual check is the only option, is there an option where I would need to do manual labeling on only a small fraction of the files, say 50 out of 550 and classification would still produce average to good result? I don't aim at perfect result, but I aim at something less time-consuming due to limited time.

I also played with tf-idf which outputs terms, but of course, not really what I need, so that was a bit random.

Thanks in advance for your help. Any specific ideas and algorithms would be very welcome.

• It would help to do some analysis of the scripts to identify aspects that distinguish the various categories. Once you do this manually for some examples, you could consider writing some rules based on the observations. The rest of the examples can be labeled using the rules. For a model-based approach, if you label a small set of examples (~50), then a simple model (Naive Bayes, etc.) can potentially be trained on these. – raghu May 12 '18 at 13:28
• what kind of rules do you mean? – A.D. May 12 '18 at 20:15
• The rules can target specific patterns that occur in the scripts. These can be individual words, phrases, or multiple words (not necessarily consecutive) that indicate a specific category. This can be used if the analysis shows common patterns across scripts in a category. – raghu May 13 '18 at 4:36
• @raghu can you please write the first comment as an answer, so I can mark it as such? :) And thanks for your help :) – A.D. May 14 '18 at 11:02