Skip to main content
Data Science

Return to Question

Post Closed as "Needs more focus" by Toros91, Stephen Rauch♦, Kiritee Gak, Aditya, Sean Owen
added 13 characters in body
Source Link
A.D.
A.D.
  • 205
  • 1
  • 6

I need to classify MOOC video scripts into one of 5 classes which specify the intent of the sentence, e.g. explanation, example, summary etc., sentence-wise.

I have 11 courses extracted and selected equal ratio of files from every course based on the number of files I have for each of them. Say course 1 has 100 files and course 2 has 5 times less, then I selected data for labeling such that the labeled daya for course 1 will be 5 times more than the labeled data for course 2.

Due to lack of time, I wanted to label 1/5th of my data, which is about ~1800~8800 sentences in 111 files. My question is.. can this be enough to predict the rest of the data, i.e.or approx 4 x 1800~29600 sentences more.. maybe if i slowly introduce it to the model, not at once.. or i don't know.. any ideas are welcome, since I am not too experiences. Also specific text classification algorithms you think work best, would be very helpful to mention too. I am open to trying several, as long as their implementation is not too much time-consuming. I will of course try the most common ones first such as Naive Bayes and SVM.

Oh also, another noob question - do I need to convert my text into vectors for some reason? Or will the classifier do it's own work based on the textual data?

I need to classify MOOC video scripts into one of 5 classes which specify the intent of the sentence, e.g. explanation, example, summary etc., sentence-wise.

I have 11 courses extracted and selected equal ratio of files from every course based on the number of files I have for each of them. Say course 1 has 100 files and course 2 has 5 times less, then I selected data for labeling such that the labeled daya for course 1 will be 5 times more than the labeled data for course 2.

Due to lack of time, I wanted to label 1/5th of my data, which is about ~1800 sentences. My question is.. can this be enough to predict the rest of the data, i.e. approx 4 x 1800 sentences more.. maybe if i slowly introduce it to the model, not at once.. or i don't know.. any ideas are welcome, since I am not too experiences. Also specific text classification algorithms you think work best, would be very helpful to mention too. I am open to trying several, as long as their implementation is not too much time-consuming. I will of course try the most common ones first such as Naive Bayes and SVM.

Oh also, another noob question - do I need to convert my text into vectors for some reason? Or will the classifier do it's own work based on the textual data?

I need to classify MOOC video scripts into one of 5 classes which specify the intent of the sentence, e.g. explanation, example, summary etc., sentence-wise.

I have 11 courses extracted and selected equal ratio of files from every course based on the number of files I have for each of them. Say course 1 has 100 files and course 2 has 5 times less, then I selected data for labeling such that the labeled daya for course 1 will be 5 times more than the labeled data for course 2.

Due to lack of time, I wanted to label 1/5th of my data, which is about ~8800 sentences in 111 files. My question is.. can this be enough to predict the rest of the data, or approx ~29600 sentences more.. maybe if i slowly introduce it to the model, not at once.. or i don't know.. any ideas are welcome, since I am not too experiences. Also specific text classification algorithms you think work best, would be very helpful to mention too. I am open to trying several, as long as their implementation is not too much time-consuming. I will of course try the most common ones first such as Naive Bayes and SVM.

Oh also, another noob question - do I need to convert my text into vectors for some reason? Or will the classifier do it's own work based on the textual data?

Source Link
A.D.
A.D.
  • 205
  • 1
  • 6

Would it work to label 1/5th of my data and once I learn a classifier to slowly provide the unlabeled data?

I need to classify MOOC video scripts into one of 5 classes which specify the intent of the sentence, e.g. explanation, example, summary etc., sentence-wise.

I have 11 courses extracted and selected equal ratio of files from every course based on the number of files I have for each of them. Say course 1 has 100 files and course 2 has 5 times less, then I selected data for labeling such that the labeled daya for course 1 will be 5 times more than the labeled data for course 2.

Due to lack of time, I wanted to label 1/5th of my data, which is about ~1800 sentences. My question is.. can this be enough to predict the rest of the data, i.e. approx 4 x 1800 sentences more.. maybe if i slowly introduce it to the model, not at once.. or i don't know.. any ideas are welcome, since I am not too experiences. Also specific text classification algorithms you think work best, would be very helpful to mention too. I am open to trying several, as long as their implementation is not too much time-consuming. I will of course try the most common ones first such as Naive Bayes and SVM.

Oh also, another noob question - do I need to convert my text into vectors for some reason? Or will the classifier do it's own work based on the textual data?