0
$\begingroup$

Let's say I have been given 1000 documents and 6 labels from someone. My job is to label each of these 1000 documents into 1 of the 6 labels which are words not numbers. How can I automate or semi-automate this process using data science??

Improve this question
$\endgroup$
3
$\begingroup$

Semi-supervised learning. You label 1% manually, let the algorithm learn, then it labels unknown data, learns from it and labels again.

Improve this answer
$\endgroup$
1
$\begingroup$

You have two options. Supervised learning where you will have to label the data manually and then use those data points to train a model and predict the remaining instances.

Or, you can use unsupervised learning, these are techniques which do not need a label. You can use k-means to cluster your data into $k=6$ labels. Then you can associate these clusters with the label based on your experience.

Improve this answer
$\endgroup$
3
  • $\begingroup$ How to use k-means...the centroids are initialised randomly so they won't cluster the documents according to my labels? Will it be right here to not initialize centroids randomly?? $\endgroup$ – Rishabh Baid Nov 1 '18 at 6:26
  • $\begingroup$ It's best to randomly initialize them to avoid introducing bias. Let the centroids converge. Then attribute each cluster with one of your labels. $\endgroup$ – JahKnows Nov 1 '18 at 6:29
  • $\begingroup$ I tried doing exactly that..But I find at the end of the K-means program(random initialization) that there are a number of documents in the same cluster which wouldn't be in the same cluster...if I was doing the labeling manually. $\endgroup$ – Rishabh Baid Nov 1 '18 at 8:12

Not the answer you're looking for? Browse other questions tagged or ask your own question.