I have a small unlabeled textual dataset and I would like to classify all document in 2 categories.

I understand and have used supervised classification. I have searched a lot but still cannot understand how unsupervised binary classification works. Can you give me an example or a simple explanation ? Or some paper, document or code that explains it well ?

Thank you.


Correct me if I'm wrong, but your unsupervised classification is not much different than clustering. This is because since it is unsupervised, you do not actually have a user-provided indication of what your classes should be. Thus an unsupervised machine learning system would guess its own classes, and an straightforward way would be to make these classes correspond to the clusters.

In your case you could represent each document as bag-of-words. Then, if necessary, you can do feature reduction (e.g. using PCA). Finally, you could use k-means (in fact 2-means, since you want binary classification). Assigning a new element to any of the two clusters is your classifier.


Differences in the data

The difference between unsupervised and supervised learning is that in the case of unsupervised learning your dataset is not labeled.

Differences in the optimization

When you are performing supervised learning you are trying to minimize and objective relative to your current model prediction and the ground truth of your labeled example. The procedure is the following:

  • Pick an example
  • Perform a prediction
  • Calculate the loss
  • Adapt the model weights
  • Pick another example...

In unsupervised learning, you are trying to draw inferences from the data. That's where you need to tweak your vocabulary to understand things better. Instead of performing a binary classification you will instead perform a clustering with K clusters, in your case K=2. So the objective is a little different. For instance instead of minimizing a logloss, you'll probably need to maximize the differences between your 2 cluster by adapting a decision boundary. An example procedure might be:

  • Choose a naive boundary
  • Assign each example into their cluster given the decision boundary
  • Adapt the boundary to maximize the clusters distances

See this great medium post to discover some clustering techniques.

About your use case

Without getting into details what you can try for instance in your use case is to to create an embedding of each document given the vocabulary inside to get a dense representation of the dataset and then perform a clustering algorithm on the dataset.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.