Differences in the data
The difference between unsupervised and supervised learning is that in the case of unsupervised learning your dataset is not labeled.
Differences in the optimization
When you are performing supervised learning you are trying to minimize and objective relative to your current model prediction and the ground truth of your labeled example. The procedure is the following:
- Pick an example
- Perform a prediction
- Calculate the loss
- Adapt the model weights
- Pick another example...
In unsupervised learning, you are trying to draw inferences from the data. That's where you need to tweak your vocabulary to understand things better. Instead of performing a binary classification you will instead perform a clustering with K clusters, in your case
K=2. So the objective is a little different. For instance instead of minimizing a logloss, you'll probably need to maximize the differences between your 2 cluster by adapting a decision boundary. An example procedure might be:
- Choose a naive boundary
- Assign each example into their cluster given the decision boundary
- Adapt the boundary to maximize the clusters distances
See this great medium post to discover some clustering techniques.
About your use case
Without getting into details what you can try for instance in your use case is to to create an embedding of each document given the vocabulary inside to get a dense representation of the dataset and then perform a clustering algorithm on the dataset.