3
$\begingroup$

Long story short, I have 200 vectors as a result of TF-IDF (Term Frequency - Inverse Document Frequency) on thousands of keywords in hundreds of vectors. The total number of unique keywords that I got is 745 keywords, meaning that there are 745 dimensions/axes. Now, I was wondering how does K-means clustering work on those 200 vectors? Is it accurate to say that K-Means is clustering those 200 vectors by the keywords weight similarity?

$\endgroup$
1
$\begingroup$

K-means clusters points (in your case represented as 745 dimensional vectors) by their similarity, that is some distance measure between points (usually the Euclidean distance).

TF-IDF produces a vector from a sentence or document, where each entry (axis) represents the frequency of a word divided by the frequency of it's occurrence in all sentences or documents, hence the name. Other weighting scheme are possible as well, see here.

If two vectors are very close to each other, it means that the content of their documents is very similar. Therefore it's likely they end up in the same cluster. In contrast, if two vectors are far away, the words in each document might be completely different or the frequency of words might be different.

So the distance between vectors can be interpreted as measure of similarity between the documents content.

| improve this answer | |
$\endgroup$
  • $\begingroup$ to elaborate your last line: if, and only if I have two keywords (which also means I have 2 dimensions or 2 axes), and the vector1 --which is also the k-centroid of cluster A-- consists of two weighted keywords with numerical weights 4.3 and 2.2. At the same time, I have vector2 --which is also the k-centroid of cluster B-- consist of two weighted keywords as well but this time with numerical weights 1.3 and 4.2. Then for vector3 which consists of numerical weight 3.3 and 3.2, it would be clustered to cluster A because the first keyword which weights more makes it lean to cluster A, right? $\endgroup$ – Jack Zaki Zakiul Fahmi Jailani Jul 10 at 16:31
0
$\begingroup$

Long story short:

The distance measure between two vectors used in k-means algorithm is / can be user-provided. So, it can mean whatever the user wants it to mean by providing the appropriate measure.

Now, for TF-IDF and a geometric measure (eg Euclidean metric) one can say that two feature vectors will be close if the have similar frequencies for their respective keywords.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.