1
$\begingroup$

I have a database of tags given by users to the product. For example

user; product; tag
1; A; Tag1
1; A; Tag2
2; A; Tag1
2; B; Tag1
..
..

I am trying to cluster tags which are given together to any product. At the end I want to visualize them like PCA plot where I can see clusters of tags which are 'closer' (tendency of user to assign these tags together) to each other. So far I can think of applying t-SNE or simply PCA to get some kind of clustering with available tags.

For this purpose, I made frequency matrix like following

first_tag; second_tag; occurrence
Tag1; Tag2; 2393
Tag1; Tag3; 38
Tag2; Tag3; 8393
..
..

I am stuck here. I don't know how to proceed with clustering. I simply tried visualizing it with networkx library of python where I made edges with 'first_tag' and 'second_tag' and 'occurrence' as its weight. But it was futile exercise, I couldn't get anything out of it. I thought of using sklearn.decomposition PCA but I am struggling to convert this data-set into proper matrix which can be fed to such algorithms. I can always make n x n matrix by making rows and columns equal to number of tags but it was causing very slow and sometimes out of memory errors. Any other elegant solution for this?

$\endgroup$
1
$\begingroup$

Instead of clustering, association rules is likely to be much more appropriate on such data.

Nevertheless it's obviously possible to treat cooccurrences as a similarity measure and cluster this for example using a variant of hierarchical clustering (for similarities, not distances!) Or by transforming the similarities to distances. You can also try to treat these as affinities for tSNE or MDS, for example (again, your usual library will likely not just work, but you will need to write some code yourself), but it's probably much easier to simply visualize the cooccurence matrix as a "heat map", after arranging similar tags close to each other with hierarchical clustering.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.