1
$\begingroup$

I am scraping reviews off Amazon with the intent to perform sentiment analysis to classify them into positve, negative and neutral. Now the data I would get would be text and unlabeled.

My approach to this problem would be as following:-

1.) Label the data using clustering algorithms like DBScan, HDBScan or KMeans. The number of clusters would obviously be 3.

2.) Train a Classification algorithm on the labelled data.

Now I have never performed clustering on text data but I am familiar with the basics of clustering. So my question is:

1. Is my approach correct?

2. Any articles/blogs/tutorials I can follow for text based clustering since I am kinda new to this?

PS: I am familiar with both NLP and Clustering algo's but I have never performed Clustering on text data.

$\endgroup$

1 Answer 1

1
$\begingroup$

In my opinion there are two main problems with your approach:

  • The clustering is extremely unlikely to correspond to sentiment, unless the features that you use for clustering are specifically engineered to represent sentiment. In general text clustering tend to group documents by common words, i.e. similar topic. This might lead to different categories of reviews by type of product, for example.
  • The second and I think most important issue is that without any labelled data, you can't evaluate the system. A common mistake would be to use the classes obtained from the clustering in order to evaluate the classification model: this doesn't evaluate the full task of sentiment analysis since there's no way to know how well the clustering represents sentiment. The proper method is to manually annotate a random subset of documents for the purpose of evaluation.

Also in general the second part with the classification model is not needed because the unsupervised clustering model can directly be applied to new instances.

$\endgroup$
4
  • $\begingroup$ So the only way this could work is if I MANUALLY label the data and then perform classification? $\endgroup$
    – spectre
    Dec 26, 2021 at 7:11
  • $\begingroup$ @spectre no it's not the only way, a very common way for sentiment analysis is to use a pre-trained model, either directly to obtain the result or as a first step for training a custom model. For Amazon reviews I think that this should work fairly well, it's quite standard. However a really clean evaluation requires at least a small subset of manual annotation, otherwise you just assume that the pre-trained model is correct. $\endgroup$
    – Erwan
    Dec 26, 2021 at 13:23
  • $\begingroup$ I recently found a library TextBlob which can give the sentiment score of a string. I think I will use that for labelling the dataset and then proceed further with step 2 mentioned above. But then again we return to your aforementioned point of otherwise you just assume that the pre-trained model is correct. $\endgroup$
    – spectre
    Dec 26, 2021 at 14:04
  • $\begingroup$ @spectre this is why I'd suggest at least a small sample of manually annotated documents (maybe 100 to 200), just to evaluate how well the pretrained model does with your data. Maybe you could also design some more advanced system using semi-supervised learning, but I don't have a clear idea how with this particular case. $\endgroup$
    – Erwan
    Dec 27, 2021 at 17:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.