I am facing some issues with a text classification problem and need your help to understand the best way to approach it.

The problem statement is as follows: Assume I have a set of sentences describing a certain product (assume they are furniture items). Each of the sentences needs to be tagged with a class coming from a roster of 10 possible classes. The 10 classes refer to possible attributes of the item, such as softness, brightness etc. On top of this, sentences contain sentiment information on each item (good, bad, neutral). Example: "I am satisfied with the performance of this couch, it is really soft".

The first question is: how would you approach the problem of labeling each sentence with both classes AND sentiment?

Now, to make things even more complex, assume each sentence can describe up to 3 items, with relative sentiment. How would you approach the multi-ouput sentiment+class problem in this case?

Thanks a lot in advance for any help!

  • $\begingroup$ The way you are using the vocabulary is a bit confusing. Your database consists of what kind of data? You then have 10 possible classes for each of the three outputs? $\endgroup$
    – JahKnows
    Mar 17 '17 at 16:51
  • $\begingroup$ Have you made any progress on this ? I m trying to develop a machine learning model to solve a multi-label classification + sentiment analysis problem. Any help would be appreciated. $\endgroup$ Apr 18 '18 at 16:21

The "low hanging fruit" approach is "bag of words".

Lemmatize the sentence and use the lemma frequencies as features.

Build two models:

  1. Sentiment (or reuse StanfordCoreNLP)
  2. Topic (for this you need a training sample - a set of sentences labeled with the topic).

Now you can map every sentence to topic and sentiment and this will tell you whether the customer liked or disliked (sentiment) the softness (topic).

Multi-topic sentences

Usually a sentence has a single topic, but, if you do require multiple topics per sentence, you can use predict_proba or similar and use all classes with probability over, say, 0.3.

Class proliferation

The above approach fails on sentences like

I liked the softness but hated the design.

because the total (average) sentiment of the sentence is probably neutral and we miss out softness-good and design-bad.

It is thus tempting to triple the number of classes (topic-sentiment) to accommodate reviews like

The table combines pleasant design, flimsy build and neutral color.

The problem is that building models for very many classes requires a lot of training data. IOW, you will need to manually annotate a lot of reviews - and even then the model quality if unlikely to meet your expectations.

Real NLP

If you really need to handle multi-topic sentences, you might want to consider a "deeper" approach.

Specifically, take a look at the parse tree of the sentence.

  1. split it into constituent phrases
  2. use the topic model to identify the unique topic of the each phrase
  3. use the sentiment model on the phrase
  • $\begingroup$ Thanks sds. Dual sentiment/topic modeling is the approach I was considering as well, but as far as I understand it has the issue that I'd be unable to correctly map multiple sentiments to different topics on the same sentence. In other words, sentiment classification is bound to the sentence, and I'm unable to use fuzzy classification. To solve this, I was thinking of increasing classes multiplicity by creating e.g. "softeness-positive", "softness-negative" etc. kind of classes. I can't think of any other method right now but curious to hear your opinion on this? $\endgroup$
    – intoML
    Mar 21 '17 at 14:22
  • $\begingroup$ @intoML: I don't think this is a viable approach - very many classes require a LOT of training data. Please see edit. $\endgroup$
    – sds
    Mar 21 '17 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.