0
$\begingroup$

I am trying to build a multi class text-classifier that classifies whether the tweet belongs to one of the categories ( Advise or Science or others )
let the input be any tweet like this ,
Input :

The goal of teaching should not be to help the students learn how to 
memorize and spit out information under academic pressure. Brain

The purpose of teaching is to inspire the desire for learning in them and 
make them able to think, understand, and question.

#maths #richardfeyman

Output :
( Learning : 98% , others : 1 % , Science : 1 % )
For now i came up with this
Idea 1 :
Maybe i can use LDA to get the topics of the tweet and perform semantic matching between these labels ( learning , science , others ) so one with the highest score can be choosen as right one (argMax(...) ).
What do you think about the above idea ?
Can anyone please enlighten me or point me in the right direction .

$\endgroup$
2
$\begingroup$

LDA is amazing for topic modelling, but naming the topics (regardless of what algorithm you use) is inevitably a disappointing process, because the labels you’re thinking of are very subjective and the algorithm has little respect for what words we want to go in each topic :p

The way I would approach your problem is:

  • Train an LDA topic model, using grid search rather than intuition to select the right number of topics
  • Transform your training examples into k-dimensional vectors using your trained topic model
  • Add your training vectors to a k-d tree and classify new tweets using a k-nearest neighbours approach
$\endgroup$
1
  • $\begingroup$ thanks for your answer i would definitely try this approach for sure. $\endgroup$ – guru_007 Apr 16 '20 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.