1
$\begingroup$

This is in continuation of my earlier post.

In my previous model, I used just two features which couldn't fare well and gave 71% prediction (accuracy) score. Now, I'm trying to consider another important field "subject" from the data set. I'm planning to use NLTK package along with Tf-Idf.

I generated the below using NLTK (FreqDist) on the subject field - enter image description here Here is my idea: I'll tokenize the subject field, apply stopwords, then Tf-Idf and get the top N most frequent words and use them as features. The Tf-Idf score will be my values. In total, I'll have (N+2) features.

But, one downside is - the feature matrix will be huge (say 200+ features) and sparse. Is that a concern? Does this design make sense?

Your thoughts please.

Improve this question
$\endgroup$
0

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.