0
$\begingroup$

I want to prepare a deep learning model for text classification(Document classification), But in my training content many place have name, address, brand name, etc.. which will do confuse to model as deep learning is work with sequence of words, I was thinking if we will train model by removing these details, but in predict content when we get name, address, brand name then it also confuse...

Can someone give the idea how i handle this situation in deep learning?

And which deep learning algorithm and library should use for best accuracy and speed for text classification.

$\endgroup$
1
$\begingroup$

Normally if you have enough training data this shouldn't be an issue, because the model will realize that these named entities are not relevant for classifying the text.

Anyway if you want to remove them you would need to first detect them with Named Entity Recognition, there are many implementations. Then you can simply remove them.

Using DL classification as a first option is questionable imho:

  • it usually requires larger training data
  • it takes much longer to train than traditional models
  • it's harder to understand what happens when things go wrong

I'm old school so I'd recommend starting with simple methods, for instance TFIDF vectors with Naive Bayes or Decision Trees.

$\endgroup$
2
  • $\begingroup$ I implemented Naive Bayes with TFIDF but it does not gives good accuracy, So i thought try using deep learning. Anyway thanks for your idea. $\endgroup$
    – Rajesh das
    Sep 25 '20 at 13:29
  • $\begingroup$ Performance depends (1) on the task: for instance, how easy is it for you to find the correct label given a document? If it's not easy for you, chances are that it's not easy for ML either. (2) on the data: how many documents? how many features do you use? If you have more features than documents it's likely to overfit (and in this case it's probably going to be worse with DL). $\endgroup$
    – Erwan
    Sep 25 '20 at 13:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.