I do the (text) topic classification using TfidfVectorizer and SGDClassifier, literally I want to classify the website into categories (like Sport, Business etc).
Now, the problem is, that each website might fit into multiple categories eg (IT and Eshop, Sport and Eshop) etc.
The question can have 2 parts on is how to do it technically (Python, SKLEARN) another is a theoretical question.
Let me explain the latter.
As far as I understand how the text classification works, it cannot work for multiclass (eg adding another Eshop class output) because, the IT eshop is full of IT keywords => it is easy to classify it as IT related site.
But if i take 3 IT eshops sites and add it in the testing data both as IT and Eshop category, the keywords (word vectors) for these both categories will be same, and literally only IT focused.
Let's say I add 3 more Sport eshops and the problem is still same. The new category Eshop won't get only the clean expected keywords like "Shopping cart, Checkout" etc, but it will keep the same word vectors for IT and Sport.
Am I right, yes?
Even if I had some extra solo classification to mark the eshops or some meta categories like Eshop, Small business, the problem is still same, it will still keep the top words which are more or less related to topic like IT, Sport.
So the question is, how to do the multi class classification (I am ok with the 1 class output of SGDClassifier as long as it will mark more classes with .predict_proba() method)
Then how to deal with this problem of multi classes like IT + Eshop, or IT + something else.
And then I realize that Eshop is maybe too much "meta" category so maybe it needs some other logic or idea, than TfidfVectorizer. (Maybe some category related keywords across all documents in same category as I mentioned above, or other method like Neural Network or I don't know).
