I am performing an online news classification. The idea is to recognize group of news of the same topic. My algorithm has these steps:
1) I go through a group of feeds from news sites and I recognize news links.
2) For each new link, I extract the content using dragnet, and then I tokenize it.
3) I find the vector representation of all the old news and the last one using TfidfVectorizer from sklearn.
4) I find the nearest neighbor in my dataset computing euclidean distance from the last news vector representation and all the vector representations of the old news.
I have a problem using TfidfVectorizer because it weights more the special words that only appear in a few news, like Apple, and news that talk about Aple are grouped together even when they deal with different topics.
So, Is there a common approach more efficient than the one I am using?