Let's imagine this list:
corpus = ['cat','banana','dog','horse','apple','tiger','snake']
I am looking for a way to build a vectorizer with dynamic clustering. I'm referring to a process like such:
- Take the first item as a class.
- Look at the second, evaluate if it belongs to the same class (it doesn't but it doesn't know yet), and classify it in class .
- Get to the third item and evaluate that two of those three are far more closely related, therefore create class  for fruits and reevaluate all of its previous assumptions.
- Go over the whole corpus in this way.
- Then, if 'plane' is added to the corpus, can again evaluate it doesn't fit the previous classes and create a class.
So far, I worked with Multinominal Naive Bayes and Support Vector Machine in order to solve this problem. Both of them worked fine on discriminating between datasets that were already labeled. They however failed to achieve what I was looking for. The accuracy is great, but the classes are non-dynamic.
The clustering is destined to be used for authorship identification on conversational datasets spanning different topics and themes. I tried models like Word2vec but the result was not right either.
As far as expectations go, I'd like to know if anybody ever build something similar, or if there are already models and vectorizers out there that I could use to accomplish such a task. (it is entirely possible that SVM already allows for dynamic clustering and I just didn't get it while reading its documentation).
In short: How would you go about this problem?