Dynamic clustering in NLP dataset

Question

Let's imagine this list:

corpus = ['cat','banana','dog','horse','apple','tiger','snake']

I am looking for a way to build a vectorizer with dynamic clustering. I'm referring to a process like such:

1. Take the first item as a class[0].
2. Look at the second, evaluate if it belongs to the same class (it doesn't but it doesn't know yet), and classify it in class [0].
3. Get to the third item and evaluate that two of those three are far more closely related, therefore create class [1] for fruits and reevaluate all of its previous assumptions.
4. Go over the whole corpus in this way.
5. Then, if 'plane' is added to the corpus, can again evaluate it doesn't fit the previous classes and create a class[2].

So far, I worked with Multinominal Naive Bayes and Support Vector Machine in order to solve this problem. Both of them worked fine on discriminating between datasets that were already labeled. They however failed to achieve what I was looking for. The accuracy is great, but the classes are non-dynamic.

The clustering is destined to be used for authorship identification on conversational datasets spanning different topics and themes. I tried models like Word2vec but the result was not right either.

As far as expectations go, I'd like to know if anybody ever build something similar, or if there are already models and vectorizers out there that I could use to accomplish such a task. (it is entirely possible that SVM already allows for dynamic clustering and I just didn't get it while reading its documentation).

In short: How would you go about this problem?

Answer 1

You are describing the Chinese restaurant process, analogous to seating customers at tables in a restaurant. Customer one (item one) sits at table one (cluster one). The next customer (item two) can either sit at table one (cluster one) or sit at table two (cluster two). This process is repeated for each item.

There needs to be a decision rule to either join an existing cluster or create a new cluster. One example of a decision rule is to measure similarity and use a threshold to decide.

Answer 2

Yeah, there are Algorithms such as DBSCAN or HDBSCAN could be suitable for this task. My approach would be like this I would first gonna vectorize the words in the corpus, so that they can be represented as numerical data. Then I am gonna use word embedding model such as Word2Vec or GloVe. They are quite simple to use. Once I get the numerical vectors for each word, I am gonna apply the clustering algorithm to the data to identify clusters of words that are related to each other. The clustering method would need to be re-run as new words are introduced to the corpus to take into account the fresh information and modify the clusters as necessary.