I have a problem of clustering huge amount of sentences into groups by their meanings. This is similar to a problem when you have lots of sentences and want to group them by their meanings.
What algorithms are suggested to do this? I don't know number of clusters in advance (and as more data is coming clusters can change as well), what features are normally used to represent each sentence?
I'm trying now the simplest features with just list of words and distance between sentences defined as:
$|A \cup B$ \ $A \cap B|$/$|A \cup B|$
(A and B are corresponding sets of words in sentence A and B)
Does it make sense at all?
I'm trying to apply Mean-Shift algorithm from scikit library to this distance, as it does not require number of clusters in advance.
If anyone will advise better methods/approaches for the problem - it will be very much appreciated as I'm still new to the topic.