SBERT Embeddings from Conversations

Question

I have a dataset consisting of text-based conversations between two humans. One conversation has on average 20 turns and can look as follows:

Person 1: Do you like cooking?
Person 2: Yes. I like cooking very much. I got this hobby when I was 12 years sold.
Person 1: Why do you like it?
Person 2: I have no idea. I like cooking by myself. I like to taste delicious food.
...

With SBERT I can get the embeddings of one turn (e.g., "Hello there, how are you doing?"). Is it also possible to get one embedding with SBERT for several turns or a whole conversation (20 turns)? Are there other models which are capable to do this or are more recent? Afterward, I would like to project the embedding to 2D or 3D space and apply clustering.

Answer 1

SBERT supports batch inference, which means that it can process multiple sentences at the same time. For instance (source):

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')

#Our sentences we like to encode
sentences = ['This framework generates embeddings for each input sentence',
    'Sentences are passed as a list of string.', 
    'The quick brown fox jumps over the lazy dog.']

#Sentences are encoded by calling model.encode()
embeddings = model.encode(sentences)

#Print the embeddings
for sentence, embedding in zip(sentences, embeddings):
    print("Sentence:", sentence)
    print("Embedding:", embedding)
    print("")

The limit for how many sentences you can fit will depend on their length and the amount of memory in your GPU/CPU. Note that the needed amount of memory is quadratic on the sequence length. You should find the appropriate limit for the number of sentences empirically.

About projecting the embeddings and clustering, I suggest you use UMAP for the dimensionality reduction and then k-means for clustering.