I have a text column in my data frame which contains paragraph(s) having multiple and variable sentences in each instance/example/row of the dataframe. Then, I created the sentence tokens of that paragraph using
sent_tokenizer of nltk and put it into another column.
So my data frame looks like this:
index text class 0 ["Hello i live in berlin", 'I'm xxx'] 1 1 ["My name is xx", "I have a cat", "Love is life"] 0
now when I'm using:
from sentence_transformers import SentenceTransformer model = SentenceTransformer('distilbert-base-nli-stsb-mean-tokens') sentences = df['text'] sentences = sentences.tolist() embeddings = model.encode(sentences)
TypeError: expected string or bytes-like object
The encode method is not taking a list of list of sentences as an argument.