K-means clustering on the data frame having only one column

Question

unsup_df is a DataFrame which has only one column: review.

I want to form 2 clusters of the reviews. One positive and one negative.

from sklearn.cluster import KMeans

tfidf_vectorizer = TfidfVectorizer()  
tfidf_matrix = tfidf_vectorizer.fit_transform(unsup_df)  
num_clusters = 2  
km = KMeans(n_clusters=num_clusters)  
km.fit(tfidf_matrix)   
clusters = km.labels_.tolist()

The above piece of code is throwing an error:

ValueError: n_samples=1 should be >= n_clusters=2

on the line km.fit(tfidf_matrix)

Answer 1

Here is how to fit k-means to a single-dimensional text data in Pandas:

import pandas as pd
from sklearn.cluster import KMeans
from sklearn.feature_extraction.text import TfidfVectorizer

df = pd.DataFrame({"corpus": ["I am Sam. Sam-I-am",
                              "That Sam-I-am! That Sam-I-am! I do not like that Sam-I-am",
                              "Do you like green eggs and ham?",
                              "I do not like them, Sam-I-am. I do not like green eggs and ham"]})

X = TfidfVectorizer().fit_transform(df.corpus)
km = KMeans(n_clusters=2).fit(X)
km.labels_.tolist()

Answer 2

Your unsup_df must be in the wrong shape. Otherwise, it should work.