# Clustering with k-means for text classification based on similarity

I have a column that contains all texts that I would like to cluster in order to find some patterns/similarity among each other.

Text
Word2vec is a two-layer neural net that processes text by “vectorizing” words.

Its input is a text corpus and its output is a set of vectors: feature vectors that represent words in that corpus.

While Word2vec is not a deep neural network, it turns text into a numerical form that deep neural networks can understand.

Documentation and examples for common text utilities to control alignment, wrapping, weight, and more.

NounEdit. text (countable and uncountable, plural texts). A writing consisting of multiple glyphs, characters, symbols or sentences.

Fish are gill-bearing aquatic craniate animals that lack limbs with digits.


I tried to determine the optimal number of clusters with elbow method, but unfortunately with no success. So I remove the stopwords and vectorised texts in the column:

import re
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.cluster import KMeans
from nltk.tokenize import word_tokenize
from nltk.stem import WordNetLemmatizer
import nltk
from nltk.corpus import stopwords
from sklearn.pipeline import Pipeline

stop_words = stopwords.words('english')

def preprocessing(line):
line = re.sub(r"[^a-zA-Z]", " ", line.lower())
words = word_tokenize(line)
words_lemmed = [WordNetLemmatizer().lemmatize(w) for w in words if w not in stop_words]
return words_lemmed

vect =TfidfVectorizer(tokenizer=preprocessing)
vect_text=vect.fit_transform(df['Text'])
kmeans =KMeans(n_clusters=3).fit(vect_text)

n_clusters=3, n_init=10, n_jobs=1, precompute_distances='auto',


Is it right to cluster the texts above based on their similarity using this approach?