Naive Bayes as a baseline model in an NLP task

Question

I want to use the Naive Bayes model as a baseline in an classification task that I am working. I found this really useful tutorial: https://www.geeksforgeeks.org/applying-multinomial-naive-bayes-to-nlp-problems/ and I want to apply it into my problem.

My dataset has a dataframe form with rows the texts and coloums the labels, original text, preprocessed text, etc.

The code that I have is this one

# cleaning texts
import pandas as pd
import re
import nltk
from nltk.corpus import stopwords
from nltk.stem.porter import PorterStemmer
from sklearn.feature_extraction.text import CountVectorizer
 
nltk.download('stopwords')
 
corpus = []
for i in range(0, len(df)):
    text = df['Preprocessed_Text'][i]
    text = ''.join(text)
    corpus.append(text)

# creating bag of words model
cv = CountVectorizer(max_features = 1500)
 
X = cv.fit_transform(corpus).toarray()
y = df.iloc[:, 5].values

# splitting the data set into training set and test set
from sklearn.cross_validation import train_test_split
 
X_train, X_test, y_train, y_test = train_test_split(
           X, y, test_size = 0.25, random_state = 0)

from sklearn import naive_bayes
# fitting naive bayes to the training set
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import confusion_matrix
 
#classifier = GaussianNB()
classifier = MultinomialNB()
classifier.fit(X_train, y_train)
 
classifier.score(X_test, y_test)

I have the following two questions:

1. In the pre-processed part the is it correct this one text = ''.join(text)? Or a space (text = ' '.join(text)) is needed?

2. By this approach we apply first the bag-of-words in the whole dataset and then split it into training and testing. Is there any way to do this in the opposite way? Therefore, first splitting the texts into training and testing and then apply bag of words.

Update: I have already split the texts into training validation and testing in a listing form, thus

# evaluation sets
texts_train
auhors_train

texts_valid
authors_valid

texts_test
authors_test

Then I'm applying the Bag of Words method only in the training and transform the other evaluation sets with the following way

# creating bag of words model
from sklearn.feature_extraction.text import CountVectorizer

cv = CountVectorizer(max_features = 1500)
 
cv.fit_transform(texts_train).toarray()

bow_train = cv.transform(texts_train)
bow_vald = cv.transform(texts_valid)
bow_test = cv.transform(texts_test)

Last step is to fit a classifier and measure its accuracy

# Fitting naive bayes to the training set
from sklearn import naive_bayes
from sklearn.naive_bayes import MultinomialNB

classifier = MultinomialNB()
classifier.fit(bow_train, authors_train)
 
classifier.score(bow_vald, authors_valid)

Answer 1

1. Assuming that the Preprocessed_Text column contains a regular string, you don't have to do any kind of join since you variable text is a single string.
2. It's indeed recommended to calculate the bag of words representation only on the training set. It's "cleaner" in the sense that it prevents any possible data leakage, and it's more coherent with respect to applying the model to any fresh test set. However in this case there might be out-of-vocabulary words in the test set, this is normal. You can do this by splitting the "preprocessed text" data first, then calling cv.fit_transform() only on the training instances. Later the test set instances are encoded using cv.transform(), which just maps the new text using the previous representation.

Answer 2

1. You should use space in the join statement. Otherwise the returned text will not be readable as it will show without any space in between the words Example - With ' '.join(text) - It is raining With ''.join(text) - Itisraining
2. You can do both way.