When to model a problem by using the Bayes' theorem?

Question

I have a labeled training dataset where each observation has a sentence either in English or in French as its predictors and its label (target value) is whether this sentence is English or French. The test set includes again some sentences either in English or in French but without labels.

A friend of mine suggested that we should model this problem by using the Bayes' theorem since we have have some prior values (labeled observations in training set). I agree that this can work too but I cannot really understand his argument "we should model this problem by using the Bayes' theorem since we have have some prior values".

This is because in my mind every labeled observation can be considered as a prior value and every prior value can be considered as a labeled observation so you can also apply any machine learning classification algorithm e.g. decision trees) in these cases.

Is this right in general or at least for this specific problem?

Why Bayes' theorem modeling comes up as the best solution for the problem which I described above?

Answer 1

You can model your problem with Bayes' theorem. In particular, naive Bayes classifer can be used for binary classification of text data.

Priors in a naive Bayes classifer refers to the base rate for the different classes (i.e., Are there more English or French sentences?)

Naive Bayes may or may not be the best solution. Typically, the best solution is chosen empirically by using predictive performance on a hold-out dataset.

Answer 2

Bayesian classifiers are very good at using words to tokens to classify. use MultinomialNB like any other classifier to predict outcomes. I found for my case that LogisticRegression outperform MultinomialNB, but your dataset may produce different results

paragraph="Herbert Simon research and concepts increased computer scientist understanding of reasoning and increased the computer's ability too solve problems and proof theorems . Herbert Simon , Al Newell , Clifford Shaw proposals were radical and affect computer scientist today . In Simon’s book , “Models of my life” , Simon demonstrated the Logical Theorem algorithm could prove certain mathematical theorems . Simon said , “This was the task to get a system to discover proof for a theorem , not simply to test the proof . We picked logic just because I happened to have Principia Mathematica sitting on my shelf and I was using it to see what was involved in finding a proof of anything . ” Alfred North Whitehead and Bertrand Russell book Principia Mathematica contained theorems considered to form the foundation of mathematical logic . Simeon evolved Logic theorem into General problem solver . GPS is currently used in robotics and gives the robot amazing problem solving capabilities . Many mathematicians considered some of LTs proofs superior to those previously published"

sentences = nltk.sent_tokenize(paragraph)

words=[]
for sentence in sentences:
    word_list=nltk.word_tokenize(sentence)
    
    #print(word_list)
    for i in range(0, len(word_list)-1):
        words.append(word_list[i])
    
print(words)

def return_weights(vocab, original_vocab, vector, vector_index):
    zipped = dict(zip(vector[vector_index].indices, vector[vector_index].data))
    
    # Let's transform that zipped dict into a series
    zipped_series = pd.Series({vocab[i]:zipped[i] for i in vector[vector_index].indices})

# Let's sort the series to pull out the top n weighted words
zipped_index = zipped_series.sort_values(ascending=False).index

return [original_vocab[i] for i in zipped_index]
NUMERIC_COLUMNS=[]
LABELS=[]
def combine_text_columns(data_frame, to_drop=NUMERIC_COLUMNS + LABELS):
    """ converts all text in each row of data_frame to single vector """
    
    # Drop non-text columns that are in the df
    to_drop = set(to_drop) & set(data_frame.columns.tolist())
    text_data =data_frame.drop(to_drop,axis=1)
    
    # Replace nans with blanks
    text_data.fillna("",inplace=True)
    
    # Join all text items in a row that have a space in between
    return text_data.apply(lambda x: " ".join(x), axis=1)


get_text_data=FunctionTransformer(combine_text_columns,validate=False)

pipeline = Pipeline([
    ('vect', CountVectorizer(stop_words='english',lowercase=True)),
    ("tfidf1", TfidfTransformer()),
    ##('vectorizer',TfidfVectorizer(stop_words='english')),
    ##('chi', SelectKBest()),
    
     ('scale', MaxAbsScaler()),
    #('clf', LogisticRegression(C=1e5)),
    ('clf', MultinomialNB()) 
    #('clf', SGDClassifier(loss='hinge', penalty='l2',alpha=1e-3, random_state=42, max_iter=5, tol=None)),
])

sentences = nltk.sent_tokenize(paragraph)

tfidf_vec = TfidfVectorizer(stop_words='english')
text_tfidf = tfidf_vec.fit_transform(sentences)

shape=text_tfidf.get_shape()
vocab= {v:k for k,v in tfidf_vec.vocabulary_.items()}

df=pd.DataFrame(columns=['Index','Text','Tfidf','Target'])
for index in np.arange(shape[0]):
    weights=return_weights(vocab,tfidf_vec.vocabulary_,text_tfidf,index)
    target=vocab.get(np.max(weights))
    index=len(df)
    #df.loc[index]=[text_tfidf[index].toarray(),target]
    df.loc[index]=[index,sentences[index],text_tfidf[index].toarray(),target]

df.set_index('Index')
print(df.head(10))    

#X=df[['Index','Text']].values
#y=df['Target'].values.astype(str)

encoder = LabelEncoder()
df['Target']=encoder.fit_transform(df['Target'])

train,test=train_test_split(df,test_size=.6,random_state=42, shuffle=True)
pipeline.fit(train['Text'],train['Target'])

predictions=pipeline.predict(test['Text'])
print(test['Target'],predictions)

score = f1_score(test['Target'],predictions,pos_label='positive',average='micro')
print("score of Naive Bayes algo is :" , score)