# classifier predicts only one class [closed]

I have a classification that has to predict three different classes: gcc,icc, clang. The prblem is that if I use a blind test set to do a submission, when I look athe the prediction I have on it I find that most of the predictions are only of one type. So I have the following situation:

My code is the following:

#pakages
import numpy as np
import pandas as pd
import json as j
import re
import nltk
from nltk.tokenize import word_tokenize

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import CountVectorizer

from sklearn.naive_bayes import GaussianNB
from sklearn.metrics import confusion_matrix, classification_report
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier

#for visualizing data
import matplotlib.pyplot as plt
import seaborn as sns; sns.set(font_scale=1.2)

%matplotlib inline

print('Libraries imported.')

dataFrame = pd.read_json('training_dataset.jsonl',lines = True)
dataFrame

dataFrame['opcodes'] = dataFrame['instructions'].apply(lambda x:[i.split()[0] for i in x])

df = dataFrame[["opcodes", "compiler"]].copy()
df

df['opcodes'] = [" ".join(opcode) for opcode in df['opcodes'].values]

from sklearn.feature_extraction.text import TfidfVectorizer

tfidf_vectorizer=TfidfVectorizer(ngram_range=(2,2),min_df=2,max_df=0.5)

df_x = df['opcodes']

X_all = tfidf_vectorizer.fit_transform(df_x)
y_all = dataFrame['compiler']

X_train, X_test, y_train, y_test = train_test_split(X_all, y_all,
test_size=0.2, random_state=15)

from sklearn import svm
from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier

clf = RandomForestClassifier(n_estimators=100).fit(X_train,y_train)

y_pred = clf.predict(X_test)

acc = clf.score(X_test, y_test)
print("Accuracy %.3f" %acc)


Now, using a random forest classifier I get an accuracy of 0.89, but the problem is that the model predicts mostly the class gcc, so I guess something is wrong.

I was almost soing a submission when I noticed this.

Does this happen because I didn't consider the fact that I have a multiclass classification problem? I really don't understand.

[EDIT] The dataset is balanced.

[EDIT 2] When try to do the prediction on the blind test set I do the following:

test = pd.read_json('test_dataset_blind.jsonl',lines = True)
test

test['instructions'] = [" ".join(opcode) for opcode in
test['instructions'].values]

tfidf_vectorizer=TfidfVectorizer(ngram_range=(2,2),min_df=2,max_df=0.5)
df_x_new = test['instructions']

X_new = tfidf_vectorizer.fit_transform(df_x)

new_pred_class = clf.predict(X_new)[:3000]
len(new_pred_class)

sub_compiler = pd.DataFrame({'instructions':
test['instructions'],'compiler':new_pred_class})
sub_compiler.to_csv('sub_compiler.csv')


[EDIT 4]anctually the problem seems to be on how I select the data to test. Infact, if I do new_pred_class = clf.predict(X_new)[-3000:], I get:

 so this time mostly the clang are predicted.

I have 30000 rows in the original dataset and 3000 in the blind test set.

[EDIT 5]So I think I have to reorganize the select the new_pred_class taking random values from clf.predict(X_new)[:3000], so what I mean is that I should take 3000 random values. How could I do that? Thank's again.

EDIT 6] if I do type(X_new) I have scipy.sparse.csr.csr_matrix

• try clf.predict(X_new.sample(3000))` – oW_ Nov 9 '19 at 21:45
• Thank's for the answer. It gives me the error : AttributeError: sample not found – J.D. Nov 9 '19 at 21:51
• I solved using .resample(). Thank you. – J.D. Nov 9 '19 at 22:31