I am using information gain feature selection technique to get different features subset sizes for my dataset, like so:
fs1 = SelectKBest(score_func=mutual_info_classif, k=10) fs1.fit(X_train, y_train) X_train_fs1 = fs1.transform(X_train) X_test_fs1 = fs1.transform(X_test) fs2 = SelectKBest(score_func=mutual_info_classif, k=20) fs2.fit(X_train, y_train) X_train_fs2 = fs2.transform(X_train) X_test_fs2 = fs2.transform(X_test) fs3 = SelectKBest(score_func=mutual_info_classif, k=30) fs3.fit(X_train, y_train) X_train_fs3 = fs3.transform(X_train) X_test_fs3 = fs3.transform(X_test)
I am then testing the performance of 4 different algorithms (Logistic Regression, SVM, AdaBoost and Decision Trees) using different subset sizes of feature selected features (subset 1 has k=10, so 10 features, subset 2 has 20 features, etc.). To evaluate the model's performance, I am calculating the Precision, Recall and AUC, like so:
def compareAlgorithms(X_train, y_train, score): # Compare Algorithms seed = 7 # prepare models models =  models.append(('LR', LogisticRegression())) models.append(('SVM', SVC())) models.append(('Linear SVC', LinearSVC())) models.append(('ADABOOST', AdaBoostClassifier())) models.append(('DT', DecisionTreeClassifier())) # evaluate each model in turn results =  names =  scoring = score print(score, ":") for name, model in models: skf = StratifiedKFold(n_splits=5, shuffle=False, random_state=seed) #kfold = model_selection.KFold(n_splits=5, random_state=seed) cv_results = model_selection.cross_val_score(model, X_train, y_train, cv=skf, scoring=scoring) results.append(cv_results) names.append(name) msg = "%s: %f (%f)" % (name, cv_results.mean(), cv_results.std()) print(msg) return results, names
As I now have lots of results, I am trying to plot the results to better visualise which algorithm is performing better with which subset. I want to create plots similar to the ones I found in this article:
I have tried using matplotlib to do this, but have found it quite difficult seeing as I am trying to plot different classifiers on different features subset. I can (sort of) plot a line plot of the algorithms performance for one data subset using this function:
def plot(results,names, score): import matplotlib.pyplot as plt # plot for algorithm comparison fig = plt.figure() fig.suptitle(score) ax = fig.add_subplot(111) plt.plot(results) ax.set_xticklabels(names) plt.show()
Which results in this plot:
The problem with the above plot (beside the overlapping model names below which I am working on fixing) is that it is for one feature subset.
Can anyone help me do a plot like the one in the paper I have attached and maybe direct me to useful resources for someone just starting to learn about data visualisation?