Sklearn SVM - how to get a list of the wrong predictions?

Question

I am not an expert user. I know that I can obtain the confusion matrix, but I would like to obtain a list of the rows that have been classified in a wrong way in order to study them after classification.

On stackoverflow I found this Can I get a list of wrong predictions in SVM score function in scikit-learn but I am not sure to have understood everything.

This is an example code.

# importing necessary libraries
from sklearn import datasets
from sklearn.metrics import confusion_matrix
from sklearn.model_selection import train_test_split

# loading the iris dataset
iris = datasets.load_iris()

# X -> features, y -> label
X = iris.data
y = iris.target

# dividing X, y into train and test data
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 0)

# training a linear SVM classifier
from sklearn.svm import SVC
svm_model_linear = SVC(kernel = 'linear', C = 1).fit(X_train, y_train)
svm_predictions = svm_model_linear.predict(X_test)

# model accuracy for X_test  
accuracy = svm_model_linear.score(X_test, y_test)

# creating a confusion matrix
cm = confusion_matrix(y_test, svm_predictions)

To iterate through the rows and to find the wrong ones, the proposed solution is:

predictions = clf.predict(inputs)
for input, prediction, label in zip(inputs, predictions, labels):
  if prediction != label:
    print(input, 'has been classified as ', prediction, 'and should be ', label) 

I didn't understand what is "input"/"inputs". If I adapt this code to my code, like this:

for input, prediction, label in zip (X_test, svm_predictions, y_test):
  if prediction != label:
    print(input, 'has been classified as ', prediction, 'and should be ', label)

I obtain:

[6.  2.7 5.1 1.6] has been classified as  2 and should be  1

Is the row 6 the wrong row? What are the numbers after the 6.? I am asking this because I am using the same code on a dataset that is bigger than this one, so I would like to be sure that I am doing the right things. I am not posting the other dataset because unfortunately I can't, but the problem there is that I obtained something like this:

  (0, 253)  0.5339655767137572
  (0, 601)  0.27665553856928027
  (0, 1107) 0.7989633757962163 has been classified as  7 and should be  3
  (0, 885)  0.3034934766501018
  (0, 1295) 0.6432561790864061
  (0, 1871) 0.7029318585026516 has been classified as  7 and should be  6
  (0, 1020) 1.0 has been classified as  3 and should be  8

When I count every line of this last output, I obtain the double of the lines of the test set... So I am not sure that I am analysing exactly the wrong list of predicted results… I hope to have been enough clear.

Answer 1

Welcome to SE:DataScience.

Here [6. 2.7 5.1 1.6] is the feature of the input instance which is classified wrongly. It is one row from your input feature X = iris.data.

The message means: your SVM use the input feature [6. 2.7 5.1 1.6] to predict a label, and it predicts label=2. The ground truth is label=1.

If you want to print the indices of rows that are classified wrongly, you can use

for row_index, (input, prediction, label) in enumerate(zip (X_test, svm_predictions, y_test)):
  if prediction != label:
    print('Row', row_index, 'has been classified as ', prediction, 'and should be ', label)

Answer 2

The following method works for all kinds of classification problem.

Use list comprehension to find all indices of wrong prediction:

indices = [i for i in range(len(y_test)) if y_test[i] != y_pred[i]]

wrong predictions will then be:

wrong_predictions = test_dataframe.iloc[indices,:]

You can also make indices a new column of wrong_predictions, it would be convenient to compare :)

Answer 3

Welcome.

In addition to what user12075 mentioned, you could do:

indices = np.arange(y.shape[0])
X_train, X_test, y_train, y_test, idx_train, idx_test = train_test_split(X, y, indices, stratify=y, test_size=0.3,
                                                                         random_state=42)

Then,

for input, prediction, label in zip (indices[idx_test], svm_predictions, y_test):
  if prediction != label:
    print(input, 'has been classified as ', prediction, 'and should be ', label)