I am not an expert user. I know that I can obtain the confusion matrix, but I would like to obtain a list of the rows that have been classified in a wrong way in order to study them after classification.
On stackoverflow I found this Can I get a list of wrong predictions in SVM score function in scikit-learn but I am not sure to have understood everything.
This is an example code.
# importing necessary libraries from sklearn import datasets from sklearn.metrics import confusion_matrix from sklearn.model_selection import train_test_split # loading the iris dataset iris = datasets.load_iris() # X -> features, y -> label X = iris.data y = iris.target # dividing X, y into train and test data X_train, X_test, y_train, y_test = train_test_split(X, y, random_state = 0) # training a linear SVM classifier from sklearn.svm import SVC svm_model_linear = SVC(kernel = 'linear', C = 1).fit(X_train, y_train) svm_predictions = svm_model_linear.predict(X_test) # model accuracy for X_test accuracy = svm_model_linear.score(X_test, y_test) # creating a confusion matrix cm = confusion_matrix(y_test, svm_predictions)
To iterate through the rows and to find the wrong ones, the proposed solution is:
predictions = clf.predict(inputs) for input, prediction, label in zip(inputs, predictions, labels): if prediction != label: print(input, 'has been classified as ', prediction, 'and should be ', label)
I didn't understand what is "input"/"inputs". If I adapt this code to my code, like this:
for input, prediction, label in zip (X_test, svm_predictions, y_test): if prediction != label: print(input, 'has been classified as ', prediction, 'and should be ', label)
[6. 2.7 5.1 1.6] has been classified as 2 and should be 1
Is the row 6 the wrong row? What are the numbers after the 6.? I am asking this because I am using the same code on a dataset that is bigger than this one, so I would like to be sure that I am doing the right things. I am not posting the other dataset because unfortunately I can't, but the problem there is that I obtained something like this:
(0, 253) 0.5339655767137572 (0, 601) 0.27665553856928027 (0, 1107) 0.7989633757962163 has been classified as 7 and should be 3 (0, 885) 0.3034934766501018 (0, 1295) 0.6432561790864061 (0, 1871) 0.7029318585026516 has been classified as 7 and should be 6 (0, 1020) 1.0 has been classified as 3 and should be 8
When I count every line of this last output, I obtain the double of the lines of the test set... So I am not sure that I am analysing exactly the wrong list of predicted results… I hope to have been enough clear.