Visualize the predicted and actual class after training and testing

Question

The data set X has 10 features with 50 instances labelled as 0 and 1. Considering only 6 instances as an example here, let YPred labels are [1,0,1,0,1,1] and the actual ground truth labels are YTest = [0,0,1,1,1,0]. I cannot draw a decision boundary after classification since the data set is multi-dimensional. In the following code, pred = predict(svmModel, X(testIdx,:)); runs k times.

I cannot understand which predicted class labels should be taken after the cross-validation ends to say that this is the final "good" prediction.

Answer 1

The whole essence of cross-validation is to check the model with various sets of data and to know how well it predicts with 'unseen' data. So, with the 10 folds, you create 10 different training and test/validation set. But, the model remains the same.

What you are trying to ask is which model to choose. There is only one model. We are only checking the model that's already built using cross-validation.

After checking the model you have built, use the complete dataset to train and predict the labels on test set using this final model.

Usually, the dataset is split into training, validation and test set. Test set is not touched till the final model has been built. The test set you create during cross-validation is actually the validation set. The ability of the model to predict well is validated on this set.

Hope it helps!

Update: fitcsvm() is what trains the data. This is the modelling bit. What you are doing is right. What you need to understand is that cross-validation is used for checking the model. There would be cases where one model wouldn't predict as well as another one. Testing with only one test would not give good insights on how well the model is working. This is why, we use cross-validation - to test the model with different training and test sets.

10 folds for only 50 records might be an overkill. 2-3 folds would suffice.