1
$\begingroup$

The data set X has 10 features with 50 instances labelled as 0 and 1. Considering only 6 instances as an example here, let YPred labels are [1,0,1,0,1,1] and the actual ground truth labels are YTest = [0,0,1,1,1,0]. I cannot draw a decision boundary after classification since the data set is multi-dimensional. In the following code, pred = predict(svmModel, X(testIdx,:)); runs k times.

I cannot understand which predicted class labels should be taken after the cross-validation ends to say that this is the final "good" prediction.

$\endgroup$

1 Answer 1

2
$\begingroup$

The whole essence of cross-validation is to check the model with various sets of data and to know how well it predicts with 'unseen' data. So, with the 10 folds, you create 10 different training and test/validation set. But, the model remains the same.

What you are trying to ask is which model to choose. There is only one model. We are only checking the model that's already built using cross-validation.

After checking the model you have built, use the complete dataset to train and predict the labels on test set using this final model.

Usually, the dataset is split into training, validation and test set. Test set is not touched till the final model has been built. The test set you create during cross-validation is actually the validation set. The ability of the model to predict well is validated on this set.

Hope it helps!

Update: fitcsvm() is what trains the data. This is the modelling bit. What you are doing is right. What you need to understand is that cross-validation is used for checking the model. There would be cases where one model wouldn't predict as well as another one. Testing with only one test would not give good insights on how well the model is working. This is why, we use cross-validation - to test the model with different training and test sets.

10 folds for only 50 records might be an overkill. 2-3 folds would suffice.

$\endgroup$
3
  • $\begingroup$ Thank you for your help again. I don't quite follow how to train the dataset again since the model is already build. MATLAB has 2 functions - fitcsvm() which is used in the training and predict which is used in testing. How do I use fitcsvm again I don't know. $\endgroup$
    – Srishti M
    Jun 11, 2018 at 16:26
  • $\begingroup$ Thank you for the update. But the sentence "After checking the model you have built, use the complete dataset to train" is unclear. After using fitcscm() to train and then using predict() on the validation set, how and why do I again use the complete dataset to re-train as you mentioned in this sentence? $\endgroup$
    – Srishti M
    Jun 12, 2018 at 15:23
  • $\begingroup$ Okay. As mentioned in the next paragraph (which shouldn't have been in the same paragraph, my bad), this is done when the training dataset and test dataset can be clearly distinguished. What you are currently predicting on is the validation dataset. So, when you have a test set that has been untouched during modelling and kept aside for the final prediction, you use the training and validation set as a whole and run the model on this dataset. Why do you need to do this? The more data, the better! $\endgroup$
    – aathiraks
    Jun 12, 2018 at 15:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.