I have a dataset of 39 medical MR images, and I have to build a model to classify the tumor type. so is it suitable to use k-fold cross validation for validating the model? if so, what would be the number of K?

  • 1
    $\begingroup$ LOOCV (k=1) is best given your data. $\endgroup$ – user2974951 Dec 5 '18 at 14:00
  • $\begingroup$ thank you sir for your quick reply, so it is not recomended to use k-fold cross validation? $\endgroup$ – gin Dec 5 '18 at 14:12
  • $\begingroup$ LOOCV is a type of CV when k=n (not k=1 as in my previous comment), so they are similar, but LOOCV will be better in this case since you have a very small dataset and k-fold CV will partion it even further into smaller datasets. $\endgroup$ – user2974951 Dec 5 '18 at 14:19
  • $\begingroup$ you mean by n is the data samples? in my case 39? $\endgroup$ – gin Dec 5 '18 at 15:39
  • $\begingroup$ Yes, n is sample size. $\endgroup$ – user2974951 Dec 6 '18 at 6:30

Given the size of your data set, the best approach to cross validation is the Leave-One-Out method. You haven't discussed the language or package you used for your model, but generally speaking you set the $k$ equal to the number of records. In your case, that 39. This will cause your model to train on 38 instances and predict the 39th, with each instance eventually will receive a classification.

  • $\begingroup$ Thank you sir for your reply, I am using matlab. so you're suggesting using Leave-One-Out method with k=39? $\endgroup$ – gin Dec 5 '18 at 14:14
  • $\begingroup$ I am not sure about MATLAB, but if it has a generic Cross Validation function, you should probably set $k=39$. $\endgroup$ – Skiddles Dec 5 '18 at 14:15
  • $\begingroup$ Did that work for you? I have found that sometimes the measures get out of whack with LOO. It could have been user error, but I just didn't have time to work through it so changed $k$ to something smaller. Of course, as the number of instances increases, using LOO takes a lot longer to complete and it is questionable if a smaller $k$ is any less valuable overall. You really wouldn't want to try LOO with a sizable data set. $\endgroup$ – Skiddles Dec 5 '18 at 15:13
  • $\begingroup$ No I haven't tried. to be honest, I haven't started building the system yet. I just wanted to gather information before starting. so you mean that i should make k smaller than 39 and try the best outcome? $\endgroup$ – gin Dec 5 '18 at 15:42
  • $\begingroup$ Sorry to give you mixed messages. Since your data set is so small, I would set $k=39$ because it will give the model the most data to train with (38 records). If you had 100,000 records, I'd probably set $k=5$ depending on the type of model you are developing. Does that make sense? $\endgroup$ – Skiddles Dec 5 '18 at 15:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.