I have features extracted from a small dataset, would like to reduce the dimensions by using LDA. Also want to do a SVM classification with k-fold cross-validation.

My question is: What would be the best practice: to do LDA before the CV, or to do LDA within the CV (i.e. to each train and test fold)?


It depends on what you want to achieve.

If you want to visualize the results of your SVM classification, then you should do it after.

If you want to reduce noise, speed up training ... or whatever reason you want to reduce the dimensionality of your problem. You should do it before.

One idea here that can be useful, is to do LDA inside a pipeline and choosing the best hyperparameter within the CV.

In this example you can check a PCA and then a logistic regression. Your case will be similar but with LDA and SVM.

  • $\begingroup$ The goal is to classify two-classes with the SVM. $\endgroup$ – user3885769 Jan 21 at 12:50
  • $\begingroup$ Thanks, I will check the link out. Hence, in this case, doing LDA before the SVM k-fold CV would be the best practice, right? $\endgroup$ – user3885769 Jan 21 at 14:09
  • 1
    $\begingroup$ Yes, it would be better. I am not sure If I would say best practice. For me, best practice would be doing it inside a pipeline and letting CV choosing the best hyper parameter. $\endgroup$ – Carlos Mougan Jan 21 at 14:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.