I am implementing Linear Discriminant Analysis in R, which parameters can be tunned in cross validation set up? In regularized mode called penalizedLDA there are parameters which are optimised but I want to know which parameters are turned in case of simple LDA method?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
LDA has a closed-form solution and therefore has no hyperparameters. The solution can be obtained using the empirical sample class covariance matrix. Shrinkage is used when there are not enough samples. In that case the empirical covariance matrix is often not a very good estimator.
-
$\begingroup$ I made a SVM classifier where I have a nested cross-validation setup for hyper-parameter running. so If I want to compare the accuracy, It is a recommended practice to build the same framework for all the classifiers when predictive performance has to be compared. So, If I use LDA then I can compare it with SVM performance with nested C.V for parameter running? $\endgroup$ Commented Aug 3, 2017 at 21:55
-
$\begingroup$ I have a dataset with 55 observations and 180 features which are non-collinear so can I use PenalisedLDA? $\endgroup$ Commented Aug 3, 2017 at 22:00
-
1$\begingroup$ Yes, compare it on the same test set. And yes, regularization is recommended if you have more features than samples. Whether LDA is appropriate for your dataset also depends on how well your data fits the assumptions of LDA. $\endgroup$– oW_Commented Aug 3, 2017 at 22:13