# What is the best way to optimize the parameters in a Sklearn classifier, when I have little data?

What is the best way to optimize the parameters in a Sklearn classifier when I only have a data set with 684 rows and 177 columns, and the column I want to predict has 3 labels?

I know I should split my data into training, validation, and test set, and then find the parameters to train the training set that maximizes a metric in the validation set and use this optimized classifier in the test set. However, when I did this using a decision tree classifier, the parameters that worked best for the validation set showed a worse result in the test set than the default parameters. So what is the best way to find the best parameters in this data set? I do not know if it helps, but my data set is very sparse.

• You could look into bagging and boosting. Also, decision tree classifiers have very high variance. You could use a random forest (which aggregates decision tree classifiers to reduce variance), but I would suggest thinking about using a different model. – Andrew Maurer Aug 8 '19 at 4:14

Basically you would apply a multiclass logit with lasso penalty in your case. Alternatively you could also check boosting with lasso (aka $$l1$$ penalty). Here is some playcode with the iris data.