2
$\begingroup$

Dropout is a widely used technique in deep learning. Dropout was built for neural networks, but I wonder if other prediction models can use this idea as well as a regularizer. Do you know of any similar technique in linear regression, SVMs or tree-based methods?

$\endgroup$
2
$\begingroup$

Random forests could be thought of as using a kind of dropout-esque technique as each split node only considers a random subset of the features, effectively 'dropping out' the other ones. Also, sometimes in large tree ensembles, each tree is only given a random subset of features to begin with, akin to dropout on the input layer of a neural network.

$\endgroup$
  • $\begingroup$ Yeah, I guess it is difficult with tree-based methods because they are non parametric, and dropout strongly relies on the parameters. $\endgroup$ – David Masip Apr 20 '18 at 22:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.