Dropout is a widely used technique in deep learning. Dropout was built for neural networks, but I wonder if other prediction models can use this idea as well as a regularizer. Do you know of any similar technique in linear regression, SVMs or tree-based methods?
$\begingroup$
$\endgroup$
asked Apr 20, 2018 at 14:30
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
Random forests could be thought of as using a kind of dropout-esque technique as each split node only considers a random subset of the features, effectively 'dropping out' the other ones. Also, sometimes in large tree ensembles, each tree is only given a random subset of features to begin with, akin to dropout on the input layer of a neural network.
-
$\begingroup$ Yeah, I guess it is difficult with tree-based methods because they are non parametric, and dropout strongly relies on the parameters. $\endgroup$ Apr 20, 2018 at 22:50