- Dropout is also usually compared with neural networks ensembles. It seems it has some of the performance benefits of training and averaging several neural networks.
- Dropout is easier to calibrate than regularization. There is only one hyperparameter which is the dropout rate and people widely use 0.5 while training (and then 1.0 on evaluation of course :)), see e.g. this TensorFlow example.
Anyhow, I am a little skeptical of neural networks empirical studies. There are just too many hyperparameters to fine tune, from the topology of the network to the gradient descent optimization procedure to activation functions and whatever it is you are testing like regularization. Then, the entire thing is stochastic and usually performance gains are so small that you can hardly statistical test for differences. Many authors do not even bother doing statistical testing. They just average cross-validation and declare whatever model had the highest decimal point gain to be the winner.
You may find a study promoting dropout only to be contradicted by another promoting regularization.
I think it all boils down to aesthetics preferences. Dropout IMHO sounds more biological plausible than regularization. It also seems easier to calibrate. So, I personally prefer it when using a framework like TensorFlow. If we have to use our own neural network, which we often do, we will use regularization because it was easier to implement.