I have just read the paper from Ian Goodfellow et al. titled "Maxout Networks".

It seems that the Maxout activation should be quite powerful, as it can approximate any convex function, i.e. Relu, function that is used in many state-of-the-art models. Also, Maxout, in theory, should work better with dropout than other activation functions.

I am therefore wondering, what is the intuition for using Relu instead of Maxout.

  • 1
    $\begingroup$ Not enough bang for your variable-number buck? More variables means more variance. Welcome to the site! $\endgroup$ – Emre Nov 16 '17 at 22:29
  • $\begingroup$ Agree with @Emre that more variables can lead to overfitting. $\endgroup$ – tom Nov 16 '17 at 22:57
  • $\begingroup$ Thanks! So assuming that we have much more data, maxout should work better than relu, because overfitting will not be an issue? $\endgroup$ – Kuba Nov 17 '17 at 8:51
  • $\begingroup$ Perhaps, but is it the best, or even a good use of model complexity? That is the question. $\endgroup$ – Emre Nov 17 '17 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.