2
$\begingroup$

I have just read the paper from Ian Goodfellow et al. titled "Maxout Networks".

It seems that the Maxout activation should be quite powerful, as it can approximate any convex function, i.e. Relu, function that is used in many state-of-the-art models. Also, Maxout, in theory, should work better with dropout than other activation functions.

I am therefore wondering, what is the intuition for using Relu instead of Maxout.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Not enough bang for your variable-number buck? More variables means more variance. Welcome to the site! $\endgroup$
    – Emre
    Commented Nov 16, 2017 at 22:29
  • $\begingroup$ Agree with @Emre that more variables can lead to overfitting. $\endgroup$
    – tom
    Commented Nov 16, 2017 at 22:57
  • $\begingroup$ Thanks! So assuming that we have much more data, maxout should work better than relu, because overfitting will not be an issue? $\endgroup$
    – Kuba
    Commented Nov 17, 2017 at 8:51
  • $\begingroup$ Perhaps, but is it the best, or even a good use of model complexity? That is the question. $\endgroup$
    – Emre
    Commented Nov 17, 2017 at 19:17

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.