6
$\begingroup$

Does RELU means to change pixel value to 0 if it is negative anywhere , and later if we apply maximum pooling then what is the use of RELU because in this step we choose maximum value so no matter it is negative or zero if there is any positive value must be greater then 0 or negative number.

$\endgroup$
4
$\begingroup$

Yes RELU means changing the pixel value to zero if it is negative. It matters if we apply RELU or not before max pooling. Let us say your max pooling window is of size 2x2. If an image has only negative values in 2x2 square at particular location then RELU plus max pooling gives a zero as output where as only max pooling would give the maximum negative number.

$\endgroup$
1
  • $\begingroup$ Perhaps worth noting the existence of the maxout transfer function which does do things how the OP describes them, and does work. $\endgroup$ – Neil Slater Jan 12 '17 at 10:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.