Is there a closed form solution for a ReLU that is bounded with a maximum value at 1? I am trying to produce output values for pixel intensities 0 <= x <= 1, but my outputs are producing values greater than 1. How can I counteract this?
Being itself a non-linearity, the very advantage of ReLUs is their linear regime, that prevents it from saturating too early.
Having said that, a couple of alternatives for your case are to either clip the output of the ReLUs or to place sigmoid (or tanh, adjusting the output range) after them (or instead of them).