For a particular task, I need a convex activation function with the following properties:
- f''(x) > 0
- 0 <= f(x) <= 1
- f(x) is monotonic
- f(x) is not "exploding" i.e. avoiding functions such as f(x) = x²
The only example I have in mind for this is the softplus activation function. Would you have anything else in mind?