Modulo as activation function in neural network

Question

Can we use a modulo function $f(x)$ as activation function in a neural network? Modulo function is monotonic and continuous (just like Relu) except at a finite number of points in the domain of our input data. By modulo function $f(x)$ I mean

\begin{equation} f(x) = \begin{bmatrix} \vdots \\ x+a \ \ \ \if \ -a < x < 0 \\ x \ \ \ \ \ \ \ \if\ \ \ \ 0 < x < a\\ x-a \ \ \ \if \ \ \ \ a < x < 2a \\ \vdots \\ \end{bmatrix} \end{equation} where a is a positive constant number and could be treated as hyperparameter for simplicity.

I want my output to take values between [0,1] and I am sampling the output from a gaussian distribution 𝑁(𝜇, $\sigma^2$) where 𝜇, $\sigma^2$ are the outputs of neural network. Hence the output may go outside the range [0,1]. I don't want to do clipping because it will create further problems in my network

I am new to Latex, sorry for not using a good formating.

Answer 1

The modulo function is not monotonic. Even though it has a positive derivative except at points where it is not differentiable. It is generally not recommended to use a non-convex Activation function. Hence, it is better to avoid it or change the structure of problem.