I have encountered PELU (Parametric Exponential Linear Unit) in the literature, but I can't find practical applications of it.

Moreover, I have some questions about how it works:

  • Are its parameters learned, or are they hyperparameters?

  • Is a Keras Custom layer required for a TensorFlow implementation?

  • In the original paper, it was applied to convolutional models only. Could you share links to applications outside Computer Vision?

  • Computational burden aside, is its performance proven (outside original paper) to be superior to more classical activations from the ReLU family?


1 Answer 1


I have not used PELU activation function by myself, so dont know much about its performance benefits but can say

  • a, b, and c looks to be hyperparameters only.
  • and it can be directly implemented in tensorflow 2.0 as

    tf.cond(h, lambda:c*h, lambda:a(tf.exp(h/b)-1))
  • $\begingroup$ This is a very elegant and Pythonic implementation $\endgroup$
    – Leevo
    Commented Jan 9, 2020 at 10:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.