0
$\begingroup$

I have encountered PELU (Parametric Exponential Linear Unit) in the literature, but I can't find practical applications of it.

Moreover, I have some questions about how it works:

  • Are its parameters learned, or are they hyperparameters?

  • Is a Keras Custom layer required for a TensorFlow implementation?

  • In the original paper, it was applied to convolutional models only. Could you share links to applications outside Computer Vision?

  • Computational burden aside, is its performance proven (outside original paper) to be superior to more classical activations from the ReLU family?

$\endgroup$
1
$\begingroup$

I have not used PELU activation function by myself, so dont know much about its performance benefits but can say

  • a, b, and c looks to be hyperparameters only.
  • and it can be directly implemented in tensorflow 2.0 as

    tf.cond(h, lambda:c*h, lambda:a(tf.exp(h/b)-1))
    
|improve this answer|||||
$\endgroup$
  • $\begingroup$ This is a very elegant and Pythonic implementation $\endgroup$ – Leevo Jan 9 at 10:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.