0
$\begingroup$

I am attempting to formulate my own activation function. However, I'm new to neural networks, am not yet ready to test it, but would want to know if I already landed on a better activation function than my benchmark before pushing through for a successful study.

These are their graphs. Mine is in green and the benchmark is in purple.enter image description here

Is it possible to tell which one is better based on these graphs?

Thanks!

Improve this question
$\endgroup$
1
  • $\begingroup$ Both functions are smooth, non-monotonic, and bounded. $\endgroup$
    – jwho
    Commented Oct 19, 2022 at 20:28

1 Answer 1

Reset to default
2
$\begingroup$

tl;dr No.

The choice of an activation function is highly dependent on the task at hand, so there isn't necessarily a "better" in the general sense, let alone a signal you could get from a chart.

The chart also doesn't tell you if the activation or its derivative is easy/inexpensive to compute relative to the other, which can be a consideration.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.