5
$\begingroup$

Scikit-learn lists these as the implemented activation functions for it's multi-layer perceptron classifier:

‘identity’, no-op activation, useful to implement linear bottleneck, returns f(x) = x
‘logistic’, the logistic sigmoid function, returns f(x) = 1 / (1 + exp(-x)).
‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x).
‘relu’, the rectified linear unit function, returns f(x) = max(0, x)

Does anyone know if it is possible to implement a custom activation function? If not, can someone point me to a library where this is possible?

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ sklearn wasn't made specifically for this task, so it is missing features like this. I recommend pyTorch instead; it's the latest and greatest in the field, and it's simple. Here's a relevant discussion. Welcome to the site and good luck. $\endgroup$
    – Emre
    Apr 27, 2017 at 22:53
  • $\begingroup$ Thanks so much Emre! I just got pyTorch up and running and am fiddling with the forward function right now $\endgroup$
    – pennydreams
    Apr 28, 2017 at 17:47
  • $\begingroup$ @pennydreams did you ever get to completing the function and how did you implement it? I am trying something similar and could use some pointers! $\endgroup$
    – dsforlife84
    Nov 6, 2017 at 13:40
  • $\begingroup$ @dsforlife84 As Emre stated, it doesn't seem to be possible to implement a custom activation function in scikit-learn's MPLClassifier. I'd look at tensorflow or pytorch for implementing neural nets with custom activation functions. $\endgroup$
    – pennydreams
    Nov 7, 2017 at 14:42

1 Answer 1

Reset to default
2
$\begingroup$

Although @Emre's comment that sklearn wasn't specifically made for the task is correct, nothing stops you from introducing another activation function by implementing it in sklearn/neural_network/_base.py. You should be able to find the file in your installation.

To this end, you would define your single-parameter function and add it in the ACTVATIONS list. Refer to the existing implementations for some guidance how to do it.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Looks like we'd also need to implement its derivative $\endgroup$
    – Benitok
    Feb 24, 2021 at 4:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.