Scikit-learn lists these as the implemented activation functions for it's multi-layer perceptron classifier:

‘identity’, no-op activation, useful to implement linear bottleneck, returns f(x) = x
‘logistic’, the logistic sigmoid function, returns f(x) = 1 / (1 + exp(-x)).
‘tanh’, the hyperbolic tan function, returns f(x) = tanh(x).
‘relu’, the rectified linear unit function, returns f(x) = max(0, x)

Does anyone know if it is possible to implement a custom activation function? If not, can someone point me to a library where this is possible?

  • 2
    $\begingroup$ sklearn wasn't made specifically for this task, so it is missing features like this. I recommend pyTorch instead; it's the latest and greatest in the field, and it's simple. Here's a relevant discussion. Welcome to the site and good luck. $\endgroup$
    – Emre
    Apr 27 '17 at 22:53
  • $\begingroup$ Thanks so much Emre! I just got pyTorch up and running and am fiddling with the forward function right now $\endgroup$ Apr 28 '17 at 17:47
  • $\begingroup$ @pennydreams did you ever get to completing the function and how did you implement it? I am trying something similar and could use some pointers! $\endgroup$ Nov 6 '17 at 13:40
  • $\begingroup$ @dsforlife84 As Emre stated, it doesn't seem to be possible to implement a custom activation function in scikit-learn's MPLClassifier. I'd look at tensorflow or pytorch for implementing neural nets with custom activation functions. $\endgroup$ Nov 7 '17 at 14:42

Although @Emre's comment that sklearn wasn't specifically made for the task is correct, nothing stops you from introducing another activation function by implementing it in sklearn/neural_network/_base.py. You should be able to find the file in your installation.

To this end, you would define your single-parameter function and add it in the ACTVATIONS list. Refer to the existing implementations for some guidance how to do it.

  • 1
    $\begingroup$ Looks like we'd also need to implement its derivative $\endgroup$
    – Benitok
    Feb 24 at 4:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.