I have a pretty specific question regarding the caret package however I still hope to finde help here.

I recently worked with the caret package and trained a multilayer perceptron with method = 'mlp'.
I looked up the github page of Max Kuhn (developer of caret), and it says that you only need to tune one hyperparameter: the size (number of neurons in the hidden layer). Which is really convinient.

However it further states that caret for the training builds on the RSNNS Package (by Bergmeier). The mlp model implemented in this RSNNS package has additional tunable parameters over just the size hyperparameter (i.e. learnFunc,hiddenActFunc,Std_Backpropagation, maxit).

So I asked myself what values caret uses for those parameters? Default values or are those optmizied?


It appears that the defaults are used, except for lin, which is inferred from the type of the target variable: [source code]

Note too that you can set any of the other RSNNS parameters through the dots.

| improve this answer | |
  • $\begingroup$ Thank you. Since I am not a Data Science expert would you recommend tuning another parameter like the Activation Function or # of iterarion. As Background we get around 90% Accuracy in a classification problem which is pretty reasonable for our field. $\endgroup$ – SysRIP Dec 13 '19 at 8:07
  • $\begingroup$ I guess that would be new questions? $\endgroup$ – SysRIP Dec 13 '19 at 12:40
  • $\begingroup$ @SysRIP, which hyperparameters to tune is certainly another question (probably already asked); but on topic here, I think that caret having deemed all the other defaults as good enough to not include as parameters indicates that size is a good enough start. RSNNS indicates where you might go next, with "The defaults that are set for initialization and update functions usually don't have to be changed": that leaves activations, learning function, number of iterations. RSNNS seems to be missing some modern options, e.g. RELU activation and adam optimizer? $\endgroup$ – Ben Reiniger Dec 13 '19 at 14:53
  • $\begingroup$ Okay thanks a lot that was really helpfull. I wish I could gift you a upvote. $\endgroup$ – SysRIP Dec 13 '19 at 15:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.