2
$\begingroup$

I am trying to implement Python's MLPClassifier with 10 fold cross-validation using gridsearchCV function. Here is a chunk of my code:

parameters={
'learning_rate': ["constant", "invscaling", "adaptive"],
'hidden_layer_sizes': [(100,1), (100,2), (100,3)],
'alpha': [10.0 ** -np.arange(1, 7)],
'activation': ["logistic", "relu", "Tanh"]
}

clf
= gridSearchCV(estimator=MLPClassifier,param_grid=parameters,n_jobs=-1,verbose=2,cv=10)

Though,I am not sure if hidden_layer_sizes: [(100,1), (100,2), (100,3)] is correct. Here, I am trying to tune 'hidden layer size' & 'number of neurons'. I would like to give this 'tuple' parameter for hidden_layer_sizez: 1, 2, 3, and neurons: 10, 20, 30,...,100.

But I do not know if it is the correct way to do it. Therefore, I am choosing default neurons to be 100 in each layer.

Can anyone advise please?

$\endgroup$
  • $\begingroup$ I am very new to Python and was going through this post. My query is similar and response on setting up of Hidden layers helped a lot. However, I am unable to set up alpha in same way as mentioned above. Any help on this? I mean when I am setting up alpha as alpha': [10.0 ** -np.arange(1, 7)], it is giving me error. $\endgroup$ – Abhik Jha Mar 4 '18 at 13:14
  • $\begingroup$ the alpha parameter of the MLPClassifier is a scalar. [10.0 ** -np.arange(1, 7)], is a vector. Which works because it is passed to gridSearchCV which then passes each element of the vector to a new classifier. Have you set it up in the same way? $\endgroup$ – S van Balen Mar 4 '18 at 14:03
4
$\begingroup$

A tuple of the form $(i_1, i_2, i_3, ... , i_n)$ gives you a network with $n$ hidden layers, where $i_k$ gives you the number of neurons in the $k$th hidden layer.

If you want three hidden layers with $10,30$ and $20$ neurons, your tuple would need to look like $(10,30,20)$.

$(100,1)$ would mean that the second hidden layer only has one neuron.

$\endgroup$
  • $\begingroup$ Thanks for your reply, but I still do not get it. I would like to tune two things simultaneously; 'Number of layers ranging from 1 to 3', and 'Number of neurons in each layer ranging as 10, 20, 30, 40, 50, 100'. Can you please show in my above example code how to do it? Alternately, let's say I fix on 3 hidden layers. Now, I want to tune only neurons ranging as 10, 20, 30, 40, 50, 100 $\endgroup$ – zx mnb Jun 16 '17 at 18:04
  • 1
    $\begingroup$ you need to spell out all the combinations [(10,),...(100,),...(100,100,100)]. you can use itertools to generate them, e.g. [x for x in itertools.product((10,20,30,40,50,100),repeat=3)] if you really want all the combinations of of neurons for 3 hidden layers... $\endgroup$ – oW_ Jun 16 '17 at 18:09
  • 1
    $\begingroup$ Thanks! I get it now what you mean, however, can you please provide a working example in my above code? $\endgroup$ – zx mnb Jun 16 '17 at 18:17
  • 1
    $\begingroup$ 'hidden_layer_sizes': [x for x in itertools.product((10,20,30,40,50,100),repeat=3)] ? $\endgroup$ – oW_ Jun 16 '17 at 18:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.