How to implement Python's MLPClassifier with gridsearchCV?

I am trying to implement Python's MLPClassifier with 10 fold cross-validation using gridsearchCV function. Here is a chunk of my code:

parameters={
'hidden_layer_sizes': [(100,1), (100,2), (100,3)],
'alpha': [10.0 ** -np.arange(1, 7)],
'activation': ["logistic", "relu", "Tanh"]
}

clf
= gridSearchCV(estimator=MLPClassifier,param_grid=parameters,n_jobs=-1,verbose=2,cv=10)


Though,I am not sure if hidden_layer_sizes: [(100,1), (100,2), (100,3)] is correct. Here, I am trying to tune 'hidden layer size' & 'number of neurons'. I would like to give this 'tuple' parameter for hidden_layer_sizez: 1, 2, 3, and neurons: 10, 20, 30,...,100.

But I do not know if it is the correct way to do it. Therefore, I am choosing default neurons to be 100 in each layer.

• I am very new to Python and was going through this post. My query is similar and response on setting up of Hidden layers helped a lot. However, I am unable to set up alpha in same way as mentioned above. Any help on this? I mean when I am setting up alpha as alpha': [10.0 ** -np.arange(1, 7)], it is giving me error. – Abhik Jha Mar 4 '18 at 13:14
• the alpha parameter of the MLPClassifier is a scalar. [10.0 ** -np.arange(1, 7)], is a vector. Which works because it is passed to gridSearchCV which then passes each element of the vector to a new classifier. Have you set it up in the same way? – S van Balen Mar 4 '18 at 14:03
• 'hidden_layer_sizes': [x for x in product(range(1,100), range(1,3))] – ปิยวัฒน์ โคตรพรม Sep 28 '20 at 16:30

A tuple of the form $(i_1, i_2, i_3, ... , i_n)$ gives you a network with $n$ hidden layers, where $i_k$ gives you the number of neurons in the $k$th hidden layer.
If you want three hidden layers with $10,30$ and $20$ neurons, your tuple would need to look like $(10,30,20)$.
$(100,1)$ would mean that the second hidden layer only has one neuron.
• you need to spell out all the combinations [(10,),...(100,),...(100,100,100)]. you can use itertools to generate them, e.g. [x for x in itertools.product((10,20,30,40,50,100),repeat=3)] if you really want all the combinations of of neurons for 3 hidden layers... – oW_ Jun 16 '17 at 18:09
• 'hidden_layer_sizes': [x for x in itertools.product((10,20,30,40,50,100),repeat=3)] ? – oW_ Jun 16 '17 at 18:22