I am just getting touch with Multi-layer Perceptron. And, I got this accuracy when classifying the DEAP data with MLP. However, I have no idea how to adjust the hyperparameters for improving the result.

Here is the detail of my code and result:

enter image description here.

from sklearn.neural_network import MLPClassifier

import numpy as np
import scipy.io
x_vals = data['all_data'][:,0:320]

y_vals_new = np.array([0 if each=='Neg'  else 1 if each =='Neu' else 2 for each in data['all_data'][:,320]])
y_vals_Arousal = np.array([3 if each=='Pas'  else 4 if each =='Neu' else 5 for each in data['all_data'][:,321]])

DEAP_x_train = x_vals[:-256]        #using 80% of whole data for training
DEAP_x_test = x_vals[-256:]         #using 20% of whole data for testing
DEAP_y_train = y_vals_new[:-256]     ##Valence
DEAP_y_test = y_vals_new[-256:]
DEAP_y_train_A = y_vals_Arousal[:-256]   ### Arousal
DEAP_y_test_A = y_vals_Arousal[-256:]

mlp = MLPClassifier(solver='adam', activation='relu',alpha=1e-4,hidden_layer_sizes=(50,50,50), random_state=1,max_iter=11,verbose=10,learning_rate_init=.1)

mlp.fit(DEAP_x_train, DEAP_y_train)

print (mlp.score(DEAP_x_test,DEAP_y_test))
print (mlp.n_layers_)
print (mlp.n_iter_)
print (mlp.loss_)

If you are using SKlearn, you can use their hyper-parameter optimization tools.

For example, you can use:

If you use GridSearchCV, you can do the following:

1) Choose your classifier

from sklearn.neural_network import MLPClassifier
mlp = MLPClassifier(max_iter=100)

2) Define a hyper-parameter space to search. (All the values that you want to try out.)

parameter_space = {
    'hidden_layer_sizes': [(50,50,50), (50,100,50), (100,)],
    'activation': ['tanh', 'relu'],
    'solver': ['sgd', 'adam'],
    'alpha': [0.0001, 0.05],
    'learning_rate': ['constant','adaptive'],

Note: the max_iter=100 that you defined on the initializer is not in the grid. So, that number will be constant, while the ones in the grid will be searched.

3) Run the search:

from sklearn.model_selection import GridSearchCV

clf = GridSearchCV(mlp, parameter_space, n_jobs=-1, cv=3)
clf.fit(DEAP_x_train, DEAP_y_train)

Note: the parameter n_jobs is to define how many CPU cores from your computer to use (-1 is for all the cores available). The cv is the number of splits for cross-validation.

4) See the best results:

# Best paramete set
print('Best parameters found:\n', clf.best_params_)

# All results
means = clf.cv_results_['mean_test_score']
stds = clf.cv_results_['std_test_score']
for mean, std, params in zip(means, stds, clf.cv_results_['params']):
    print("%0.3f (+/-%0.03f) for %r" % (mean, std * 2, params))

5) Now you can use the clf to make new predictions. For example, check the performance on your test set.

y_true, y_pred = DEAP_y_test , clf.predict(DEAP_x_test)

from sklearn.metrics import classification_report
print('Results on the test set:')
print(classification_report(y_true, y_pred))
  • $\begingroup$ Thanks for yr detailed comments. I really appreciate it. However, this error have occurred while running. ValueError: Invalid parameter learning_rate for estimator MLPClassifier..... I have already checked this optional choices for each MLPClassifier parameter. And, it's definitely correct. Could you give me any clues about this issue? $\endgroup$ – Irving.ren Jul 28 '18 at 5:48
  • $\begingroup$ There were some weird spaces in the keys of the parameter_space. If you try again, it should work! $\endgroup$ – Bruno Lubascher Jul 28 '18 at 9:39
  • $\begingroup$ Yes, I just found it. Finally, it works! Thanks for yr advise! $\endgroup$ – Irving.ren Jul 28 '18 at 10:55
  • $\begingroup$ my pleasure, any time :) $\endgroup$ – Bruno Lubascher Jul 28 '18 at 11:05
  • 1
    $\begingroup$ If you check the documentation (scikit-learn.org/stable/modules/generated/…), you will see hidden_layer_sizes : tuple, length = n_layers - 2, default (100,). This means that you define the inners layers, but the input and output layers you don't. That means that your input layer will be the size of each item in DEAP_x and the output layer will have the size of the number of possible values in DEAP_y. The input/ouput layer size is not defined on the __init__ but rather on the fit. $\endgroup$ – Bruno Lubascher Jul 28 '18 at 12:09

As a complement to the very practical answer of @BrunoGL, I'd like to give a more theoretical answer. I'd like to suggest everyone trying to adjust hyperparameters of a simple Neural Network to read Efficient Backprop, by Lecun and others (http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf). Yes it's from 1998, no it's not outdated.

It covers the impact of the main hyperparameters you have to set (activation, solver, learning rate, batches), commons traps, the problems you may encouter if you fall into them, how to spot those problems and how to solve them. A must read for everyone that want to tune a Neural Network. Plus, it's free.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.