1
$\begingroup$

I am just getting touch with Multi-layer Perceptron. And, I got this accuracy when classifying the DEAP data with MLP. However, I have no idea how to adjust the hyperparameters for improving the result.

Here is the detail of my code and result:

enter image description here.

from sklearn.neural_network import MLPClassifier

import numpy as np
import scipy.io
x_vals = data['all_data'][:,0:320]

y_vals_new = np.array([0 if each=='Neg'  else 1 if each =='Neu' else 2 for each in data['all_data'][:,320]])
y_vals_Arousal = np.array([3 if each=='Pas'  else 4 if each =='Neu' else 5 for each in data['all_data'][:,321]])

DEAP_x_train = x_vals[:-256]        #using 80% of whole data for training
DEAP_x_test = x_vals[-256:]         #using 20% of whole data for testing
DEAP_y_train = y_vals_new[:-256]     ##Valence
DEAP_y_test = y_vals_new[-256:]
DEAP_y_train_A = y_vals_Arousal[:-256]   ### Arousal
DEAP_y_test_A = y_vals_Arousal[-256:]

mlp = MLPClassifier(solver='adam', activation='relu',alpha=1e-4,hidden_layer_sizes=(50,50,50), random_state=1,max_iter=11,verbose=10,learning_rate_init=.1)

mlp.fit(DEAP_x_train, DEAP_y_train)

print (mlp.score(DEAP_x_test,DEAP_y_test))
print (mlp.n_layers_)
print (mlp.n_iter_)
print (mlp.loss_)
$\endgroup$
9
$\begingroup$

If you are using SKlearn, you can use their hyper-parameter optimization tools.

For example, you can use:

If you use GridSearchCV, you can do the following:

1) Choose your classifier

from sklearn.neural_network import MLPClassifier
mlp = MLPClassifier(max_iter=100)

2) Define a hyper-parameter space to search. (All the values that you want to try out.)

parameter_space = {
    'hidden_layer_sizes': [(50,50,50), (50,100,50), (100,)],
    'activation': ['tanh', 'relu'],
    'solver': ['sgd', 'adam'],
    'alpha': [0.0001, 0.05],
    'learning_rate': ['constant','adaptive'],
}

Note: the max_iter=100 that you defined on the initializer is not in the grid. So, that number will be constant, while the ones in the grid will be searched.

3) Run the search:

from sklearn.model_selection import GridSearchCV

clf = GridSearchCV(mlp, parameter_space, n_jobs=-1, cv=3)
clf.fit(DEAP_x_train, DEAP_y_train)

Note: the parameter n_jobs is to define how many CPU cores from your computer to use. The cv is the number of splits for cross-validation.

4) See the best results:

# Best paramete set
print('Best parameters found:\n', clf.best_params_)

# All results
means = clf.cv_results_['mean_test_score']
stds = clf.cv_results_['std_test_score']
for mean, std, params in zip(means, stds, clf.cv_results_['params']):
    print("%0.3f (+/-%0.03f) for %r" % (mean, std * 2, params))

5) Now you can use the clf to make new predictions. For example, check the performance on your test set.

y_true, y_pred = DEAP_y_test , clf.predict(DEAP_x_test)

from sklearn.metrics import classification_report
print('Results on the test set:')
print(classification_report(y_true, y_pred))
$\endgroup$
  • $\begingroup$ Thanks for yr detailed comments. I really appreciate it. However, this error have occurred while running. ValueError: Invalid parameter learning_rate for estimator MLPClassifier..... I have already checked this optional choices for each MLPClassifier parameter. And, it's definitely correct. Could you give me any clues about this issue? $\endgroup$ – 任凯盟 Jul 28 '18 at 5:48
  • $\begingroup$ There were some weird spaces in the keys of the parameter_space. If you try again, it should work! $\endgroup$ – BrunoGL Jul 28 '18 at 9:39
  • $\begingroup$ Yes, I just found it. Finally, it works! Thanks for yr advise! $\endgroup$ – 任凯盟 Jul 28 '18 at 10:55
  • $\begingroup$ my pleasure, any time :) $\endgroup$ – BrunoGL Jul 28 '18 at 11:05
  • $\begingroup$ Thanks for yr kindness.I still have another question about hidden_layer_sizes parameter. Ex: (50,50,50) represent 3 hidden layers with 50 hidden units each,right? How about the input and output layer? How did i initialize it via the hyperparameter? $\endgroup$ – 任凯盟 Jul 28 '18 at 11:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.