I have a multiclass(7 labels) classification problem implemented in MLP. I am trying to classify 7 cancers based on some data. The overall accuracy is quite low, around 58%. However, some of the cancers is classified with accuracy around 90% for different parameters. Below cancer 1,2,3 etc means different types of cancer, For example 1= breast cancer, 2 = lung cancer etc. Now, for different parameter settings I get different classification accuracy. For example,

1. hyper parameters

    learning_rate = 0.001
    training_epochs = 10
    batch_size = 100
    hidden_size = 256
    #overall accuracy 53%, cancer 2 accuracy 91%, cancer 5 accuracy 88%, 
     #cancer 6 accuracy 89% 

2. hyper parameters

    learning_rate = 0.01
    training_epochs = 30
    batch_size = 100
    hidden_size = 128
    #overall accuracy 56%, cancer 2 accuracy 86%, cancer 5 accuracy 93%, 
    #caner 6 accuracy 75%

As you can see, for different parameter settings I am getting totally different results. Cancer 1,3,4,7 have very low accuracy, so I excluded them. But cancer 2, 5,6 have comparatively better results. But, for cancer 6, the results vary by great number depending on the parameter settings.

An important note is, here overall accuracy is not important but if I can classify 2-3 cancers with more than 90% accuracy that is more important. So my question is, how do I interpret the results? In my paper how should I show the results? which parameter settings should I show/use? Or should I show different parameter settings for different cancer types? So basically, how to handle this type of situations?

  • $\begingroup$ How stable are these outcomes? If you re-run the model several times, what's the mean and std of these? As far as reporting, disease classification typically deals with very unbalanced classes and in this case accuracy is not a good metric. See this, for example $\endgroup$ – Sophie Searcy - Metis Jan 19 '18 at 15:29
  • $\begingroup$ Dataset is balanced. 10% difference between no of samples of different classes. $\endgroup$ – asdfkjasdfjk Jan 19 '18 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.