1
$\begingroup$

I have a question that my single deep neural network model gives above 90% accuracy for one data set and the same model gives an accuracy between 70-80% for the other data set. I want to know why this variation in accuracy happens in spite of the fact that i am using the same deep neural network model.

$\endgroup$
  • $\begingroup$ It happens because the data is different, of course. Are the two data sets equally sampled/distributed? If the model has seen one of the data sets during training, then the answer is overfitting, but there could be different reasons. $\endgroup$ – Mephy Sep 5 '18 at 13:09
  • $\begingroup$ Thanx Mephy for your valuable response $\endgroup$ – Anam Habib Sep 5 '18 at 14:42
  • $\begingroup$ do you retrain the model on the new dataset? $\endgroup$ – Mohammad Athar Sep 5 '18 at 19:38
  • $\begingroup$ Yes,i am using the same model on a new dataset. $\endgroup$ – Anam Habib Sep 6 '18 at 2:52
  • $\begingroup$ I have a question. Are you training one model on one dataset or two models with same architecture with both the datasets? $\endgroup$ – Mohit Motwani Sep 6 '18 at 7:38
1
$\begingroup$

I can think of one reason. Because you're using different data sets, the properties of your data set are likely to be different. So obviously what the model will learn will be different. And maybe the architecture of your network suits one data set more than the other. But also if you try training the model on your other data set(with less accuracy) for more epochs, it's possible that the accuracy will improve. Your model could be just be slow to understand this data set.

It's like when you see a human you can easily recognize a face. But when you see a software code, it's not always easy to find the bug. But if you're trained well enough you'll eventually be faster and better in recognizing the bugs.

I hope it makes sense to you.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanx @Mohit Motwani for your valuable response but i want to know that how should i know that what properties of dataset effects the accuracy of a model? $\endgroup$ – Anam Habib Sep 5 '18 at 14:39
  • $\begingroup$ That's the thing about Neural Networks, it's difficult to understand what they are learning. For example, if you're doing an image classification task, you don't know what features it's using to distinguish the classes. So maybe you can try visualizing like in this paper cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf. There is no one method to do this(or that I'm aware of). You might have to do some trial and error depending on your application. $\endgroup$ – Mohit Motwani Sep 6 '18 at 5:44
  • $\begingroup$ ok Thanx @Mohit Motwani $\endgroup$ – Anam Habib Sep 7 '18 at 11:21
1
$\begingroup$

I'm assuming you're taking a certain neural net (NN) architecture and doing a train/test/validate cycle on dataset A to get 90% accuracy, then you reset your parameters and train/test/validate on dataset B to get 70% accuracy

To expand what @mohit-motwani says:

There does not exist a machine learning model that will approximate all functions to any desired degree. By corollary, there is no single neural net architecture that will best fit any given data.

Try tweaking various hyperparameters and see if that helps. Activation functions, convolution size, padding, pooling, even a few more/less hidden layers can drastically change your results.

NNs have been getting a lot of praise lately as "automated learning machines" but in reality, you still have to know the theory behind them, and what's under the hood to fully appreciate how they work. You'll get better with experience, but don't think any one NN architecture is "best"

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.