0
$\begingroup$

I was recently trying to train a convolutional neural network to classify people as Hispanic or white (for learning purposes). I couldn't find a good dataset of just those two races, so I had to manually scrape images from the web. I ended up with 48 total images, 24 of them Hispanic and 24 white. When I trained my network, I tried many different architectures and hyperparameters, but the accuracy stayed at exactly 50%. I learned this was because the network was outputting 1 every time, classifying every image as Hispanic because it found a local optima for the cost function. I believe this is because of my 50/50 data split, although I may be wrong. If if is because of the split, how does this not happen in the real world? For example, if I had 10 million images, half of them white and the other half Hispanic, how does the network avoid falling prey to the same trap?

$\endgroup$
0
$\begingroup$

First of all your dataset is very small to train a deep learning model you would need at least 10k images to train the model and produce good results. Secondly, if you still think that your cost function gets stuck in local minima then you might consider using Nesterov momentum.

| improve this answer | |
$\endgroup$
  • $\begingroup$ I wasn't trying to make this a very accurate model, it was more so for my own experimentation. The question was asking what was causing the network to reach the local optima. $\endgroup$ – achandra03 Sep 18 '19 at 1:01
  • $\begingroup$ Could be the loss function, cost function, your dataset etc $\endgroup$ – Syed Nauyan Rashid Sep 18 '19 at 5:28
0
$\begingroup$

In general, the cost function of a neural network is non-convex, so it's mathematically okay that you're converging at a local minima. (To convince you that it's non convex, take any permutation of the set of weights and it would also yield you to the same point in space). As a suggestion, try using a linear classifier, or a multilayer perceptron and compare your results.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.