1
$\begingroup$

I am currently trying to train my backpropagation to classify 150k training pairs. Each training pair is a vector of 18 Bipolar numbers and it runs through 2 hidden layers with a final output of 1 number(18-18-18-1).

When I feed my neural network with only a couple thousand training pairs it can classify them 100% but when I try to feed more it makes many errors. Does anyone have any recommendations for what can help a back prop net handle more training pairs? Should I do batch updating of weights or update them for every training pair(what I do now)? Also whats the best way to test for convergence with so many training pairs(when summing up error it's ~40k).Any help is much appreciated.

(FYI: each training pair is a loan account I am trying to evaluate with data on balance, age, how long it's been open, average income of zipcode and the target is whether or not it was paid)

$\endgroup$
  • $\begingroup$ Can you show your learning curve? $\endgroup$ – Emre May 11 '17 at 17:34
  • $\begingroup$ I forgot to add my output/target is unbalanced its 85% not paid(-1) and 15% paid(1) and once it gets over 20k training patterns it classifies everything as unpaid because that is the majority. I tried to over feed it the paid accounts with a training set that's 50%paid 50% unpaid and then it makes errors saying way more paid during testing. $\endgroup$ – matt standley May 11 '17 at 18:33
  • $\begingroup$ What error metric are you using? Are you talking about training or validation error (please post learning curves!)? Have you centered and scaled your inputs? Have you tried giving it more hidden units (18 isn't much, depending on the complexity of the problem - try 2x 500)? (And just wondering - what's a bipolar number?) $\endgroup$ – stmax May 12 '17 at 14:17
  • $\begingroup$ I am using 1/n∑(target - output)^2 to calculate mean squared error for each epoch where n is the number of training pairs. By bipolar number I mean it is either 1 or -1. I am going to try to post learning curves, how do I make them/what variables should I use to make them? Thanks for the tip to make the net 2x500 it is working better with more hidden neurons. Also, what do you mean by scale the inputs? $\endgroup$ – matt standley May 12 '17 at 18:50
  • $\begingroup$ Mean squared error is normally used for regression problems, you seem to have a classification problem ("paid" vs "not paid"). In that case the binary crossentropy loss function would be more appropiate (make sure the target is encoded with 0 and 1). Learning curves: calculate training and validation loss after each epoch, then plot losses over epoch number. Scaling: neural nets want all features to be in the same range (like -1 to +1) - that's already the case with bipolar numbers though. What's the activation function of your output neuron? Should be sigmoid or tanh, definitelly not linear. $\endgroup$ – stmax May 12 '17 at 20:10
1
$\begingroup$

Typical things to look for when training neural networks:

  • Learning Curve: a plot of the error on the validation set X size of training set. The error must decrease as the network is trained, so if you see fluctuations or an increase something is wrong.
  • Learning Rate: the learning rate influences the convergence of gradient descent. A small learning rate will make the model overfit the data while a big one may make it underfit.
  • Precision: of the classified examples in the training set, how many did the network got right, or more precisely, how many did it classify as expected?
  • Recall: of all the positive examples, in your case the paid ones, how many did it classify as paid?

If you notice the performance of the network is bad you may:

  • Add other features that make the model fit the data better - for example add average family income to the features
  • Get more training examples
  • Some papers suggest deeper networks are able to catch subtleties about the model they are trying to fit, so experimenting with the network architecture is an option.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.