# Help training Back-propagation Neural Network With 150k training pairs

I am currently trying to train my backpropagation to classify 150k training pairs. Each training pair is a vector of 18 Bipolar numbers and it runs through 2 hidden layers with a final output of 1 number(18-18-18-1).

When I feed my neural network with only a couple thousand training pairs it can classify them 100% but when I try to feed more it makes many errors. Does anyone have any recommendations for what can help a back prop net handle more training pairs? Should I do batch updating of weights or update them for every training pair(what I do now)? Also whats the best way to test for convergence with so many training pairs(when summing up error it's ~40k).Any help is much appreciated.

(FYI: each training pair is a loan account I am trying to evaluate with data on balance, age, how long it's been open, average income of zipcode and the target is whether or not it was paid)

• Can you show your learning curve?
– Emre
May 11 '17 at 17:34
• I forgot to add my output/target is unbalanced its 85% not paid(-1) and 15% paid(1) and once it gets over 20k training patterns it classifies everything as unpaid because that is the majority. I tried to over feed it the paid accounts with a training set that's 50%paid 50% unpaid and then it makes errors saying way more paid during testing. May 11 '17 at 18:33
• What error metric are you using? Are you talking about training or validation error (please post learning curves!)? Have you centered and scaled your inputs? Have you tried giving it more hidden units (18 isn't much, depending on the complexity of the problem - try 2x 500)? (And just wondering - what's a bipolar number?) May 12 '17 at 14:17
• I am using 1/n∑(target - output)^2 to calculate mean squared error for each epoch where n is the number of training pairs. By bipolar number I mean it is either 1 or -1. I am going to try to post learning curves, how do I make them/what variables should I use to make them? Thanks for the tip to make the net 2x500 it is working better with more hidden neurons. Also, what do you mean by scale the inputs? May 12 '17 at 18:50
• Mean squared error is normally used for regression problems, you seem to have a classification problem ("paid" vs "not paid"). In that case the binary crossentropy loss function would be more appropiate (make sure the target is encoded with 0 and 1). Learning curves: calculate training and validation loss after each epoch, then plot losses over epoch number. Scaling: neural nets want all features to be in the same range (like -1 to +1) - that's already the case with bipolar numbers though. What's the activation function of your output neuron? Should be sigmoid or tanh, definitelly not linear. May 12 '17 at 20:10

• Recall: of all the positive examples, in your case the paid ones, how many did it classify as paid?