0
$\begingroup$

I am trying to train neural network (brain.js) a multiplication table. It is not going too well: requires lots of hidden layers, iterations and very small error threshold, and the results are still incorrect. What am I doing wrong?

console.clear();
let net = new brain.NeuralNetwork({hiddenLayers: [128]});
let training_data = [
    {input:[2,2], output:[4/81]},
    {input:[2,3], output:[6/81]},
    {input:[2,4], output:[8/81]},
    {input:[2,5], output:[10/81]},
];
net.train(training_data, {
    errorThresh: 0.0000001,  // error threshold to reach before completion
    iterations: 500000,   // maximum training iterations 
    log: true,           // console.log() progress periodically 
    logPeriod: 10000,       // number of iterations between logging 
    learningRate: 0.01    // learning rate 
});
for(let i = 2; i < 3; i++){
    for(let j = 2; j < 10; j++){
        console.log(i + " * " + j + " = " + net.run([i,j])[0]*81);      
    }
}

The output is:

2 * 2 = 4.265779517591

2 * 3 = 5.825156696140766

2 * 4 = 7.789406754076481

2 * 5 = 10.152319103479385

2 * 6 = 12.860703930258751

2 * 7 = 15.819618478417397

2 * 8 = 18.90912254154682

2 * 9 = 22.006048411130905

After receiving the answer from SRJ I have changed the neural net so, that it now has 4 hidden layers and receives 10 000 inputs:

console.clear();
let net = new brain.NeuralNetwork({hiddenLayers: [4]});
let training_data = [
];

for(let i = 0; i < 100; i++){
    for(let j = 0; j < 100; j++){
        training_data.push({input:[i,j], output: [i*j/10000]});
    }
}
net.train(training_data, {
    errorThresh: 0.001,  // error threshold to reach before completion
    iterations: 20000,   // maximum training iterations 
    log: true,           // console.log() progress periodically 
    logPeriod: 100,       // number of iterations between logging 
    learningRate: 0.0001    // learning rate 
});
let random_numbers = [Math.floor(Math.random()*50), Math.floor(Math.random()*50)]
console.log(random_numbers[0] + " * " + random_numbers[1] + " = " + net.run([random_numbers[0], random_numbers[0]])[0] * 10000);

Still does not work. The output: "29 * 42 = 728.9011776447296". (while in reality should be 1218).

$\endgroup$
  • $\begingroup$ Hi and welcome! Note that programming issues are generally off-topic here. I am not sure if this is a programming issue or not, so I will not close this post as off-topic. I suggest that you read our on-topic page to understand which questions you can ask here. $\endgroup$ – nbro Jun 18 at 9:41
  • $\begingroup$ Thank you! I know, that my question is more suitable for online forum, but as I know, there is no such of for neural networks and especially brain.js:) $\endgroup$ – Jaroslav Tavgen Jun 18 at 9:51
  • $\begingroup$ There's Data Science SE and Stack Overflow. Those sites are more suitable to ask questions that involve code or programming issues. $\endgroup$ – nbro Jun 18 at 9:52
1
$\begingroup$

As your function is linear and you have a hidden size of 128, Your model might be suffering from overfitting. Try to lower the hidden layer to 2 or 4 and try to regularize the model using dropout. And also neural net requires a lot of data to train. With small data neural net does not perform well. So if you are unwilling to provide extra data ,you should go for linear model like linearRegressor from sklearn.It will do your task just fine with proper hyperparameter. But if you want to train a neural net then you have to provide a huge amount of data and also choose proper hidden size and regularizer.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you VERY much for you answer! What is the minimum amount of data that should be supplied to NN? $\endgroup$ – Jaroslav Tavgen Jun 18 at 7:03
  • $\begingroup$ It depends on the size of your network and the type of your task. I think use a hidden layer of size 2 or 4 with the current data and test the performance .If it doesn't improve then keep adding data gradually until it improves. $\endgroup$ – SrJ Jun 18 at 7:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.