I am a beginner and I have developed code in Octave to train a neural network (as part of Andrew Ng's coursera course).

Now I would like to generate some datasets in Excel for simple mathematical functions and try and train my neural network implementation.

I tried y = x^2 and y = sin(x), but my training accuracy is only around 67%.

I have varied the number of hidden layer units, varied the lambda, but still don't get good results. Where could I be wrong?


Without code you could have made to many mistakes to answer your question directly.

However, I suggest two things:

  1. Gradient checking (e.g. this explanation, but there are many more - pretty sure also in the course)
  2. XOR problem with a 2:2:1 network with only sigmoid activations. Plot the error surface. Compare the decision surface with that of this interactive demo.

It should look similar to this: enter image description here

However, please keep in mind that the problem might not be your implementation, but rather the network architecture / hyperparameters such as the number of epochs you're training or the training data.

Also, very important: I doubt that you will get good results for $y = x^2$, except if $x$ is restricted to $[-1,1]$ or something similar simple. Please keep the domain of your output layer in mind.

  • $\begingroup$ Martin could you clarify why restricting your domain is necessary in training the network? $\endgroup$
    – mburke05
    May 18 '16 at 23:04
  • $\begingroup$ One the one hand you have to have a look at the range of values of the activation function of the last layer. Now you might simply take a linear output, but then the last weights might get enormous. Even small fluctuations at previous layers might so have a huge impact on the last one and this on the output $\endgroup$ May 18 '16 at 23:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.