2
$\begingroup$

Let's say you train a neural network on Input = 1/Output = 2; Input = 4/Output = 8.

How would you train a machine to recognize that it needs to multiply the input by 2, from scratch?

$\endgroup$
1
  • $\begingroup$ Do you just want a NN to function over inputs of 1:2 or all integers or integers 0:9? Are there any constraints on number of nodes or hidden layers? $\endgroup$ May 13 '16 at 13:06
1
$\begingroup$

You need to limit the network model to one which naturally would generalise your problem.

For a simple multiplication, this would be a single linear neuron. You can also train bias so that the network represents y = Ax + B.

If you take that very simple network give it just 2 or 3 examples to learn from, and train it using least squares error, it will quickly converge to the function you want. A single layer linear network like this can generalise to y = Ax + B where x, y and B are vectors, and A is a matrix - i.e. it can solve simultaneous linear equations provided you supply enough examples to do so (usually number of examples equals number of dimensions of the vectors).

This is actually really trivial, and if you train such a simple network using any modern NN library, it will be very fast to learn your linear function. However, in the general case of having a specific non-linear function and wanting the network to learn it in a general sense, it is not possible. The network can be made to learn to generalise reasonably well near where you have provided example inputs and outputs, but will output incorrect results when you give it an x value far away from the training examples.

It is important to note that the network does not learn the math formula for your function. It finds the best approximation given its own internal model. But by picking a network where the internal model is a good match to the function you want to learn, it is going to generalise well to it, and may be able to get it exactly if there is no noise in the examples and enough of them.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.