identifying input sequence using neural network

Question

I fount simple neural scenario sayinh : network having 3 layers, 3 inputs and 2 outputs. It should be trained to recognize a simple pattern - if inputs are correspondingly 6.0,7.0 and 8.0 then outputs should be 3.0 and 4.0, otherwise outputs should be 23.0

What I understand is, hidden layer should Check input pattern in sequence with simple if else condition.

if in1 == 6 and in2 == 7 and in3 ==8:
    out1, out2 = 3, 4
else
    out = 5

Do I understand the problem correctly or something I am missing?

Answer 1

A layer in a neural network does not process logic of the kind you present. It cannot "check the input pattern with a simple if else pattern". You could make a model that processed your example inputs to example outputs as you show, but it would not be a neural network.

In machine-learning, neural networks have a formal, numeric structure. There are a few variations, but essentially in your diagram:

• Each circle is a "neuron" which may have a numeric output value, also called its activation.

• Each arrow is a connection between neurons that shows where values are passed between neurons. Each connection has a strength or weight associated with it.

• When you have many arrows going into a neuron, you can calculate its new activation starting with the sum of weights times the activation values of neurons that they are connected from. Call this sum $z$. Then the activation of the neuron you are looking at is $a = f(z)$ where f is an "activation function" (also called "transfer function").

The above very roughly covers how many neural networks are constructed. As you can see though, there is no room in that for having your if/else statement.

Instead, neural networks like the one you show (technically a feed forward network also called a multi-layer perceptron or MLP) are most often trained by example. This involves repeatedly giving the network inputs where you know the output you want. To start with the network will give completely incorrect outputs, and you then adjust the weights to try and make the result closer. This is repeated very many times until the network becomes the best model it can for your purpose. A very common algorithm used to train a neural network in this way is backpropagation and gradient descent.

For your problem as described, a neural network is not a good model, and would not be used in the real world. That is because you already know the full logic of a simpler model that covers all possible inputs. At best with a neural network you will get an approximation to that already-perfect model.

However, if this is for the purposes of understanding neural networks, your approach would be roughly this:

1. Generate some training data with example inputs and outputs. Most example outputs will be the 23,23 default values, but when inputs are the special values 6,7,8 then the training outputs should be 3 and 4.

2. Build an initial network. Normally you would use an appropriate library - it is important you learn how the library works and understand any limitations. E.g. many activation functions have a maximum output of 1, so cannot learn the values 3,4, or 23.

3. Train the network to fit the example data. There are very many options there, depending on the library you are using. Usually it is just a matter of selecting parameters for the learning that help it progress smoothly. You may have to run training for a large number of repeats over the whole dataset (called epochs)

4. Test that your network works as you expect. Even if you have trained it well, the response of the network is never as precise as your problem statement. If you gave it inputs of 5.8,7.2,8.1 (i.e. close to 6,7,8) then it would likely output something like 7.2,8.5 and not 23,23 . . . it will interpolate between values.