1
$\begingroup$

I want to predict using a multi layer perceptron. My data have 3 attributes x,y,z. My window size (input) is 3. That is, using 3 sets of values for x,y,z to predict next set of x,y,z. What should be my number of input neurons?

Should it be 6 input neurons for 3 sets of values of x,y,z?
or should it be 3 input neurons and each neuron has 3 inputs for x,y,z?

$\endgroup$
1
$\begingroup$

My understanding of the question is that you are using a basic MLP, feed-forward neural network, to work with data that is a 3-dimensional vector which you are calling $(x,y,z)$ - although note that most notation you read would call the whole observation $x$ which would be a vector $(x_1, x_2, x_3)$. You want to accept an input of 1st, 2nd, 3rd observations of this data (a window on some possibly longer series), and predict the 4th observation.

For the given architecture, the prediction is made immediately on the supplied input - a MLP has no "memory" for previous inputs - so you will need 3 x 3 = 9 inputs to represent each (x,y,z) in the series. For a MLP it doesn't really matter how you arrange this input - e.g. you could have $(x^{(1)}, y^{(1)}, z^{(1)},x^{(2)}, y^{(2)}, z^{(2)},x^{(3)}, y^{(3)}, z^{(3)})$ or $(x^{(1)}, x^{(2)}, x^{(3)},y^{(1)}, y^{(2)}, y^{(3)},z^{(1)}, z^{(2)}, z^{(3)})$ or any other arrangement as long as you are consistent for every training example and prediction.

You don't usually want or need to somehow combine the x, y and z values to create a pre-processed layer with 3 inputs, or create a special layer that combines $x^{(1)}, x^{(2)}, x^{(3)}$. You might do that if your understanding of how the sequence worked allowed you to come up with features that modelled something important. However, the more usual approach is to allow the neural network to figure out how the multiple inputs combine from the training data.

You may want to look at alternative architecture of Recurrent Neural Network (RNN), which has more options for dealing with predictions of series. For a RNN, you would have 3 input neurons, and you would run the network 3 times in sequence and read the prediction after running it the third time.

You have noticed correctly that a MLP has no "understanding" internally that your x in (x,y,z) at step 1 is the same type of variable as the x in steps 2 and 3. The MLP would have 9 separate inputs and would ignore any direct relations, or even that there is a sequence being predicted. Whilst in a RNN, both those details are built into the model as core assumptions. In some cases that could be an important advantage to the RNN. But it is still worth trying the simpler MLP first and testing whether its predictions are good enough for purpose.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.