2
$\begingroup$

I am going through the tutorial at the link below which uses MNIST handwritten digit database.

https://machinelearningmastery.com/handwritten-digit-recognition-using-convolutional-neural-networks-python-keras/

The 28x28 sized image data has to be reshaped into a 1D vector of 784 pixels. 28x28=784. Why does the multilayer perceptron insist on only a 1D vector of input data? There are no problems with convolutional and recurrent neural networks accepting input shape of higher dimensions.

$\endgroup$
2
$\begingroup$

You could say every type of neural network gets 1d input data. It's just more convenient to think about 2d-CNNs taking 2d data because the convolution operation is best illustrated by moving squares across a grid, and similarly for max-pooling.

But you could easily write out all the multiplications, additions, and max operations you performed in one line of algebra. And in this line you could easily flatten the 2d input indices into 1d indices. The information passed in and the calculations and output of the network would be exactly the same, but this representation would lose its real-world interpretation so there's no reason to think of it this way.

The point is that there are different ways of representing exactly the same information, and we choose a 1d representation for FNN inputs and sometimes higher dimensions for CNN inputs because it corresponds to the physical structure of the real-world problem and it is easiest think about. This is why most libraries have you input the data in these shapes. But if you think about what's happening at the lowest level of computation on your computer you won't necessarily find the same structure.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Thanks for your answer. Upvoted. Is this 1d input data requirement for MLP a constraint imposed by keras python library rather than a constraint of MLP architecture? $\endgroup$ – user781486 Aug 18 '18 at 2:39
  • 1
    $\begingroup$ It makes sense to think of perceptron input layers as one-dimensional. They get dotted with the weight vector to compute the input to the activation function. However it's still just a mathematical convention. Maybe your input data is 2d in the real world so you could think of the 2d input data getting element-wise multiplied by a 2d weight matrix and then collected into a sum etc. Keras won't let you do this, so in practice you would flatten your input data before feeding it to the NN. The point is that the information the same - so don't worry about what shape you are forced to pass it in as $\endgroup$ – Imran Aug 18 '18 at 2:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.