This image looks pretty familiar to anyone getting acquainted with neural networks, and on first glance it makes a lot of intuitive sense.

enter image description here

But on second, third, fourth, etc glance, some questions pop up! I hope I'm not the only one with these concerns.

There are 3 circles representing Input nodes. Does this mean that the data sent into the NN has to be separated into multiple batches? I don't hear much talk of that, or any kind of emphasis / reasoning. Why don't we send the data through the network all at once? Can't the video card handle it? (Or is that the reason why right there, that the card can't handle it?).

There are 4 circles representing Hidden nodes. Each previous node points to
a Hidden one. The way this is show, it is confusing:

Does this mean that data coming in is somehow merged together? I see 3 arrows all coalescing together into the next layer. That's what it looks like visually.

Is there some order, or sequence here? There's no indication of this process happening in some order. It just looks like a mess of arrows pointing from Input to Hidden. Shouldn't be be concerned with the order in which these activations flow? After all, we are looking at a network graph.

There are 2 circles representing Output nodes. Is this showing that a NN can output 2 different sets of data?? Why do we want multiple outputs? We started with 3 blocks of data, now we have 2. How are 2 separate blocks of data going to help us make a binary choice on, say, image classification?

The more I look at this image, the more I dislike it. Yet I see it everywhere, in many introductory / intermediate discussions of NN. I'm hoping there is a reason why such an unclear image is so popular.


1 Answer 1


You are making one mistake which cascades on towards other mistakes. The multiple inputs are different features of the same sample. Let's say we want to regress house prices based on some features of a house. These three nodes could represent the rating of the neighborhood, the surface area and the number of bedrooms for example. Then you take a weighted sum to every of the hidden nodes. Every hidden node takes different weights over these features. The order you mention is per layer, since we need the output of the previous layer as input for the next one, these dependencies force this order.

With regards to output, you can have a combined loss. In case of binary classification you would probably just have a sigmoid activation at the end, however if you would have multiclass classification you would have a softmax layer at the end, one output for every potential class. There are a lot of other things you could want to output, for example a bounding box so you would need coordinates of where the network thinks the box is, or even per pixel if it's part of the object or not, then you would have a m x n grid as output.

EDIT: All the things you mentioned about the GPU etcetera, that is way too far for what this image represents. It's one network for one sample. I do agree it would have been better to just have one output node because most more basic examples in supervised learning have just 1 target.

  • $\begingroup$ So each of the initial inputs represent a single feature? So if it's a 23x23 image, then instead of 3, there would be 529? That sounds very weird...I thought it would be one single image for each input. Which brings me back to the beginning -- is this clumsy picture showing us a NN being built on 3 training examples? Or 3 thirds of a complete dataset? $\endgroup$ Commented Sep 6, 2016 at 22:31
  • $\begingroup$ Yes it would be 529 features, how else would the network distinguish the different values of the pixels? If it's a colored image there would be even more. With regards to your last question, this picture shows a NN with 3 features. Nowhere does it indicate how many instances it has been trained on, which is the usual situation. Network design and training are somewhat seperated issues $\endgroup$ Commented Sep 7, 2016 at 4:41
  • $\begingroup$ Ahhhh...you are totally right. I didn't understand what the input really was. Even though it could be, say, 529 pixels, each pixel will make it to each node in the next layer. So each image still gets represented at each level in this manner. Now it all becomes so much more clear. Thanks!! $\endgroup$ Commented Sep 7, 2016 at 8:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.