Graphs demonstrating the structure of neural networks are very unclear

Question

This image looks pretty familiar to anyone getting acquainted with neural networks, and on first glance it makes a lot of intuitive sense.

But on second, third, fourth, etc glance, some questions pop up! I hope I'm not the only one with these concerns.

There are 3 circles representing Input nodes. Does this mean that the data sent into the NN has to be separated into multiple batches? I don't hear much talk of that, or any kind of emphasis / reasoning. Why don't we send the data through the network all at once? Can't the video card handle it? (Or is that the reason why right there, that the card can't handle it?).

There are 4 circles representing Hidden nodes. Each previous node points to
a Hidden one. The way this is show, it is confusing:

Does this mean that data coming in is somehow merged together? I see 3 arrows all coalescing together into the next layer. That's what it looks like visually.

Is there some order, or sequence here? There's no indication of this process happening in some order. It just looks like a mess of arrows pointing from Input to Hidden. Shouldn't be be concerned with the order in which these activations flow? After all, we are looking at a network graph.

There are 2 circles representing Output nodes. Is this showing that a NN can output 2 different sets of data?? Why do we want multiple outputs? We started with 3 blocks of data, now we have 2. How are 2 separate blocks of data going to help us make a binary choice on, say, image classification?

The more I look at this image, the more I dislike it. Yet I see it everywhere, in many introductory / intermediate discussions of NN. I'm hoping there is a reason why such an unclear image is so popular.

Answer 1

You are making one mistake which cascades on towards other mistakes. The multiple inputs are different features of the same sample. Let's say we want to regress house prices based on some features of a house. These three nodes could represent the rating of the neighborhood, the surface area and the number of bedrooms for example. Then you take a weighted sum to every of the hidden nodes. Every hidden node takes different weights over these features. The order you mention is per layer, since we need the output of the previous layer as input for the next one, these dependencies force this order.

With regards to output, you can have a combined loss. In case of binary classification you would probably just have a sigmoid activation at the end, however if you would have multiclass classification you would have a softmax layer at the end, one output for every potential class. There are a lot of other things you could want to output, for example a bounding box so you would need coordinates of where the network thinks the box is, or even per pixel if it's part of the object or not, then you would have a m x n grid as output.

EDIT: All the things you mentioned about the GPU etcetera, that is way too far for what this image represents. It's one network for one sample. I do agree it would have been better to just have one output node because most more basic examples in supervised learning have just 1 target.