I guess you have not figured out the concept of dropout very well. First, the reason we apply it is that we add some noise to the architecture in order not be dependant on any special node. The reason is that it was observed that while training a network, after overfitting, the weights for some of neurons increases and cause the network to be dependant on them. By exploiting dropout, we are not dependant on any node anymore due to it is possible to drop it while training.
Now, answers to your question. First, you have to bear this point in mind that the probability shows the chance of dropping a node in a layer. Consequently, chance 0.5 does not mean you, for instance, will have those two nodes. It just means after employing dropout, the chance of dropping for each node is half. Dropout is used for layers. It is customary to use it in fully connected layers. You set the hyper-parameter and it is the chance of keeping the nodes in the layer. While testing, you don't drop any node. We don't multiply neurons to the probability. The probability specifies the chance of existence of that node.
Okey-doke! I update the answer. As you can read in the paper,
At test time, it is not feasible to explicitly average the predictions from exponentially
many thinned models. However, a very simple approximate averaging method works well in
practice. The idea is to use a single neural net at test time without dropout. The weights
of this network are scaled-down versions of the trained weights. If a unit is retained with
probability p during training, the outgoing weights of that unit are multiplied by p at test
time as shown in Figure 2. This ensures that for any hidden unit the expected output (under
the distribution used to drop units at training time) is the same as the actual output at
test time. By doing this scaling, 2n networks with shared weights can be combined into
a single neural network to be used at test time. We found that training a network with
dropout and using this approximate averaging method at test time leads to signicantly
lower generalization error on a wide variety of classication problems compared to training
with other regularization methods.
I guess the easiest way to understand it is to watch this video. As you can see there are different implementations for that but the reason it is multiplied is that for any hidden unit the expected output (under the distribution used to drop units at training time) is the same as the actual output at test time. To be concise, it is done in order not to change the distribution that the outputs of the layer have.
