How does dropout work during testing in neural network?

Question

The below paragraph is picked from the textbook Hands-On Machine Learning with sci-kit learn & Tensorflow. I couldn't understand what the author is trying to convey. It would be really grateful if someone can provide an explanation for below paragraph with an example. I have referred these links Dropout in Deep Neural Networks, https://www.coursera.org/lecture/deep-neural-network/dropout-regularization-eM33A, that somehow aligns with the same objective. However, i am seeking good explanations. Please help me.

Suppose p = 50%, in which case during testing a neuron will be connected to twice as many input neurons as it was (on average) during training. To compensate for this fact, we need to multiply each neuron's input connection weights by 0.5 after training. if we don't, each neuron will get a total input signal roughly twice as large as what the network was trained on, and it unlikely to perform well. More generally, we need to multiply each input connection weight by the keep probability(1-p) after training. Alternatively, we can divide each neuron's output by keep probability during training (these alternatives are not perfectly equivalent, but they work equally well)

Answer 1

During training, p neuron activations (usually, p=0.5, so 50%) are dropped. Doing this at the testing stage is not our goal (the goal is to achieve a better generalization). From the other hand, keeping all activations will lead to an input that is unexpected to the network, more precisely, too high (50% higher) input activations for the following layer.

Consider the neurons at the output layer. During training, each neuron usually get activations only from two neurons from the hidden layer (while being connected to four), due to dropout. Now, imagine we finished the training and remove dropout. Now activations of the output neurons will be computed based on four values from the hidden layer. This is likely to put the output neurons in unusual regime, so they will produce too large absolute values, being overexcited.

To avoid this, the trick is to multiply the input connections' weights of the last layer by 1-p (so, by 0.5). Alternatively, one can multiply the outputs of the hidden layer by 1-p, which is basically the same.