Return to Answer

Imagine you have a position where there are two legal moves. The first move is a dead-lost for you, however, the second move gives you a winning advantage. You'll also need to reach a depth of 10 to see the loss/win.

• First move: forced loss for you (need to see 10 moves ahead to see that)
• Second move: forced win for you (need to see 10 moves ahread to see that)

Our second policy network will need less evaluations to pick move 2 because it's prior probability given by the policy network is correct in the first place.

Imagine you have a position where there are two legal moves. The first move is a dead-lost for you, however, the second move gives you a winning advantage.

• First move: forced loss for you
• Second move: forced win for you

Our second policy network will need less iterations to pick move 2 because it's prior probability given by the policy network is correct in the first place.

Now, we know the our second move will eventually be chosen. When it does happen, the value network gives a +1000. This will increase Q, which makes the second move much more likely be chosen in the later simulations.

Now, we know our second move will eventually be chosen. When it does happen, the value network gives a +1000. This will increase Q, which makes the second move much more likely be chosen in the later simulations.

So AlphaGo is much more likely to pick the losing move to simulate here (in the very first simulation). In our first simulation, we will also use the value network to get a score for the simulation. In the paper, it's:

Given enough simulations, the number of times the second move is chosen for simulation should be more than the number of times the first move.

Everything here is very similar to Bayesian analysis. We start off with some prior probability (given by the policy network), then we generate data to move the probability (given by the value network).

So AlphaGo is much more likely to pick the losing move to simulate (in our very first simulation). In our first simulation, we will also use the value network to get a score for the simulation. In the paper, it's:

Given enough simulations, the number of times the second move is chosen for simulation should be more than the number of times the first move is chosen.

Everything here is very similar to Bayesian analysis. We start off with some prior probability (given by the policy network), then we generate data to move the probability distirubtion (given by the value network).