I think the OP was confusing about AlphaGo with alpha-beta. In alpha-beta, you'd indeed use the policy network for helping with pruning, but not here. Again, there is no pruning as the algorithm relies on Monte-Carlo tree search (MCTS). Anyone who thinks my answer is too long might skip to the summary section, where I state why the two networks are not redundant. In the following example, I'll do some simplification to make my ideas easier to understand. **Example:** Imagine you have a position where there are two legal moves. The first move is a dead-lost for you, however, the second move gives you a winning advantage. You'll also need to reach a depth of 10 to see the loss/win. * First move: forced loss for you (need to see 10 moves ahead to see that) * Second move: forced win for you (need to see 10 moves ahread to see that) **Evaluation network** Let's assume the evaluation network Google gives you is perfect. It can evaluate any leaf position in our example perfectly. We won't change our value network in the example. To simplify our example, let's assume our value network gives: - -1000 for any leaf position which is a loss for you - +1000 for any leaf position which is a win for you **Policy network** Let's assume Google gives you two policy networks. The probabilities generated for our position is: - Policy 1: 0.9 for move 1 and 0.1 for move 2 - Policy 2: 0.2 for move 1 and 0.8 for move 2. Note that our first policy network gives **incorrect** prior probability for our example. It gives 0.9 for move 1, which is a losing move. This is fine because not even Google could train a perfect policy network. **Playing with the first policy network** AlphaGo needs to generate a simulation with Monte-Carlo, and it needs to choose move 1 or 2. Now, AlphaGo draws a uniform-distributed random variable, and it'll pick: - Move 1 if the random number is <= 0.9 - Move 2 if the random number is > 0.9 So AlphaGo is much more likely to pick the losing move to simulate (in our very first simulation). In our first simulation, we will also use the value network to get a score for the simulation. In the paper, it's: [![enter image description here][1]][1] This value would be -1000, because this simulation would lead to a loss. Now, AlphaGo needs to generate the second simulation. Again, the first move would be much more likely to pick. But eventually, the second move would be pick because: - Our prior probability for the second move is 0.1, not zero - AlphaGo is encouraged to try moves which haven't been explored much. In the paper, this is done by this equation: [![enter image description here][2]][2] Note that `N` is the number of moves searched for the move and it's in the denominator. The more likely our first move is searched, the smaller the `u` function is. Thus, the probability for selecting our second move improves because AlphaGo actually picks a move by this equation: [![enter image description here][3]][3] [![enter image description here][4]][4] This is the **key** equation. Please look at it carefully: - It has a term `P` for the prior probability (given by the policy network) - It has a term `Q` for the evaluation scores (given by the value network) Now, we know our second move will eventually be chosen. When it does happen, the value network gives a +1000. This will increase `Q`, which makes the second move much more **likely** be chosen in the later simulations. Given enough simulations, the number of times the second move is chosen for simulation should be more than the number of times the first move is chosen. Finally, the move that AlphaGo decides to make is (quoted from the paper): > Once the search is complete, the algorithm chooses the most visited move from the root position. **Playing with the second policy network** Our second policy network will need less evaluations to pick move 2 because it's prior probability given by the policy network is correct in the first place. **Remarks** Everything here is very similar to `Bayesian` analysis. We start off with some prior probability (given by the policy network), then we generate data to move the probability distirubtion (given by the value network). **Summaries** - Policy network is used to generate prior probabilities to guide what move the Monte-Carlo search should pick - Value network is used to generate data to validate the policy network. If the policy network is bad, AlphaGo would need more computing resources to converge (if ever). - You can think of it like Bayesian analysis [1]: https://i.sstatic.net/kqpt7.gif [2]: https://i.sstatic.net/sdkQf.gif [3]: https://i.sstatic.net/jZviV.gif [4]: https://i.sstatic.net/lw5cz.gif