2
$\begingroup$

The following text is quoted from the AlphaGo Zero Paper 2017 from Nature. My question is regarding the eight features.

The input to the neural network is a 19 × 19 × 17 image stack comprising 17 binary feature planes. Eight feature planes, Xt, consist of binary values indicating the presence of the current player’s stones (Xit=1 if intersection i contains a stone of the player’s colour at time­step t;

This sounds like the t means time and the eight features are the latest 8 board states. If this is the case, this will be my question.

0 if the intersec­tion is empty, contains an opponent stone, or if t < 0). A further 8 feature planes, Yt, represent the corresponding features for the opponent’s stones. The final feature plane, C, represents the colour to play, and has a constant value of either 1 if black is to play or 0 if white is to play. These planes are concatenated together to give input features st = [Xt, Yt, Xt−1, Yt−1,..., Xt−7, Yt−7, C].

I have the feeling the eight features should be the combination of rotated (4) and reflected (2 ) board states of the current board state. Please correct me if I am wrong.

History features Xt, Yt are necessary, because Go is not fully observable solely from the current stones, as repetitions are forbidden; similarly, the colour feature C is necessary, because the komi is not observable.

Board repetition can be easily prevented by keeping tracking of all previous board states and when a new move is attempted, just compare against all previous board states. The same is for komi. Keeping 8 previous board states to prevent repetition doesn't sound right to me. What if the repetition happened to the last 9th board state? If this is not a serious paper published to the Nature, I would highly doubt this is wrong.

$\endgroup$

1 Answer 1

2
$\begingroup$

This sounds like the t means time and the eight features are the latest 8 board states.

That is correct. The description in the article is clear.

I have the feeling the eight features should be the combination of rotated (4) and reflected (2) board states of the current board state. Please correct me if I am wrong.

That is wrong.

This data augmentation can be done by modifying state data stored from each episode, and applied during weight update step (either randomly or exhaustively), because any rotation/reflection of a board should have the same expected value. It is not necessary to provide that augmentation at the feature level.

Board repetition can be easily prevented by keeping tracking of all previous board states and when a new move is attempted, just compare against all previous board states.

A game engine can do that, as could bespoke code added to the agent that was specific to Go. However, a neural network cannot do that unless those previous states are part of the input.

The same is for komi.

If you want the board to correctly predict the value of a state, the nerual network needs to have the current player identity encoded in some way. Having it tracked outside of the neural network's inputs (i.e. outside of state representation) does not help with this.

What if the repetition happened to the last 9th board state?

Most typically in Go, the repetition rule has effects over a relatively short duration, because moves add stones in new positions. The relevant rules are called ko and superko, with superko being optional. When applying the standard ko rule, the repetition that it blocks is only likely to be achievable through play on short timescales (2 steps) that are covered by the AlphaGo Zero representation.

In the unlikely event that a play is blocked due to superko rule and repetition from earlier than 8 steps ago, the game engine would prevent a play that the agent may have assigned a high value to, and the agent could get the value wrong. I suspect that this has little or no impact on performance, as loops in Go board state of this duration are rare and do not occur much in practice.

$\endgroup$
8
  • $\begingroup$ If the game engine can handle repetition, why bother adding 14x19x19 input features to the neural network. The repetition should be ruled out just like when the stones need to be removed from the board when their liberty is 0. This should be the rule of the game. $\endgroup$
    – Qian Chen
    Sep 14, 2020 at 7:44
  • $\begingroup$ I don't understand how the neural network could prevent repetition using the previous 8 steps as input features. The neural network only gives feed back in terms of how good or how bad the move is. The repetition prevention should be game rules that should be hard coded somewhere, I think. $\endgroup$
    – Qian Chen
    Sep 14, 2020 at 7:59
  • $\begingroup$ @QianChen: The neural network needs to calculate expected value of a state. The value is different when a board position seen first and second time (because the second time would not be allowed). The rules are already in the game engine, but the game engine does not return a value for a state. In order to get the correct value for the state, the state encoding given to the NN function needs to account for the impact of the rules. By your argument there would be no need to have any information given to the neural network, because the validity of moves is handled by the game engine. $\endgroup$ Sep 14, 2020 at 8:10
  • 1
    $\begingroup$ @QianChen: The value function is approximate. It is better the more repetition it can handle, but there are diministing returns. Being able to spot ko is really important for accuracy of the value functions, but being able to spot beyond 8 steps is much less important (because it will very rarely occur, if ever, during near optimal play, and not affect the value score much at all). I don't know why Deep Mind chose 8 steps specifically. However, they need at least 2 for the most basic ko detection. Again this is for the valuation of a state, not to model the rules. $\endgroup$ Sep 14, 2020 at 8:31
  • 1
    $\begingroup$ @QianChen You say "the value network will give the answer if the ko is not handled properly." - it will give an answer, but it will be the wrong answer (it would be some average of when ko applies and when it does not). Just like with whose turn it is, if the NN does not have data in the input that allows it to adjust the value for what is really going on, it cannot do so. The NN in AlphaGo Zero does not have any memory (it is not e.g. a RNN), so if anything in the context of the current board position would affect the predicted value, it is needed as an input. $\endgroup$ Sep 14, 2020 at 12:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.