1
$\begingroup$

I read the AlphaGo Zero paper and I didn't found nothing about it in there. But I would like to know if AlphaGo Zero can adapt to the way the oponent plays (oponent profile) or something like this. Thanks!!

$\endgroup$
1
$\begingroup$

But I would like to know if AlphaGo Zero can adapt to the way the oponent plays (oponent profile) or something like this.

That is not included in the algorithm as written, where the "profile" of the opponent is effectively AlphaGo Zero itself (learned through self play).

It is not clear whether adapting play style to a given opponent would offer any advantage. It would be difficult to assess because AlphaGo Zero is such a strong player, that it will win a large percentage of games against human players as-is. Seeking and measuring any improvement, except versus earlier versions of itself, would be quite hard.

However, there are a likely a few places in the code where learned play style of an opponent could in theory allow AlphaGo Zero to be more efficient. The most obvious is in the "rollout" policy (I'm not 100% sure if they use the same term), where the algorithm simulates and samples different possible trajectories through the game in order to predict likely outcomes.

The current rollout policy in AlphaGo is learned through self play. But it is just a neural network that predicts probability of making plays given board state. It could easily be adjusted in a supervised learning fashion, based on sampled plays from an opponent. If it could be learned accurately, then it should make searches more efficient and accurate - the impossible but ideal situation being that it predicted opponents' move exactly and thus could quickly find the ultimate counter to their actions. In fact the original AlphaGo rollout policy did model human play in this way. It was based on large database of many human master level play moves, not a single player. The Deep Mind team did suggest in their paper that this gave better results at the time than a self-play policy - they tried both and the human database was better. Since then, AlphaGo Zero has surpassed the performance of original AlphaGo without the database of human moves.

$\endgroup$
4
  • $\begingroup$ Hello @Neil Slater, thanks for your so good answer, I really appreciate this!! I agree with you the AlphaGo doesnt need some kind of adaptation in your game style against humans, but against previous versions or others automatic agent players this could be a crucial difference. And about use a large database from a player to adapt the game style of the AlphaGo, I think that's not really interesting, because we know the goal to construct automatic agent players is to build an efficient autonomous system that can learn and play for itself (at least I think the goal is that). $\endgroup$ May 19 '18 at 17:04
  • $\begingroup$ And so I think it's more interesting to make or apply some kind of a rule-based system in a real time way (the agent collects informations of the oponent style game and makes use of this informations to try to antecipate the oponent moves and adapt your style). Do you think it's a good idea considering board games? $\endgroup$ May 19 '18 at 17:04
  • $\begingroup$ @MatheusPrandini: It is possible to re-train the rollout network online whilst working against a player. However, I am not convinced that the network is fast learning and flexible enough to extrapolate a player's "style" from seeing only a few plays. So it could not infer the type of move a player makes late in the game from seeing early moves (at least not without specific training to do that in general). That is still an area where current machine learning is behind human behaviour. $\endgroup$ May 19 '18 at 17:49
  • $\begingroup$ Yeah, I agree with you. But I'm very interisting and motivated to study something like this to apply in my Master's degree research. Thanks for the comments! $\endgroup$ May 20 '18 at 12:34
1
$\begingroup$

It's not important to adjust to the opponent, as Go is only about winning or losing. There is no bigger reward the faster / the more obvious it wins.

Or to put it different: only the current board situation is important in a min-Max setting (although the value approximation of a state admittedly depends a bit on the opponent)

$\endgroup$
1
  • $\begingroup$ Hello @Martin Thoma, thanks for you answer!! Well, I see your point but I think its interesting to use some kind of a rule-based system in an automatic agent player, because the more you can antecipate the oponent moves the more will be the chances of win the game. And this fact reflects directly in the eficient of the agent player. I think this can be used or implemented on any board game and can bring good results. $\endgroup$ May 19 '18 at 16:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.