I'm working on building an AI for a chess-like game. I've implemented a Monty Carlo Tree Search (for the early game) and Rainbow DQN (for the mid to late game), and will be implementing Alpha Beta pruning and an Endgame Tablebase.

Part of my plan is that, during training, moves which are dictated by the Monty Carlo Tree Search won't be added to the memory of DQN, as those early position, moves, and transitions would recur frequently and seem unlikely to help with creating a strong evaluation function of mid to late game positions (instead seeming more likely to disrupt the process).

I also thought I might apply this idea to the moves that were dictated by the tablebase, with any state that can be looked up in the tablebase, being stored as a terminal state in the memory of the Rainbow DQN. In my mind this carries the advantage of making the rewards a little less sparse, as they would be spread over less moves. These moves also seem less valuable to the creation of a strong evaluation function of mid to late game positions.

Do these "optimizations" sound reasonable?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.