My issue derives from the challenge of solving a seemingly easy-looking game. To spare you the full catalogue of rules, here is a short summary of the game:
- Single player card game
- You go through a standard deck of cards and need to choose an action for each card (without knowing the next cards)
- For each of the choices you have a huge variant of possibilities (about ~15 stacks) to apply the cards to - given certain suit and rank combos you get points (lots of rules underlying - thus this description represents the gist setting of the game)
If the cards were known, you could (by backpropagation) calculate the perfect choice in order to get the maximum amount of points. Intuitively, I tried writing an algorithm to simulate as many moves as possible into the future to determine the best choice. Even with various pre- and post-pruning methods (such as trivial mini-max), it would take hours to calculate one decision in a semi-reliable manner (considering most rewards are only "seen" about 15+ moves after the choice).
For the last couple of weeks, I abandoned this basic approach and went ahead to look for viable reinforcement learning ideas. To say the least, I felt completely lost due to various problems, i.e. integrating the "card count" into my state space. Should I include the full track of cards remaining (more detailed) or revert to a well-known implementation (Hi-Lo System in Blackjack for example.. has many similarities to this project).
All in all, I wanted to ask for a usable proposal in order to solve a immense-state space ("continuous"?) problem with self, reinforcement learning. I peeked into DeepQ learning or alike, but couldn't find good literature in regards to solo-player card games like Blackjack which require you to keep a card count. Besides that, the delayed reward makes it harder to compare to such projects.