For a side project I'm trying to build a (simplified) AI for Heroes Of Might and Magic, using (as a starting point) deep Q-learning. But I'm having trouble to understand how the "state space" is supposed to be represented. Here's an idea of what the battle looks like enter image description here

Roughly speaking:

  • there are 2 opposing armies
  • each armies has troops
  • each troop occupies an hex and contains a number of soldiers
  • the more soldiers, the more HP and attack a "troop" has
  • each troop attacks in turns, and during its turn it can either move, or move and attack an adjacent enemy troop
  • the army that manages to kill all enemy troops win

I've simplified this such that I reduced the board size and am only considering 1 troop per army. However I am not sure how I should "formalize" the problem into a RL problem, especially concerning the "state" of the game. For state, I can either:

  • have a state that exposes the Hit Point, position, attack, etc of the enemy and my own
  • have a state that is a NMP tensor that contains all the info of the board. Ie the tensor at position (0,0) would contain a 0-vector, but the tensor at position (1,2) where an enemy is would contain a vector of all the enemy statistics

It seems to me that the first solution would be simpler but complex to generalize to a variable number of enemy units Second solution has all the infos but the state is getting big and very sparse. Also I'm not sure it's the best to input the state in terms of absolute positions, rather than relative position to the unit that has to take an action.

What do you think ? Are there any resources to understand better how such a problem should be transformed into a RL problem?

  • $\begingroup$ It would be better suited for the AI SE. It's usually a really difficult problem as you usually should start with replicating a fully functionnal environnement (i.e either recoding the battle aspect of the game, or somehow interface your algo to the game). Maybe start with another game (chess) with a simpler set of rules and for which you can find online ressources. $\endgroup$ Commented Nov 13, 2020 at 13:08


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.