For a side project I'm trying to build a (simplified) AI for Heroes Of Might and Magic, using (as a starting point) deep Q-learning. But I'm having trouble to understand how the "state space" is supposed to be represented.
Here's an idea of what the battle looks like
Roughly speaking:
- there are 2 opposing armies
- each armies has troops
- each troop occupies an hex and contains a number of soldiers
- the more soldiers, the more HP and attack a "troop" has
- each troop attacks in turns, and during its turn it can either move, or move and attack an adjacent enemy troop
- the army that manages to kill all enemy troops win
I've simplified this such that I reduced the board size and am only considering 1 troop per army. However I am not sure how I should "formalize" the problem into a RL problem, especially concerning the "state" of the game. For state, I can either:
- have a state that exposes the Hit Point, position, attack, etc of the enemy and my own
- have a state that is a NMP tensor that contains all the info of the board. Ie the tensor at position (0,0) would contain a 0-vector, but the tensor at position (1,2) where an enemy is would contain a vector of all the enemy statistics
It seems to me that the first solution would be simpler but complex to generalize to a variable number of enemy units Second solution has all the infos but the state is getting big and very sparse. Also I'm not sure it's the best to input the state in terms of absolute positions, rather than relative position to the unit that has to take an action.
What do you think ? Are there any resources to understand better how such a problem should be transformed into a RL problem?
