I'm working in an RL environment where not all actions are always available. In this case, depending on the state where the environment is at, some of the actions are not available for the agent to choose.
I could find this work that deals with this situation: https://ojs.aaai.org/index.php/AAAI/article/view/5740
Before moving on and starting implementing it, I'd like to ask if you have any other suggestions of solutions to deal with such a scenario.
I'd appreciate any pointer.
You could use an out-of-the-box learning algorithm if the availability of actions is deterministic and depends only on the state the agent is in, i.e. if availability of actions does not violate the Markovian assumption. You need only substitute the full action space with the 'available action space' in any place where the action space shows up. As an example, consider the update rule in Q learning, then replace the Q(S,A) with Q(S, A') where S' indicates the available actions for that particular S.
Please note that the referenced paper targets a setting in which availability of actions is not Markovian.
