I am having some confusion as to whether the action should be included as part of the state input to an agent in a reinforcement learning setting (state-action pair). As from my observation, this is not completely clear as different agents/environments combinations might have different performances if action was included/excluded from input states (I might be wrong).
For my specific problem:
- the agent can't influence/control the states through its actions (similar to the case of a simple multi-armed bandit)
- the action space is discrete
- I am using a DQN based approach
I would also appreciate a general overview/rules of thumb of when to include/exclude actions as state inputs.
ps. when i say "different agents/environments combinations" in the beginning I mean using different agents to solve the same env or same agent to solve different env.