I am new to reinforcement learning and I am tryng to understand the multi armed bandit problem.
I think I have understood that it consists in choosing the bandit that maximizes the future reward.
My doubt is in the implementation. In all the examples I have seen, the reward from each bandit is set with a probability distribution. My question is:
Can I set the rewards of a multi armed bandit problem with deterministic values?
I think I have understood that it consists in choosing the bandit that maximizes the future reward.
Yes, in expectation.
The target of finding the best action is often easy to get eventually correct - you could for instance try each action 1000 times in turn and calculate the average reward from it. So, there are usually two other important goals that bandit algorithms try to solve:
Finding the best action with the least number of trials.
Scoring the highest reward during learning. This may also be expressed as "minimising regret", the idea of looking back once you know what the best action is and figuring out how far away from optimal all your previous choices were.
Can I set the rewards of a multi armed bandit problem with deterministic values?
You can. A deterministic value is a special case of a probability distribution - it has $p(X = r) = 1$ and $p(X \ne r) = 0$ - so you can even use exactly the same description of the problem.
However, deterministic mulit-armed bandits are not very challenging. If you know in advance that the results are deterministic then you could try each action once and then pick the maximising result from then on. Although technically this is still a learning algorithm, it is not a very interesting one.
Deterministic contextual bandits, where the agent is given an observation that is known to influence the rewards obtained from different actions, do still have some challenge when solving for deterministic rewards. The agent then has to learn the mapping from input features to rewards whilst exploring different actions.
