I think I have understood that it consists in choosing the bandit that maximizes the future reward.
Yes, in expectation.
The target of finding the best action is often easy to get eventually correct - you could for instance try each action 1000 times in turn and calculate the average reward from it. So, there are usually two other important goals that ...
Because we use cumulative reward (or the approximated Q/V of it, but not the immediate reward of (s,a)) in order to calculate gradient-contributed by each trajectory, which we collected by follow the most recent policy.
using a 50 x 50 matrix I get 2500 cells, and in the construction of the neural network I have a 2500x2500 parameters + 2500 for a total of 6252500.
I think that slows down operations.
and there are only 3 Danse layers, the last one is size 4 because of the possible actions that are 4.
Is it possible to reduce the time of operation by adding more Danse layers?...