I am recently learning RL. I see most algorithms are to estimate the Q function. I would like to know why not simply train a model that takes current states as input and outputs actions.
Take ANN for example. Perhaps we can record the action taken and calculate discounted awards for every iteration, and let the optimizer adjust weight of the ANN model. Is this impossible? Why or Why is fitting the Q function better?
It is not true that most RL algorithms are to estimate the Q function. A famous RL algorithm is the policy-gradient method which does exactly what you mentioned. There is a neural net that takes in state and outputs distribution over what action should be taken next. The parameters of this network are then trained via policy gradient estimates, so this network is never explicitly modeling the Q-function or the predicted rewards at all.
That said, the variance of these policy gradients tends to be extremely high, so in practice people use Actor-Critic training which optimizes both the policy network and a Q-network jointly to stabilize training.
