In Q-learning, Q is an array of expected rewards for (state, action) combinations.
It seems to me the same result could be achieved while slightly simplifying the algorithm, if instead of associating reward with taking a given action in a state, we just associate it with the state. Then a reward would come to be associated with taking action A in state S, by virtue of the fact that this leads to state T, which has the reward. (This would correspond to the intuitive idea that an action is taken not for its own sake, but in order to steer the world into a desirable state.)
Is there a reason this is not done? All the references I have found so far, simply take it for granted that Q maps (state, action) pairs to expected rewards, without commenting on other possibilities.