In Reinforcement Learning, while creating transition samples (state, action, next_state, reward), where:

  • Agent: The learning agent
  • Environment: The trainer

The environment gives two feedback to the agent: reward and next state. Can I as the environment, randomly assign next_states from the total state space to my agent. ? How can I decide what are the allowed next_state(s) from a given state?

New contributor
DK818 is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.

It depends on your environment. For example, If you environment is a straight line. Let's say that the states are 1, 2, 3 ... 10. The agent here can either move left or right. Now, if the agent is at state 4, and it moves right, according to the logic of the environment, next_state has to be 5.

Now, if you include some sort of a condition that, if the agent reaches state 5, the agent will be randomly moved to any other position, then you can return the next_state to be any random number.

Long story short, the next_state is dependent on what the environment is.

New contributor
Parthapratim Neog is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
  • If we are randomly selecting the next state, how is the training of model (agent) going to converge? Because in the Deep Q learning approach., usually, the Q value is a function of the current reward + Q value of the next state. And for a given current state if the next state keeps on changing every time, then the best-suited action will be difficult to find for a given current state. Is my understanding correct? – DK818 yesterday
  • I think this is an interesting topic to discuss, on how an agent will learn in an environment like this. A simple example environment would be a ludo game environment where your next_state would be determined by the value of dice ( which is random chosen between 1 and 6 ). I am not really sure, and qualified enough to answer this new question. Maybe you can put up another question on this. – Parthapratim Neog yesterday

Your Answer

DK818 is a new contributor. Be nice, and check out our Code of Conduct.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.