1
$\begingroup$

In Reinforcement Learning, while creating transition samples (state, action, next_state, reward), where:

  • Agent: The learning agent
  • Environment: The trainer

The environment gives two feedback to the agent: reward and next state. Can I as the environment, randomly assign next_states from the total state space to my agent. ? How can I decide what are the allowed next_state(s) from a given state?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

It depends on your environment. For example, If you environment is a straight line. Let's say that the states are 1, 2, 3 ... 10. The agent here can either move left or right. Now, if the agent is at state 4, and it moves right, according to the logic of the environment, next_state has to be 5.

Now, if you include some sort of a condition that, if the agent reaches state 5, the agent will be randomly moved to any other position, then you can return the next_state to be any random number.

Long story short, the next_state is dependent on what the environment is.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ If we are randomly selecting the next state, how is the training of model (agent) going to converge? Because in the Deep Q learning approach., usually, the Q value is a function of the current reward + Q value of the next state. And for a given current state if the next state keeps on changing every time, then the best-suited action will be difficult to find for a given current state. Is my understanding correct? $\endgroup$
    – DK818
    Dec 12, 2018 at 1:58
  • 1
    $\begingroup$ I think this is an interesting topic to discuss, on how an agent will learn in an environment like this. A simple example environment would be a ludo game environment where your next_state would be determined by the value of dice ( which is random chosen between 1 and 6 ). I am not really sure, and qualified enough to answer this new question. Maybe you can put up another question on this. $\endgroup$
    – Parthapratim Neog
    Dec 12, 2018 at 6:21
  • 1
    $\begingroup$ so can a human act as an environment and provide next states during training? But this will not ensure that from the initial state 's' if action 'a' is taken then it will always go to next state s'. because now next state s' will be a function of human input and it will be inconsistent. I think this kind of setup will never converge $\endgroup$
    – DK818
    Dec 18, 2018 at 2:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.