Questions tagged [dynamic-programming]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
0 answers
16 views

Consistency error in visualization of policy improvement in Sutton & Barto's book?

Sutton & Barto introduce in their foundational book on "Reinforcement Learning: An Introduction" in the context of Dynamic Programming algorithms for policy evaluation and improvement. ...
Steve's user avatar
  • 41
1 vote
0 answers
18 views

Structured policies in dynamic programming: solving a toy example

I am trying to solve a dynamic programming toy example. Here is the prompt: imagine you arrive in a new city for $N$ days and every night need to pick a restaurant to get dinner at. The qualities of ...
learningowl's user avatar
0 votes
1 answer
492 views

Bellman operator and contraction property

Currently, I am learning about Bellman Operator in Dynamic Programming and Reinforcement Learning. I would like to know why is Bellman operator contraction with respect to infinity norm? Why not ...
lra's user avatar
  • 21
3 votes
1 answer
118 views

Confusion about the Bellman Equation

In some resources, the belman equation is shown as below: $v_{\pi}(s) = \sum\limits_{a}\pi(a|s)\sum\limits_{s',r}p(s',r|s,a)\big[r+\gamma v_{\pi}(s')\big] $ The thing that I confused is that, the $\pi$...
datatech's user avatar
1 vote
1 answer
84 views

Coding a Content Addressable Memory on a GPU

I´m trying to code a CAM or more simply a dictionary storing the pointer of the data accessible by a key. I try to do it with a GPU but all attempts have been inefficient compared on using System....
Izar Urdin's user avatar
2 votes
2 answers
113 views

Idenitity between TD(0) algorithm and Policy Evaluation in Dynamic Programming when alpha is equal to 1

TD(0) algorithm is defined as the iterative update of the following: $$ V(s) \leftarrow V(s) + \alpha({r + \gamma V(s')} - V(s) ) $$ Now, if we assume alpha to be equal to 1, we get the traditional ...
Tommaso Bendinelli's user avatar
1 vote
0 answers
450 views

Reducing the training time of an RL agent

I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my ...
chink's user avatar
  • 555
3 votes
1 answer
142 views

About the time differences in the Bellman equation

I am trying to grasp fundamental mathematics behind the Reinforcement Learning and so far I have unterstood how the Value Iteration and Policy algorithms do converge (contractions, etc.) I have still ...
Ufuk Can Bicici's user avatar
4 votes
1 answer
3k views

What is the difference between dynamic programming and Q-learning?

What is the difference between the DP-based algorithm and Q-learning?
user10296606's user avatar
  • 1,834