2
$\begingroup$

I'm attempting to model a simple process using a Markov Decision Process.

Let $A$ be a set of $3$ actions : $ A \in \{b,s\}$. $T(s,a,s')$ represents the probability of if in state $s$ , take action $a$ and end up in state $s'$

Notation for the MDP diagram is as follows :

enter image description here

Here is my MDP diagram which models 7 states:

The outgoing actions for each state sum to 1.

enter image description here

$T(1,b,2) = .7 $

$T(1,b,3) = .3 $

$T(1,s,4) = .9 $

$T(1,s,5) = .05 $

$T(1,s,6) = .05 $

I've tried to keep this as simple as possible to check my understanding. Are my representations & probabilities correct ?

Improve this question
$\endgroup$
2
  • $\begingroup$ What is action 'h`? That is not being modeled. $\endgroup$
    – Brian Spiering
    Feb 9, 2020 at 23:49
  • $\begingroup$ @BrianSpiering I've not included 'h' , it can be modeled but I've not included it. I've removed 'h' for clarity, thanks. $\endgroup$
    – blue-sky
    Feb 10, 2020 at 19:52

1 Answer 1

Reset to default
1
$\begingroup$

Looks 'correct' to me, in the sense that it satisfies the requirements for being an MDP. Whether it models the underlying real-world problem correctly cannot be validated with the information given here.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.