I have a sequential data from time T1 to T6. The rows contain the sequence of states for 50 customers. There are only 3 states in my data. For example, it looks like this:

        T1  T2  T3  T4  T5  T6
Cust1   C   B   C   A   A   C

My transition matrix X looks like this:

    A   B   C
A   0.3 0.6 0.1
B   0.5 0.2 0.3
C   0.7 0.1 0.2

Now, we see that at time T6 the state is at C which corresponds to c=[0 0 1] vector. I am now predicting T7 by doing the matrix multiplication: c * X which gives me [0.7 0.1 0.2]. Based on this, I decide that the state at T7 would be A (highest prob. value).

For T8, I use the result of the probability vector I got above and do [0.7 0.1 0.2]*X = [0.4 0.44 0.14] and decide that the state is B.

My question is: Am I doing something wrong? Am I contradicting the memoryless property of the Markov Chains?


2 Answers 2


I am not sure what you actually want to do, but if you want to simulate the Markov chain, than you really have to bring probabilities into play.

Thus, in each step, you would use the transition matrix to determine the probability for each of the possible target states - A, B and C in your example. You would then draw a value - let me call it U - from a uniform distribution on [0,1] and determine the next state based on that value.

In your example, you would proceed from C to A if U is between 0 and 0.7 (and therefore with probability 0.7), to B if U is between 0.7 and 0.8 (and therefore with probability 0.8 - 0.7 = 0.1) and stay at C if U is between 0.8 and 1.0 (with probability 1.0 - 0.8 = 0.2)


I think you are not doing anything wrong, the markov property is satisfied when the prediction can be solely based on the present state. I do not think you are contradicting this "memoryless" condition (which applies to the whole chain) when you predict what could probaly happen after a last state that is already just predicted.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.