Problem: Lets say we have an irreducible Markov chain. Given a failure state or non desirable state F and a current state S Is it possible to find how far we are from the failure state F.

Example: if we try to model an engine failure using Markov process and using historic data we have a state transition matrix P, one of which is a state where the engine was failed . Now , in a dynamic system, if the current state S and a state transition matrix are given, can we calculate how far we are from failure state F.


x(n+1) = (x)P^n

Right hand side of the equation gives us a vector v which consists of probabilities of transitioning to different states at stage n. With different value of n, the values in this vector v will change one of the values in v will be the probability of going to state F. I want to find for what n , transitioning probability to F is maximum.

  • $\begingroup$ (This question belongs to math.stackexchange..) You can get an approximate solution by simply computing your probability for all n for which L1 distance between x P^n and the stationary distribution x* is more than epsilon. This assumes your chain is also aperiodic. $\endgroup$
    – Valentas
    May 29, 2015 at 8:43


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.