How to estimate the transition probabilities for a Markov Chain when time intervals are non-equally spaced

Question

I've been given a dataset with a number of observable states. I am trying to apply a Finite State Markov Chain to model the system, but I found that I can't estimate the transition probabilities if the observed states were sampled using different time intervals. How can I find these probabilities?

I will try to make the question a more clear. I have samples collected in random intervals during a 6-month period. This samples represent the "quality" of a system, which is ranked from 0 to 15 in discrete intervals i.e. (0,1,2...15). I need to model de system using a FSMC to mimic the system's behavior. So far, I have estimated the transition probabilities between states using only the frequencies of those transitions using all the samples. I am not interested in modeling the time, however, I am not sure If I can estimate the transition probabilities in such a simple way or if I have to take the time between samples (which in my case is random) into consideration when estimating those probabilities.

Answer 1

With the limited information that you have given, it is not possible to judge the trade-off between different approaches of handling time here. I can propose the following alternatives.

• Check if you can discretize the time information and then use transition to same state assumption. For example, let us say that transition from A to B took place at 30 minutes. If you have standard time interval of 10 minutes, you can say that for interval 1 & 2 (10 minutes, 20 minutes), the transition was from state A to A whereas in the third interval the transition was from A to B. There will be problems though when multiple transitions take place in a given interval. You can either make the interval small enough or distribute the transition probability equally in such case.

• Is the time information significant to whatever that you are trying to model? Otherwise you can ignore the time information.