Questions tagged [markov-process]

A Markov process is a stochastic process for which the Markov property holds: If you know the current state, then the next state is independent of all past states.

Filter by
Sorted by
Tagged with
0
votes
0answers
523 views

HMM - Matlab for data set to detect anomaly

I have a dataset of oil temperatures. The time series consist of 100 hours of measurement at every second. There is an anomaly in the data that I would like to detect using Hidden Markov Models (HMM). ...
1
vote
0answers
1k views

Markov chain modelling?

I am working on a personal loan dataset. For each loan, we recorded its credit status monthly after it was drawn by the borrower. Let's say there were 6 status coded by A-F. My project is to use ...
17
votes
1answer
2k views

Visualization of multiple Markov models

I am working on a project where we compare over 10 different Markov models, each representing a different treatment plan. Most often single models are visualized with a decision tree or transition ...
3
votes
1answer
129 views

Using dhmm_em to form the hmm of mfccs' from song clips

I was working on a project on music genre classification and decided to use an hmm to model my data. After extracting the mfccs' from multiple song clips I get multiple matrices of size 13 * x (...
1
vote
0answers
276 views

Online methods for sequence prediction

A ml beginner here, so please bear with me. If I understand correctly RNNs seem to be the go to method right now for sequence prediction for a given input (single/as a sequence). But I do not have ...
3
votes
3answers
3k views

Data visualization of frequencies of state transitions (possibly in R?)

I am working on some experimental data, which can be of types A, B and C. Now I observe this data for 5 time points, and I can see them move between A to B, B to C,... etc. I see such transitions for ...
4
votes
1answer
7k views

Markov switching models

What are some reference sources for understanding Markov switching models?
7
votes
3answers
944 views

How do I choose a discount factor in Markov Decision Problems?

I'm referring to the gamma in the Value function:
2
votes
1answer
266 views

Can I apply Hidden Markov Models this way?

I have just gotten my feet wet with Hidden Markov Models. Now I want to apply them to tell whether a transaction from an ATM is suspicious or not. I have great confusion in defining my Hidden States. ...
0
votes
2answers
3k views

Can HMM be used as a binary classifier?

I have some time-series data, which I need to use to predict a binary label for a given time-stamp. I was initially doing it using a Logistic Regression Model but I later figured that a time-series ...
17
votes
5answers
15k views

Python library to implement Hidden Markov Models

What stable Python library can I use to implement Hidden Markov Models? I need it to be reasonably well documented, because I've never really used this model before. Alternatively, is there a more ...
2
votes
1answer
119 views

How can I rank paths through an HMM?

I have a profile hidden Markov model that I use to identify all instances of a user-defined pattern of symbols in a long sequence of symbols. I use the Viterbi algorithm to find the most probable path ...
3
votes
1answer
2k views

emission probability using hmmlearn package in python

I am learning hmm and try to implement it in Python hmmlearn package(http://hmmlearn.github.io/hmmlearn/hmm.html#building-hmm-and-generating-samples). However I am not quite understand what the ...
3
votes
1answer
157 views

When to stop calculating values of each cell in the grid in Reinforcement Learning(dynamic programming) applied on gridworld

Considering application of Reinforcement learning(dynamic programming method performing value iteration) on grid world, in each of the iteration, I go through each of the cell of the grid and update ...
4
votes
1answer
178 views

Markov Chains: How much steps to conclude a Transition Matrix

I have just learned Markov Chains which I am using to model a real world problem. The model comprises 3 states [a b c]. For now I am collection data and calculating ...
3
votes
0answers
64 views

Identifying sequences of behavioral interactions between multiple individuals

I'm wondering if anyone might have some novel insights as to the best way to analyze the following data. It's a problem I've been thinking about in the back of my mind for a while, so I thought that ...
0
votes
1answer
68 views

Matlab simulation through FIS and Markov Process

I need to simulate for an academical project how the traffic fluxes (input/output with respect to a monitored area, measured in number of cars) of a city area evolves in correspondence of an event (i....

1
2