Questions tagged [markov-process]

A Markov process is a stochastic process for which the Markov property holds: If you know the current state, then the next state is independent of all past states.

67 questions
Filter by
Sorted by
Tagged with
523 views

HMM - Matlab for data set to detect anomaly

I have a dataset of oil temperatures. The time series consist of 100 hours of measurement at every second. There is an anomaly in the data that I would like to detect using Hidden Markov Models (HMM). ...
1k views

Markov chain modelling?

I am working on a personal loan dataset. For each loan, we recorded its credit status monthly after it was drawn by the borrower. Let's say there were 6 status coded by A-F. My project is to use ...
2k views

Visualization of multiple Markov models

I am working on a project where we compare over 10 different Markov models, each representing a different treatment plan. Most often single models are visualized with a decision tree or transition ...
129 views

Using dhmm_em to form the hmm of mfccs' from song clips

I was working on a project on music genre classification and decided to use an hmm to model my data. After extracting the mfccs' from multiple song clips I get multiple matrices of size 13 * x (...
276 views

Online methods for sequence prediction

A ml beginner here, so please bear with me. If I understand correctly RNNs seem to be the go to method right now for sequence prediction for a given input (single/as a sequence). But I do not have ...
3k views

Data visualization of frequencies of state transitions (possibly in R?)

I am working on some experimental data, which can be of types A, B and C. Now I observe this data for 5 time points, and I can see them move between A to B, B to C,... etc. I see such transitions for ...
7k views

Markov switching models

What are some reference sources for understanding Markov switching models?
944 views

How do I choose a discount factor in Markov Decision Problems?

I'm referring to the gamma in the Value function:
266 views

Can I apply Hidden Markov Models this way?

I have just gotten my feet wet with Hidden Markov Models. Now I want to apply them to tell whether a transaction from an ATM is suspicious or not. I have great confusion in defining my Hidden States. ...
3k views

Can HMM be used as a binary classifier?

I have some time-series data, which I need to use to predict a binary label for a given time-stamp. I was initially doing it using a Logistic Regression Model but I later figured that a time-series ...
15k views

Python library to implement Hidden Markov Models

What stable Python library can I use to implement Hidden Markov Models? I need it to be reasonably well documented, because I've never really used this model before. Alternatively, is there a more ...
119 views

How can I rank paths through an HMM?

I have a profile hidden Markov model that I use to identify all instances of a user-defined pattern of symbols in a long sequence of symbols. I use the Viterbi algorithm to find the most probable path ...
2k views

emission probability using hmmlearn package in python

I am learning hmm and try to implement it in Python hmmlearn package(http://hmmlearn.github.io/hmmlearn/hmm.html#building-hmm-and-generating-samples). However I am not quite understand what the ...
157 views

When to stop calculating values of each cell in the grid in Reinforcement Learning(dynamic programming) applied on gridworld

Considering application of Reinforcement learning(dynamic programming method performing value iteration) on grid world, in each of the iteration, I go through each of the cell of the grid and update ...
178 views

Markov Chains: How much steps to conclude a Transition Matrix

I have just learned Markov Chains which I am using to model a real world problem. The model comprises 3 states [a b c]. For now I am collection data and calculating ...