# Tag Info

### Can Reinforcement Learning learn to be deceptive?

There is definitely a lot of work to do on the NLP and knowledgebase side of things before you can realise your agent. However, as the question suggests, we can ignore those details and focus on: Can ...
• 29.1k

### Python library to implement Hidden Markov Models

As an update on this question, I believe the accepted answer is not the best as of 2017. As suggested in comments by Kyle, hmmlearn is currently the library to go ...
• 456
Accepted

### What is the relationship between MDP and RL?

What is the relationship between Markov Decision Processes and Reinforcement Learning? In Reinforcement Learning (RL), the problem to resolve is described as a Markov Decision Process (MDP). ...
• 29.1k

### Python library to implement Hidden Markov Models

pomegranate library has support for HMM and the documentation is really helpful. After trying with many hmm libraries in python, I find this to be quite good.
Accepted

### Equations in "Intoduction to RL": What is the meaning and difference between E, and E with subscript?

In general, the expectation is taken with respect to some random variable X. Often, when dealing with a single random variable, it can be implicitly inferred over which random variable it is being ...

### What are the differences between Reinforcement Learning (RL) and Supervised Learning?

What are difference between Reinforcement Learning (RL) and Supervised Learning? The main difference is to do with how "correct" or optimal results are learned: In Supervised Learning, the learning ...
• 29.1k
Accepted

### Reinforcement Learning - How are these state values in MRP calculated?

As it is such a small MRP, it is possible to solve it quickly and analytically using simultaneous equations based on the Bellman equation: $$v(s) = \sum_{r,s'} p(r,s'|s)(r + v(s'))$$ and ...
• 29.1k
Accepted

### Reward dependent on (state, action) versus (state, action, successor state)

Your intuition is correct. In the most general case (Sutton's definitions), the model of the environment consists of the state transition distribution and the reward distribution. The latter one is ...
• 2,121

### Markov Chains for sequential data

Because there is very little data HMM will probably overfit (depends on the number of states and letters). I would go with a simple markov chain as it has less parameters and you dont need to tune ...
• 149
Accepted

### Markov Chains for sequential data

If you know what the state history is, you don't need a 'hidden' Markov model, you just need a Markov model (or some other mechanism). The 'hidden' part implies a distinction between some sequence of ...
• 2,248

### Can Reinforcement Learning learn to be deceptive?

What is a great deception? It could be defined as a believable set of information aiming to a final deceitful objective. Just like any RL model, you can maximize a score thanks to small rewards ...
• 4,814
Accepted

### How can I rank paths through an HMM?

Use log-probabilities, then use k-shortest paths. The probability of a path is the product of the probabilities on its edges. If you log-transform everything (so that each edge is annotated with the ...
• 3,401
Accepted

### Using dhmm_em to form the hmm of mfccs' from song clips

Edit: It turns out that the above hmm can only be applied to discrete values which is possibly why I was getting errors. It seems I have to use another function for 'gaussian output' instead, ...
• 456
Accepted

### finding themes from text documents

The best method to find themes in a collection of documents is topic modeling. Topic modeling finds the hidden (aka, latent) themes beyond just keyword counts. There are many approaches to topic ...
• 22.1k
Accepted

### Should reinforcement learning always assume (PO)MDP?

How does reinforcement learning algorithms work without the assumption of (PO)MDPs? It doesn't. The theory of reinforcement learning is tied very strongly to an underlying MDP framework. The RNN-...
• 29.1k
Accepted

### Evaluating value functions in RL

In a discrete probability space, the expectation of a random variable (RV) is a sum over all possible values multiplied by their individual probabilities. Here, your RV is $q_\pi(s,a)$, with $s$ being ...

### Predict how many days late or early someone will finish their work

If we assume that each task delivery is independent of eachother, and the process does not change a lot over time (stationary), we can treat this as a standard regression problem. Since this is about ...
• 1,527
Accepted

### Q-learning when minimising a total cost instead of maximising a total reward

I have a decision problem where the results are measured as a cost that I want to minimise. It seems like a good fit to Q-learning, but I am not sure how to adjust it to deal with a cost instead of a ...
• 29.1k
Accepted

### Difference between $Q(s,a)$ ,$V^*(s)$ and $V^\pi(s)$ in Markov Decision Process?

Your confusion seems to come from mixing up between some policy $\pi$ and an optimal policy $\pi^*$. Your summary is generally correct, but missing these extra details. Let me try go through it again. ...
• 171
1 vote

### Reinforcement Learning control with known dynamic equation

Reinforcement learning (RL) is completely based around MDPs, to the point where its definition is essentially "RL is a collection of algorithms that can learn about action choices within a MDP ...
• 29.1k
1 vote

### Artificially increasing frequency weight of word ending characters in word building

One was to evaluate the code is to run thousands of simulations and look the histogram of word length frequency. Then parameterize your code so long word bias can be changed up and down. Rerun the ...
• 22.1k
1 vote
Accepted

### MDP - RL, Multiple rewards for the same state possible?

it is possible that performing an action $a$ that takes us to state $s′$, could result in multiple rewards? Yes, that is true the general case that any $(s,a)$ pair can result in a range of results ...
• 29.1k
1 vote
Accepted

### Using Policy Iteration on an automaton

Policy Iteration is essentially a two step process: Evaluate the current policy by calculating $v(s)$ for every non-terminal state. Internally this requires multiple loops over all the states until ...
• 29.1k
1 vote

### What could be some Classification techniques to classify a tree of webpages given the category of each webpage

There are of course a number of ways this can be done, such as majority voting or some other rule-based algorithm, however it can also be done through supervised learning since you have some labels ...
• 8,986
1 vote

### What is the optimal value of a Markov Decision process with Single actions at each state?

Hi guys thanks for your comments and sorry for the slow reply. I cannot reply to them directly because I am not allowed. @Constantinos, to your point regarding b: b-->b, b-->a being different ...
1 vote

### Simple Markov Chains Memoryless Property Question

I am not sure what you actually want to do, but if you want to simulate the Markov chain, than you really have to bring probabilities into play. Thus, in each step, you would use the transition ...
1 vote

### Markov Chains for sequential data

1) You can use HMM , SSM or UCM if u have assumption that transition is happening from some hidden state. Considering 20 data points, I wonder how model fitting will happen. 2) markov chain would ...
1 vote
Accepted

### Viterbi-like algorithm suggesting top-N probable state sequences implementation

Apparently, I misunderstood your question. There are several methods for finding the k-best paths with extending versions of the Viterbi algorithm. My first advice would be to look at this question ...
• 456
1 vote

### Visualization of multiple Markov models

If we limit the question on comparing two graphs, I can propose a way based on adjacency matrices comparison. There is a sample notebook: graph_diff.ipynb To summarize: Having two graphs, ...
• 164
1 vote

### How to estimate the transition probabilities for a Markov Chain when time intervals are non-equally spaced

With the limited information that you have given, it is not possible to judge the trade-off between different approaches of handling time here. I can propose the following alternatives. Check if you ...
• 2,018

Only top scored, non community-wiki answers of a minimum length are eligible