9
votes
Can Reinforcement Learning learn to be deceptive?
There is definitely a lot of work to do on the NLP and knowledgebase side of things before you can realise your agent. However, as the question suggests, we can ignore those details and focus on: Can ...
- 29.1k
8
votes
Python library to implement Hidden Markov Models
As an update on this question, I believe the accepted answer is not the best as of 2017.
As suggested in comments by Kyle, hmmlearn is currently the library to go ...
- 456
7
votes
Accepted
What is the relationship between MDP and RL?
What is the relationship between Markov Decision Processes and Reinforcement Learning?
In Reinforcement Learning (RL), the problem to resolve is described as a Markov Decision Process (MDP). ...
- 29.1k
6
votes
Python library to implement Hidden Markov Models
pomegranate library has support for HMM and the documentation is really helpful. After trying with many hmm libraries in python, I find this to be quite good.
4
votes
Accepted
Equations in "Intoduction to RL": What is the meaning and difference between E, and E with subscript?
In general, the expectation is taken with respect to some random variable X. Often, when dealing with a single random variable, it can be implicitly inferred over which random variable it is being ...
- 156
4
votes
What are the differences between Reinforcement Learning (RL) and Supervised Learning?
What are difference between Reinforcement Learning (RL) and Supervised Learning?
The main difference is to do with how "correct" or optimal results are learned:
In Supervised Learning, the learning ...
- 29.1k
3
votes
Accepted
Reinforcement Learning - How are these state values in MRP calculated?
As it is such a small MRP, it is possible to solve it quickly and analytically using simultaneous equations based on the Bellman equation:
$$v(s) = \sum_{r,s'} p(r,s'|s)(r + v(s'))$$
and ...
- 29.1k
3
votes
Accepted
Reward dependent on (state, action) versus (state, action, successor state)
Your intuition is correct. In the most general case (Sutton's definitions), the model of the environment consists of the state transition distribution and the reward distribution. The latter one is ...
- 2,121
3
votes
Markov Chains for sequential data
Because there is very little data HMM will probably overfit (depends on the number of states and letters). I would go with a simple markov chain as it has less parameters and you dont need to tune ...
- 149
3
votes
Accepted
Markov Chains for sequential data
If you know what the state history is, you don't need a 'hidden' Markov model, you just need a Markov model (or some other mechanism). The 'hidden' part implies a distinction between some sequence of ...
- 2,248
3
votes
Can Reinforcement Learning learn to be deceptive?
What is a great deception? It could be defined as a believable set of information aiming to a final deceitful objective.
Just like any RL model, you can maximize a score thanks to small rewards ...
- 4,814
2
votes
Accepted
How can I rank paths through an HMM?
Use log-probabilities, then use k-shortest paths.
The probability of a path is the product of the probabilities on its edges. If you log-transform everything (so that each edge is annotated with the ...
- 3,401
2
votes
Accepted
Using dhmm_em to form the hmm of mfccs' from song clips
Edit: It turns out that the above hmm can only be applied to discrete
values which is possibly why I was getting errors. It seems I have to
use another function for 'gaussian output' instead, ...
- 456
2
votes
Accepted
finding themes from text documents
The best method to find themes in a collection of documents is topic modeling. Topic modeling finds the hidden (aka, latent) themes beyond just keyword counts.
There are many approaches to topic ...
- 22.1k
2
votes
Accepted
Should reinforcement learning always assume (PO)MDP?
How does reinforcement learning algorithms work without the assumption of (PO)MDPs?
It doesn't. The theory of reinforcement learning is tied very strongly to an underlying MDP framework. The RNN-...
- 29.1k
2
votes
Accepted
Evaluating value functions in RL
In a discrete probability space, the expectation of a random variable (RV) is a sum over all possible values multiplied by their individual probabilities. Here, your RV is $q_\pi(s,a)$, with $s$ being ...
- 156
2
votes
Predict how many days late or early someone will finish their work
If we assume that each task delivery is independent of eachother, and the process does not change a lot over time (stationary), we can treat this as a standard regression problem.
Since this is about ...
- 1,527
2
votes
Accepted
Q-learning when minimising a total cost instead of maximising a total reward
I have a decision problem where the results are measured as a cost that I want to minimise. It seems like a good fit to Q-learning, but I am not sure how to adjust it to deal with a cost instead of a ...
- 29.1k
2
votes
Accepted
Difference between $Q(s,a)$ ,$V^*(s)$ and $V^\pi(s)$ in Markov Decision Process?
Your confusion seems to come from mixing up between some policy $\pi$ and an optimal policy $\pi^*$. Your summary is generally correct, but missing these extra details.
Let me try go through it again. ...
- 171
1
vote
Reinforcement Learning control with known dynamic equation
Reinforcement learning (RL) is completely based around MDPs, to the point where its definition is essentially "RL is a collection of algorithms that can learn about action choices within a MDP ...
- 29.1k
1
vote
Artificially increasing frequency weight of word ending characters in word building
One was to evaluate the code is to run thousands of simulations and look the histogram of word length frequency.
Then parameterize your code so long word bias can be changed up and down. Rerun the ...
- 22.1k
1
vote
Accepted
MDP - RL, Multiple rewards for the same state possible?
it is possible that performing an action $a$ that takes us to state $s′$, could result in multiple rewards?
Yes, that is true the general case that any $(s,a)$ pair can result in a range of results ...
- 29.1k
1
vote
Accepted
Using Policy Iteration on an automaton
Policy Iteration is essentially a two step process:
Evaluate the current policy by calculating $v(s)$ for every non-terminal state.
Internally this requires multiple loops over all the states until ...
- 29.1k
1
vote
What could be some Classification techniques to classify a tree of webpages given the category of each webpage
There are of course a number of ways this can be done, such as majority voting or some other rule-based algorithm, however it can also be done through supervised learning since you have some labels ...
- 8,986
1
vote
What is the optimal value of a Markov Decision process with Single actions at each state?
Hi guys thanks for your comments and sorry for the slow reply. I cannot reply to them directly because I am not allowed.
@Constantinos, to your point regarding b: b-->b, b-->a being different ...
- 11
1
vote
Simple Markov Chains Memoryless Property Question
I am not sure what you actually want to do, but if you want to simulate the Markov chain, than you really have to bring probabilities into play.
Thus, in each step, you would use the transition ...
- 66
1
vote
Markov Chains for sequential data
1) You can use HMM , SSM or UCM if u have assumption that transition is happening from some hidden state. Considering 20 data points, I wonder how model fitting will happen.
2) markov chain would ...
- 425
1
vote
Accepted
Viterbi-like algorithm suggesting top-N probable state sequences implementation
Apparently, I misunderstood your question.
There are several methods for finding the k-best paths with extending versions of the Viterbi algorithm.
My first advice would be to look at this question ...
- 456
1
vote
Visualization of multiple Markov models
If we limit the question on comparing two graphs, I can propose a way based on adjacency matrices comparison. There is a sample notebook: graph_diff.ipynb
To summarize:
Having two graphs,
...
- 164
1
vote
How to estimate the transition probabilities for a Markov Chain when time intervals are non-equally spaced
With the limited information that you have given, it is not possible to judge the trade-off between different approaches of handling time here. I can propose the following alternatives.
Check if you ...
- 2,018
Only top scored, non community-wiki answers of a minimum length are eligible