Skip to main content
39 votes
Accepted

What is Monte Carlo dropout?

Let's start with normal dropout, i.e. dropout only at training time. Here, dropout serves as a regularization to avoid overfitting. During test time, dropout is not applied; instead, all nodes/...
user12075's user avatar
  • 2,264
6 votes
Accepted

Estimating the value of $\pi$ with a Monte Carlo dartboard: $<$ or $\leq$?

Short answer: Both formulations lead to the same answer. Mathematical explanation: In order to understand that let us look at two similar problems. Imagine we want to integrate a function $f(x)=x^2$ ...
MachineLearner's user avatar
2 votes

What is the intuition behind using Monte Carlo to solve a differential equation

The trick is to convert ODE/PDE into an integral equation and then Monte Carlo comes to play. Here are some examples. http://jotterbach.github.io/2018/08/08/MonteCarloODE/
Tatyana's user avatar
  • 61
2 votes

How to handle differences between training and deploying of an RL agent

If I train an agent for taking actions for 15 mins during the training process, is it ok I make my agent take actions at every 5 min interval during deployment? It is impossible to say in general. ...
Neil Slater's user avatar
  • 28.9k
2 votes
Accepted

Different results every time I train a reinforcement learning agent

A part of the agent consists of taking random actions. So there is a % chance that the agent will take a random action instead of an action based on the training. This is called "exploration". This ...
user1939088's user avatar
2 votes

Evaluating a trained Reinforcement Learning Agent?

If your goal is optimal control, then you will want to measure the agent by how well it does at the task. You should use some aggregate measure of reward, such as total reward per episode (aka "return"...
Neil Slater's user avatar
  • 28.9k
2 votes
Accepted

What visualization I should choose for Monte Carlo simulations in timeline events?

Instead of (or additionally to) histogram you may check the density plot of the distribution of the intervals between the events. In the r snipped below the ...
Marmite Bomber's user avatar
1 vote
Accepted

Hamiltonian Monte Carlo - Is it generally a good method for obtaining random variables in Machine Learning?

Hamiltonian Monte Carlo (HMC) comes up mostly in Bayesian statistics. It belongs to a general class of sampling algorithms called Markov Chain Monte Carlo (MCMC). I think there is some confusion about ...
aranglol's user avatar
  • 2,196
1 vote

CDF/PDF vs Monte Carlo

You are interested in the distribution of $D := \theta_{A} - \theta_{B} | W$, where $\theta_{A}$ and $\theta_{B}$ are parameters from your model representing the posterior click-through rates under ...
aranglol's user avatar
  • 2,196
1 vote

How can I deal with a computationally expensive simulator method in Sequential Monte Carlo/Approximate Bayesian Computation?

Maybe you could try variational inference. Specifically Variational Sequential Monte Carlo. This would give you a much lighter computation, as it does not rely on sampling. The authors of Variational ...
noe's user avatar
  • 26.7k
1 vote
Accepted

Quantifying the performance of Stepwise Regression ran on Monte Carlo generated datasets & comparing them to your method of interest

The first step is to quantify the total number of 'Positives', i.e., the total number of structural factors explaining each dataset, and this is straight-forward since you already have that number ...
Marlen's user avatar
  • 167
1 vote

Which Model for predicting flight delays is appropriate except Random Forest and Decision Tree? (Monte Carlo?)

Weather is responsible for 90% of the flight delays. How is it possible to make reliable predictions with just 10% of the remaining causes? (if their data is available) You have an existing map called ...
Nicolas Martin's user avatar
1 vote

What is the most common practice of generating (X,Y) from an arbitrary CDF or PDF?

I am no expert in the field but I can imagine 3 different cases : Variables are independent in which case you can use your formula on each You have access to some chain rules decomposition that ...
mprouveur's user avatar
  • 338
1 vote

Having a reward structure which gives high positive rewards compared to the negative rewards

Is my understanding correct and is it recommended to have such a reward structure for my use case ? Your understanding is not correct, and setting extremely high rewards for the goal state in this ...
Neil Slater's user avatar
  • 28.9k
1 vote
Accepted

How to formulate reward of an rl agent with two objectives

In general it is not possible to simultaneously optimise two separate objective functions. Your approach of adding weights (your coefficients) to each objective, then summing the scaled objectives, is ...
Neil Slater's user avatar
  • 28.9k
1 vote

MCMC for finding Bayesian Neural Network

There are in fact a bunch of papers on this topic. I'd recommend this one and this one. Not sure what you want to do with your Bayesian Neural Network, but maybe this one is useful too.
matthiaw91's user avatar
  • 1,545
1 vote
Accepted

Evaluating value functions in RL

In a discrete probability space, the expectation of a random variable (RV) is a sum over all possible values multiplied by their individual probabilities. Here, your RV is $q_\pi(s,a)$, with $s$ being ...
Andrei Poehlmann's user avatar
1 vote

How is Importance-Sampling Used in Off-Policy Monte Carlo Prediction?

After some head-scratching, I think I was able to make sense of the "transformation" that the author uses in equation 5.4 to yield the correct expectation for $v_{\pi}(s)$. I've introduced some ...
Jack's user avatar
  • 43

Only top scored, non community-wiki answers of a minimum length are eligible