8
votes
Which tribe does Probabilistic Graphical Models fall under?
Probabilistic Graphical Models (PGMs) are:
Connectionist: RBMs are PGMs and neural networks (source)
Bayesian: Bayes Networks are bayesian (Wikipedia article)
Symbolist: Markov Logic Networks (source)...
- 18.8k
7
votes
Accepted
What makes a Tree-Structured Parzen Estimator "tree-structured?"
It means that your hyperparameter space is tree-like: the value chosen for one hyperparameter determines what hyperparameter will be chosen next and what values are available for it.
From a HyperOpt ...
- 11.2k
7
votes
Accepted
Changing the batch size during training
Efficient use of resources
It is a balancing game with the learning rate, and one reason you don't normally see people do this is that you want to utilise as much of the GPU as possible.
It is ...
- 14.7k
5
votes
What is the meaning of likelihood?
All of the answers here, including the accepted one, are conspicuously confused. I down-voted the accepted answer but downvotes of users who lack reputation in this "community" are not counted. I have ...
- 153
5
votes
When to use bayesian linear regression instead of linear regression?
The Bayesian approach should be used in the case of:
Strong priors - You have preexisting data and / or domain knowledge that you want to incorporate into the analysis.
Distributional estimates - ...
- 20.5k
4
votes
Effect of outliers on Naive Bayes
There are different flavors of Naive Bayes, so the answer depends a bit on the use case.
One potential issue with outliers is that unseen observations can lead to 0 probabilities. For example, ...
oW_♦
- 6,264
4
votes
generalized likelihood ratio test (GLRT)
Likelihood-ratio tests are a mainstay of classical hypothesis testing. The idea is to form the likelihoods of the two hypotheses under consideration, and choose the one with the highest likelihood if ...
- 10.5k
4
votes
When to use BayesianSearchCV and how it works?
As mentioned in that Kaggle notebook, you can use it pretty much as just a drop-in replacement for other search methods (grid or random). Bayesian searches still are random searches over a predefined ...
- 11.2k
4
votes
why naive is needed in Naive Bayes ,what happens if naive is not included in Bayes theorem?
I do not think your formulation is correct. What you have described are just conditional distributions for each word in the sentence but not the joint conditional distribution, given a specific class.
...
- 1,954
4
votes
Accepted
Opinions on an LSTM hyper-parameter tuning process I am using
First of all you might want to know there is a "new" Keras tuner, which includes BayesianOptimization, so building an LSTM with keras and optimizing its hyperparams is completely a plug-in task with ...
- 2,696
4
votes
Accepted
What is the difference between maximum likelihood hypothesis and maximum a posteriori hypothesis?
I'll try and provide some intuition for you here, instead of focusing on the mechanics of the math behind the methods.
Imagine you are evaluating whether a coin is fair or not, so you collect a ...
- 376
3
votes
Accepted
Is deep learning a must in a Data Science MSc programme?
No, it's not problematic. Most data scientists do not need or use deep learning. Deep learning is very popular right now, but that does not mean it's widely used. Deep learning can lead to substantial ...
- 4,139
3
votes
Creating a posterior distribution for classic coin flipping in python using grid search
In short
The problem assumes a uniform prior distribution function. All possible $P(H)$ are equally likely. Because they are standardizing the probability distribution function at the end it does not ...
- 8,776
3
votes
Accepted
Replacing missing value by class conditional mean
Your intuition about 'no effect' is true in some sense. But this replacement may be not the best use of the information you have.
The choice of missing value treatment depends on your initial ...
- 1,541
3
votes
Accepted
Problem understanding probabilistic generative models for classification
My understanding is:
The first line is OK - it derives from Bayes Rule
Assume that this probability follows a logistic function, that is that
$$P(C_1|x) = \frac{1}{1+exp(-a)}$$
Then if
$$ y = \...
- 251
2
votes
Which tribe does Probabilistic Graphical Models fall under?
Based on this paper (Qi, Szummer, and Minka. 2005. Bayesian Conditional Random Fields):
In this paper, we propose Bayesian Conditional Random Fields (BCRF), a novel Bayesian approach to training ...
- 123
2
votes
Which tribe does Probabilistic Graphical Models fall under?
Only Domingos knows for sure, since he invented this taxonomy, but I'd guess it would fall under "connectionists" (which he associates with neural networks), since graphs are all about connections (...
- 10.5k
2
votes
Ensemble learning
Ensemble learning is categorized into 4 different classes: bagging, boosting, stacking, hierarchy classification and sometimes they consider grading as another category. Each one of these categories ...
- 1,969
2
votes
Accepted
Where can we find the application of bayes's theorem in Bayesian optimiation with gaussian processing
It is a 49 page long paper, so following observations are based only on a cursory reading.
The optimisation is for finding best value of parameters for cost function of machine learning models.
...
- 1,988
2
votes
Accepted
Probability of event given two depandant events
Let $A$, $B$, $C$ be three (potentially dependent) events. By the probability chain rule:
$$ P(A,B,C)=P(A|B,C)P(B,C)=P(A|B,C)P(B|C)P(C) $$
So we can do
\begin{align}
P(A|B,C)
&= \frac{ P(A,B,C)}{...
- 1,010
2
votes
Accepted
How do Bayesian methods do automatic feature selection?
Your interpretation of L1 regularization sounds right. Is it used to perform feature selection? Yes, in the broad sense that this 'encourages' coefficients in the linear model to be 0, and those ...
- 6,595
2
votes
What is the meaning of likelihood?
Likelihood and probability are two very different concepts:
One talk about probabilities when the distribution is already known and one want to know how probable an event is.
Likelihood on the ...
- 239
2
votes
Accepted
Bayesian regularization vs dropout for basic ann
It actually makes perfect sense to use both. Gal et al. provided a nice theory on how to interpret dropout through a Bayesian lense. In a nutshell, if you use dropout + regularization you are ...
- 1,515
2
votes
Accepted
Mean estimation for nested location data
I don't know if that is the case, but if some kind of continuity assumptions are realistic, you could try to move away from categorical variables (block) to continuous variables (longitude and ...
- 309
2
votes
Mean estimation for nested location data
One option is to move to a more rigorous geographic information system (GIS) data structure.
For example, both plus codes and H3 are designed for nested location data. If your data is reformated to ...
- 20.5k
2
votes
Accepted
Why do machine learning engineers insist on training with more data than validation set?
The reasoning will be: "The more data for training the better". Then you have to keep in mind that the validation/hold-out set has to resemble how it should work on production/testing. The ...
- 6,082
2
votes
Accepted
What is the num_initial_points argument for Bayesian Optimization with Keras Tuner?
The Bayesian optimization algorithm selects points to test based on a balance between exploring uncertain regions and exploiting high-performing regions. But before you've tested very many points, ...
- 11.2k
2
votes
Does the Bayesian MAP give a probability distribution over unseen data?
Bayesian MAP gives a point estimate $\mathbf{w}_{MAP}$ for model parameter $\mathbf{w}$, which implies
$p(\mathbf{w}|\mathbf{t},\cdots) = \left\{\begin{matrix}
1 & \mathbf{w}=\mathbf{w}_{MAP}\\
0 ...
- 9,247
2
votes
Accepted
How useful is Bayesian Inference
This may be an unpopular opinion to some, but in my experience Bayesian statistics is not particularly useful in data science in industry, for a couple of reasons:
A Bayesian approach is very useful ...
- 601
2
votes
How useful is Bayesian Inference
I think these real-world industrial applications of Bayesian analysis might be helpful to you:
"Bayesian Product Ranking at Wayfair", David J. Harris, Wayfair blog, Jan. 20, 2020
"...
- 21
Only top scored, non community-wiki answers of a minimum length are eligible