Skip to main content
Share Your Experience: Take the 2024 Developer Survey
8 votes

Which tribe does Probabilistic Graphical Models fall under?

Probabilistic Graphical Models (PGMs) are: Connectionist: RBMs are PGMs and neural networks (source) Bayesian: Bayes Networks are bayesian (Wikipedia article) Symbolist: Markov Logic Networks (source)...
Martin Thoma's user avatar
  • 18.9k
7 votes
Accepted

What makes a Tree-Structured Parzen Estimator "tree-structured?"

It means that your hyperparameter space is tree-like: the value chosen for one hyperparameter determines what hyperparameter will be chosen next and what values are available for it. From a HyperOpt ...
Ben Reiniger's user avatar
  • 11.9k
7 votes
Accepted

Changing the batch size during training

Efficient use of resources It is a balancing game with the learning rate, and one reason you don't normally see people do this is that you want to utilise as much of the GPU as possible. It is ...
n1k31t4's user avatar
  • 14.9k
5 votes

What is the meaning of likelihood?

All of the answers here, including the accepted one, are conspicuously confused. I down-voted the accepted answer but downvotes of users who lack reputation in this "community" are not counted. I have ...
Michael Hardy's user avatar
5 votes

When to use bayesian linear regression instead of linear regression?

The Bayesian approach should be used in the case of: Strong priors - You have preexisting data and / or domain knowledge that you want to incorporate into the analysis. Distributional estimates - ...
Brian Spiering's user avatar
4 votes

Effect of outliers on Naive Bayes

There are different flavors of Naive Bayes, so the answer depends a bit on the use case. One potential issue with outliers is that unseen observations can lead to 0 probabilities. For example, ...
oW_'s user avatar
  • 6,377
4 votes

generalized likelihood ratio test (GLRT)

Likelihood-ratio tests are a mainstay of classical hypothesis testing. The idea is to form the likelihoods of the two hypotheses under consideration, and choose the one with the highest likelihood if ...
Emre's user avatar
  • 10.5k
4 votes

When to use BayesianSearchCV and how it works?

As mentioned in that Kaggle notebook, you can use it pretty much as just a drop-in replacement for other search methods (grid or random). Bayesian searches still are random searches over a predefined ...
Ben Reiniger's user avatar
  • 11.9k
4 votes

why naive is needed in Naive Bayes ,what happens if naive is not included in Bayes theorem?

I do not think your formulation is correct. What you have described are just conditional distributions for each word in the sentence but not the joint conditional distribution, given a specific class. ...
aranglol's user avatar
  • 2,196
4 votes
Accepted

Opinions on an LSTM hyper-parameter tuning process I am using

First of all you might want to know there is a "new" Keras tuner, which includes BayesianOptimization, so building an LSTM with keras and optimizing its hyperparams is completely a plug-in task with ...
German C M's user avatar
  • 2,724
4 votes
Accepted

What is the difference between maximum likelihood hypothesis and maximum a posteriori hypothesis?

I'll try and provide some intuition for you here, instead of focusing on the mechanics of the math behind the methods. Imagine you are evaluating whether a coin is fair or not, so you collect a ...
Brandon Donehoo's user avatar
3 votes
Accepted

Is deep learning a must in a Data Science MSc programme?

No, it's not problematic. Most data scientists do not need or use deep learning. Deep learning is very popular right now, but that does not mean it's widely used. Deep learning can lead to substantial ...
Ryan Zotti's user avatar
  • 4,149
3 votes

Creating a posterior distribution for classic coin flipping in python using grid search

In short The problem assumes a uniform prior distribution function. All possible $P(H)$ are equally likely. Because they are standardizing the probability distribution function at the end it does not ...
JahKnows's user avatar
  • 8,916
3 votes
Accepted

Replacing missing value by class conditional mean

Your intuition about 'no effect' is true in some sense. But this replacement may be not the best use of the information you have. The choice of missing value treatment depends on your initial ...
David Dale's user avatar
  • 1,551
3 votes
Accepted

Problem understanding probabilistic generative models for classification

My understanding is: The first line is OK - it derives from Bayes Rule Assume that this probability follows a logistic function, that is that $$P(C_1|x) = \frac{1}{1+exp(-a)}$$ Then if $$ y = \...
Jacques Wainer's user avatar
2 votes

Which tribe does Probabilistic Graphical Models fall under?

Based on this paper (Qi, Szummer, and Minka. 2005. Bayesian Conditional Random Fields): In this paper, we propose Bayesian Conditional Random Fields (BCRF), a novel Bayesian approach to training ...
justhalf's user avatar
  • 123
2 votes

Which tribe does Probabilistic Graphical Models fall under?

Only Domingos knows for sure, since he invented this taxonomy, but I'd guess it would fall under "connectionists" (which he associates with neural networks), since graphs are all about connections (...
Emre's user avatar
  • 10.5k
2 votes

Ensemble learning

Ensemble learning is categorized into 4 different classes: bagging, boosting, stacking, hierarchy classification and sometimes they consider grading as another category. Each one of these categories ...
Bashar Haddad's user avatar
2 votes
Accepted

Where can we find the application of bayes's theorem in Bayesian optimiation with gaussian processing

It is a 49 page long paper, so following observations are based only on a cursory reading. The optimisation is for finding best value of parameters for cost function of machine learning models. ...
hssay's user avatar
  • 1,998
2 votes
Accepted

Probability of event given two depandant events

Let $A$, $B$, $C$ be three (potentially dependent) events. By the probability chain rule: $$ P(A,B,C)=P(A|B,C)P(B,C)=P(A|B,C)P(B|C)P(C) $$ So we can do \begin{align} P(A|B,C) &= \frac{ P(A,B,C)}{...
user3658307's user avatar
  • 1,020
2 votes
Accepted

How do Bayesian methods do automatic feature selection?

Your interpretation of L1 regularization sounds right. Is it used to perform feature selection? Yes, in the broad sense that this 'encourages' coefficients in the linear model to be 0, and those ...
Sean Owen's user avatar
  • 6,595
2 votes

What is the meaning of likelihood?

Likelihood and probability are two very different concepts: One talk about probabilities when the distribution is already known and one want to know how probable an event is. Likelihood on the ...
Jeanba's user avatar
  • 239
2 votes
Accepted

Bayesian regularization vs dropout for basic ann

It actually makes perfect sense to use both. Gal et al. provided a nice theory on how to interpret dropout through a Bayesian lense. In a nutshell, if you use dropout + regularization you are ...
matthiaw91's user avatar
  • 1,545
2 votes
Accepted

Mean estimation for nested location data

I don't know if that is the case, but if some kind of continuity assumptions are realistic, you could try to move away from categorical variables (block) to continuous variables (longitude and ...
NiklasvMoers's user avatar
2 votes

Mean estimation for nested location data

One option is to move to a more rigorous geographic information system (GIS) data structure. For example, both plus codes and H3 are designed for nested location data. If your data is reformated to ...
Brian Spiering's user avatar
2 votes
Accepted

Why do machine learning engineers insist on training with more data than validation set?

The reasoning will be: "The more data for training the better". Then you have to keep in mind that the validation/hold-out set has to resemble how it should work on production/testing. The ...
Carlos Mougan's user avatar
2 votes
Accepted

What is the num_initial_points argument for Bayesian Optimization with Keras Tuner?

The Bayesian optimization algorithm selects points to test based on a balance between exploring uncertain regions and exploiting high-performing regions. But before you've tested very many points, ...
Ben Reiniger's user avatar
  • 11.9k
2 votes

Does the Bayesian MAP give a probability distribution over unseen data?

Bayesian MAP gives a point estimate $\mathbf{w}_{MAP}$ for model parameter $\mathbf{w}$, which implies $p(\mathbf{w}|\mathbf{t},\cdots) = \left\{\begin{matrix} 1 & \mathbf{w}=\mathbf{w}_{MAP}\\ 0 ...
Esmailian's user avatar
  • 9,322
2 votes
Accepted

How useful is Bayesian Inference

This may be an unpopular opinion to some, but in my experience Bayesian statistics is not particularly useful in data science in industry, for a couple of reasons: A Bayesian approach is very useful ...
Robert Long's user avatar
2 votes

How useful is Bayesian Inference

I think these real-world industrial applications of Bayesian analysis might be helpful to you: "Bayesian Product Ranking at Wayfair", David J. Harris, Wayfair blog, Jan. 20, 2020 "...
Michael T.'s user avatar

Only top scored, non community-wiki answers of a minimum length are eligible