Questions tagged [bayesian]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
9
votes
2answers
4k views

What makes a Tree-Structured Parzen Estimator “tree-structured?”

From what I understand the Tree-Structured Parzen Estimator (TPE) creates two probability models based on hyperparameters that exceed the performance of some threshold and hyperparameters that don't. ...
6
votes
3answers
630 views

Which tribe does Probabilistic Graphical Models fall under?

Pedro Domingos in "The Master Algorithm" listed five tribes of machine learning algorithms: Symbolists Connectionists Evolutionaries Bayesians Analogizers Which category do probabilistic ...
5
votes
2answers
1k views

Effect of outliers on Naive Bayes

Are Naive Bayes algorithms affected by outliers in the data? Suppose there is a data set, does one need to remove outliers before applying Naive Bayes?
5
votes
2answers
112 views

Bayesian optimisation in deeplearning

Has anyone tried using Bayesian optimisation to get best learning rates, and other hyperparameters for deeplearning. How to change the parameters between the training. Any examples on callbacks? Can ...
5
votes
1answer
1k views

Modeling uncertainty from Logistic Regression

Logistic regression is a part in a simulation pipeline that I use for some scenario analysis. The dataset that this is based on is not small but relatively noisy, and only one explanatory variable/...
4
votes
2answers
155 views

Mean estimation for nested location data

I want to estimate the average income for a location. I have nested data in the following way: A block is inside a neighborhood, which is inside a zipcode, which is inside a district, which is inside ...
4
votes
1answer
43 views

Bayesian regularization vs dropout for basic ann

Does it make sense conceptually to apply dropout to an artificial neutral network while also applying bayesian regularization? On one hand I would think that technically this should work just fine, ...
4
votes
1answer
300 views

How to best estimate the coefficients of a confusion matrix in case of strong class imbalance?

I have a trained binary classifier (forget about how this was trained and think of it as a magical black box) and I would like to measure its classification performance (e.g. compute a confusion ...
4
votes
1answer
1k views

Why the estimated Lasso coefficients of almost all variables are equal to zero?

I would greatly appreciate if you could guide me. In fact, I used "Bayesian Optimization " to tune hyper-parameters of Lasso but the estimated Lasso coefficients of almost all variables are equal to ...
3
votes
1answer
77 views

When to use bayesian linear regression instead of linear regression?

When does it make sense to use a bayesian approach, maybe in context to linear regression? To be more concrete: Assume you measure a certain number of devices and you wanna' check the linear ...
3
votes
2answers
2k views

generalized likelihood ratio test (GLRT)

I am having trouble in understanding the generalized likelihood ratio test (GLRT). Can anyone explain what it is to me, or point me toward an easy-to-understand reference? Is it a supervised or ...
3
votes
1answer
56 views

Why do machine learning engineers insist on training with more data than validation set?

Among my colleagues I have noticed a curious insistence on training with, say, 70% or 80% of data and validating on the remainder. The reason it is curious to me is the lack of any theoretical ...
3
votes
3answers
305 views

What is the meaning of likelihood?

I am studying Bayes probability applied to machine learning, and I have encoutered the concept of likelihood, which I don't understand. I have seen that the Bayes rule is: $P(A|B)=\frac{P(B|A)P(A)}{...
3
votes
1answer
910 views

LSTM: Converting to Bayesian Deep Neural Network

Starting from Yarin Gal's research paper on using Dropout as a Bayesian Approximation (https://arxiv.org/pdf/1506.02142.pdf), I am trying to apply this concept to my Sequence Prediction model. My ...
3
votes
1answer
71 views

How is bayesian risk computed to prune decision trees?

I've been trying to follow this paper on Bayesian Risk Pruning. I'm not very familiar with this type of pruning, but I'm wondering a few things: (1) The paper describes risk-rates to be defined per ...
3
votes
0answers
31 views

If I use Gibbs sampling with a Bayesian model, what do I have to check is memoryless?

Right now I am trying to better understand how Bayesian modeling works with just the basics. I found through reading tutorials that some very basic Bayesian models like Bayesian Hierarchical Modeling ...
3
votes
2answers
395 views

Is k-means with Mahalanobis a valid option for clustering?

I want more info into if k-means with Mahalanobis distance is a mathematically/methodologically correct option for datasets with different variance clusters. The steps are: Create aggregate datasets (...
3
votes
1answer
81 views

How to update the posterior belief when we are observing a stream of correlated data from a fixed but unknown data source

I want to build a [probabilistic] model that aims to infer the true value of an unknown categorical variable, $y \in \{1,2,..., K\}$. We have a dataset $(X,y): \mathbb{R}^d\rightarrow \{1,2,..., K\}$ ...
3
votes
0answers
473 views

Implement gaussian mixture model with stochastic variational inference

I am trying to implement Gaussian Mixture model with stochastic variational inference, following this paper. This is the pgm of Gaussian Mixture. According to the paper, the full algorithm of ...
2
votes
1answer
73 views

why naive is needed in Naive Bayes ,what happens if naive is not included in Bayes theorem?

Im trying to understand why naive is needed in Naive Bayes and everyone says Naive Bayes assumes the input features (predictors) are not correlated hence they are not dependent on each other . i want ...
2
votes
1answer
156 views

When to use BayesianSearchCV and how it works?

Can somebody highlight when to use BayesianSearchCV and how it works? I have seen the implementation of same on kaggle and wanted to explore it further. Below is the link where the implementation ...
2
votes
1answer
385 views

Replacing missing value by class conditional mean

I have two classes, $p(x|y=0)$ and $p(x|y=1)$ with ${{\mu }_{0}}$ and ${{\mu }_{1}}$ as mean and shared covariance matrix $\Sigma $. Now, I have a missing feature ${{x}_{n}}$ for a particular ...
2
votes
1answer
103 views

Is deep learning a must in a Data Science MSc programme? [closed]

I am reading the programme outline of this two-year MSc in Data Science and I found that it has no deep learning content (as in many other european ones). I am no expert but as far as I've seen I ...
2
votes
1answer
49 views

Probability of event given two depandant events [closed]

Is there anyway, to compute the probability of the event given two dependent events? I know that Bayes can help if those events are independent, but what if condition events are dependent?
2
votes
1answer
54 views

Where can we find the application of bayes's theorem in Bayesian optimiation with gaussian processing

I am trying to learn bayesian optimisation by following this tutorial. However, until now I don't get the relation between bayes's theorem to the gaussian process formalism. Any ideas?
2
votes
1answer
18 views

How to calculate probability of non independence using bayes theorem?

i looked into one of the post about naive bayes calulation of naive part Predit the class label for instance (A=1,B=2,C=2) using naive Bayes classifcation. Let C1 be class 1 and C2 be class 2. For ...
2
votes
1answer
24 views

Good introductory reference for Bayesian Non-parametric (Dirichlet Process / Chinese Restaurant Process)

I am looking for a recommendation for basic introductory material on Bayesian Non-parametric methods, specifically Dirichlet Process / Chinese Restaurant Process. I am looking for material which ...
2
votes
1answer
127 views

MCMC for finding Bayesian Neural Network

Is someone familiar with such an approach: Suppose I want to build a bayesian neural network, with distributions over my parameters instead of point estimates. First I train my network with standard ...
2
votes
1answer
93 views

Using predicted probabilities and bayesian inference to update beliefs

I'm currently working on a project to predict the likelihood of an outcome. I'd like to implement a system where the belief the event will happen is updated after running the model on new data. ...
2
votes
1answer
66 views

How Do Bayesian Methods in Machine Learning Help With the Problem of Limited Data? Can This Be Used for Image Classification/Recognition? [closed]

When reading about machine learning, I've often come across information stating that Bayesian methods in machine learning are effective when you only possess a limited amount of data. As someone who ...
2
votes
0answers
106 views

What are the tradeoffs between Bayesian Deep Learning and Deep Gaussain Processes?

I understand the differences between Deep Gaussian Processes(DGPs) and Bayesian Deep Learning(BDL): DGPs are essentially feed-forward neural networks where each node is a Gaussian Processes, which BDL ...
2
votes
1answer
762 views

Linear Discriminant Analysis + bayesian theorem = LDA classifier??

I am new to machine learning and as I learn about Linear Discriminant Analysis, I can't see how it is used as a classifier. I can understand the difference between LDA and PCA and I can see how LDA ...
2
votes
0answers
388 views

Bayesian linear regression / categorical variable / Laplace prior

I'm trying to do feature selection in the bayesian framework with a Laplace prior with the following code in Python; Code: ...
2
votes
0answers
54 views

information leakage when using empirical Bayesian to generate a predictor

Consider the following problem: I want to predict the next bat of a set of baseball player. I have a training data set, where it contains the historical bat records (0-1 encoded, which is our target ...
1
vote
3answers
47 views

Why prior in MAP could be ignored?

A posterior $p(\theta\vert x) = \frac{p(x \vert \theta)p(\theta)}{p(x)} $ Many materials say that since the $p(x)$ is a constant, the $p(x)$ can be ignored. Thus, $p(\theta\vert x) \propto p(x \vert ...
1
vote
1answer
54 views

What is the difference between maximum likelihood hypothesis and maximum a posteriori hypothesis?

I am a student and I am studying machine learning. I am focusing on the concept of Bayesian learning and I have studied the maximum likelihood hypothesis and the maximum a posteriori hypothesis. I ...
1
vote
1answer
88 views

Problem understanding probabilistic generative models for classification

I am a student and I am studying machine learning. I am focusing on probabilistic generative models for classification and I am having some troubles understanding this topic. In the slide of my ...
1
vote
2answers
875 views

Opinions on an LSTM hyper-parameter tuning process I am using

I am training an LSTM to predict a price chart. I am using Bayesian optimization to speed things slightly since I have a large number of hyperparameters and only my CPU as a resource. Making 100 ...
1
vote
2answers
70 views

Parallel hyperparameter optimization techniques?

Most hyperparameter optimization technique want to evaluate points one by one. I have an expensive optimization problem, but i can run hundreds of evaluations in parallel. The dimension of the problem ...
1
vote
1answer
926 views

Creating a posterior distribution for classic coin flipping in python using grid search

I'm reading the book "Bayesian Analysis with Python" and the author provides some python code designed to show the grid search method of obtaining an approximate posterior distribution for the classic ...
1
vote
1answer
21 views

the probability distribution of dependent variables

There are three variables, X3 is a function of X1 and X2, ...
1
vote
1answer
50 views

Bayesian Neural net with non probibalistic Data?

Is it possible to construct a Bayesian Neural Network without Probability Distributions as dependent Variable for purpose of predictive modeling? I mean, if id like to Infer on a Specific Value, like ...
1
vote
1answer
73 views

ML algorithm where variable importance depends upon other variables - specifying conditionality

I am designing an algorithm that can detect cheating in a game of chess. I have a database of players who have been flagged by moderators as cheating, along with a number of game-level ...
1
vote
1answer
24 views

Visualize n-dimensional bayesian optimization results

I am working on a 6-dimensional bayesian optimization problem using (skopt's gp_minimize). After the optimizer ran for j iterations I would like to somehow visualize the "progress/result" of ...
1
vote
0answers
9 views

Tracking time-series latency using conjugate priors

I need to do a project using Bayesian statistics for a class and I am trying to apply it to my work. I help manage a time series database with 40,000+ different time series that we collect. The time ...
1
vote
0answers
44 views

Poisson model with overdisperssion

I'm working with a dataset $X$ (of length $N$) of count data, which looks like: I developed a statistical model which can be improved, so I'm asking for any suggestions, for instance, differnet ...
1
vote
0answers
29 views

Custom Loss Function for Mixing Sparse and Dense Features for a Prediction Problem

I have a largely uncorrelated feature space of about 40 dichotomous features, using which I'm trying to predict a continuous target variable. Now, some of these features are very sparse (Active less ...
1
vote
0answers
19 views

Improve confidence interval accuracy

I am doing a linear regression on log-transformed data and I use the bayesian approach to model the predictive distribution and construct my 90% prediction Interval. The problem with this approach is ...
1
vote
0answers
63 views

Firebase AB testing algorithm

We have run an AB test at firebase which has the following results: I was also building my own Bayesian AB-test suite and was wondering how they came to these conclusions. What I was doing was ...
1
vote
0answers
29 views

GMM with Dirichlet prior

I'm learning about the variational inference - mean field approximation on from this online course DeepBayes2019 page 30 The probabilistic model is written as follows: $p(X, Z \mid \pi, \mu, \lambda) =...