Questions tagged [bayesian]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2
votes
1answer
78 views

How do Bayesian methods in machine learning help with the problem of limited data? Can this be used for image classification? [closed]

When reading about machine learning, I've often come across information stating that Bayesian methods in machine learning are effective when you only possess a limited amount of data. As someone who ...
3
votes
1answer
84 views

How to update the posterior belief when we are observing a stream of correlated data from a fixed but unknown data source

I want to build a [probabilistic] model that aims to infer the true value of an unknown categorical variable, $y \in \{1,2,..., K\}$. We have a dataset $(X,y): \mathbb{R}^d\rightarrow \{1,2,..., K\}$ ...
0
votes
1answer
18 views

Update of mean and variance of weights

I'm trying to understand the Bayes by Backprop algorithm from the paper Weight Uncertainty in Neural Networks, the idea is to make a NN in which each weight has it's own probability distribution. I ...
0
votes
1answer
33 views

High Recall but too low Precision result in imbalanced data

I was training a model using XGBoost Classifier on heavy imbalanced data base with 232:1 of binary class. Because my training data contains 750k rows and 320 features (after doing many feature ...
0
votes
0answers
17 views

Is it a good idea to use the mean and standard deviation of coefficients from other models as my prior in Bayesian Regression?

I have a dataset that I’ve been playing around with for school I have gotten very good results with a bunch of methods (Ridge, Lasso, ElasticNet, SVM, Bagging, Stacking and NN even) Now I’m having a ...
4
votes
1answer
22 views

Does the Bayesian MAP give a probability distribution over unseen data?

I'm working my way through the Bayesian world. So far I've understood that the MLE or the MPA are point estimates, therefore ...
1
vote
1answer
28 views

Combining multiple probabilities from a classifier. Propagating probabilities

Let's say I have trained a classifier that classifies images of animals into 10 different classes. And let's say that I have 20 different images of a particular animal and because I know the ...
0
votes
0answers
27 views

Calibration curve motivation

I struggle to understand the mathematical motivation for the binary classification model calibration curve. Why do we assume that the predicted probabilities should be consistent with the proportion ...
0
votes
1answer
81 views

How to test dev set on Time Series data via forecasting

I'm implementing $3$ Bayesian Deep Learning models (links below) for my masters. I'm supposed to test them on a civil engineering time series data. My models ...
0
votes
0answers
25 views

Bayes Classification Problem

Let we have 2 classes with their densties : p(x| w1)=$$\tfrac12e^{-|x|} $$ P(x|w2) = $$e^{-2|x-2|}$$ Where x is a one dimension feature vector . Now it is required to be known where to place the ...
1
vote
0answers
19 views

Algorithm to determine a single output value based on multiple input values [closed]

The main challenge is the lack of data. Input values come from tests results of patients. A patient takes a breath test at an interval during a timespan. The result values can range from 0 to ~200, ...
3
votes
1answer
76 views

How is bayesian risk computed to prune decision trees?

I've been trying to follow this paper on Bayesian Risk Pruning. I'm not very familiar with this type of pruning, but I'm wondering a few things: (1) The paper describes risk-rates to be defined per ...
3
votes
2answers
722 views

Is k-means with Mahalanobis a valid option for clustering?

I want more info into if k-means with Mahalanobis distance is a mathematically/methodologically correct option for datasets with different variance clusters. The steps are: Create aggregate datasets (...
0
votes
1answer
38 views

What is the num_initial_points argument for Bayesian Optimization with Keras Tuner?

I've implemented the following code to run Keras-Tuner with Bayesian Optimization: ...
1
vote
1answer
34 views

Visualize n-dimensional bayesian optimization results

I am working on a 6-dimensional bayesian optimization problem using (skopt's gp_minimize). After the optimizer ran for j iterations I would like to somehow visualize the "progress/result" of ...
6
votes
3answers
641 views

Which tribe does Probabilistic Graphical Models fall under?

Pedro Domingos in "The Master Algorithm" listed five tribes of machine learning algorithms: Symbolists Connectionists Evolutionaries Bayesians Analogizers Which category do probabilistic ...
0
votes
1answer
31 views

Hyperparameter tuning with Bayesian-Optimization

I'm using LightGBM for the regression problem and here is my code. ...
0
votes
1answer
197 views

Estimating class prevalence in unlabelled data after predicting labels with a binary classifier

I'm looking to get an estimate of the prevalence of 1's (i.e. the rate of positive labels) in a very large dataset that I have. However, I am hoping to report this percentage as a 95% credible ...
6
votes
1answer
234 views

Changing the batch size during training

The choice of batch size is in some sense the measure of stochasticity : On one hand, smaller batch sizes make the gradient descent more stochastic, the SGD can deviate significantly from the exact ...
1
vote
2answers
72 views

Parallel hyperparameter optimization techniques?

Most hyperparameter optimization technique want to evaluate points one by one. I have an expensive optimization problem, but i can run hundreds of evaluations in parallel. The dimension of the problem ...
1
vote
1answer
428 views

How to do hidden variable learning in Bayesian Network with Python?

I learned how to use libpgm in general for Bayesian inference and learning, but I do not understand if I can use it for learning with hidden variable. More precisely, I am trying to implement approach ...
3
votes
1answer
84 views

Why do machine learning engineers insist on training with more data than validation set?

Among my colleagues I have noticed a curious insistence on training with, say, 70% or 80% of data and validating on the remainder. The reason it is curious to me is the lack of any theoretical ...
0
votes
0answers
40 views

Practical difference between Bayesian Neural Networks and Feed Forward Neural Network with Gaussian Noise

To my understanding, a BNN's weights come from a Gaussian with trained mean and standard deviation, while a FFNN of the following form, comes from a learned weight, which acts as a 'mean', and is ...
3
votes
1answer
93 views

When to use bayesian linear regression instead of linear regression?

When does it make sense to use a bayesian approach, maybe in context to linear regression? To be more concrete: Assume you measure a certain number of devices and you wanna' check the linear ...
0
votes
0answers
17 views

I think a learning rate schedule would be counter-productive with AdaBelief. Am I wrong?

I am inclined to believe the concept of a learning rate schedule is overcome by the improvements of Adabelief over Adam. My code is on Github; please check it out and attempt to replicate my results, ...
1
vote
0answers
9 views

Tracking time-series latency using conjugate priors

I need to do a project using Bayesian statistics for a class and I am trying to apply it to my work. I help manage a time series database with 40,000+ different time series that we collect. The time ...
0
votes
2answers
32 views

Books about statistical inference [closed]

I'm currently taking a course "Introduction to Machine Learning" which covers the following topics: linear regression, overfitting, classification problems, parametric & non-parametric ...
0
votes
0answers
31 views

Bayesian Regression Model

I am new to Bayesian modeling. I am running Bayesian regression model in R using brm function from brms library, which is powered by STAN. I have a data with 10 million records. I took 10% sample out ...
1
vote
1answer
84 views

What is the difference between maximum likelihood hypothesis and maximum a posteriori hypothesis?

I am a student and I am studying machine learning. I am focusing on the concept of Bayesian learning and I have studied the maximum likelihood hypothesis and the maximum a posteriori hypothesis. I ...
1
vote
1answer
93 views

Problem understanding probabilistic generative models for classification

I am a student and I am studying machine learning. I am focusing on probabilistic generative models for classification and I am having some troubles understanding this topic. In the slide of my ...
0
votes
1answer
39 views

Hyperparameter tuning of neural networks using Bayesian Optimization

One of the assumptions for finding good hyperparameters using Bayesian optimization (GP) is that the unknown function is smooth. Is this assumption valid for neural networks or at least for most of ...
1
vote
0answers
45 views

Poisson model with overdisperssion

I'm working with a dataset $X$ (of length $N$) of count data, which looks like: I developed a statistical model which can be improved, so I'm asking for any suggestions, for instance, differnet ...
0
votes
1answer
83 views

What makes the posterior intractable?

In the setting of Variational AutoEncoders, i.e. when we want to find the posterior distribution over the data generating, latent variable z, given some ...
1
vote
0answers
36 views

Custom Loss Function for Mixing Sparse and Dense Features for a Prediction Problem

I have a largely uncorrelated feature space of about 40 dichotomous features, using which I'm trying to predict a continuous target variable. Now, some of these features are very sparse (Active less ...
1
vote
1answer
49 views

Probability of event given two depandant events [closed]

Is there anyway, to compute the probability of the event given two dependent events? I know that Bayes can help if those events are independent, but what if condition events are dependent?
1
vote
0answers
21 views

Improve confidence interval accuracy

I am doing a linear regression on log-transformed data and I use the bayesian approach to model the predictive distribution and construct my 90% prediction Interval. The problem with this approach is ...
0
votes
1answer
108 views

BPR TripletLoss Recommender System

I am trying to modify the code of this repo to build a recommender system based on BPR triplet loss. In particular I modified the TripletLoss layer class like this ...
0
votes
0answers
11 views

Machine learning. Input: set of distributions, Output: distribution

I have a set of features where typical machine learning techniques do not work very good. All features have very different distributions, some heteroscedasticity is also present. The distribution ...
4
votes
2answers
155 views

Mean estimation for nested location data

I want to estimate the average income for a location. I have nested data in the following way: A block is inside a neighborhood, which is inside a zipcode, which is inside a district, which is inside ...
1
vote
2answers
2k views

Opinions on an LSTM hyper-parameter tuning process I am using

I am training an LSTM to predict a price chart. I am using Bayesian optimization to speed things slightly since I have a large number of hyperparameters and only my CPU as a resource. Making 100 ...
2
votes
1answer
27 views

Good introductory reference for Bayesian Non-parametric (Dirichlet Process / Chinese Restaurant Process)

I am looking for a recommendation for basic introductory material on Bayesian Non-parametric methods, specifically Dirichlet Process / Chinese Restaurant Process. I am looking for material which ...
0
votes
0answers
57 views

Classification of OLS regression coefficients

A variable $A$ (reaction time) is log-normally distributed, i.e. $\log(A) \sim \mathcal{N}(0,\sigma^2)$ and is linearly dependent of $n$ variables $X = (X_1,\ldots,X_n)$, i.e. \begin{align} A &= \...
1
vote
0answers
72 views

Firebase AB testing algorithm

We have run an AB test at firebase which has the following results: I was also building my own Bayesian AB-test suite and was wondering how they came to these conclusions. What I was doing was ...
1
vote
1answer
52 views

Bayesian Neural net with non probibalistic Data?

Is it possible to construct a Bayesian Neural Network without Probability Distributions as dependent Variable for purpose of predictive modeling? I mean, if id like to Infer on a Specific Value, like ...
3
votes
1answer
967 views

LSTM: Converting to Bayesian Deep Neural Network

Starting from Yarin Gal's research paper on using Dropout as a Bayesian Approximation (https://arxiv.org/pdf/1506.02142.pdf), I am trying to apply this concept to my Sequence Prediction model. My ...
1
vote
0answers
39 views

Specifying priors in rstanarm for hierarchical model

We are given the model $$ \begin{align*} y_{ij} & \sim \mathsf{Normal}(\alpha_j + \beta x_i, \sigma^2)\\ \alpha_j & \sim \mathsf{Normal}(\gamma_0 + \gamma_1 u_j, \tau^2) \end{align*} $$ with ...
2
votes
1answer
106 views

Using predicted probabilities and bayesian inference to update beliefs

I'm currently working on a project to predict the likelihood of an outcome. I'd like to implement a system where the belief the event will happen is updated after running the model on new data. ...
4
votes
1answer
51 views

Bayesian regularization vs dropout for basic ann

Does it make sense conceptually to apply dropout to an artificial neutral network while also applying bayesian regularization? On one hand I would think that technically this should work just fine, ...
2
votes
1answer
20 views

How to calculate probability of non independence using bayes theorem?

i looked into one of the post about naive bayes calulation of naive part Predit the class label for instance (A=1,B=2,C=2) using naive Bayes classifcation. Let C1 be class 1 and C2 be class 2. For ...
2
votes
1answer
75 views

why naive is needed in Naive Bayes ,what happens if naive is not included in Bayes theorem?

Im trying to understand why naive is needed in Naive Bayes and everyone says Naive Bayes assumes the input features (predictors) are not correlated hence they are not dependent on each other . i want ...