Questions tagged [probability]

Probability is the branch of mathematics concerning numerical descriptions of how likely an event is to occur or how likely it is that a proposition is true.

Filter by
Sorted by
Tagged with
-1
votes
0answers
13 views

how can i plot probability distribution of my classes in the way below?

all, i would like to plot the following: i have a binary classification problem where i am using xgboost as my 'model' below: ...
0
votes
1answer
10 views

how can i interpret kernel density plots from classification?

all, i have a classification problem where i am predicting likelihood of client defaulting on loan. i plotted the predicted probabilities from my model, and then plotted against the label '1' for ...
0
votes
0answers
18 views

How to develop a likelihood based prediction model to predict chance of rain in a particular hour of a year

I have time-series weather data (from 2005-2018) of temperature and rainfall (every one-hour interval) from three different location weather stations. I want to predict/see what likely be a chance of ...
-1
votes
0answers
15 views

Odds ratios explanation and question

I am very new to the odds ratios and I am trying to learn by myself and by watching videos and reading articles and trying to figure out a solution for one of my statistics questions. So I have read ...
0
votes
0answers
42 views

Evaluation of Mixture Density Networks

I have programmed an Mixture Density Network model to a market price. As input I have many numerical and categorical properties. The output of the network is a probability distribution (shape, ...
0
votes
1answer
17 views

Which has higher probability smoothing or unsmoothing? [closed]

I was reading about smoothing in NLP from this amazing book https://web.stanford.edu/~jurafsky/slp3/ (chapter 3). Then I came across a question in Chapter3 exercise (3.2). I calculated the probability ...
0
votes
1answer
33 views

Understanding output probabilites of xgboost in multiclass problems

I would like to understand the output probabilities of a xgboost classifier (or any other decision tree ensemble based classifier) in the case of a multiclass problem. For example: We have 5 different ...
0
votes
0answers
14 views

non-deep learning alternatives to mixture density network

I am currently using a Mixture density network to fit a distribution to a large dataset. The input to my network contains many features, while the output is a simple one-dimensional probability ...
1
vote
1answer
23 views

Probability of Gaussian Naive Bayes

How would I go about attaching a probability to the prediction outputted by a Gaussian Naive Bayes model ? I'm asking because the predict_proba function U can use with sklearn's Gaussian Naive Bayes ...
0
votes
0answers
18 views

Relation between optimal value function and optimal action value function

In equation 3.17 of Sutton and Barto's book: $$q_*(s, a)=\mathbb{E}[R_{t+1} + \gamma v_*(S_{t+1}) \mid S_t = s, A_t = a]$$ $G_{t+1}$ here have been replaced with $v_*(S_{t+1})$, but no reason has ...
-1
votes
0answers
13 views

Hypothesis Testing

I have a doubt in Type 1 and type 2 errors in Hypothesis. Below is the question asked to me. Consider the null hypothesis that a process produces no more than the maximum permissible rate of ...
2
votes
2answers
19 views

Notation for features (general notation for continuous and discrete random variables)

I'm looking for the right notation for features from different types. Let us say that my samples as $m$ features that can be modeled with $X_1,...,X_m$. The features Don't share the same distribution (...
1
vote
1answer
13 views

What is the concept of Normalized Mutual Information in the evaluation of Clustering?

I know what mutual information basically is but not quite sure about why and how it is used in the context of evaluation of clustering mechanisms ? Can someone please explain the intuition behind it ? ...
0
votes
0answers
9 views

How to compare ensemble spread with RMSE in order to find data points not included in the interpolation region?

I am currently working on the refinement of a huge dataset. The concept is that I want to reduce as many training data as possible. I have generated more than 1 million data without labels. To give ...
0
votes
1answer
20 views

Finding gender affinity for businesses

What are the different models I can use to find the gender affinity of businesses using yelp dataset-- https://www.kaggle.com/yelp-dataset/yelp-dataset . I need to find Probablity (Male buying from a ...
1
vote
0answers
13 views

How do data scientists reason under ambiguity?

I am looking for real-wrold examples of how data scientists build models and make decisions under situations of extreme ambiguity as described by Cassie Kozyrkov here: A decision under ambiguity ...
0
votes
1answer
31 views

Fitting a pandas dataframe to a Poisson Distribution

I have a simple dataframe df2 that consist of indices and one column of values. I want to fit this dataframe to a poisson distribution. Below is the code I am using: ...
0
votes
0answers
9 views

How to calculate probability of a users belonging to a single group over time?

Please bear with me, I am new to statistics and data science. I am trying to do a customer segmentation based on multiple features I am continually collecting on each customer over time. I expect ...
0
votes
1answer
35 views

how to compute bernoulli entropy?

I am reading gail implementation code in openai baselines. they compute bernoulli entropy as one of the loss in adversary network loss function. In their code, they implement bernoulli entropy as ...
2
votes
2answers
59 views

why we sample when predicting with Recurent Neural Network

I trained a Recurrent Neural Network to predict the next word in a sentence. I trained and now I want to predict, but there is something I am not getting well. I saw it in many tutorials even in the ...
0
votes
0answers
12 views

How to calculate two dimensional conditional probabilities?

I generated random data using Pareto distribution. I calculated with that data the conditional kernel estimator (Gaussian Kernel Density): ...
0
votes
0answers
7 views

Validating periods in a time series

i'm currently looking for an algorithm which is capable of solving the following task: A time series of discrete values (i.e., counts of crimes) is evenly spaced among 365 days. My task is to find ...
0
votes
0answers
11 views

Ranking graph's nodes by score propagation

Problem I have the following directed tripartite graph $G(E\cup V\cup P, A)$, where there is a many-to-one symmetric relationship between the subsets V and E - $e\in E,v\in V,[e, v]\in A \iff [v, e]\...
0
votes
0answers
9 views

Estimate machine count for API dependency

I have a SDK, with many APIs, which is used by many apps. Those apps are installed on many machines. I get data containing machine Id, app name and API info . The data is sampled such that 2% of the ...
1
vote
0answers
36 views

“Smearing” probabilities or how to handle imprecise locations for canonically classification-type problems

I am trying to predict failures at different nodes on a line. Each node has different weather features and hardware/configuration features. For a little under half of the historical failures I have, I ...
0
votes
2answers
15 views

Regressor to estimate probability from samples

What is a good way to estimate probabilities in a dataset of samples? Each sample is a measurement, that is usually 1 or 0. The goal is to calculate probabilies based on all feature rows. Simple ...
0
votes
0answers
8 views

How to calculate Probability from MSE results?

I am using a MSE classifier to classify some binary images into 4 classes, I extract 8 dimensional features for classification and normalize them to the max feature of them, so the max value for any ...
1
vote
1answer
24 views

Clarifying Probability Mass Function (PMF)

I am currently reading Deep Learning book, and I want to get better understanding of probability theory. In chapter 3.3.1 of Deep Learning book it states that: Often we associate each random ...
0
votes
1answer
17 views

How to perform link prediction in text based relationship data

I need to establish if there is a link between 2 columns from two different datasets with one matching column, where; ...
1
vote
1answer
33 views

Convolution and Pooling as Infinitely strong priors

I am currently reading about Convolutional Neural Networks in Deep Learning Book. I am stuck on section 9.4 titled "Convolution and Pooling as Infinitely String Priors". Could someone please ...
0
votes
1answer
108 views

How to compute denominator in Naive Bayes?

Suppose we have class C_k and input feature vector x in dataset How to calculate probability ...
1
vote
0answers
13 views

Practical example and working of Laplace Smoothing or Linear Interpolation in Natural Language Processing (NLP)

Let us suppose we have a document where total_words = 50 (for example -> is,the,now,is,am,here,now) total_unique_words = 40 (for example -> is,the,am,here,now) How can we apply the ...
1
vote
1answer
17 views

Understanding these probabilities, and reading the pdf and cdf

UPDATE: this is a math question but as i would like to codify it, I thought the DS SE might be appropriate. I am trying to sketch out this problem statement. Where am I going wrong since I don't see ...
2
votes
0answers
24 views

overlapping of surfaces with holes

I have two disks, of equal diameter. Both disks have a 'spatter' of holes, in random positions (from a two-dimensional - i.e., surface - uniform distribution; in other words, holes close to the sides ...
2
votes
1answer
25 views

Control which features are used for every task in multioutput classification?

I would like to perform a multiclass-multioutput classification task, on vectorized textual data. I started by using a random forest classifier in a multioutput startegy: ...
0
votes
0answers
22 views

Bias of an estimator?

What is p(X;0)actually modelling? What's the difference between the two types of theta? What's happening in this equation and did expectation convert into summation ? Source:https://subscription....
2
votes
1answer
66 views

why naive is needed in Naive Bayes ,what happens if naive is not included in Bayes theorem?

Im trying to understand why naive is needed in Naive Bayes and everyone says Naive Bayes assumes the input features (predictors) are not correlated hence they are not dependent on each other . i want ...
2
votes
0answers
15 views

Connection between prob output LogisticReg/SVM and ROC

I have the following ROC generated using LPOCV and Logistic regression or SVM (l2 norm). Now, let's say I have a test set containing 10 patients and I get that the probabilities of those patients to ...
0
votes
1answer
29 views

What is the name of this statistical interaction?

What is the name of the following statistical / informational interaction: given A, I know exactly what B is. given B, I know to some extent what A is. I'm not looking for a probability but rather ...
0
votes
1answer
261 views

XGBoost: How to set the probability threshold for multi class classification

I am using the XGBoost for classification of text data. There are the 3 different classes in the training dataset. ...
0
votes
0answers
9 views

Is it possible to fit GMM so that mean lies outside of image bounding box?

I m currently reading https://arxiv.org/pdf/1411.3159.pdf They take gradient maps and fit Gaussian Mixture Model. Then they evaluate how often is the activation center inside of the image bounding box....
0
votes
1answer
30 views

How to interpret axis of Histogram and distribution curve?

After doing statistical analysis of data I got the below figures: The data contains a line length in meters which is represented with Line(m). The purpose here is to confirm the distribution behavior ...
0
votes
0answers
11 views

Identify franchise names in a list of names

I am experimenting with a string vector of 200.000 observations to create a feature that determines if the name belongs to a franchise (Burger King, Mc'Donalds, etc). The present strategy is to ...
0
votes
0answers
26 views

Gaussian mixture models: find correct prior

This task might be pretty simple to the most of you. However, I am struggling to calculate the PDF for the Gaussian distributions. Help is very much appreciated!
1
vote
0answers
7 views

How does one use activation function with greater than [-1;1] range for binary classification?

In Efficient Backprop (http://yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf), Lecun and others propose to use activation function that don't reach target values on their asypmptotes. They explain (§ 4....
1
vote
1answer
19 views

Rules/Methods to use for determining user groups with no review data

I’m working on an analytical problem where I have customers who deal with end consumers. End consumers can leave customers reviews based on their experience (Star ratings, text etc). I’m trying to ...
1
vote
0answers
23 views

Calculate marginal probability distributions of a dataset

I have a dataset with 'n' features and a label(binary) corresponding each entry. I am using a predicitive model to predict these labels using those 'n' features. Now, I wish to know the marginal ...
3
votes
1answer
114 views

Confidence value for face recognition

In the context of face recognition I have the following histogram: blue bins count the comparison distances for "self matches" (comparing two images of the same person). Orange bins count the ...
2
votes
1answer
28 views

Synthetic time series generation according to some distribution

I'm trying to develop a change detection model that uses sliding windows. Given a time series with some features I've a sliding widows that analyses that time period and compares with a successive ...
2
votes
0answers
29 views

Difference between Gibbs sampling and variational Bayes inference

After reading in blogs and books, I came to the conclusion that Gibbs sampling and variation Bayes are methods for estimating or inference of posterior. Below link described but it's difficult to ...

1
2 3 4 5