# Questions tagged [distribution]

The tag has no usage guidance.

170 questions
Filter by
Sorted by
Tagged with
112 views

### How to combine data having similar distribution?

I have a collection of time series data with data points of around 2 years of daily data. I am thinking of a way to increase the number of data points in it so that the neural network gets a better ...
18 views

### Probability distribution for lstm and mlp

I have total of 6300 samples, 5800 of which are training data, and 500 of which are testing data. We compare the performance of LSTM and multilayer perceptron (MLP) with one hidden layer in terms of ...
1 vote
220 views

### How to find the distribution of a single variable based on population distribution

I am trying to find the age distribution of the subset of married people of a population. However, I realise that just visualising the married count by age (Figure 1) is highly dependent on the ...
688 views

### Analysis of probability distribution of each features and Machine Learning

While I know that probability distributions are for hypothesis testing, confidence level constructions, etc. They definitely have many roles in statistical analysis. However, it is not obvious to me ...
55 views

### How to measure statistical similarity or discrepancy between a dataset and a distribution?

Is any way to measure statistical similarity or discrepancy between a dataset and a distribution? I have do some research, but find most of method are intended to describe discrepancy between data and ...
1 vote
184 views

### Binomial family in logistic regression

I was asked in an interview why do we use the binomial distribution in logistic regression and how is it related to the class that we are predicting? Could anyone explain, without any mathematical ...
94 views

### Non IID variables and SVM Classifier

I am training an SVM model to predict the trend of stock prices (one-day ahead predictions. Classification task). It Had completely slipped from my mind that SVMs assume IID data until I had a ...
4k views

### Boxplots or violinplots?

This is quite a general question, perhaps somewhat opinion-based. In most papers people use boxplots to visualize a certain distribution, yet violinplots are able to give more information. ...
226 views

### Getting a balanced sample across many variables

Let’s say each element in my population has several attributes. Let’s call then A, B, C, D, E, F. Let’s say, for simplicity, each attribute has 10 values (but could be any number between 2 and 30). ...
1k views

### derivation for expected value for variance

Hi Im taking a course about probability distribution in datascience and below is derivation of the expected value for the variance Variance = expected value of the squared difference from mean for ...
1 vote
209 views

### Generating a set of different scenarios based on some initial observations

I have a in my hands 3 different time series which model 3 different scenarios (base, downside, upside). Every of this time-series depends on a set of 11 different attributes, which take values for ...
1k views

### Normal vs Uniform Distribution for machine learning

I have a dataset that follows Zipf's law such that the majority of the values are concentrated at one end, with the remaining items containing a very small percentage. Training on the dataset as is ...
1 vote
62 views

### Finding the worst affected industry due to COVID in terms of unemployment

My goal is to find the worst affected industries from COVID—19 in terms unemployment. In terms of the data I will use for this task, I have a time series county-wise unemployment rate data of each ...
1 vote
60 views

### Is there a measure to compare features on the basis of normality

I have a dataset of cars and it has many features including 'acceleration’, ‘horsepower’, and ‘mpg'. I am supposed to check which of these features is the most similar to a normal distribution, so I ...
74 views

### difference btw data distribution and frquency distribution

I have a dataset with 'n' features and corresponding labels(binary in nature). How can I calculate the data distribution and frequency distribution of the same? What is the difference btw the two?
129 views

### Co-joining multi-peak histograms

I am analysing a bunch of data files which represent responsiveness of cells to addition of a drug. If a drug is not added, cell responds normally, if it is added, it shows abnormal patterns: , . We ...
1 vote
89 views

### How to draw a sample from data set with respect to a given categorical or numerical variable based on given freely chosen distribution? (Python)

Say I have a data set for some past period. Now new data appears and for a given variable in the data and we find that the distributions have shifted (for example with "age" it would be that suddenly ...
71 views

### Using Maximum Mean Discrepancy (MMD) to Compare Kernel Density Estimates (KDEs)

I'm interested in comparing two Kernel Density Estimates (KDEs) and I've come across the Maximum Mean Discrepancy (MMD) metric ...
1k views

### A/B Testing (Binomial Distribution vs Random Distribution)

When performing an A/B test for the number of clicks for users viewing (each view is an impression) two variants of an ad, a binomial distribution can be assumed where each variant has a constant ...
54 views

### Is Maximum Mean Discrepancy (MMD) suitable for comparing distributions with different sample sizes?

I'm working on a project where I need to compare the similarity of two probability distributions using MMD. However, the two datasets have different sample sizes. I've read that MMD can be affected by ...
14 views

### Two overlap gaussian mixtures come from one or two generations of data?

Reviewing for a ML exam, my teacher asked last year a philosophical question about gaussians and generations of data which I couldn’t find no good answer. Imagine that an oracle can say with 100% ...
35 views

### Binary Classification Dataset Distribution

I have a question regarding the distribution of my dataset. I have a binary classification task which I need to label. I now want to know how I should distribute the labels for optimal training / ...
26 views

### What can I do when validation data and test data has different distribution in imbalance classification?

I am building classification model for bio (scRNA) data. Datasets in this field, for example, dataset A has 1, 2 classes, dataset B has 2, 3 classes kind of that. So I integrated datasets for training ...
31 views

### Bayesian Linear regression for non-gaussian distribution

Which distribution should be considered for non-gaussian distribution while building Bayesian Linear regression. I am trying to build this model using Pymc3 library but since I have just started to ...
8 views

### Grouped Time Restricted Demand Regression with value cap

so I am working on quite an interesting regression task that I haven't encountered before. Our company sells products (steel) in tons. We offer contracts where the customer orders a certain amount of ...
1k views

### Does t-test require Standard Deviation of sample for calculation

Might be a novice question, but the main difference between a t-test and z-test, I was able to understand, is that the z-test calculation requires the SD value of the population where as in a t-test, ...
363 views

### What is a good metric for comparing a single value to a distribution?

I'd like to compare single values from one distribution to another distribution, effectively transforming the second distribution in such way, that its values reflect both datasets simultaneously, i.e....
21 views

### What distribution-shift datasets exist for ML?

What datasets there are which contain a (labelled) distribution shift or which distribution-shifted datasets share classes? Currently I have found some interesting results using CIFAR10 and the ...
50 views

### Train neural network to predict multiple distributions

I aim to train a neural network to predict 2 distributions (10 quantiles, i.e. deciles) at 5 time points. So my y is of shape: ...
19 views

### Using neural networks to learn total distributions from linearly transformed samples

I am trying to explore the use of neural network based models to replace a cumbersome challenge with a faster more approximate method. The problem involves learning a distribution termed the "...
2k views

### What is the difference between Covariate Shift, Label Shift, Concept Shift, Concept Drift, and Prior Probability Shift?

As a beginner in MLOps, I was overwhelmed by some confusing definitions. As far as I understand, when we have a classifier or regressor with y = f(X) function: <...
71 views

### How do I fix mis-rendered matplotlib?

How do I correct my data or format it so that it is presentable, and fix my graphs? Dataset is 345551 rows × 7 columns. I am using numpy, pandas, seaborn and matplot lib. It seems that my pricing ...
1 vote
2k views

### Data Analytics how to read ECDF graph

Hi there, My question is about how to read ECDF graphs. I am still quite unsure what the jumps / zig-zags in the graph mean and what is happening when there is a horizontal line and so on. I would be ...
2k views

### The tag on feature-scaling says "zero mean" and unit variance. Is it correct?

The tag on feature scaling says: Popular feature scaling types include scaling the data to have zero mean and unit variance, and scaling the data between a given minimum and maximum value. It is ...
62 views

### For standard deviation's formula, why does division by sample size come before square rooting?

Why is this used to calculate a sample's standard deviation: $s=\sqrt{\frac{1}{N-1}\sum_{i=1}^N(x_i-\bar{x})^2}$ and not something like: $s=\frac{1}{N-1}\sqrt{\sum_{i=1}^N(x_i-\bar{x})^2}$ I ...
1 vote
123 views

### Univariate Outlier Detection

Let's say I have a dataset with the following format: customerid product orders_in_last7days orders_in_last6days orders_in_last5days orders_in_last4days orders_in_last3days orders_in_last2days ...
2k views

### How to plot categorical variables with a pie chart

I am concerned with a single column (fruit) from my df: ...
64 views

### Distribution check - different results using plt.hist and seaborn.displot

I have an apparently very simple task: checking the distribution of values contained in an array data. Only thing is, I get two completely different results by ...
1 vote
59 views

### What is the distribution of the goal feature in this dataset

https://www.kaggle.com/kemical/kickstarter-projects I was checking the distribution of the "goal" feature in this 2016 KickStarter data set. I figure goal would be more normally distributed ...
3k views

### Regression: How to deal with positive skewness in continuous target variable

I'm working on a regression problem. My aim is to "learn" the distribution of a continuous target $y$ as good as possible to make predictions. My model looks like: $$y_i=\beta X_i + u_i.$$ $y$ is ...
1 vote
60 views

### Understanding Syntactic divergence

I am trying to read a paper https://arxiv.org/abs/2004.14444 Section 6.1 of the paper describes Syntactic divergence. I have confusion regarding the distribution graphs and split of the dataset. If a ...
1 vote
25 views

### Strange behaviour around zero for predicted distributions from a deep learning regression model

Can somebody help make sense of these very odd distributions that I obtained from my trained deep learning regression model? The model was trained with either MAE or MSE loss, which is what the ...
98 views

### What is the next after finding best distribution for my data?

I found the distribution of my data with "distfit" library for python. But what now? The best distribution that describes my data is "weibull" distribution. But I don't know what ...
889 views

### How to make a gaussian distribution in python considering mean. variance. skewness and kurtosis?

np.random.normal(mean,sigma,size) allows to create a gaussian distribution based only on mean and variance. I want to create a distribution based on ...
51 views

### Multiclass classification (gradient boosted trees) predictions distribution using softmax()

Let's consider a multiclass problem where the target is composed by 20% class 'A', 50% of class 'B' and 30% of class 'C'. The model is trained and then the class predictions are obtained via the ...
105 views

### How can I obtain the mean of a Poisson distribution given the first improbable point of the distribution?

I generated a Poisson distribution with mean equal to 3 and 10000 samples by using np.random.poisson(3,10000). The plot is the following: from this plot I see that ...
199 views

### Compute the pdf from pandas kde

I have data (features/targets in machine learning terminology), e.g. X1(t), X2(t), ... XN(t) and dependent variable y(t). I can use pandas to plot the kde's of the independent variables (X1(t),...). I ...
14k views

### Plotting different values in pandas histogram with different colors

I am working on a dataset. The dataset consists of 16 different features each feature having values belonging to the set (0, 1, 2). In order to check the distribution of values in each column, I used <...
1 vote