Questions tagged [distribution]
The distribution tag has no usage guidance.
91
questions
2
votes
0answers
9 views
How to estimate the marginal distribution of a class with respect to one predictor in a classification task?
I have a dataset with a binary dependent variable $y \in \{0,1\}$ and a set of predictors $x1,x2,..,t$. Here, $t$ is the time in minutes (in 24 hrs, that is $t \in (0,1440)$). I want to estimate the ...
2
votes
1answer
22 views
Generating random numbers from best probability distributions?
I have done some statistical distributions on real-world data. The distribution fitting gives me the below results (lognormal fits the best on data based on chi-squared test):
...
0
votes
0answers
17 views
How to do normalize distribution technique on a simple matrix?
This is just a practice about normal distribution in MATLAB and not a real project.
I created a simple matrix and want to test normal distribution on it, it's
...
0
votes
1answer
18 views
How to set and check normal distribution on a data set?
Sorry if my question is simple, I have a data set with two class and want to check and set normal distribution on it(if it was necessary, in MATLAB).
But the question is that I should use it on every ...
0
votes
1answer
25 views
How does $\chi^2$ feature selection work?
I can't find the information how $\chi^2$ are used to select numerical features for a model.
...
0
votes
1answer
26 views
The distribution of dataset train and test are the differents, how to fix this?
I am new in data science and like some help to understand my problem. For instance, I have two signals non-stationary for the same condition (figure 1).
I acquisition them at different times(in the ...
0
votes
1answer
16 views
Can continuous dataset have negative values? [closed]
As in the description, I want to choose continuos dataset with temperatures and the minimun value is -40 and maximum is 51. Is that OK? Can we have negative values in continuous distribution?
1
vote
0answers
20 views
Use of PYMC distribution.dist()
In pymc3 documentation, it specifies that the .dist() member for distribution allows the distribution to be used without a model for sampling and use of the logp functions e.g.
...
0
votes
0answers
13 views
When to use t-distribution instead of normal distribution?
According to the Student's t-distribution wiki article the t-distribution is used instead of the Normal distribution "when estimating the mean of a normally distributed population in situations where ...
0
votes
0answers
18 views
Using normal distribution data with high amplitude
I am using some data from a csv file to train a model which detects credit card fraud. The data set is from Kaggle and has 284,xxx samples each with 30 features.
After some visualization in Python, ...
1
vote
1answer
26 views
Why do train, test, validation datasets need to have the same distribution?
I've found a lot of martial on how to deal with differently distributed train/test/validation data sets, however I'm struggling to find out why they they need to have the same distribution.
Can ...
0
votes
0answers
18 views
How to avoid dataset shift or covariate shift? [duplicate]
I wonder If there can occur a dataset shift or covariate shift through the following train_test_split-method?
...
0
votes
0answers
4 views
the metric that can be used to characterize and compare different histogram
There is a set of different data set, each of which corresponds to a histogram. Are there any metrics that can be used to characterize the shape of each histogram, and further compare different ...
1
vote
0answers
13 views
Co-joining multi-peak histograms
I am analysing a bunch of data files which represent responsiveness of cells to addition of a drug. If a drug is not added, cell responds normally, if it is added, it shows abnormal patterns:
, .
We ...
0
votes
0answers
9 views
How to estimate claims for warranty analytics where continuous improvement reduces warranty claims?
I have a situation where a company who has warranty claims on its product and they want to know that is it possible for them to extend that. Now the case is that they do improvement on the existing ...
0
votes
1answer
24 views
Predicted and true values distributions comparison
Is this alarming when a distribution of predicted values differs from a distribution of true values? I use xgbregressor and get the following plots
Usage of boxcox doesn't improve the case.
My data ...
0
votes
0answers
15 views
Normalization of Distribution of numeric features and target
Here, I am talking about the Normalization of Distribution of Features/Target, not normalization/scaling of them.
I've been seeing Notebooks which normalize some features and the target variable, ...
0
votes
0answers
31 views
dataset shift, covariate shift or sample selection bias in small subsets?
I try to show from which training data size on which machine learning method (CNN, SVM etc.) achieves better performance.
For this I would like to use subsets of different sizes from the datasets of ...
2
votes
0answers
46 views
difference between normal skewed distribution and skewed distribution
From what I've read normal skewed is a distribution that has all the properties of normal distribution and is skewed:
according to this resource:https://www.statisticshowto.datasciencecentral.com/...
1
vote
0answers
50 views
How to combine data having similar distribution?
I have a collection of time series data with data points of around 2 years of daily data. I am thinking of a way to increase the number of data points in it so that the neural network gets a better ...
2
votes
0answers
19 views
Is it possible to train probabilistic model to return several distributions?
I have nonlinear data of function y(x), which is let's say parabolic. At some points of x there are several y's (look at the picture).
Is it possible to train a probabilistic model to return several ...
1
vote
2answers
74 views
Fitting a distribution to data
I have a large data set of property values at two points in time. I have calculated the change in value between those two points. I want to assess which distribution most closely matches the variation ...
1
vote
1answer
26 views
How this visualisation was made?
I really like how this visualization represents the survey participants.
Is any tool for that? (Or R/python library?)
6
votes
3answers
154 views
xgboost: Is there a way to perform regression on rates/percentages data?
I have a dependent variable, $Y$, that is made up of rates/percentages data, so each value is between $0$ and $1$. I was attracted to the xgboost library because it allows focusing in on specific ...
0
votes
1answer
26 views
How to generate 12 independent random weights which all add up to one
I'm using Palisade's @Risk software with a triangular distribution to generate 12 random weights which must add up to one, but I get a lot of negative numbers. Is there a straightforward way to set ...
0
votes
1answer
74 views
CUDA 8.0 is compatible with my GeForce GTX 670M Wikipedia says, but TensorFlow rises an error: GTX 670M's Compute Capability is < 3.0
According to Wikipedia, the GeForce GTX 670M has a Compute Capability of 2.1 (and a Fermi micro-architecture), which is confirmed by TensorFlow (I can read "2.1" in the error it rises).
Wikipedia ...
1
vote
1answer
21 views
Why do seaborn.dist and pyplot.hist generate two different looking histograms on the same data?
I'm looking at telecom customers data. Two of the variables I'm looking at currently are:
Monthly Charges - The total amount charged to the customer monthly.
Is Senior Citizen - Whether the customer ...
0
votes
0answers
10 views
Which distribution should I use for Naive Bayes algorithm(Gaussian or Rayleigh)? What to do with categorical data?
I am predicting whether credit card application of an individual would be approved or not given his/her credentials.
I have the following dataset:
The variable descriptions are as follows:
I need ...
0
votes
1answer
49 views
Effect of skewness in data
I am preparing classification model. Many of numeric variables are positives skewed. Should I change a distribution of variables to be more Gaussian?
0
votes
2answers
19 views
How do I fix mis-rendered matplotlib?
How do I correct my data or format it so that it is presentable, and fix my graphs?
Dataset is 345551 rows × 7 columns.
I am using numpy, pandas, seaborn and matplot lib.
It seems that my pricing ...
0
votes
0answers
11 views
Compare large prediction with observed values
I have test scores for a lot of schools and I created a performance index calculated using the last score of a state test. This index is supossed to predict the performance of the school in the next ...
0
votes
1answer
139 views
A2C Continuous for Pendulum-v0 working implementation, negation for loss and entropy calculation
very good implementation of A2C continuous for Pendulum-v0
Code has snippet to stop execution when mean of last 10 or 20 is higher than -20 but the results look like:
...
2
votes
1answer
39 views
How to scale or standardize data that is mostly 0 (ranges from 0-1)?
I am relatively new to data science and big data munging in general. I currently have various columns of data that range from $0-1$, but most of the values in each ...
1
vote
0answers
19 views
F-test for comparing the mean of two groups
What would be the procedure, in terms of F-test, if I would like to check if the mean of one group is greater than the other group (alternative hypothesis: $\mu_1 > \mu_2$)?
And also what would ...
3
votes
1answer
235 views
Plotting Weibull distribution on Wind Speed
I have a 720 hourly set of wind speed and wind direction data and I want to fit the Weibull Distribution on it. After searching for some time, I wrote the following code in Python to get my ...
0
votes
0answers
16 views
How does t-sne scale work? Is it possible to compare scales of two t-sne plots?
I have used t-sne for two convolutional layers of binary classifier (VGG16 like CNN). I want to compare them, but I am not sure if that is okay as the scales of these two outputs from t-SNE plots can ...
0
votes
0answers
14 views
Normalization in SVM classifier
I am trying to normalize my features for a classification model with 3 class outputs. There are two kinds of features. First is medical test results and second is patient information such as age. The ...
0
votes
0answers
54 views
Normalization and probability density function
I am wondering is it reasonable to calculate the probability density function ( or calculate some measures of the probability distribution such as skewness) of data after standardizing the data (or ...
7
votes
2answers
423 views
how to check the distribution of the training set and testing set are similar
I have been playing the Kaggle Competition and I find there is a situation that the distribution of the training set and testing set are different, so I am wondering how to check the distribution of ...
0
votes
0answers
12 views
create joint prob distribution or empirical relation for two variables
There are two variables, X1 and X2. The experimental study shows that they are highly correlated. Are there any reliable ways to create an empirical mapping(or equation) between X1 and X2. Assuming ...
0
votes
1answer
43 views
Distribution of error values in linear regression vs logistic regression
Why do error values in linear regression have to be normally distributed and why not in logistic regression?
2
votes
0answers
45 views
How to create a prediction interval with the fact that the residuals follow a specific distribution (in python)
I am looking at a software development pipeline where I am predicting the lead time of different products flowing through the pipeline.
After applying a boxcox transformation on the lead time (...
2
votes
1answer
188 views
Normal distribution instead of Logistic distribution for classification
Logistic regression, based on the logistic function $\sigma(x) =
\frac{1}{1 + \exp(-x)}$, can be seen as a hypothesis testing problem. Where the reference distribution is the standard Logistic ...
0
votes
0answers
15 views
Is DQN independent from type of input distribution?Why?
Can we say:
Since DQN is online learning, it is independent of type of input distribution?
2
votes
2answers
83 views
How to visualize change in a distribution with a few outliers that account for a very large percent of the total?
I'm working on an edtech product where some of our traffic lands on webpages about textbooks.
Textbooks belong to subjects like Algebra, Calculus and Spanish. In each of our subjects, we have "...
0
votes
1answer
87 views
Clustering by Distributions of Groups of observations
I have data as shown below. Like the groups shown A,B,C.. there are roughly 5400 groups of data and 15 observations for every group (I sampled 6-7 for each group and pasted them here).
...
1
vote
0answers
13 views
How to elegantly caclulate probability distribution parameters for a particular random variable given some observed data?
I am using bnlearn package and R to learn Bayesian Network structure and also fit it using Maximum Likelihood estimation(MLE). bn.fit uses MLE to learn[as i understand] a generalized probability ...
4
votes
1answer
636 views
How can I plot/display a dataset or an image distribution?
I want to view a specific image or a dataset's distribution, and see if they are different.
Does simply writing something like :
...
1
vote
0answers
84 views
Two-sample K–S test Kolmogorov–Smirnov test in Python
For my evaluation, I have three different time-series data of the following format with different characterstics where the first column is timestamp and the second column is the value.
...
1
vote
0answers
34 views
Probability Distribution of the Process Duration Time based on the Process Status Measurements
I have data of execution of several processes. Each execution is identified by the process_id.
The duration of the execution is measured by external agents, testing ...
