Questions tagged [distribution]
The distribution tag has no usage guidance.
119
questions
0
votes
0answers
14 views
How to find a probability distribution the parameters of which do not impact each other like mean and variance in normal distribution do?
I need to find a probability distribution to fit my data. My data has two important features, duration and activity count. Duration means how long one sequence lasts and activity count means the ...
0
votes
1answer
20 views
How to interpret KDE distribution graph?
I would like to know how to interpret this distribution graph. I have been doing an exercise from the book called 'Python for Finance Cookbook' by Eryk Lewinson. It does not give an in-depth ...
0
votes
0answers
19 views
What observations can you perceive from the following table?
I am preparing my thesis presentation and I want to give a clear understanding of the data that I have collected for the jury members. I understood the data very well, but I feel like I am lacking a ...
0
votes
1answer
17 views
[under/over]-sampling teaches model the wrong distribution?
TLDR: Will under/oversampling during the training phase teach the model the wrong distribution and adversely affect accuracy?
Let us assume you want to train a classifier to differentiate between ...
0
votes
1answer
26 views
Unable to transform the variable to normal distribution
I have a variable called 'list_price' in my data ,which is not normally distributed.
Here is the screenshot of how the distribution looks like before transformation
Tried transforming it with the ...
0
votes
0answers
13 views
Calculating probability from a distribution
I have a histogram/count plots of average time a property is in the market. Given this distribution, I'd like to calculate the probability that a similar property will be leased within 'x # of weeks' ...
0
votes
1answer
11 views
Getting a balanced sample across many variables
Let’s say each element in my population has several attributes.
Let’s call then A, B, C, D, E, F.
Let’s say, for simplicity, each attribute has 10 values (but could be any number between 2 and 30).
...
0
votes
1answer
31 views
Finding the worst affected industry due to COVID in terms of unemployment
My goal is to find the worst affected industries from COVID—19 in terms unemployment.
In terms of the data I will use for this task, I have a time series county-wise unemployment rate data of each ...
1
vote
1answer
21 views
Can the extent of variability within a dataset be reflected through clustering?
As an example: I need to compare the extent of variability amongst houses belonging to 4 different architectural eras - I want to see how different the houses are within each group and then compare ...
0
votes
0answers
8 views
what kind of distribution is followed by word and sentence vectors generated by TFIDF ,word2vec,glove,bert,flair?
what kind of distribution is followed by word or sentence embedding vectors generated by TFID or pretrained models like word2vec,glove,bert,flair ?
is it continuous or discrete or any other ...
0
votes
1answer
26 views
What is the distribution of the goal feature in this dataset
https://www.kaggle.com/kemical/kickstarter-projects
I was checking the distribution of the "goal" feature in this 2016 KickStarter data set. I figure goal would be more normally distributed ...
1
vote
1answer
19 views
A good machine learning approach for distribution of a whole?
So I had done with different classification, regression and clustering approaches for predictions of values etc. I was wondering if there is a machine learning approach for distribution of a whole ...
0
votes
1answer
34 views
Naive Bayes vs Full Bayes model classifiers
I have a hard time to understand when Naive Bayes works better than Full Bayes.
In general, i know that naive bayes does the assumption that features are independent given the class. However, if ...
0
votes
0answers
10 views
Machine learning. Input: set of distributions, Output: distribution
I have a set of features where typical machine learning techniques do not work very good. All features have very different distributions, some heteroscedasticity is also present.
The distribution ...
0
votes
0answers
39 views
How to estimate tail distribution for computing Conditional Value at Risk in python?
How to estimate parameteres of generalized pareto distribution in python? 🙏
I solve the task of computing Value at Risk and Conditional Value at Risk.
...
1
vote
1answer
26 views
Understand how to simulate a statistics [closed]
This solution describes how to simulate statistics to find a confidence interval.
A journalist called 1000 people in town to ask who will they be voting for out of candidates A and B.
The observed ...
1
vote
1answer
16 views
How to draw a sample from data set with respect to a given categorical or numerical variable based on given freely chosen distribution? (Python)
Say I have a data set for some past period. Now new data appears and for a given variable in the data and we find that the distributions have shifted (for example with "age" it would be that suddenly ...
0
votes
0answers
13 views
Setting whiskers to min/max of data set - valid or not?
Suppose there exist several measurements of experiments under different condidtions. Therefore, the shape of probability distributions of the measurement data will differ. Now I need to compare those ...
0
votes
0answers
10 views
Modeling dependence among outputs in neural network
If I am modeling multiple outputs in a neural network, does modeling them all jointly in 1 neural network capture conditional dependencies? My understanding is that each output component would share ...
0
votes
0answers
17 views
How to distinguish Multivariate Bernoulli Distribution from Binomial Distribution,Multinoulli distribution,Multinomial distribution?
Ok While studying naive Bayes I came across this question and from the accepted answer I reach to this blog. While reading this blog I got a clear idea of how Bernoulli distribution turns to (...
0
votes
0answers
9 views
The data set missing values and percentage of data that would remain unaffected
Can not understand question and answer to it given on DS questions site (see link below):
Q3. You are given a data set. The data set has missing values which spread along 1 standard deviation from ...
1
vote
0answers
37 views
Professionals appear to interpret sample correlation (e.g. Karl Pearson) as if it represents linear correlation. Is it the correct interpretation? [closed]
I am stressed following the wrong interpretation. What is the correct way of understanding a correlation coefficient.
1
vote
1answer
51 views
0
votes
0answers
10 views
Given a dataset how do i determine the number of modes in the underlying distribution?
given some data is there a technique determine the number of modes (Unimodal distribution, bimodal distribution etc.) without plotting and visualization?
1
vote
0answers
12 views
Compare the variances of two categorical distributions in a repeated measure design
I ran two model-building procedures with different parameters on the same sample and obtained the selection of my optimized hyperparameter for each outer fold (each of the analyses had 100 outer folds ...
0
votes
0answers
11 views
Text data distributions comparison
I would like to know what is the best method to compare the different text data distributions. I am working on text classification. I built a model using the old dataset. Now, I would like to know, ...
1
vote
1answer
26 views
How is the E(X) of a Poisson distribution lambda? [closed]
I was recently learning about the Poisson distribution, and was very perplexed about the E(X) equaling lambda. Like how did lambda even come into the picture here, isn't it a symbol of wavelength?
...
1
vote
3answers
503 views
Is there a cost associated with converting Koalas dataframe to Spark dataframe?
I know that pandas works "under the hood" with numpy arrays stored in dictionaries. In contrast, Koalas works with the underlying Spark framework. Does that mean that there is no extra cost associated ...
1
vote
1answer
36 views
How to resample one dataset to conform to the distribution of another dataset?
I have two datasets with 20 features, but with different feature distributions (DS_A and DS_B). How can I sample the DS_A to make its distribution similar to DS_B, with respect to multiple features??
...
0
votes
1answer
31 views
How to interpret axis of Histogram and distribution curve?
After doing statistical analysis of data I got the below figures:
The data contains a line length in meters which is represented with Line(m). The purpose here is to confirm the distribution behavior ...
0
votes
1answer
45 views
difference btw data distribution and frquency distribution
I have a dataset with 'n' features and corresponding labels(binary in nature). How can I calculate the data distribution and frequency distribution of the same? What is the difference btw the two?
1
vote
0answers
25 views
Calculate marginal probability distributions of a dataset
I have a dataset with 'n' features and a label(binary) corresponding each entry. I am using a predicitive model to predict these labels using those 'n' features. Now, I wish to know the marginal ...
3
votes
1answer
157 views
Confidence value for face recognition
In the context of face recognition I have the following histogram:
blue bins count the comparison distances for "self matches" (comparing two images of the same person). Orange bins count the ...
2
votes
1answer
28 views
Synthetic time series generation according to some distribution
I'm trying to develop a change detection model that uses sliding windows. Given a time series with some features I've a sliding widows that analyses that time period and compares with a successive ...
1
vote
0answers
26 views
SQL Oracle - Excel's BETAINV function (Inverse cumulative function of a beta distribution) in SQL Oracle
Is there a function equal to Excel's BETAINV(probability,alpha,beta,[A],[B]) in Oracle SQL Developer? Or a similar one which I could use to return the inverse cumulative distribution function of a ...
1
vote
0answers
47 views
2
votes
1answer
20 views
Testing for gender composition between groups
I did kmeans with k=4 and I would like to find out if there are any differences in the composition of gender between the 4 clusters.
Do I use Fisher's Exact Test of Independence? Is there any ...
2
votes
1answer
22 views
evaluation metrics for multiple values per session
I have an application that executes my foo() function several times for each user session. There are 2 alternate algorithms that i can implement as "foo" function and my goal is to evaluate them based ...
6
votes
3answers
183 views
Regression: How to deal with positive skewness in continuous target variable
I'm working on a regression problem. My aim is to "learn" the distribution of a continuous target $y$ as good as possible to make predictions. My model looks like:
$$y_i=\beta X_i + u_i.$$
$y$ is ...
1
vote
1answer
42 views
Univariate Outlier Detection
Let's say I have a dataset with the following format:
customerid
product
orders_in_last7days
orders_in_last6days
orders_in_last5days
orders_in_last4days
orders_in_last3days
orders_in_last2days
...
2
votes
0answers
20 views
How to estimate the marginal distribution of a class with respect to one predictor in a classification task?
I have a dataset with a binary dependent variable $y \in \{0,1\}$ and a set of predictors $x1,x2,..,t$. Here, $t$ is the time in minutes (in 24 hrs, that is $t \in (0,1440)$). I want to estimate the ...
2
votes
1answer
22 views
Generating random numbers from best probability distributions?
I have done some statistical distributions on real-world data. The distribution fitting gives me the below results (lognormal fits the best on data based on chi-squared test):
...
0
votes
0answers
27 views
How to do normalize distribution technique on a simple matrix?
This is just a practice about normal distribution in MATLAB and not a real project.
I created a simple matrix and want to test normal distribution on it, it's
...
0
votes
1answer
20 views
How to set and check normal distribution on a data set?
Sorry if my question is simple, I have a data set with two class and want to check and set normal distribution on it(if it was necessary, in MATLAB).
But the question is that I should use it on every ...
0
votes
1answer
34 views
How does $\chi^2$ feature selection work?
I can't find the information how $\chi^2$ are used to select numerical features for a model.
...
0
votes
1answer
55 views
The distribution of dataset train and test are the differents, how to fix this?
I am new in data science and like some help to understand my problem. For instance, I have two signals non-stationary for the same condition (figure 1).
I acquisition them at different times(in the ...
0
votes
1answer
36 views
Can continuous dataset have negative values? [closed]
As in the description, I want to choose continuos dataset with temperatures and the minimun value is -40 and maximum is 51. Is that OK? Can we have negative values in continuous distribution?
1
vote
0answers
68 views
Use of PYMC distribution.dist()
In pymc3 documentation, it specifies that the .dist() member for distribution allows the distribution to be used without a model for sampling and use of the logp functions e.g.
...
0
votes
2answers
940 views
When to use t-distribution instead of normal distribution?
According to the Student's t-distribution wiki article the t-distribution is used instead of the Normal distribution "when estimating the mean of a normally distributed population in situations where ...
0
votes
0answers
23 views
Using normal distribution data with high amplitude
I am using some data from a csv file to train a model which detects credit card fraud. The data set is from Kaggle and has 284,xxx samples each with 30 features.
After some visualization in Python, ...
