Questions tagged [descriptive-statistics]
Descriptive statistics summarize features of a sample, such as mean and standard deviations, median and quartiles, the maximum and minimum. With multiple variables, may include correlations and crosstabs. Can include visual displays - boxplots, histograms, scatterplots and so on.
89
questions
0
votes
1
answer
25
views
Is there a measure to compare features on the basis of normality
I have a dataset of cars and it has many features including 'acceleration’, ‘horsepower’, and ‘mpg'.
I am supposed to check which if these features is the most similar to a normal distribution, so I ...
0
votes
1
answer
31
views
what are the best sources to learn about statistics?
am interested in quant analytics. hope statistics is heart for quant analysis.
so, what are the best sources to learn core statistics (in depth) and what is the best approach to learn statistics?
- 57
1
vote
2
answers
280
views
How to find a vector representation for each descriptor?
Cubes data is well known data for extreme classification. Each picture has a set of descriptor along with it. Total data set has 312 descriptor. You will find list of descriptior in this file.
My ...
- 61
0
votes
1
answer
27
views
Best practise XAI: understand features which build up cluster and explain underlaying structure
I want to cluster my data and show which features were used to define the clusters to show the structure in my data.
To explain the use case: Imaging I have data from many products and I want show the ...
- 113
15
votes
5
answers
1k
views
How can I ensure anonymity with queries to small datasets?
I'm building a service that will contain personal data relating to real people.
Initially the dataset will be quite small, and as such it may be possible to identify individuals if the search ...
- 253
1
vote
1
answer
44
views
Which test statistic do you recommend for checking the difference between a dynamic for some parameter in control and treatment group?
I have two groups of patients, a control (A) and a treatment (B), and data for patients' weight in the beginning and in the end of a period of treatment in each group. I need to check if the dynamic ...
- 111
0
votes
0
answers
14
views
What are the reasons that can lead to increase Micro F1-score but decrease the Macro F1-score
I have an imbalaced multi-label classification dataset
I tried these 2 models
First I used Bernoulli Naive Bayes algorithm that nativelly supports multi-label classification I got Micro F1-score of 45%...
- 539
0
votes
0
answers
14
views
Aggregating irregular recorded accelerometer and gps data
I have a dataset containing accelerometer- and GPS data on which I want to try some classification. However, I am not sure how to aggregate this data into a larger timescale (per day or per week) as ...
0
votes
0
answers
19
views
why we use chebyshev's inequality when we can convert data into normal distribution?
When there is a normal distribution available, why don't we just converted all data into Russian distribution and then use this equation instead of going with other formula and all we have to buy to ...
0
votes
0
answers
16
views
Is this Time Series Non-Stationary? Or is the ADF test wrong?
I have two time series I want to prove if they are stationary or not. The first time series is the total hours from a YouTube channel over time and the second one is the organic hours (total hours - ...
0
votes
0
answers
7
views
Is there a dedicated name for a matrix of scatter plots representing all 2d projections of a n-d dataset?
When facing a n-dim dataset, all variables being numericals, I am used to produce a series of all 2d projection of this dataset. Do these exhaustive 2d projections over a dataset own a specific name ?
...
- 101
1
vote
1
answer
21
views
Are there any algorithm to generate a set of data that match some statistic requirements?
I was wondering if there are time-efficient algorithms that can reverse the process of basic statistics computation. What I mean is an algorithm that instead of computing the mean, SD, max-min range, ...
- 13
0
votes
0
answers
26
views
Substantive differences between using the term "more likely" vs. "higher share" when reporting survey data?
I have a question on proper language to describe differences in survey data. I'm doing some simple descriptive statistics using nationally representative survey data.
I am using svyttest in r to test ...
- 1
0
votes
1
answer
69
views
What is the difference between accuracy in percentage (classification metric) and MPE (regression metric) as evaluation metrices?
What is the difference between accuracy in percentage (classification metric) and MPE (regression metric) as evaluation metrices?
0
votes
2
answers
244
views
Missing Values Correlation
Is it worth it to study missingness correlation between columns?
If you have strongly correlated missing values (say between two columns, A and B), how will this change or shape the way you look at ...
0
votes
1
answer
37
views
How to count words in a dataframe?
I would like to count how many Male and female who answer (ex. Biking / Cycling).
Below is the sample data:
- 1
0
votes
1
answer
179
views
PSI where not to use
From what I understand PSI is used for continuous data. Generally, equal sized bins are created to compare two data set, and number of buckets is usually 10. Is that for a reason, why 10 bucket? Also, ...
- 1
1
vote
0
answers
10
views
Trying to get an idea of decile system
I was checking a Linkedin profile and I came across this statement in the “Grade” section. It’s a little confusing. It is as follows:
“Grade: gpa within the first decile”.
Is this statement correct? ...
- 455
1
vote
1
answer
11
views
what is a collective term for things as screening failures, no response and drop-outs
Hello wondering what is a collective term (in terms of statistics) for things as screening failures, no response and drop-outs?
- 143
1
vote
0
answers
57
views
Null and 0 Values Handling - Basic Statistics [duplicate]
I am trying to do some descriptive statistics and basic graphs on relatively small data sets derived from environmental sample collection and/or chemical research. It turns out that sample points end ...
- 111
0
votes
2
answers
509
views
A measure of redundancy in mutual information
Mutual information quantifies to what degree $X$ decreases the uncertainty about $Y$. However, to my understanding, it does not quantify "in how many ways" $X$ decreases the uncertainty. E.g....
- 33
1
vote
0
answers
74
views
Estimating the (normal?) distribution of the grayscale values of a region in camera footage
The problem:
Given a small ($25\times25$) region on a $1080$p image extracted from camera footage of a railway system, I am trying to detect whether there is a train present or not. I proceed to take ...
- 111
1
vote
1
answer
39
views
Applying an algorithm on it's own training data for descriptive purposes?
Good Day, I am newer to data science so I am not confident in this. To set up the question I will describe my data and approach.
Data
I don't want to share specific data examples as I want to try and ...
- 159
1
vote
1
answer
52
views
Understanding math notation in infoGAN paper
I'm reading this paper about mutual information in infoGAN infoGAN_paper_link and already have the code to run it. I pretty much found code for it which is fine and dandy except for the fact that I ...
- 359
1
vote
1
answer
3k
views
Calculating correlation for categorical variables
I am struggling to find out a suitable way to calculate correlation coefficient for categorical variables. Pearson's coefficient is not supported for categorical ...
- 189
1
vote
1
answer
58
views
What is the best way to cluster this kind of data?
I have data that looks like this:
The chart on the left is the trend and the smaller chart on the right is the box plot showing the distribution of means. Each color is the output of a particular ...
- 113
1
vote
1
answer
108
views
Accuracy over different sample sizes from dataset
What I'm trying to do is predict how much more data would help in a classification task.
So, what I'm doing is bootstrapping entries in my dataset to get a sample, with a specified size. Then, I fine-...
- 308
2
votes
2
answers
2k
views
p-value and effect size
Is it correct to say that the lower the p-value is the higher is the difference between the two means of the two groups in the t-test?
For example, if I apply the t-test between two groups of ...
1
vote
0
answers
37
views
paired t-test shows no difference between median and Wilcoxon test p value shows that there is a difference between median values ? How to interpret?
I have a dataset. I wanted to do paired t test on it. So I carried out normality test and it showed that it does not follow normal distribution. So I used Wilcoxon test in place of paired t test. The ...
- 151
1
vote
1
answer
96
views
How to bin a distribution data reported with different frequencies ( salaries ), showing mixed linearity and non-linearity?
I am researching on pay-scales, and wish to receive advise to treat data of salaries.
Objective
My interest is to approximate the salary corresponding to different hierarchical levels in an ...
- 165
0
votes
0
answers
36
views
Data calculation/normalisation /data summarisation
Clustering is done based on houses data and say suppose cluster 1 contains 5 houses.
Appliance 1 is used in 2 houses out of 5 and appliance 2 is used in 4 out of 5 houses.Each appliance produces an ...
- 9
2
votes
0
answers
22
views
Statistical method to find the value which preserves the most information inside "most" of data points. (resize images to a common height)
So I have this data of around 88K images and I found out some interesting properties for my images.
...
- 323
1
vote
1
answer
39
views
How to get a (descriptive) overview of a large database?
I'm facing a data framework with
~ 20 k observations and
151 variables across
2078 subjects
At first I am primarily interested in how the data looks like related to a single parameter. But I cannot ...
- 540
2
votes
3
answers
1k
views
A/B Testing (Binomial Distribution vs Random Distribution)
When performing an A/B test for the number of clicks for users viewing (each view is an impression) two variants of an ad, a binomial distribution can be assumed where each variant has a constant ...
- 121
1
vote
0
answers
20
views
Normalise the scores by small and large numbers division
I am working on a scoring problem use-case where I will score each task based on few aspects and take an average of scores of tasks in each experiment. For each experiment I will have 1 to N number of ...
- 153
1
vote
1
answer
480
views
Can a Box plot be used for finding the useful features from the dataset? [closed]
I am reading a book by professor Trevor Hastie and professor Robert Tibshirani called "Introduction to Statistical Learning". In the applied section of the chapter 4, there is a question 11(...
0
votes
1
answer
41
views
Analysis of passed/not passed students by enrolled year
I am analysing some data from my University about successful students through years. The dataset looks like as follows
...
- 295
2
votes
1
answer
139
views
Problem regarding calculating correlation approach?
There are 3 columns say product_id, product_type, and price_drop. ...
- 23
2
votes
3
answers
383
views
How do I interpret the output of linear regression model in R?
I have the following linear regression model and its analysis. There are a few errors, but I am not very sure about the errors. I have not succeeded in finding them so far.
First, the 95% confidence ...
- 121
0
votes
2
answers
488
views
Calculating Median and Mode of numerical variables for different subgroups in R
I have customer call data and I want to get the median and mode for the call success rate for different subgroups.
My variables are: Customer ID, Employment Status (Retired, Employed, Unemployed), ...
- 53
3
votes
2
answers
579
views
Distribution vs Histogram [closed]
I am not very good at understanding statistics jargon
I tried to read articles on statistics every article says distribution and show a picture of bell curve. I understand that x axis of bell curve ...
- 389
0
votes
1
answer
14
views
Getting the earliest date (duplicates due to several call ids and agents)
I am trying to get the earliest call interaction, with only the agent that interacted at that call and the customer id. Using the following code, I still get duplicates:
...
- 53
0
votes
0
answers
32
views
Exploratory statistics, how to idenify and remove driver (bias)
I am looking at customer data, and created frequency tables (+histograms) for customers with different professional statuses and what the best time is to reach them. Status ranges here from employed, ...
- 53
1
vote
0
answers
17
views
Coefficient of variance problem
I am doing a project and I am facing a problem I have no idea how to solve.
I have a data series of stock prices, and, based on this series I obtained another series, of so-called rentabilities (which ...
- 11
1
vote
1
answer
100
views
Drastic drop in Somers' D ? Why?
I came across to find the correlation between the ratings assigned by two coaches to a same group of 40 players.
I have tabulated the results as below:
The Somers' D is 50%.
However, for the case ...
- 11
0
votes
1
answer
100
views
Insights betwwen two columns/variables in Dataframe
I have data in two columns one is range of old credit score (Input score range) and new credit score (cvsc100). How do i find insights from both of them ? where the old is range of values and other ...
- 9
0
votes
1
answer
41
views
(Basic) statistics
I got a misunderstanding regarding basic statistics (I think) but I can't get my head around that:
I did an online survey regarding the usage of some application. The user could answer 1 (I don't know ...
1
vote
0
answers
39
views
How to handle and report patient characteristic statistic with missing data in essay?
I'm now working on an clinical trial essay with some doctors. When generating patient characteristic statistic form, I found basic patient characteristic data has some missing.
The missing datas are ...
- 11
0
votes
1
answer
141
views
How to interpret axis of Histogram and distribution curve?
After doing statistical analysis of data I got the below figures:
The data contains a line length in meters which is represented with Line(m). The purpose here is to confirm the distribution behavior ...
- 2,487
1
vote
0
answers
31
views
Seeking advice: How would you define "top performer" with a set of data?
still consider myself a noobie in ds here, so I may be asking a very easy and stupid question, but would really appreciate some guidance.
The hypothetical situation is that, you have a website where ...
