Skip to main content

Questions tagged [descriptive-statistics]

Descriptive statistics summarize features of a sample, such as mean and standard deviations, median and quartiles, the maximum and minimum. With multiple variables, may include correlations and crosstabs. Can include visual displays - boxplots, histograms, scatterplots and so on.

Filter by
Sorted by
Tagged with
0 votes
1 answer
49 views

What is the best practice to detect constant and nearly constant data over time?

I'm experimenting with the characterization of data over time. I generated some synthesized data with certain periodic patterns over time every 5mins (granularity of 5mins= data generated with the ...
Mario's user avatar
  • 400
1 vote
2 answers
60 views

Is there a measure to compare features on the basis of normality

I have a dataset of cars and it has many features including 'acceleration’, ‘horsepower’, and ‘mpg'. I am supposed to check which of these features is the most similar to a normal distribution, so I ...
Tareq Ewaida's user avatar
0 votes
1 answer
46 views

what are the best sources to learn about statistics?

am interested in quant analytics. hope statistics is heart for quant analysis. so, what are the best sources to learn core statistics (in depth) and what is the best approach to learn statistics?
tovijayak's user avatar
1 vote
2 answers
291 views

How to find a vector representation for each descriptor?

Cubes data is well known data for extreme classification. Each picture has a set of descriptor along with it. Total data set has 312 descriptor. You will find list of descriptior in this file. My ...
A. H.'s user avatar
  • 61
0 votes
1 answer
28 views

Best practise XAI: understand features which build up cluster and explain underlaying structure

I want to cluster my data and show which features were used to define the clusters to show the structure in my data. To explain the use case: Imaging I have data from many products and I want show the ...
soph's user avatar
  • 113
15 votes
5 answers
2k views

How can I ensure anonymity with queries to small datasets?

I'm building a service that will contain personal data relating to real people. Initially the dataset will be quite small, and as such it may be possible to identify individuals if the search ...
mal's user avatar
  • 253
1 vote
1 answer
49 views

Which test statistic do you recommend for checking the difference between a dynamic for some parameter in control and treatment group?

I have two groups of patients, a control (A) and a treatment (B), and data for patients' weight in the beginning and in the end of a period of treatment in each group. I need to check if the dynamic ...
Vlad's user avatar
  • 111
1 vote
1 answer
21 views

Are there any algorithm to generate a set of data that match some statistic requirements?

I was wondering if there are time-efficient algorithms that can reverse the process of basic statistics computation. What I mean is an algorithm that instead of computing the mean, SD, max-min range, ...
Anselme's user avatar
  • 13
0 votes
1 answer
121 views

What is the difference between accuracy in percentage (classification metric) and MPE (regression metric) as evaluation metrices?

What is the difference between accuracy in percentage (classification metric) and MPE (regression metric) as evaluation metrices?
Mehmet Gökdelen's user avatar
0 votes
2 answers
316 views

Missing Values Correlation

Is it worth it to study missingness correlation between columns? If you have strongly correlated missing values (say between two columns, A and B), how will this change or shape the way you look at ...
dbzadnen khiari's user avatar
0 votes
1 answer
42 views

How to count words in a dataframe?

I would like to count how many Male and female who answer (ex. Biking / Cycling). Below is the sample data:
Jasmine's user avatar
0 votes
1 answer
244 views

PSI where not to use

From what I understand PSI is used for continuous data. Generally, equal sized bins are created to compare two data set, and number of buckets is usually 10. Is that for a reason, why 10 bucket? Also, ...
tndr's user avatar
  • 1
1 vote
0 answers
10 views

Trying to get an idea of decile system

I was checking a Linkedin profile and I came across this statement in the “Grade” section. It’s a little confusing. It is as follows: “Grade: gpa within the first decile”. Is this statement correct? ...
NaveganTeX's user avatar
1 vote
1 answer
11 views

what is a collective term for things as screening failures, no response and drop-outs

Hello wondering what is a collective term (in terms of statistics) for things as screening failures, no response and drop-outs?
freshman_2021's user avatar
1 vote
0 answers
57 views

Null and 0 Values Handling - Basic Statistics [duplicate]

I am trying to do some descriptive statistics and basic graphs on relatively small data sets derived from environmental sample collection and/or chemical research. It turns out that sample points end ...
Valy's user avatar
  • 111
0 votes
2 answers
574 views

A measure of redundancy in mutual information

Mutual information quantifies to what degree $X$ decreases the uncertainty about $Y$. However, to my understanding, it does not quantify "in how many ways" $X$ decreases the uncertainty. E.g....
user1767774's user avatar
1 vote
0 answers
74 views

Estimating the (normal?) distribution of the grayscale values of a region in camera footage

The problem: Given a small ($25\times25$) region on a $1080$p image extracted from camera footage of a railway system, I am trying to detect whether there is a train present or not. I proceed to take ...
John's user avatar
  • 111
1 vote
1 answer
39 views

Applying an algorithm on it's own training data for descriptive purposes?

Good Day, I am newer to data science so I am not confident in this. To set up the question I will describe my data and approach. Data I don't want to share specific data examples as I want to try and ...
Reily Bourne's user avatar
1 vote
1 answer
57 views

Understanding math notation in infoGAN paper

I'm reading this paper about mutual information in infoGAN infoGAN_paper_link and already have the code to run it. I pretty much found code for it which is fine and dandy except for the fact that I ...
zipline86's user avatar
  • 359
1 vote
1 answer
3k views

Calculating correlation for categorical variables

I am struggling to find out a suitable way to calculate correlation coefficient for categorical variables. Pearson's coefficient is not supported for categorical ...
Ricky's user avatar
  • 189
1 vote
1 answer
60 views

What is the best way to cluster this kind of data?

I have data that looks like this: The chart on the left is the trend and the smaller chart on the right is the box plot showing the distribution of means. Each color is the output of a particular ...
thentangler's user avatar
1 vote
1 answer
119 views

Accuracy over different sample sizes from dataset

What I'm trying to do is predict how much more data would help in a classification task. So, what I'm doing is bootstrapping entries in my dataset to get a sample, with a specified size. Then, I fine-...
MartinM's user avatar
  • 308
2 votes
2 answers
3k views

p-value and effect size

Is it correct to say that the lower the p-value is the higher is the difference between the two means of the two groups in the t-test? For example, if I apply the t-test between two groups of ...
HelpNeederStudent's user avatar
1 vote
0 answers
38 views

paired t-test shows no difference between median and Wilcoxon test p value shows that there is a difference between median values ? How to interpret?

I have a dataset. I wanted to do paired t test on it. So I carried out normality test and it showed that it does not follow normal distribution. So I used Wilcoxon test in place of paired t test. The ...
pinky's user avatar
  • 151
1 vote
1 answer
119 views

How to bin a distribution data reported with different frequencies ( salaries ), showing mixed linearity and non-linearity?

I am researching on pay-scales, and wish to receive advise to treat data of salaries. Objective My interest is to approximate the salary corresponding to different hierarchical levels in an ...
user305883's user avatar
0 votes
0 answers
36 views

Data calculation/normalisation /data summarisation

Clustering is done based on houses data and say suppose cluster 1 contains 5 houses. Appliance 1 is used in 2 houses out of 5 and appliance 2 is used in 4 out of 5 houses.Each appliance produces an ...
User06's user avatar
  • 9
2 votes
0 answers
26 views

Statistical method to find the value which preserves the most information inside "most" of data points. (resize images to a common height)

So I have this data of around 88K images and I found out some interesting properties for my images. ...
Deshwal's user avatar
  • 323
1 vote
1 answer
39 views

How to get a (descriptive) overview of a large database?

I'm facing a data framework with ~ 20 k observations and 151 variables across 2078 subjects At first I am primarily interested in how the data looks like related to a single parameter. But I cannot ...
Ben's user avatar
  • 550
2 votes
3 answers
1k views

A/B Testing (Binomial Distribution vs Random Distribution)

When performing an A/B test for the number of clicks for users viewing (each view is an impression) two variants of an ad, a binomial distribution can be assumed where each variant has a constant ...
DD.'s user avatar
  • 121
1 vote
0 answers
20 views

Normalise the scores by small and large numbers division

I am working on a scoring problem use-case where I will score each task based on few aspects and take an average of scores of tasks in each experiment. For each experiment I will have 1 to N number of ...
Jack Daniel's user avatar
1 vote
1 answer
738 views

Can a Box plot be used for finding the useful features from the dataset? [closed]

I am reading a book by professor Trevor Hastie and professor Robert Tibshirani called "Introduction to Statistical Learning". In the applied section of the chapter 4, there is a question 11(...
Adarsh Namdev's user avatar
0 votes
1 answer
41 views

Analysis of passed/not passed students by enrolled year

I am analysing some data from my University about successful students through years. The dataset looks like as follows ...
V_sqrt's user avatar
  • 295
2 votes
1 answer
150 views

Problem regarding calculating correlation approach?

There are 3 columns say product_id, product_type, and price_drop. ...
Vipul Jain's user avatar
2 votes
3 answers
440 views

How do I interpret the output of linear regression model in R?

I have the following linear regression model and its analysis. There are a few errors, but I am not very sure about the errors. I have not succeeded in finding them so far. First, the 95% confidence ...
Jayden Rice's user avatar
0 votes
2 answers
557 views

Calculating Median and Mode of numerical variables for different subgroups in R

I have customer call data and I want to get the median and mode for the call success rate for different subgroups. My variables are: Customer ID, Employment Status (Retired, Employed, Unemployed), ...
ColRow's user avatar
  • 53
3 votes
2 answers
652 views

Distribution vs Histogram [closed]

I am not very good at understanding statistics jargon I tried to read articles on statistics every article says distribution and show a picture of bell curve. I understand that x axis of bell curve ...
Hitesh Somani's user avatar
0 votes
1 answer
14 views

Getting the earliest date (duplicates due to several call ids and agents)

I am trying to get the earliest call interaction, with only the agent that interacted at that call and the customer id. Using the following code, I still get duplicates: ...
ColRow's user avatar
  • 53
0 votes
0 answers
32 views

Exploratory statistics, how to idenify and remove driver (bias)

I am looking at customer data, and created frequency tables (+histograms) for customers with different professional statuses and what the best time is to reach them. Status ranges here from employed, ...
ColRow's user avatar
  • 53
1 vote
0 answers
17 views

Coefficient of variance problem

I am doing a project and I am facing a problem I have no idea how to solve. I have a data series of stock prices, and, based on this series I obtained another series, of so-called rentabilities (which ...
ada109's user avatar
  • 11
2 votes
1 answer
107 views

Drastic drop in Somers' D ? Why?

I came across to find the correlation between the ratings assigned by two coaches to a same group of 40 players. I have tabulated the results as below: The Somers' D is 50%. However, for the case ...
CoolJohnTo's user avatar
0 votes
1 answer
109 views

Insights betwwen two columns/variables in Dataframe

I have data in two columns one is range of old credit score (Input score range) and new credit score (cvsc100). How do i find insights from both of them ? where the old is range of values and other ...
marco's user avatar
  • 9
0 votes
1 answer
41 views

(Basic) statistics

I got a misunderstanding regarding basic statistics (I think) but I can't get my head around that: I did an online survey regarding the usage of some application. The user could answer 1 (I don't know ...
Klaus Ebner's user avatar
1 vote
0 answers
39 views

How to handle and report patient characteristic statistic with missing data in essay?

I'm now working on an clinical trial essay with some doctors. When generating patient characteristic statistic form, I found basic patient characteristic data has some missing. The missing datas are ...
Jensen Fan's user avatar
0 votes
1 answer
143 views

How to interpret axis of Histogram and distribution curve?

After doing statistical analysis of data I got the below figures: The data contains a line length in meters which is represented with Line(m). The purpose here is to confirm the distribution behavior ...
Muhammad Ali's user avatar
  • 2,487
1 vote
0 answers
31 views

Seeking advice: How would you define "top performer" with a set of data?

still consider myself a noobie in ds here, so I may be asking a very easy and stupid question, but would really appreciate some guidance. The hypothetical situation is that, you have a website where ...
alwaysaskingquestions's user avatar
3 votes
2 answers
47 views

evaluation metrics for multiple values per session

I have an application that executes my foo() function several times for each user session. There are 2 alternate algorithms that i can implement as "foo" function and my goal is to evaluate them based ...
sbr's user avatar
  • 131
7 votes
2 answers
258 views

Which statistical test tells which classifier performs better than the other?

I have 3 classifiers: A, B and C. According to accuracy, specificity, sensitivity, f-score, and g-mean, say classifier B performs best. Now I want to statistically validate this claim. How should I do ...
girl101's user avatar
  • 1,161
2 votes
1 answer
82 views

Generating random numbers from best probability distributions?

I have done some statistical distributions on real-world data. The distribution fitting gives me the below results (lognormal fits the best on data based on chi-squared test): ...
Muhammad Ali's user avatar
  • 2,487
0 votes
4 answers
849 views

What do you call the ratio of positive to negative samples?

I am working with a binary classifier and I want to express the "balance" or "skewness" of the training data using a metric. I want to reflect this ratio in a report, like this: ...
Eric McLachlan's user avatar
0 votes
1 answer
102 views

Analysis for basic weight training analysis?

TL;DR: I'm doing a fairly basic project which involves exercise. It seems that descriptive statistics and basic data vis (ex: line graph) would be most appropriate for this project, but I wonder if ...
stevemn's user avatar