Questions tagged [descriptive-statistics]

Descriptive statistics summarize features of a sample, such as mean and standard deviations, median and quartiles, the maximum and minimum. With multiple variables, may include correlations and crosstabs. Can include visual displays - boxplots, histograms, scatterplots and so on.

Filter by
Sorted by
Tagged with
1
vote
0answers
28 views

Seeking advice: How would you define “top performer” with a set of data?

still consider myself a noobie in ds here, so I may be asking a very easy and stupid question, but would really appreciate some guidance. The hypothetical situation is that, you have a website where ...
1
vote
1answer
19 views

evaluation metrics for multiple values per session

I have an application that executes my foo() function several times for each user session. There are 2 alternate algorithms that i can implement as "foo" function and my goal is to evaluate them based ...
7
votes
2answers
96 views

Which statistical test tells which classifier performs better than the other?

I have 3 classifiers: A, B and C. According to accuracy, specificity, sensitivity, f-score, and g-mean, say classifier B performs best. Now I want to statistically validate this claim. How should I do ...
0
votes
0answers
8 views

How to summarize/describe feature trends with fix number of scalars?

I'm trying to make feature extraction from features with numbers over different timespans (like stock values movement) and I want to encode these trends into a fix number of descriptive scalars. I ...
2
votes
1answer
22 views

Generating random numbers from best probability distributions?

I have done some statistical distributions on real-world data. The distribution fitting gives me the below results (lognormal fits the best on data based on chi-squared test): ...
0
votes
0answers
9 views

What is an appropriate statistical method for determining correlations between variables with likert scale data?

I am conducting an exploratory analysis with a dataset of about 200 records (all likert scales (numeric)). I aim to determine the correlations between the different variables that the responses ...
0
votes
4answers
80 views

What do you call the ratio of positive to negative samples?

I am working with a binary classifier and I want to express the "balance" or "skewness" of the training data using a metric. I want to reflect this ratio in a report, like this: ...
0
votes
0answers
10 views

Wavelet transform to identify 'meaningful' data changes

Can I use the wavelet transform to find the 'meaningful' changes in a series of 1d data? For example I have a series of integers representing angles of line segments. The majority of these values are ...
0
votes
0answers
34 views

Data Analysis on normally distributed salary with Outliers

I have created normally distributed data with 2 outliers ...
0
votes
1answer
16 views

Analysis for basic weight training analysis?

TL;DR: I'm doing a fairly basic project which involves exercise. It seems that descriptive statistics and basic data vis (ex: line graph) would be most appropriate for this project, but I wonder if ...
0
votes
3answers
35 views

Aggregate several attributes into one using PCA

I am working on a soccer dataset. As we all know, soccer has a lot of different metrics related to the game. We have Key passes, Accurate Passes, Shots on Target, Saves, Clearances and so on... I have ...
0
votes
1answer
20 views

Normal distribution and QQ plot

I have data that I plotted with a normal distribution and a QQ plot. I was wondering especially in QQ plot it seems that 95.4% of the data is normally distributed. My question is what does numbers ...
1
vote
1answer
70 views

Skew and Kurtosis are so similar?

I've been taking the histogram of the optical flow in videos and plotting the kurtosis and skewness of each frame. At the end of the video, I noticed that the skewness and kurtosis follow each other - ...
0
votes
0answers
23 views

t-test with a repeated variable

I want to conduct the t-test on a data I have to check if there is a significant difference in the "fear" emotion between valid and ...
0
votes
0answers
8 views

Analysis on the 5 segment of the data to differentiate from each other using R

I have 1000 customers operational data which is already divided into segments. Now I have to do analysis on different segments to find out how they differ from each other. Why they fall into their ...
0
votes
1answer
32 views

Is it possible to estimate

For example, if I want to know a page views/relative prevalence of a Reddit post, and I have the upvote and downvote data. Is it valid that I use the upvote and downvote to predict that, even if I ...
0
votes
0answers
17 views

Best technique for testing if an event causes a trend?

Example - A major holiday is coming up and the frequency of a product being purchased starts increasing significantly as it gets closer to the day of the holiday. Then after the holiday, the frequency ...
0
votes
0answers
31 views

How to graphically present the statistical interpretations when using Kolmogorov–Smirnov test (K-S) test?

I am referring to my previous question (Any suitable way to describe the distributions of 2 Pandas Dataframes visually/graphically?) related to visualising the Pandas dataframes to show that they have ...
0
votes
1answer
15 views

Looking for help calculating a probability formula

How do I put this into a calculator or a excel spreadsheet formula. I have never done this math before, but I want to figure it out.
7
votes
5answers
2k views

When to use mean vs median

I'm new to data science and stats, so this might seems like a beginner question. I'm working on a dataset where I've user's Twitter followers gain per day. I want to measure the average growth he had ...
0
votes
1answer
15 views

What is the first tool to learn start your your data science projects? [closed]

What is the first tool to learn start your your data science projects?
3
votes
1answer
40 views

How do I handle string feature while performing model generation

I have data which looks like this ...
1
vote
1answer
105 views

Binary Classification without machine learning tools

I have to write a binary classifier for my company that should be as simple as possible and doesn't use machine learning libraries (and I also should not code too sophisticated algorithms by myself). ...
0
votes
1answer
133 views

A dataset has skewness = 1 with missing data. Standard deviation around median is 1.5. How much data will be unaffected?

There is no other description about the data, if it is univariate, bivariate, etc. neither the type of distribution is given. I recently came across this question, I would like to know how skewness ...
1
vote
1answer
60 views

Does Box plot with many outliers effects the result?

I am using Haberman's cancer survival dataset https://www.kaggle.com/gilsousa/habermans-survival-data-set to draw a box plot. Here Surv_status is the target variable which has two classes and ...
0
votes
1answer
3k views

What Does the Normalization Factor Mean in the AdaBoost Algorithm?

I am studying the AdaBoost algorithm. The update rule for a weak hypothesis is: $Dt+1(i) = Dt(i)exp(−αtyiht(xi))/zt $ where $zt$ is a normalization factor chosen so that $Dt+1$ is a distribution. ...
3
votes
1answer
361 views

Churn definition for non-contractual services

I am computing churn definition i.e, the number of days after which we will say a customer has churned in fashion retail, etc. Currently, I am using the transaction dates to get the average days ...
2
votes
0answers
197 views

Is Inferential and Descriptive Statistics for Machine Learning? [closed]

I am starting with Machine Learning and many people suggested that I should first start with Descriptive and Inferential Statistics on Udacity. I started with it. But I have a doubt that is it ...
0
votes
1answer
86 views

Comparing data sets with different measurements

I'm currently writing a thesis based on Cyber Crime, however I'm unsure of the proper to compare/analyse my data sets to talk about them in my thesis. One piece of data (https://www.pandasecurity....
3
votes
1answer
58 views

Statistically Robust Distance Measure/Metric for comparing more than two network data series

I have about 30 lists of unequal length (some of which are triplicates of the data), corresponding to metrics relating to nodes of different graphs. I want to compare their similarity using a distance ...
1
vote
1answer
653 views

Difference between Blocking and Clustering?

What is the difference between blocking and clustering? As far as I know clustering originates from the Machine Learning branch and refers to methods of grouping data points together in a "We and ...
0
votes
1answer
112 views

XGBoost: predictive, descriptive (or both) model?

I have trained an XGBoost model for prediction. The algorithm is able to calculate variable importances. I was asked why I have not analyzed these variable importances and I did not because as I ...
1
vote
1answer
1k views

How to merge columns, value count them and then plot the results?

How do I get from a dataframe with multiple columns that have similar values and need to be merged: ...
1
vote
1answer
44 views

How to select a model based on statistical analyses?

I've been statistically validating the performance of different Deep Learning models in classifying parasitized and normal cells. In the process, I could find that there is no statistically ...
3
votes
1answer
91 views

Making Use of the Target Values for Regression

Problem: I have a regression problem and I decided to useg Gradient Boosting Regression Trees to solve it. After all the preprocessing, I end up having around 130 attributes, 70K rows, and my cross-...
2
votes
0answers
53 views

Errorbars on a weighted average (of a non gaussian distribution)

I have some datapoints $x_i$ and errors $\sigma_i$ which are the absolute value of some datapoints $w_i$ with (supposed) gaussian errorbars (the $\sigma_i$). The $x_i$ are amplitudes so we did not ...
4
votes
1answer
80 views

Range to define emotions

We are capturing emotions as survey responses. We need to assign values for the responses(emotions) for analysis purposes. Is there an optimum range that can be assigned to achieve this? (like from -...
1
vote
0answers
2k views

analysing stability of a measurement over time

I have some sensor data (actually many different sensors about 90 of them) and I have their readings over some time course. I was wondering on a couple of things: 1: What metrics can I use to assess ...
2
votes
1answer
103 views

Re-bucket weekly sales data and calculate descriptive statistics

I've got sales data in weekly buckets like this: ...
0
votes
1answer
96 views

Is there any other probability distribution model than gaussian for multivariate data

Whenever we talk about the probability distribution of data having more than one feature, we have only one option i-e multivariate normal distribution. Is there any other probability distribution ...
-1
votes
1answer
114 views

Difference between summation and integration [closed]

I understand calculus and maths but when i apply statistics and add up numbers they both look kinda same Can anybody explain the difference in a little detail and simple manner please
1
vote
0answers
86 views

Interpreting statistical formulas using python (mu + sigma * data)

I am working with data using python. I tried to plot a histogram using mathplotlib. I found a few examples on how to analyze data to get more accurate graphs. So ...
1
vote
0answers
68 views

How do you determine if a value is statistically significant?

I have collected some data I need to analyze. The data is the result of a survey in which I asked approx. 180 sellers at a bazaar, how important a certain buyer's characteristic is in relation to ...
1
vote
2answers
77 views

Indepedent t-test result interpretation

I am working on 2 sample independent t-test. I have conducted analysis on test group vs control group and I have to write a report but I have few questions. 1) Do we have to take out the outliers ...
4
votes
2answers
1k views

How to measure the performance of an imputation technique

I would like to know how I can measure the performance of an imputation technique. I have read a lot about this. Most literature on the web are applying a classifier after the data has been completed. ...
2
votes
0answers
108 views

Adjusting predicted values based on average offsets from past predictions

This question is more about inference and decision making based on statistical data, so I hope I'm in the right place. If not, let me know if I should migrate this question elsewhere. I have a data ...
4
votes
0answers
1k views

How does Seaborn calculate error bars when using estimators other than the arithmetic mean?

If I create a barplot using Seaborn and specify the geometric mean or the median as the estimator, does Seaborn know to use the appropriate standard error formula to create error bars?
4
votes
2answers
132 views

Approach to creating a user profile in music web application

I am working on a use case, and I'm unsure of the best way to proceed: in order to analyze the behavior of users of a web-based music application, we retain all songs each has played since 2009. We ...