# Questions tagged [statistics]

Statistics is a scientific approach to inductive inference and prediction based on probabilistic models of the data. By extension, it covers the design of experiments and surveys to gather data for this purpose.

1,137 questions
Filter by
Sorted by
Tagged with
18 views

### Interpreting the variance of parameter estimates in linear regression

I am reading through ESL and came across this equation (3.6) where the variance of the parameter estimates are provided as $$Var(\hat{\beta}) = (X^TX)^{-1}{\sigma}^2$$ I can understand the ...
4 views

### Disaggregation algorithm that maintains mass balance?

I am working with monthly timeseries data that I need to represent as daily data. I am looking for a disaggregation algorithm or Python/R package where: Daily values smoothly increase from one month ...
• 101
1 vote
20 views

### Correlation of groups together or separately?

Apologies. I am trying to format the table but I am unable to get it in the right format. I have a tabular time-series data that I am working with and have questions on how to calculate correlation. I ...
• 11
6 views

### Tackling an automatic labeling (optimization) + classification problem

I'm aware this isn't quite programming-focused, but I did not know any other place to ask this question. It's more about approach than a technical issue. Context I have an optimization + ...
• 101
25 views

### How to understand if a model-algorithm is a machine learning ones or not?

I'm working on thesis to detect change points in a timeseries made of body movements. Im forced to not use any Machine Learning models because my colleague used ML and the professor wants to have a ...
8 views

### statistical test of cell type proportions in clusters

I have a compostion dataframe which contains spotwise compositons of cell types and their cluster informations. I am required to compare the mean abundance of cell types in each cluster with other ...
20 views

### 100% Testing Accuracy on Small Image Testing Set

I'm a little worried because I'm getting 100% accuracy on my test set. The test set is relatively small (16 images) because the dataset overall is very small (40 images), and I assume that contributes....
5 views

202 views

### Time Series Analysis and Price Elasticity

Introduction: As of now, I am a fourth year data science student. As of now, I also have my own company where I work parttime (8/12 hours per week) to gain some more experience in the domain. As you ...
• 21
22 views

### Question on theory from original GBM article

I am reading the original gradient boosting machine article and, maybe because my statistics are a bit rusty, have a few questions on one section. In section ...
• 11
21 views

### Continuous Function from Binned Data with Consistent Integrals

I have binned energy production data (in Wh) in 5-minute intervals. This means that, at the timestamp of t + 5 minutes, the value represents the amount of energy generated in the interval from t to t +...
30 views

### Generating a Sample T-Distribution in Python

I'm new to DataScience. I'm going through the stats module and the ways to create a sample T-distribution. Using the below line to generate the sample. However, my question is how is the array ...
1 vote
9 views

### How to calcualte LR_statistic for the below data

I am trying to calculate LR_statistic for two models where my log-likelihood ratio values as follows. model1.llf = -2326.9 and model2.llf = -4649.1. When read ...
1 vote
47 views

### Am I suppose to do any statistical test on results?

I am doing an email campaign. Before sending emails to users I divided my user base into treatment and control groups (50-50). Divided the groups in such a way that no difference in user behaviour. I ...
35 views

### Best practices for handling "NA" when all NA values exist due to being below the limit of detection?

I am working in R, and have a data set which has a few metabolite concentration values as continuous variables. Anywhere that the concentration was too low to be detected it simply says <LOD. This ...
• 21
1 vote
36 views

### Find out Statistical test for binary data

we are ran an email campaign for users which is called my treatment and there is control group as well. We want to measure the performance using a Metric called A. If he belongs to metric A then we ...
16 views

### Can an OCR model consistently recognize every digit of a long number correctly?

I'm working on OCR on scanned documents and we need to recognize the exact sequence of some printed numbers on it. Imagine you're reading a bank cheque serial number (16 digits) so the system needs to ...
1 vote
34 views

### Standard Error interpretation help

I performed a standard error on my machine learning model to predict protein structure. The graph Im showing here is a snippet of the actual data and I deleted some irrelevant info. The y axis is the ...
34 views

### How to Determine the Minimum Value of a Continuous Variable for Predicting Categorical variable using Logistic Regression?

I am using logistic regression to predict df['MortSubiteCardiaque'], which contains 0 and 1, based on my continuous variable df['NTProBNP']. I would like to determine the threshold for df['NTProBNP'], ...
20 views

### pyspark combined standard deviation?

I have a data frame which consists of multiple standard deviations with mean and count of each. my df is as follows ...
• 101
16 views

### Creating a project based "level of effort remaining" visualization for project tracking and management

I am looking to create a tracker to help manage the current and future progress of a large engineering project. The project is made up of many subprojects each with their own tracker. This is an excel ...
• 1
14 views

### How to find the parameters of custom regression in python code?

I want to make a time series regression that has a custom formula lets say: Yt=Beta+1/(1+(some parameter sigma))Y(t-1)+(1/(1+(some parameter sigma))**2)Y(t-2)+Beta1X(t-1)+Beta2X(t-2) how to code this ...
740 views

### Why does data science see class imbalance as a problem for supervised learning when statistics does not?

Why does data science see class imbalance as a problem in supervised learning when statistics says it is not? Data science seems to seem class imbalance as problematic and needing special techniques ...
• 4,044
26 views

### Looking for a Statistical Modelling Technique for a Credibility Scoring Model

I’m in the process of developing a model that assigns a credibility score to fatigue reports within an organization. Employees can report feeling “tired” an unlimited number of times throughout the ...
• 261