Questions tagged [hypothesis-testing]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
40 views

what is the genesis of the term - the clinical significance?

The literature suggests that p-value represents "clinical" significance. (p-value, is a number whose value can range from zero to one.)To me,p-value does not measure clinical significance....
Subhash C. Davar's user avatar
1 vote
1 answer
63 views

Which statistical procedure to use?

I'm new to Hypothesis testing. I can't really think of how to answer this - Five students take a test one year and another test the next year. Their overall grades(in %) are below for both courses. ...
a_jelly_fish's user avatar
0 votes
0 answers
19 views

AB testing: Control was performing 0.5% better than experiment set before the initiation of experiment

So we introduced a new feature in our app, that would aid conversion (hypothetically). When i tried to measure this incremental change in conversion, i split my base set of customers into control(C) ...
Narahari B M's user avatar
0 votes
0 answers
14 views

Testing Marketing vehicle effectiveness

I have a set of customers who have been targetted using a marketing vehicle. Now i want to estimate if the marketing vehicle resulted in a better conversion (did they buy after that or not). Currently,...
Gokul Anand's user avatar
1 vote
0 answers
24 views

Is there a statistical test for comparing my dataset percentages with published percentages?

I have a clinical dataset which is showing the percentage of A is 57% and B is 35% for where an tumours is initially found. However the literature is showing it is A is 73% and B is 23%. Is there a ...
Matt Smith's user avatar
0 votes
0 answers
7 views

Feature Selection using Statistical Testing on Features with Different Scales

If I want to select features based on ANOVA test for example, should I scale features to same/similar range so the results can be compared , or is it unnecessary?
jxqbbb's user avatar
  • 51
1 vote
0 answers
31 views

Can you do a power analysis to determine the sample size for a virtual species simulation which is modeled using random forests?

I am simulating virtual species on a 100x100 grid (the size for now). Each grid layer represents one environmental variable. The "suitability function" defines the probability of a presence ...
Kyle's user avatar
  • 111
2 votes
1 answer
26 views

What can we do in order to find customers that follow a specific pre-defined pattern?

Let's say I have a few customers buying product A. How can I find other customers that have the same characteristics, or something close to, these A buyers? I was thinking about using clustering ...
Aldla E Aoepql's user avatar
2 votes
1 answer
88 views

How to present a statistical justification for the choice of models with approximate accuracies?

In an experiment involving the comparison of classification algorithms, how can I assess whether there are statistically significant differences between the analyzed models? For example, the following ...
Paulo Sergio Moreira's user avatar
0 votes
1 answer
32 views

The meaning of P and degree of freedom in T-Test

I read about T-Test and how we can use it to compare between 2 models (https://towardsdatascience.com/paired-t-test-to-evaluate-machine-learning-classifiers-1f395a6c93fa) There are some issues I'm not ...
user3668129's user avatar
0 votes
0 answers
27 views

Paired T-Test to assess validity of Algorithm Selection after Nested Cross Validation

I am interested in selecting the best Algorithm (SVM, XGBoost, Random Forest, ...) for a classification problem. At present, my company is using an algorithm (let's call it ALG-A and let's say it's ...
AngelMarcos's user avatar
0 votes
0 answers
20 views

A/B test results don't hold in hold-out dataset

I'm working on a project consisting in analyzing some game data. The game consists of two players competing between them. The result of the game is either win or lose. Players can choose from ...
alexmolas's user avatar
  • 536
0 votes
1 answer
23 views

How to demonstrate two variables are orthogonal with respect to the output in a 3-D Python dataset?

I have a Python dataset with 300 samples and 3 columns: 2 independent integer variables X,Y and the dependent continuous variable ...
Adrián Pérez Diéguez's user avatar
0 votes
0 answers
30 views

Why p-value larger than level of significant should accept the hypothesis?

I hope someone explains this question to me like I'm ten. I already know why (sample mean - population mean) < critical value should accept the hypothesis.
sss's user avatar
  • 13
0 votes
0 answers
32 views

Kolmogorov-Smirnov hypothesis test

I am using this package to test the similarity of two distributions. It says: The one-sample test compares the underlying distribution F(x) of a sample against a given distribution G(x). The two-...
just_learning's user avatar
0 votes
0 answers
11 views

hypothesis testing for difference between groups of embedding vectors

In a machine learning application, I have two generators G₁ and G₂ that generate descriptions of some input z in the form of embedding vectors X and Y: Xᵢ = [x₁, x₂, ...] = G₁(zᵢ) Yⱼ = [y₁, y₂, ...] =...
Stib's user avatar
  • 1
1 vote
1 answer
17 views

Is systematic sampling a good way to fairly assess a baseline against a post experiment results group?

Apologies if this is in the wrong place. I'm a data analyst thats finding work is driving more and more into into marketing experiments and testing and just needed some help. Lets say I launch a new ...
Arcadeia's user avatar
0 votes
1 answer
68 views

"Chi-squared approximation may be incorrect" on hypothesis test

I'm doing a hypothesis test to determine if the data follows binomial distribution with p=0.31, and I'm getting a warning when running this: ...
AlePouroullis's user avatar
0 votes
1 answer
34 views

Testing the impact of events on time series

Context I am working with product data for a retail company. I have the daily impressions (number of times it was viewed online) for all products over a 30 day period (can get more data). Here is the ...
OrangeChutney's user avatar
0 votes
2 answers
86 views

How to check the impact of predictive Model

Suppose that I build an email marketing campaign where the goal is to predict which members will make purchases or not and based on that change the marketing email campaigns. After we build the model, ...
Ramin Salimi's user avatar
1 vote
2 answers
50 views

Validate Unsupervised Binary Classification

I’m working on a fully unsupervised anomaly detection problem. Since it’s completely unsupervised, I’m having hard times in defining some metrics to kind of validate the results (I run several ...
fpialcoi_o's user avatar
1 vote
1 answer
65 views

How to test the confidence for a rule based system?

I have a multi-class dataset and I generated based on it rules. That is, if certain features are seen then it must be a certain class. I chose only rules with precision 1 (with respect to the whole ...
greenButMellow's user avatar
2 votes
2 answers
49 views

Which method of correlation is appropriate for two paired lists of numbers?

I have a program which produces an image, and I use a metric to understand how accurate that image is. I choose five cases (A, B, C, D, E), and make a list of the accuracy metric for each case: ...
user745587's user avatar
1 vote
1 answer
48 views

How to test likelihood hypothesis on dataset?

How to test the following hypothesis? The larger the fare the more likely the customer is to be travailing alone. Using the data below, how would one be able to test the hypothesis? ...
IOIOIOIOIOIOI's user avatar
0 votes
2 answers
86 views

Evaluating if metric of one group is higher than the metric of another group when group sizes differ significantly

I am working with a dataset that contains data of applicant income, gender, and loan status (whether or not the person was approved for a loan). I've created the following plots from the data. The ...
Erik M's user avatar
  • 73
1 vote
1 answer
1k views

T-test against normalised or standardised data gives different results

I am studying the problem to predict popularity of a tweet, and want to test null hypothesis: there is no relationships between favorite_counts and another set of variables, like number of friends of ...
user305883's user avatar
1 vote
0 answers
62 views

statistical tests for null hypothesis - what if model is non linear?

I am reading the "An Introduction to Statistical Learning" (Gareth James & alii, Springer) as a primer to machine learning. I am reading the part in linear regressors, and learnt there ...
user305883's user avatar
1 vote
1 answer
92 views

Why do I get this result with a chi- square test?

I have a question about the chi squared independence test, I'm working on dataset and I'm interested in finding the link between the categories of product and the gender, I plot my contingency table. ...
Polaster's user avatar
1 vote
1 answer
26 views

How to test whether data is clustered wrt. subcategories?

I have a dataset of about 2000 entries, containing two numerical values, one categorical and one sub-categorical label for each entry. The data is from chemistry lab data, but for the purpose of this ...
user avatar
1 vote
0 answers
18 views

What kind of model to use to find drivers when data is aggregated and not on user level?

I have a website and have info from Google Analytics. So I can see the following "features": page url country device category (phone, desktop, etc.) Number of sessions Number of users: ...
user126224's user avatar
1 vote
0 answers
104 views

I am not sure if I used the right hypotheses test or made the right conclusions, could you give me a feedback?

I have just finish a basic statistic course and I decided to apply what I learned in a small project using the wine quality dataset. I am not sure if what I did was correct, so if possible, I would ...
panchester's user avatar
1 vote
1 answer
131 views

Cross-validation split for modelling data with timeseries behavior

Background: I have a dataset that is generated every month (it is similar with card data that contains card demography and transactions every month and new accounts can be added in the middle of data ...
zonna's user avatar
  • 73
1 vote
1 answer
46 views

What is the difference between a hypothesis test made with a normal distribution and a one sample t-test? And when to use each one?

I am studying statistics on my own and I am not understanding when to use a one sample t-test or a test with a normal distribution. As I understand, both are comparing a population mean with a sample ...
panchester's user avatar
0 votes
1 answer
72 views

How do I conduct an experiment on the new pricing if it's impossible to conduct an A/B test?

We want to introduce a new price list for the customers of our international SaaS company. Beforehand we want to test this new price list in several countries. A/B test cannot be conducted here ...
Никита Юсупов's user avatar
1 vote
1 answer
43 views

Check if distribution per week is the same

I have sales by customer (b2b) and by date. I want to check if the distribution per day inside weeks remains the same from week to week. Initial dataset Customer Date Sales Alpha 2019-02-23 527 ...
Ismail's user avatar
  • 11
0 votes
3 answers
1k views

Unequal length arrays while calculating paired tests

I have two dfs: df1 and df2 1 3 2 3 2 4 4 6 6 7 7 9 I want to ...
freshman_2021's user avatar
4 votes
2 answers
149 views

Which statistical test should I use to show significance in favour of a system?

I have a dataset of the following form: System A Rating System B Rating 4.5 5 3 4 5 3 etc. etc. I have 155 such data points gathered using a survey. Which statistical test should I use to show ...
Michael Pulis's user avatar
0 votes
1 answer
843 views

Calculating confidence interval for model accuracy in a multi-class classification problem

In the book Applied Predictive Modeling by Max Kuhn and Kjell Johnson, there is an exercise concerning the calculation of a confidence interval for model accuracy. It reads as follows. ...
Mateusz's user avatar
  • 115
2 votes
1 answer
93 views

Why rejection of a true null hypothesis is called type I error?

I’m comparing two confusion matrices: https://en.wikipedia.org/wiki/Confusion_matrix#Table_of_confusion https://en.wikipedia.org/wiki/Type_I_and_type_II_errors The 2nd is rotated, the Decision is on ...
belz's user avatar
  • 21
4 votes
2 answers
490 views

How do I know that model performance improvement is significant?

Say I am running a Machine Learning model that produces a certain result (say accuracy of 80%). I now change a minor detail in my model (say, in a Deep Learning model, increase the kernel size in one ...
emilaz's user avatar
  • 141
2 votes
0 answers
47 views

Comparing two multi-class machine learning classifiers using Stuart Maxwell Test

I need to compare 2 multi-class classifiers. So, to assess whether the difference between the two are statistically significant I have taken the following steps: obtain prediction on test data using ...
ILR's user avatar
  • 31
2 votes
0 answers
116 views

How to apply Hypothesis test on time series?

I am working on a project that is related to Wikipedia's revision history. I have around 10 time-series and each one represents how the relative frequency of positive words changes over time. I need ...
Vangelis's user avatar
4 votes
2 answers
503 views

How do you use KS-test in a data science report?

I'm writing a data science report, I want to find an exist distribution to fit the sample. I got a good looking result , but when I use KS-test to test the model, I got a low p-value,1.2e-4, ...
Carl's user avatar
  • 51
7 votes
1 answer
317 views

When to use which multiple testing correction?

There are a large number multiple testing p-value correction methods. e.g.: ...
lordy's user avatar
  • 284
1 vote
1 answer
73 views

How to deal with missing values in the survey data to perform Paired sample t test

I have a dataset where I have 100 respondents. Each respondent has to give response on service quality of Health care equipment. Is it providing efficient services to the patients? We have two ...
pinky's user avatar
  • 151
1 vote
0 answers
23 views

How to think about design of experiments in the context of more complex machine learning [closed]

I have experience with multifactor DoE, but in the context of optimizing treatment of a single or or a small number of populations. Are there any articles people recommend to help get my head around ...
user6825943's user avatar
0 votes
0 answers
38 views

Hypothesis Testing - Median Test for linear regression model given standard error and co-efficient

I need help approaching this particular problem on hypothesis testing. A researcher believes that a diet high in raw cornstarch would increase the median blood glucose levels by 20. The diet can ...
Prerna Chinawalkar's user avatar
1 vote
1 answer
91 views

How can I compare the grammatical complexity between two texts using their sentences dependency length?

This is a continuation to the following thread. I have two texts, common English texts such as news articles and informative texts versus a technical textbook. I want to compare the grammatical ...
Samir Ahmane's user avatar
2 votes
1 answer
250 views

Interpreting the results based on Granger Causality test

I am trying to use Granger Causality test: https://www.statsmodels.org/stable/generated/statsmodels.tsa.stattools.grangercausalitytests.html to assess whether "positivity score" affects ...
Darcey BM's user avatar
  • 197
2 votes
1 answer
109 views

Permutation test on two groups

I am trying to use a permutation test to test my hypothesis. I want to make sure I am understanding concept of permutation correctly. I have control and experimental group. Then I combine them and ...
haneulkim's user avatar
  • 385