Questions tagged [hypothesis-testing]
The hypothesis-testing tag has no usage guidance.
27
questions
1
vote
0answers
14 views
I am not sure if I used the right hypotheses test or made the right conclusions, could you give me a feedback?
I have just finish a basic statistic course and I decided to apply what I learned in a small project using the wine quality dataset. I am not sure if what I did was correct, so if possible, I would ...
1
vote
1answer
44 views
Cross-validation split for modelling data with timeseries behavior
Background: I have a dataset that is generated every month (it is similar with card data that contains card demography and transactions every month and new accounts can be added in the middle of data ...
1
vote
1answer
20 views
What is the difference between a hypothese teste made with a normal distribution and a one sample t-test? And When to use each one?
I am studying statistics on my own and I am not understanding when to use a one sample t test or a test with a normal distribution. As I understand, both are comparing a population mean with a sample ...
0
votes
1answer
20 views
How do I conduct an experiment on the new pricing if it's impossible to conduct an A/B test?
We want to introduce a new price list for the customers of our international SaaS company.
Beforehand we want to test this new price list in several countries.
A/B test cannot be conducted here ...
0
votes
0answers
8 views
Hypothesis testing using permutation test method, how to reverse of null hypothesis?
My goal is to prove two groups A,B comes from the same distribution therefore I could use A as control and B as experimental for ABtest.
KPI is set to CTR(#clicks/#impressions). Therefore need to ...
0
votes
0answers
14 views
Qaulity assurance framework for ML/AI
I'd like to hear how organisations are applying a quality framework to ML/AI work. I'm struggling to find good content or good practice on this.
By saying quality frameworks, for example, if I was ...
0
votes
0answers
7 views
Approach for analyzing marketing campaign
Consider my users has to do "Action A" at their own pace to be able to continue using our service. Then, we run a campaign to push the user to do "Action A" at a quicker pace by ...
1
vote
1answer
36 views
Check if distribution per week is the same
I have sales by customer (b2b) and by date. I want to check if the distribution per day inside weeks remains the same from week to week.
Initial dataset
Customer
Date
Sales
Alpha
2019-02-23
527
...
0
votes
2answers
27 views
Unequal length arrays while calculating paired tests
I have two dfs:
df1 and df2
1 3
2 3
2 4
4 6
6 7
7
9
I want to ...
3
votes
2answers
97 views
Which statistical test should I use to show significance in favour of a system?
I have a dataset of the following form:
System A Rating
System B Rating
4.5
5
3
4
5
3
etc.
etc.
I have 155 such data points gathered using a survey. Which statistical test should I use to show ...
0
votes
1answer
60 views
Calculating confidence interval for model accuracy in a multi-class classification problem
In the book Applied Predictive Modeling by Max Kuhn and Kjell Johnson, there is an exercise concerning the calculation of a confidence interval for model accuracy. It reads as follows.
...
2
votes
1answer
52 views
Why rejection of a true null hypothesis is called type I error?
I’m comparing two confusion matrices:
https://en.wikipedia.org/wiki/Confusion_matrix#Table_of_confusion
https://en.wikipedia.org/wiki/Type_I_and_type_II_errors
The 2nd is rotated, the Decision is on ...
3
votes
1answer
44 views
How do I know that model performance improvement is significant?
Say I am running a Machine Learning model that produces a certain result (say accuracy of 80%). I now change a minor detail in my model (say, in a Deep Learning model, increase the kernel size in one ...
2
votes
0answers
38 views
Comparing two multi-class machine learning classifiers using Stuart Maxwell Test
I need to compare 2 multi-class classifiers. So, to assess whether the difference between the two are statistically significant I have taken the following steps:
obtain prediction on test data using ...
0
votes
0answers
23 views
ANOVA on one hot encoded feature
Suppose I am training a model for binary classification and I have a categorical feature which has been one-hot encoded. Suppose the feature is Gender and the one ...
0
votes
0answers
7 views
How to do a t-test on rare events distributions?
I have a dataset, which contains several features (feature_1, feature_2, ..., feature_n) and ...
1
vote
0answers
19 views
How to apply Hypothesis test on time series?
I am working on a project that is related to Wikipedia's revision history. I have around 10 time-series and each one represents how the relative frequency of positive words changes over time.
I need ...
3
votes
2answers
142 views
How do you use KS-test in a data science report?
I'm writing a data science report, I want to find an exist distribution to fit the sample. I got a good looking result , but when I use KS-test to test the model, I got a low p-value,1.2e-4, ...
5
votes
1answer
107 views
When to use which multiple testing correction?
There are a large number multiple testing p-value correction methods. e.g.:
...
1
vote
1answer
36 views
How to deal with missing values in the survey data to perform Paired sample t test
I have a dataset where I have 100 respondents. Each respondent has to give response on service quality of Health care equipment. Is it providing efficient services to the patients?
We have two ...
1
vote
0answers
19 views
How to think about design of experiments in the context of more complex machine learning [closed]
I have experience with multifactor DoE, but in the context of optimizing treatment of a single or or a small number of populations. Are there any articles people recommend to help get my head around ...
0
votes
0answers
38 views
Hypothesis Testing - Median Test for linear regression model given standard error and co-efficient
I need help approaching this particular problem on hypothesis testing.
A researcher believes that a diet high in raw cornstarch would increase the median blood glucose levels by 20. The diet can ...
0
votes
0answers
15 views
How to compare between two datasets of lexical density?
I have two dataset from two different texts representing lexical density as a proportion based on a corpus. Both datasets are represented in the images below. Now, let's suppose I want to know which ...
1
vote
1answer
41 views
How can I compare the grammatical complexity between two texts using their sentences dependency length?
This is a continuation to the following thread.
I have two texts, common English texts such as news articles and informative texts versus a technical textbook. I want to compare the grammatical ...
2
votes
1answer
116 views
Interpreting the results based on Granger Causality test
I am trying to use Granger Causality test: https://www.statsmodels.org/stable/generated/statsmodels.tsa.stattools.grangercausalitytests.html
to assess whether "positivity score" affects ...
2
votes
1answer
35 views
Permutation test on two groups
I am trying to use a permutation test to test my hypothesis.
I want to make sure I am understanding concept of permutation correctly.
I have control and experimental group. Then I combine them and ...
8
votes
2answers
566 views
A/B testing: How to calculate p-value on post test segments?
My question on A/B testing is about doing post test segmentation analysis.
For example:
I run an A/B test on my website to track bounce rate. On the
treatment group, i put a video to explain ...
