Questions tagged [hypothesis-testing]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

How can I statistically measure/determine if A performs better than B?

Hi Data Science Community! I am a new Data Intern and I have been stuck on this question for a while. Here is a sample dataset I am working with: Customer Manufacturer A Spending Manufacturer B ...
user avatar
0 votes
1 answer
10 views

How do i conduct t-test for comparing the accuracy of two binary classifiers?

I am using two binary classifiers that predicts the accuracy of samples over a dataset. Accuracy is defined as ratio of correct vs incorrect predictions. Do i need to take accuracies sampled over ...
user avatar
0 votes
0 answers
19 views

Are experiments using confidence interval can be said a statistical test

I am working on some algorithm that is comparing results with other model using confidence interval , 90%. Can this be said a statistical test ? I read a article where it said about statistical test ...
user avatar
0 votes
0 answers
29 views

What kind of statistical test can be performed in a recommender system dataset that predicts the ratings for the movies?

The dataset consists of 1000s of users and users and each row of the dataset consist of user_id,movie_id and ratings the user provides to the movie. eg. 1,56,5 In my experiment i am calculating the ...
user avatar
0 votes
0 answers
15 views

Hypothesis test for classification model

I have a model that outputs 0 or 1 for interest/not-interest in a job. I'm doing an A/B/C test comparing two models (treatment groups) and none (control group). ANOVA for hypothesis testing and t-test ...
user avatar
0 votes
0 answers
6 views

What kind of hypothesis testing in Python can be used to validate that 4 job titles are significantly different based on their skillset?

I have 4 job titles, for each of which I scraped hundreds of job descriptions and classified them by if they contain words related to a predefined list of skills. For each job description, I now have ...
user avatar
1 vote
1 answer
29 views

How to test likelihood hypothesis on dataset?

How to test the following hypothesis? The larger the fare the more likely the customer is to be travailing alone. Using the data below, how would one be able to test the hypothesis? ...
user avatar
0 votes
0 answers
10 views

How to test if a curve is well described by an ellipse?

I have a set of data points in 2D, and I am trying to come up with some sort of statistical to determine if the points fall along an ellipse. My idea so far is to fit an ellipse to the points, take ...
user avatar
  • 1
0 votes
0 answers
13 views

the correct Cohen D formula for paired ttest

I have read some articles and I am not sure which is the best way to calculate the effect size for paired ttests. According to this Geff Cumming paper, using the pretest SD is the best way to go about ...
user avatar
  • 31
0 votes
0 answers
14 views

Why we need a statistical hypothesis testing for correlations between variables when we can check using scatter plots?

Need Guidance on correlation test. When we need to perform correlation between 2 variables, we generally start with scatter plot. Sometime it is suggested to perform hypothesis testing as well. I'm ...
user avatar
0 votes
0 answers
15 views

When do I need Statistical Signifcance testing and when not?

Hi there I have a handful of questions regarding statistical significance testing. As a newcomer I have sometimes topics that I do not really understand entirely. One of them is checking for ...
user avatar
0 votes
2 answers
72 views

Evaluating if metric of one group is higher than the metric of another group when group sizes differ significantly

I am working with a dataset that contains data of applicant income, gender, and loan status (whether or not the person was approved for a loan). I've created the following plots from the data. The ...
user avatar
  • 73
1 vote
2 answers
81 views

Does T-test requires Standard deviation or variance for calculation

Might be a novice question, but the main difference between a t-test and z-test, I was able to understand, is that the z-test calculation requires the SD value of the sample where as in a t-test, we ...
user avatar
  • 163
1 vote
1 answer
62 views

T-test against normalised or standardised data gives different results

I am studying the problem to predict popularity of a tweet, and want to test null hypothesis: there is no relationships between favorite_counts and another set of variables, like number of friends of ...
user avatar
1 vote
0 answers
37 views

statistical tests for null hypothesis - what if model is non linear?

I am reading the "An Introduction to Statistical Learning" (Gareth James & alii, Springer) as a primer to machine learning. I am reading the part in linear regressors, and learnt there ...
user avatar
0 votes
0 answers
27 views

Two-sided grubbs test KeyError: 355

I am trying to this two sided grubbs test by passing in a pandas.Series object and an appropriate alpha value. whenever I do the test on the whole dataset, I have ...
user avatar
  • 11
1 vote
1 answer
41 views

Why do I get this result with a chi- square test?

I have a question about the chi squared independence test, I'm working on dataset and I'm interested in finding the link between the categories of product and the gender, I plot my contingency table. ...
user avatar
0 votes
0 answers
19 views

What is the best statistical test to compare the competitiveness of 10 football/soccer leagues within a time range?

I was wondering what stat test is the best to find out which football/soccer league is the most competitive. The leagues that I would choose for this analysis are either based on the countries that ...
user avatar
1 vote
1 answer
20 views

How to test whether data is clustered wrt. subcategories?

I have a dataset of about 2000 entries, containing two numerical values, one categorical and one sub-categorical label for each entry. The data is from chemistry lab data, but for the purpose of this ...
user avatar
1 vote
0 answers
13 views

What kind of model to use to find drivers when data is aggregated and not on user level?

I have a website and have info from Google Analytics. So I can see the following "features": page url country device category (phone, desktop, etc.) Number of sessions Number of users: ...
user avatar
1 vote
0 answers
63 views

I am not sure if I used the right hypotheses test or made the right conclusions, could you give me a feedback?

I have just finish a basic statistic course and I decided to apply what I learned in a small project using the wine quality dataset. I am not sure if what I did was correct, so if possible, I would ...
user avatar
1 vote
1 answer
66 views

Cross-validation split for modelling data with timeseries behavior

Background: I have a dataset that is generated every month (it is similar with card data that contains card demography and transactions every month and new accounts can be added in the middle of data ...
user avatar
  • 63
1 vote
1 answer
27 views

What is the difference between a hypothese teste made with a normal distribution and a one sample t-test? And When to use each one?

I am studying statistics on my own and I am not understanding when to use a one sample t test or a test with a normal distribution. As I understand, both are comparing a population mean with a sample ...
user avatar
0 votes
1 answer
37 views

How do I conduct an experiment on the new pricing if it's impossible to conduct an A/B test?

We want to introduce a new price list for the customers of our international SaaS company. Beforehand we want to test this new price list in several countries. A/B test cannot be conducted here ...
user avatar
0 votes
0 answers
11 views

Hypothesis testing using permutation test method, how to reverse of null hypothesis?

My goal is to prove two groups A,B comes from the same distribution therefore I could use A as control and B as experimental for ABtest. KPI is set to CTR(#clicks/#impressions). Therefore need to ...
user avatar
  • 343
0 votes
0 answers
19 views

Qaulity assurance framework for ML/AI

I'd like to hear how organisations are applying a quality framework to ML/AI work. I'm struggling to find good content or good practice on this. By saying quality frameworks, for example, if I was ...
user avatar
0 votes
0 answers
9 views

Approach for analyzing marketing campaign

Consider my users has to do "Action A" at their own pace to be able to continue using our service. Then, we run a campaign to push the user to do "Action A" at a quicker pace by ...
user avatar
1 vote
1 answer
39 views

Check if distribution per week is the same

I have sales by customer (b2b) and by date. I want to check if the distribution per day inside weeks remains the same from week to week. Initial dataset Customer Date Sales Alpha 2019-02-23 527 ...
user avatar
  • 11
0 votes
2 answers
310 views

Unequal length arrays while calculating paired tests

I have two dfs: df1 and df2 1 3 2 3 2 4 4 6 6 7 7 9 I want to ...
user avatar
4 votes
2 answers
113 views

Which statistical test should I use to show significance in favour of a system?

I have a dataset of the following form: System A Rating System B Rating 4.5 5 3 4 5 3 etc. etc. I have 155 such data points gathered using a survey. Which statistical test should I use to show ...
user avatar
0 votes
1 answer
282 views

Calculating confidence interval for model accuracy in a multi-class classification problem

In the book Applied Predictive Modeling by Max Kuhn and Kjell Johnson, there is an exercise concerning the calculation of a confidence interval for model accuracy. It reads as follows. ...
user avatar
  • 115
2 votes
1 answer
62 views

Why rejection of a true null hypothesis is called type I error?

I’m comparing two confusion matrices: https://en.wikipedia.org/wiki/Confusion_matrix#Table_of_confusion https://en.wikipedia.org/wiki/Type_I_and_type_II_errors The 2nd is rotated, the Decision is on ...
user avatar
  • 21
3 votes
1 answer
90 views

How do I know that model performance improvement is significant?

Say I am running a Machine Learning model that produces a certain result (say accuracy of 80%). I now change a minor detail in my model (say, in a Deep Learning model, increase the kernel size in one ...
user avatar
  • 131
2 votes
0 answers
45 views

Comparing two multi-class machine learning classifiers using Stuart Maxwell Test

I need to compare 2 multi-class classifiers. So, to assess whether the difference between the two are statistically significant I have taken the following steps: obtain prediction on test data using ...
user avatar
  • 31
1 vote
0 answers
45 views

How to apply Hypothesis test on time series?

I am working on a project that is related to Wikipedia's revision history. I have around 10 time-series and each one represents how the relative frequency of positive words changes over time. I need ...
user avatar
3 votes
2 answers
260 views

How do you use KS-test in a data science report?

I'm writing a data science report, I want to find an exist distribution to fit the sample. I got a good looking result , but when I use KS-test to test the model, I got a low p-value,1.2e-4, ...
user avatar
  • 41
5 votes
1 answer
147 views

When to use which multiple testing correction?

There are a large number multiple testing p-value correction methods. e.g.: ...
user avatar
  • 234
1 vote
1 answer
41 views

How to deal with missing values in the survey data to perform Paired sample t test

I have a dataset where I have 100 respondents. Each respondent has to give response on service quality of Health care equipment. Is it providing efficient services to the patients? We have two ...
user avatar
  • 151
1 vote
0 answers
22 views

How to think about design of experiments in the context of more complex machine learning [closed]

I have experience with multifactor DoE, but in the context of optimizing treatment of a single or or a small number of populations. Are there any articles people recommend to help get my head around ...
user avatar
0 votes
0 answers
38 views

Hypothesis Testing - Median Test for linear regression model given standard error and co-efficient

I need help approaching this particular problem on hypothesis testing. A researcher believes that a diet high in raw cornstarch would increase the median blood glucose levels by 20. The diet can ...
user avatar
1 vote
1 answer
47 views

How can I compare the grammatical complexity between two texts using their sentences dependency length?

This is a continuation to the following thread. I have two texts, common English texts such as news articles and informative texts versus a technical textbook. I want to compare the grammatical ...
user avatar
2 votes
1 answer
142 views

Interpreting the results based on Granger Causality test

I am trying to use Granger Causality test: https://www.statsmodels.org/stable/generated/statsmodels.tsa.stattools.grangercausalitytests.html to assess whether "positivity score" affects ...
user avatar
2 votes
1 answer
61 views

Permutation test on two groups

I am trying to use a permutation test to test my hypothesis. I want to make sure I am understanding concept of permutation correctly. I have control and experimental group. Then I combine them and ...
user avatar
  • 343
8 votes
2 answers
721 views

A/B testing: How to calculate p-value on post test segments?

My question on A/B testing is about doing post test segmentation analysis. For example: I run an A/B test on my website to track bounce rate. On the treatment group, i put a video to explain ...
user avatar
  • 233