# The meaning of P and degree of freedom in T-Test

I read about T-Test and how we can use it to compare between 2 models (https://towardsdatascience.com/paired-t-test-to-evaluate-machine-learning-classifiers-1f395a6c93fa)

There are some issues I'm not sure I understand correctly:

1. I saw that we usually choose P (p-value) to be 0.05 or 0.01

1.1 What does it mean ?

1.2 Does it mean that if the calculated t is in the range of the T-Table we accept the null hypothesis ,i.e 95% (or 99%) of the tests, there is no difference between the 2 models ?

2. we can't use K-FOLD and T-TEST, because the samples not independent (Train data used in one fold and in the other it is the test), am I right ?

So to use T-TEST we must generate at least 30 different datasets for test (with no dependent) ?

3. Degree of freedom

3.1 If I understand correctly, more test we run (bigger N -> bigger degree of freedom) -> the more accurate the T-Test results ? am I right ?

3.2 Do we need to run at least 30 tests to use T-TEST ?

• 3.1 more N does not mean "the more accurate" the T-Test results ? Commented Sep 15, 2023 at 23:54