1
$\begingroup$

Pardon me, I agree the title of the question is not clear. I would like to know the understanding of below steps which are picked from the textbook "Hands on machine learning".

 >>> housing['income_cat'].value_counts() 
 >>> 3.0    7236
     2.0    6581
     4.0    3639
     5.0    2362
     1.0     822

If I am not wrong, the above step is to get the counts for each class. For example, for class '3' there are 7236 instances. Likewise, for class '2' there are 6581 instances.

>>>  housing['income_cat'].value_counts / len(housing) 
>>>  3.0    0.350581
     2.0    0.318847
     4.0    0.176308
     5.0    0.114438
     1.0    0.039826

Next, I was not clear, what was the intention behind the above step. By doing the above step, what am I suppose to learn ?. and

 >>> from sklearn.model_selection import StratifiedShuffleSplit

 >>> split = StratifiedShuffleSplit(n_splits=1, test_size=0.2, random_state=42)
 >>> for train_index, test_index in split.split(housing, housing["income_cat"]):
         strat_train_set = housing.loc[train_index]
         strat_test_set = housing.loc[test_index]

 >>>  strat_test_set['income_cat'].value_counts() / len(strat_test_set)
 >>>  3.0    0.350533
      2.0    0.318798
      4.0    0.176357
      5.0    0.114583
      1.0    0.039729
      Name: income_cat, dtype: float64

How come strat_test_set['income_cat'].value_counts() / len(strat_test_set) results are almost same to the results of housing['income_cat'].value_counts / len(housing) ?

$\endgroup$
3
$\begingroup$

Regarding your first question, by dividing by the length, you get the percentage of each category, and you can see that one category has a very low percentage.

If you use the regular train_test_split, the proportion of the categories will be different in each set, as the split is made randomly, and this can introduce a bias.

Imagine categories with a very low number of observations, you could even have categories missing entirely in either of the sets, which will cause trouble for your model.

The use of stratified sampling allows you to have the same proportion for the categories. That is why you see the same results: it is the goal of this sampling.

$\endgroup$
1
$\begingroup$

The answer is pretty clear as you answered your question yourself,

Also check out the normalise Param in value_counts..

What stratified sampling does is it keeps the actual percentage of each same in different folds as that in the original dataset...

(That's why they both are same..)

Why's it's done that way?

It's done so that you can reduce the overall bias in each of your sample and no one dominates over another..

Think of a worst case scenario where you have extracted few rows and all have the same value in a particular column...

It will be disastrous for the model as it hasn't seen anything else except that value in that fold...

(You can refer to the sklearn docs for Stratified Sampling..)

$\endgroup$
0
$\begingroup$

In this example, it has been validated that data distribution as claimed by http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedKFold.html (the folds are made by preserving the percentage of samples for each class.) is indeed the case.

And your data and last line of observation confirms that.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.