# Why we do random sampling when we select the training set?

The usual workflow when building a machine learning model starts with random splitting the data set into training and test set.

What I can't understand is why we do this. For example lets say we have a labeled data set of 100 data points: $$\{x_i, y_i\}_{i=1}^{100}$$

and the $$x$$ values are distributed in the following way:

$$x \in[0, 5) \rightarrow 80 \, \, \text{samples}$$

and

$$x \in[5, 10) \rightarrow 20 \, \, \text{samples}$$

Then if we random sample 80 data points to generate our training how do we know that we won't pick the 80 samples from the first interval?

I mean it is completely possible as all data points are equiprobable during sampling. And if that happens then our model wouldn't be able to see "trends" in regions outside of $$[0, 5)$$. Why then random sampling is the way to go when we do train-test split?

• Most of my models do not do a random split. My models tend to deal with time. My training might be from one time frame, validation from another. Whenever I have a model or other statistic that needs a random split, I repeat the analysis multiple times with different splits to ensure I have stability. Jun 27, 2022 at 10:10

Because this is extremely unlikely: one can calculate the probability to include $$N$$ instances with $$x>5$$ when every instance has an 80% chance to be selected for the training set, for any $$N$$. The most likely $$N$$ would be $$N=20\times 0.8 = 16$$, with probability decreasing far from this value: e.g. $$N=15$$ or $$N=17$$ is still quite likely, but for instance $$N=8$$ is very unlikely and $$N=0$$ extremely unlikely.
Btw this is an interesting experiment to run in order to understand statistics: loop over this random sampling, and see how many times you get every value of $$N$$ as a result.