It's more of a theoretical distribution, that a concrete one.
The main idea is this:
we consider all data to have an underlying distribution which generates the data. Through the procedure of creating a dataset we effectively sampled some instances from it. Now I like to think of this distribution as a theoretical notion of all possible data of this type that could ever exist.
Let me give you an example:
suppose we have the cats vs dogs dataset. This dataset contains $25000$ images of cats and dogs. Now we can consider the cat images as samples from a larger population. But what would this population include? All cat images on the web? All cat images in existence? or all cat images that could ever conceivably exist? Let's refer to this as this population as $C$. This population follows a certain distribution (not every image is an image of a cat); this distribution essentially tells us what makes a cat, a cat. Now if I were to take a picture of a cat tomorrow (let's call this $c$), I would have effectively taken a sample out of this dataset (i.e. $c \sim C$).
Where does this come into play in Machine Learning?
Well, generative models essentially try to learn this distribution, and they try to do this through its samples (i.e. our dataset). They look at its samples and try to generalize to identify what distribution spawned them. Essentially they try to answer the question what makes $c$ a sample of $C$?
Furthermore, even discriminative models make several assumptions about the data (e.g. that the samples are independent and identically distributed, that the training and test set follow the same underlying distribution)
More formally
The training and test data are generated by a probability distribution over datasets called the data-generating process. We typically make a set of assumptions known collectively as the i.i.d. assumptions. These assumptions are that the examples in each dataset are independent from each other, and that the training set and test set are identically distributed, drawn from the same probability distribution as each other. This assumption enables us to describe the data-generating process with a probability distribution over a single example. The same distribution is then used to generate every train example and every test example. We call that shared underlying distribution the data-generating distribution, denoted $p_{data}$. This probabilistic framework and the i.i.d. assumptions enables us to mathematically study the relationship between training error and test error.
- I. Goodfellow et al. "The Deep Learning Book" sec 5.2
I'd suggest reading chapter 5 from this book because the authors explain a lot of well known ML concepts (bias, variance, overfitting, underfitting etc.) through the scope of this data-generating distribution.
Edit after comment suggestion:
The question is essentially,
how does the data-generating distribution fit into the training process of a neural network?
The answer isn't so obvious, mainly because neural network classifiers are discriminatve models (i.e. they don't try to identify the data-generating distribution; rather they try to find out what features separate the classes among one another). Also I'd like to add that, as stated previously, the data-generating distribution is a theoretical concept, not a concrete one employed during training.
There is a way, through, we can tie this into the whole training procedure. Initially, consider that NNs try to minimize the cross-entropy loss between its predictions $\hat y$ and the actual labels $y$:
$$
Loss(y, \hat y) = - \sum_i y_i \, log \, \hat y_i
$$
Now let's think of $y$ and $\hat y$ not as tensors but as probability distributions. The first represents the probability that a sample would belong to class $y$, while the second represents the probability with which the network thinks a sample belongs to that class.
We can take this one step further and compute the KL divergence between $y$ and $\hat y$. This metric essentially tells us the difference between two distributions (higher values mean distributions are more different).
$$
KL \left( y \|\| \hat y \right) = \sum_i y_i \, log \, \frac{y_i}{\hat y_i}
$$
Note that minimizing cross-entropy is equivalent to minimizing the KL divergence between these two distributions. If the two distributions are identical their KL divergence has a value on $0$; this value increases the more they differ.
Minimizing the KL divergence between two distributions is the same as minimizing the JS divergence between them. This is a metric derived from KL, which can be used as a distance function between distributions (i.e. *how close $y$ is to $\hat y$).
So if you think of it this way, Neural Networks are trained to minimize the distance between the actual data-generating distribution $y$ and their perception of the data-generating distribution $\hat y$.
In order for this to be achievable, some assumptions must be held:
- The samples we have must be representative of the distribution (i.e. $y_i^{train} \sim y$).
- The test samples we'll use to evaluate our network on must follow the same distribution (i.e. $y_i^{test} \sim y$).
- The network must have sufficient capacity to learn this distribution.
- The right optimization strategy needs to be followed to minimize the distance.
- etc.
