Let's say I have 1-D data following a Gaussian distribution. I want to extract from this database the meaningful information, that is the information that lies far away from the mean.
One way to do that is to apply a function (e.g. $x \to |x-\mu|$ where $\mu$ is the mean of the distribution) and sort the values I get. But I want to keep track of how meaningful the data is as I am collecting it all along this process (as I do the bottom-up approach : I gather the samples one by one and create another (smaller) database). Is there an intuitive measure of this concept ?
I thought about comparing the histograms between the initial database and the one I create as I gather the samples. For example, the Wasserstein metric gives us a way to compare two probability distributions. An intuitive effect of the extraction I am making would be that the distance between the histograms does not decrease by much at first, as I only fill the error that does not contribute much to the distance. Is that a good approach ?
Next level : If my data is made of groups of samples (still, the entire database follows a Gaussian distribution), what would be the intuitive way to select groups that have meaningful information ?
I though about giving each group a weight that is the average of the variance of each sample that is in the group, and then sort the groups with the weights they have. But does this really keep track of the useful information ?
Next Next level : What about a n-D data following a (multivariate) Gaussian distribution ?
This is kinda theoric. If you have any questions feel free. Any article on the matter is also appreciated.