A basic assumption in machine learning is that training and test data are drawn from the same population, and thus follow the same distribution. But, in practice, this is highly unlikely. Covariate shift addresses this issue. Can someone clear the following doubts regarding this?
How does one check whether two distribution are statistically different? Can kernel density estimate (KDE) be used to estimate the probability distribution to tell the difference? Let's say I have 100 images of a specific category. The number of test images is 50, and I'm changing the number of training images from 5 to 50 in steps of 5. Can I say the probability distributions are different when using 5 training images and 50 testing images after estimating them by KDE?