Kernel Density in Scikit Learn

Question

I'm trying to understand how does the KernelDensity class in scikit-learn work. Consider the following two cases which build a kernel from two different arrays (a). I'm wondering why the result of scoring on the array b in both cases is the same? Shouldn't a kernel from 10 same points be different from that from 2 points? The one from the 10 points should indicate more density. So, why the final scoring result is the same in both cases?

case A:

a = np.array([[1],[1],[1],[1],[1],[1],[1],[1],[1],[1]])

kde = KernelDensity(bandwidth=0.1)

kde.fit(a)

b = np.array([[1]])

log_dens = kde.score_samples(b)

print('Probability is: {}'.format(np.exp(log_dens)))

Probability is: [3.9894228]

case B:

a = np.array([[1],[1]])

kde = KernelDensity(bandwidth=0.1)

kde.fit(a)

b = np.array([[1]])

log_dens = kde.score_samples(b)

print('Probability is: {}'.format(np.exp(log_dens)))

Probability is: [3.9894228]

Answer 1

First of all, the score_samples function of SKLearn's Kernel Density object returns the log of the probability density, not of probability. Therefore, its exponent isn't exactly probability - e.g. in your example you have a probability above 1, which can't be.

Second, apparently the log-probability-density is normalized by the number of points the kernel was trained on (see line 210 here, where log_density -= np.log(N).

This effectively changes the expression $\rho_K(y) = \sum_{i=1}^{N} K(y - x_i; h)$ (from the User Guide) to $\rho_K(y) = {1 \over N} \sum_{i=1}^{N} K(y - x_i; h)$, i.e. turning the sum across points in the train set to an average across the train set.

By the way, I think that this is what they meant to say in this line in the function's description:

This is normalized to be a probability density, so the value will be low for high-dimensional data.

But I agree that it's a bit unclear.