1
$\begingroup$

I'm trying to understand how does the KernelDensity class in scikit-learn work. Consider the following two cases which build a kernel from two different arrays (a). I'm wondering why the result of scoring on the array b in both cases is the same? Shouldn't a kernel from 10 same points be different from that from 2 points? The one from the 10 points should indicate more density. So, why the final scoring result is the same in both cases?

case A:

a = np.array([[1],[1],[1],[1],[1],[1],[1],[1],[1],[1]])

kde = KernelDensity(bandwidth=0.1)

kde.fit(a)

b = np.array([[1]])

log_dens = kde.score_samples(b)

print('Probability is: {}'.format(np.exp(log_dens)))

Probability is: [3.9894228]

case B:

a = np.array([[1],[1]])

kde = KernelDensity(bandwidth=0.1)

kde.fit(a)

b = np.array([[1]])

log_dens = kde.score_samples(b)

print('Probability is: {}'.format(np.exp(log_dens)))

Probability is: [3.9894228]

$\endgroup$
1
$\begingroup$

First of all, the score_samples function of SKLearn's Kernel Density object returns the log of the probability density, not of probability. Therefore, its exponent isn't exactly probability - e.g. in your example you have a probability above 1, which can't be.

Second, apparently the log-probability-density is normalized by the number of points the kernel was trained on (see line 210 here, where log_density -= np.log(N).

This effectively changes the expression $\rho_K(y) = \sum_{i=1}^{N} K(y - x_i; h)$ (from the User Guide) to $\rho_K(y) = {1 \over N} \sum_{i=1}^{N} K(y - x_i; h)$, i.e. turning the sum across points in the train set to an average across the train set.

By the way, I think that this is what they meant to say in this line in the function's description:

This is normalized to be a probability density, so the value will be low for high-dimensional data.

But I agree that it's a bit unclear.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.