You cannot just use arbitrary distance functions with k-means.
because the algorithm is not based on metric properties but on variance.
Fact is that k-means minimizes the sum of squares. This does not even give you the "smallest distances" but only the smallest squared distances. This is not the same (see: difference between median and mean) - if you want to minimize Euclidean distances, use k-medians or if you want other distances PAM (k-medoids).
You can generalize k-means to use a few more distances known as "Bergman divergences" and you can do some variant of the kernel trick. But that is not very powerful, because you don't have labels for optimizing the kernel parameters! Still, that could be what this exercise question is up to... If your "shells" are indeed centered at 0, then you can transform your data (read: kernel trick done wrong) to angle+distance from origin, and k-means may be able to cluster the projected data (dependent on the not well defined scaling of the axes).
Or the textbook did not realize that a kernel k-means has been proposed long ago. Then the argument is probably this: the mean of each shell is 0, and thus the shells cannot be distinguished. This clearly holds for unmodified k-means.