Image Similarity based on Color Palette Distribution

I am trying to compute similarity between two images based on their color palette distribution, let's say I have two sets of key value pairs as follows,

Img1: {'Brown': 14, 'White': 13, 'Black': 40, 'Gray': 31}

Img2: {'Pink': 82, 'Brown': 8, 'White': 7}

Where the numbers denote the % of that color present in the image. What would be the best way to compute similarity on a scale of 0-100 between the two images?

  • 4
    $\begingroup$ There are basically 2 possible ways to go. Simple one is to simply compare colour histograms. This question give pretty good description of several good measures/methods. But if you are going to use it in image search engine or something like that, it makes sense to also mimic human perception of colours, which is much harder task. This paper provides some cues for better human-aware comparison. $\endgroup$
    – ffriend
    Jul 27, 2014 at 22:18
  • 1
    $\begingroup$ That's a perfectly adequate answer, @ffriend. $\endgroup$
    – Emre
    Jul 28, 2014 at 1:27
  • 1
    $\begingroup$ This is a cross post from Cross Validated. Anyone know what to do? $\endgroup$
    – JenSCDC
    Jul 28, 2014 at 2:23
  • 1
    $\begingroup$ @ffriend: Please post your comment as an answer so it can be accepted. $\endgroup$
    – Nitesh
    Nov 25, 2014 at 17:29

1 Answer 1


Try using Earth Mover's Distance. It measures how much it "costs" to optimally reshape one histogram into another, where an elementary transform (using the terminology of edit distance-like distance measures) is moving a unit of mass from a bin to a bin. If you have an implementation of EMD, then your distance is dist = EMD(H1, H2, D), where

 H1 = [ 14, 13, 40, 31 ]; -- histogram #1 (first image)
 H2 = [ 82, 8, 7 ]; -- histogram #2 (second image)
 D  = [ D11, D12, D13; ... ; D41, D42, D43 ]; -- (cross-bin) ground distance

Dij is a distance from the i'th bin of the first histogram to the j'th bin of the second histogram. For example, D21 is a distance between 'White' and 'Pink'. It may be defined differently, based on which color space you want to use, and further, based on how you define what it means for two colors to be similar. If I use RGB, and I do not care much as to how human eye perceives colors, then D21 can be the \ell_2 norm of the difference [1, 1, 1] - [1, 0.05, 0.7]. (As far as I remember, the paper I mentioned uses LAB instead of RGB.)

P.S. Besides color, you may include spatial information into comparison. EMD can handle it well, as long as you properly combine similarity of colors with spatial similarity in the definition of ground distance D.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.