0
$\begingroup$

So I have a database of web-scraped cars, and I want to find the similarity between cars based on the km driven (e.g 69000), the model year (e.g 2012), and the trim of the car which will be one of three categories: base, mid, top.

What similarity measure can I use that will give me a decently accurate similarity ratio

$\endgroup$
  • 3
    $\begingroup$ I think you'll have your answer if you can define what "decently accurate similarity ratio" means $\endgroup$ – oW_ Dec 8 '17 at 21:32
  • 1
    $\begingroup$ Learn a spherical (unit norm) embedding for each car (e.g., based on the sale price) then use the cosine similarity. The result will be between -1 and +1. Welcome to the site! $\endgroup$ – Emre Dec 8 '17 at 22:17
2
$\begingroup$

I suggest you use a weighted per-attribute similarity, for instance let a and b a pair of tuples representing the attributes of some car A and another car B. For example:

a = (69000, 2012, base)
b = (71000, 2013, base)

For the km driven I would use Skm = 1 / (1 + |km(a) - km(b)|), so:

Skm = 1 / (1 + |69000 - 70000|) = 1 / 1001 = 0.0009

For the case of the year the same function could be use, then:

Syear = 1 / (1 + |2012 - 2013|) = 1 / 2 = 0.5

Last, for the trim of the car I would use a function that returns 0 if the categories were different 1 otherwise.

Strim = (base == base) = 1

Finally,

S = Wkm * Skm + Wyear * Syear + Wtrim * Strim

Where Wkm + Wyear + Wtrim = 1.0 and I would adjust the values according to what you think is decent accurate similarity ratio. Setting the values to be Wkm = 0.6, Wyear = 0.2, Wtrim = 0.2 the similarity S(a, b) = 0.30054. This similarity has the nice property that tuples with the same attributes yield a value of 1.

A good practice is to normalize the values of each attribute so you don't assign low values to any attribute for instance you could divide the columns of the km and year attributes for the maximum value, therefore for the pair of tuples in the example you will have the following transformed tuples:

a' = (0.985, 0.99, base)
b' = (1, 1, base)

Now the similarity using the same weights is S(a, b) = 0.9. Alternatively you could use a similarity matrix for the trim of the car, this is base could be more similar to mid thant to top. For example:

cat  | base | mid | top |
-------------------------
base |  1   | 0.5 | 0.2 |
-------------------------
mid  |  0.5 |  1  | 0.7 |
-------------------------
top  |  0.2 | 0.7 |  1  |

The values of the matrix have to be adjusted to fit your needs. For a more in depth comparison between similarity functions between categorical attributes (trim) see this for a comparison of different similarity measures for continuous attributes (year, km) see this.

Again to see which similarity is the best will depend on the data and on your perception, I would select a random set of pairs of cars and manually assign a similarity. Then split this set into a train set and test set. Use the train set to adjust the parameters of the different possible similarities, in the case of my proposal Wkm, Wyear and Wtrim. Then once your satisfied evaluate on the test set.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.