18
$\begingroup$

I am trying to look for a good argument on why one would use the Manhattan distance over the Euclidean distance in Machine Learning.

The closest thing I found to a good argument so far is on this MIT lecture.

At 36:15 you can see on the slides the following statement:

"Typically use Euclidean metric; Manhattan may be appropriate if different dimensions are not comparable."

Shortly after the professor says that, because the number of legs of a reptile varies from 0 to 4 (whereas the other features are binary, only vary from 0 to 1), the "number of legs" feature will end up having a much higher weight if the Euclidean distance is used. Sure enough, that is indeed right. But one would also have that problem if using the Manhattan distance (only that the problem would be slightly mitigated because we don't square the difference like we do on the Euclidean distance).

A better way to solve the above problem would be to normalize the "number of legs" feature so its value will always be between 0 and 1.

Therefore, since there is a better way to solve the problem, it felt like the argument of using the Manhattan distance in this case lacked a stronger point, at least in my opinion.

Does anyone actually know why and when someone would use Manhattan distance over Euclidean? Can anyone give me an example in which using the Manhattan distance would yield better results?

$\endgroup$
4
$\begingroup$

According to this interesting paper, Manhattan distance (L1 norm) may be preferable to Euclidean distance (L2 norm) for the case of high dimensional data:

https://bib.dbvis.de/uploadedFiles/155.pdf

The authors of the paper even go a step further and suggest to use Lk norm distances, with a fractional value of k, for very high dimensional data in order to improve the results of distance-based algorithms, like clustering.

$\endgroup$
3
$\begingroup$

I can suggest a couple ideas, from wikipedia.

  1. If you want to place less emphasis on outliers, manhattan distance will try to reduce all errors equally since the gradient has constant magnitude.
  2. If your noise is distributed Laplacian, the MLE is found by minimizing the manhattan estimate.
$\endgroup$
3
$\begingroup$

I found something which might be intuition about this problem in Hands-On Machine Learning with Scikit-Learn and TensorFlow

Both the RMSE and the MAE are ways to measure the distance between two vectors: the vector of predictions and the vector of target values. Various distance measures, or norms, are possible:

  • Computing the root of a sum of squares (RMSE) corresponds to the Euclidian norm: it is the notion of distance you are familiar with. It is also called the ℓ2 norm(...)

  • Computing the sum of absolutes (MAE) corresponds to the ℓ1 norm,(...). It is sometimes called the Manhattan norm because it measures the distance between two points in a city if you can only travel along orthogonal city blocks.

  • More generally, (... )ℓ 0 just gives the number of non-zero elements in the vector, and ℓ∞ gives the maximum absolute value in the vector.

  • The higher the norm index, the more it focuses on large values and neglects small ones. This is why the RMSE is more sensitive to outliers than the MAE. But when outliers are exponentially rare (like in a bell-shaped curve), the RMSE performs very well and is generally preferred.

$\endgroup$
2
$\begingroup$

The use of Manhattan distance depends a lot on the kind of co-ordinate system that your dataset is using. While Euclidean distance gives the shortest or minimum distance between two points, Manhattan has specific implementations.

For example, if we were to use a Chess dataset, the use of Manhattan distance is more appropriate than Euclidean distance. Another use would be when are interested in knowing the distance between houses which are few blocks apart.

Also, you might want to consider Manhattan distance if the input variables are not similar in type (such as age, gender, height, etc.). Due to the curse of dimensionality, we know that Euclidean distance becomes a poor choice as the number of dimensions increases.

So in a nutshell: Manhattan distance generally works only if the points are arranged in the form of a grid and the problem which we are working on gives more priority to the distance between the points only along with the grids, but not the geometric distance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.