When would one use Manhattan distance as opposed to Euclidean distance?

Question

I am trying to look for a good argument on why one would use the Manhattan distance over the Euclidean distance in machine learning.

The closest thing I found to a good argument so far is on this MIT lecture.

At 36:15 you can see on the slides the following statement:

"Typically use Euclidean metric; Manhattan may be appropriate if different dimensions are not comparable."

Shortly after, the professor says that, because the number of legs of a reptile varies from 0 to 4 (whereas the other features are binary, only vary from 0 to 1), the "number of legs" feature will end up having a much higher weight if the Euclidean distance is used. Sure enough, that is indeed right. But one would also have that problem if using the Manhattan distance (only that the problem would be slightly mitigated because we don't square the difference like we do on the Euclidean distance).

A better way to solve the above problem would be to normalize the "number of legs" feature so its value will always be between 0 and 1.

Therefore, since there is a better way to solve the problem, it felt like the argument of using the Manhattan distance in this case lacked a stronger point, at least in my opinion.

Does anyone actually know why and when someone would use Manhattan distance over Euclidean? Can anyone give me an example in which using the Manhattan distance would yield better results?

Answer 1

According to this interesting paper, Manhattan distance (L1 norm) may be preferable to Euclidean distance (L2 norm) for the case of high dimensional data.

The authors of the paper even go a step further and suggest to use Lk norm distances, with a fractional value of k, for very high dimensional data in order to improve the results of distance-based algorithms, like clustering.

Answer 2

I found something which might be intuition about this problem in Hands-On Machine Learning with Scikit-Learn and TensorFlow

Both the RMSE and the MAE are ways to measure the distance between two vectors: the vector of predictions and the vector of target values. Various distance measures, or norms, are possible:

• Computing the root of a sum of squares (RMSE) corresponds to the Euclidian norm: it is the notion of distance you are familiar with. It is also called the ℓ2 norm(...)

• Computing the sum of absolutes (MAE) corresponds to the ℓ1 norm,(...). It is sometimes called the Manhattan norm because it measures the distance between two points in a city if you can only travel along orthogonal city blocks.

• More generally, (... )ℓ 0 just gives the number of non-zero elements in the vector, and ℓ∞ gives the maximum absolute value in the vector.

• The higher the norm index, the more it focuses on large values and neglects small ones. This is why the RMSE is more sensitive to outliers than the MAE. But when outliers are exponentially rare (like in a bell-shaped curve), the RMSE performs very well and is generally preferred.

Answer 3

I can suggest a couple ideas, from wikipedia.

1. If you want to place less emphasis on outliers, manhattan distance will try to reduce all errors equally since the gradient has constant magnitude.
2. If your noise is distributed Laplacian, the MLE is found by minimizing the manhattan estimate.

Answer 4

The use of Manhattan distance depends a lot on the kind of co-ordinate system that your dataset is using. While Euclidean distance gives the shortest or minimum distance between two points, Manhattan has specific implementations.

For example, if we were to use a Chess dataset, the use of Manhattan distance is more appropriate than Euclidean distance. Another use would be when are interested in knowing the distance between houses which are few blocks apart.

Also, you might want to consider Manhattan distance if the input variables are not similar in type (such as age, gender, height, etc.). Due to the curse of dimensionality, we know that Euclidean distance becomes a poor choice as the number of dimensions increases.

So in a nutshell: Manhattan distance generally works only if the points are arranged in the form of a grid and the problem which we are working on gives more priority to the distance between the points only along with the grids, but not the geometric distance.

Answer 5

Manhattan distance uses the L1 Norm which encourages sparsity wrt the L2 norm. Therefore if you want to 0 out components which have poor predictive power, then they are a great choice. Whereas the L2 norm would waste time computing small deviations which wouldn’t even affect your answer.

Answer 6

Manhattan Distance is the L1 norm form (L1 norm is the sum of the magnitude of vectors in space), while Euclidean Distance is L2 Norm form (The L2 norm calculates the distance of the vector coordinate from the origin of the vector space.