How do I factor in features which are IDs?

Question

I am an absolute beginner in data science and I had this (possibly stupid) question on my mind, while reading a problem in Kaggle: Say I'm given IDs of some clients, IDs of products that they sell, and quantity of the product sold, and I'm asked to predict the quantity of a product, given the client ID, and the product ID.

Now, say the client IDs are in the range 10000 - 50000 and the product IDs are in the range 1-10.

Suppose, for a moment that the client IDs were random integers from 1-1000000 and the product IDs were random integers from 1000-2000. This isn't supposed to make the slightest change in the results, is it? After all, IDs are mere tags.

But thinking data-wise, I've just bloated up two entire columns in my data to a higher scale, and these are two features as well.

So how do I think about this? How do I factor in features which are IDs? How do I 'normalize' them?

I hope I'm not being vague here. I just don't know a better way of phrasing this question.

Answer 1

These IDs should not be represented as numerical values to your model. If you would, your model thinks 2 and 3 are closer together than 2 and 2000, while it's just an ID, the number is just a name.

Some models can deal with them but then they need to be factors or categories (like decision trees). However most models cannot deal with categories at all, there are however numerous ways to solve this problem. The most used one is probably one-hot encoding, which means for every category in your feature you add a column, and you put a 1 if it's that category and a 0 otherwise. Example:

ID | target
1  | 0
1  | 1
2  | 3
3  | 2

To:

ID_1 | ID_2 | ID_3 | target
1    | 0    | 0    | 0
1    | 0    | 0    | 1
0    | 1    | 0    | 3
0    | 0    | 1    | 2

This work very well if you have few categories, however in the case of thousands of IDs this will increase your dimensionality too much. What you can do is collect statistics about the target and other features per group and join these onto your set and then remove your categories. This is what is usually done with a high number of categories. You have to be careful not to leak any information about your target into your features though (problem called label leaking).