# One hot encoding with too many features (~ 10,000)

I am building a model to predict time off and sick leave for a specific employee.

Each of the employees has one row per day from 01/01/2013 to 31/12/2018 in the dataset flagged with 0 or 1 (if that day was taken as a time off or sick day). I am using information like location, age, job position, etc.

I was thinking to use employee id as a feature to train the model. Some users have obvious patterns, e.g, taking Monday off for some consecutive weeks. On the other hand, I feel like I do not have enough information as to have one model per employee, specially if they have being hired recently.

After researching a bit, seems that the only encoding that would make sense for employee id would be 'One Hot Encoding'. However, this would generate up to 10,000 features which doesn't seem very optimal.

Do you have any ideas that can help me build this model in a better way?

• If you have any good heuristics for segmenting employees(for example new/veteran) you can use this instead. Or at least bulk all of the new employees into one Encode value which will reduce number of categorical features Jul 21, 2019 at 10:28
• Are you expected to provide predictions also for new employees (which were not included in the training data set)? Jul 22, 2019 at 18:13
• There will be new employees in the future that can not be found in the training set. Jul 27, 2019 at 17:04

I think you are on the right track. What you are looking for is what is known as fixed effects in econometrics/statistics. Your have a model in which features X determine outcome y (so y = bX + u). On top of that you have unobserved heterogeneity (each individual employee) in your model. For each employee, the relation y <-> X may be a little different (fixed effect per employee), so that your model becomes y = b1X + b2Z + u, where Z denotes employees.

You can solve this by adding one dummy/indicator/one-hot per employee (known as least squares dummy variable fixed effects). So technically speaking, each employee gets his/her own intercept in the model. Thus the Z above is a massive matrix of one-hot encoded employee labels.

There are solutions in Stata (areg) and R (felm) which "absorb" the massive dummy variable set, so that it does not bother you when looking at the regression coefficients. If you are not interested in statistical inference (but only in making predictions), you will not look at the coefficients anyway.

I don't know the datails of your data (i.e. observations per employee), but from what you say, adding one dummy (aka one-hot feature) per employee would be the right way to go to control for unobserved heterogeneity of the employees.

• I am happy to see it is not such a crazy idea. I was not sure if it was right to have that number of features. I will give it a try. Thanks Peter. Jul 27, 2019 at 17:08

You could create embeddings for the employees. If you are using NN, just add an embedding layer for the employees. Or, you could use something similar to a collaborative filter to find embeddings for the employees. e.g, Create a matrix E = Employee x Day of the month, where each row is an employee and each column is a day of the month and

\begin{align} E_{ij} = 1 \quad &\text{if employee i worked at the day j}\\ E_{ij} = -1 \quad &\text{if employee i got a time off or sick leave } \end{align} then for each month, you can use, for instance, ALS (Alternating Least Squares) to create embeddings for the employee. You could average the embeddings out for several months and use them in your main model. This approach could also give you some kind of score for each employee that could be used in your model.

• Hi Rafael. This is a really interesting idea. I am going to take a better look and see how it would work. Thanks for your answer Jul 27, 2019 at 17:09

If you have enough relevant data, try clustering employees in unsupervised fashion, in order to replace employee ID by their class number; then, one-hot-encode the classes (or use another encoding technique: frequency encoding maybe?). Controlling the number of classes gives you control on the number of features after one-hot encoding, that is surely convenient. Also, new employees can easily be assigned to a cluster, which allows you to immediately be able to make predictions over them (you don't have to collect new data and re-train the model).

Note: A naive alternative would be to engineer some features. For instance, you mentioned taking Mondays off: just create a feature that gives the presence rate on Mondays. But this approach would likely lead to overfitting.