0
$\begingroup$

I have two datetime columns in my dataset. What I have done so far I have extracted year, month, dayofweek and hourofday from these columns.

So as you expect they will be something like this:

2015-5-5 08:21:20   ----> 2015   5   5   8

So my question is that what is the best way to normalize these numbers. Because I think that year or other numbers will dominate my machine learning model.

I have not found any article regarding this, all explaining till this point that we convert them to year month ...

Thanks.

$\endgroup$
1
$\begingroup$

You already have a good beginning. Transforming data into 4 columns. Year, month, day and hour. Now These 4 are all categoricals, you can than just apply one hot Encoding. Than no Domination will happen.

$\endgroup$
1
  • $\begingroup$ Webber The problem is that dayofYear has 365 entry and creating OHE might not be a good idea. Do you have any idea of what encoding would be the best option for high cardinality features? $\endgroup$ – sariii Nov 16 '20 at 15:22
1
$\begingroup$

It depends on what the task is (what you are trying to predict) and how the date relates or could relate to that task. If you are trying to predict cancer risk and you have the datetime of someone's birth, the time and day portion are probably irrelevant (the month too, possibly). In this scenario it makes more sense to convert the datetime to a person's age.

In other scenarios you could consider binning, for example splitting the year into 4 seasons instead of 12 months * ~30 days. And/or splitting the day into morning/noon/evening/night instead of 24 hours.

You could also convert the datetime to epoch time to obtain a single number / feature. You can look at some ways to do this in this answer.

There are more possibilities, but it really depends on what you are trying to achieve. One-hot encoding everything is not a good option in most cases.

$\endgroup$
2
  • $\begingroup$ Thanks a lot for your help. Actually, I am working on ranking problem and the datetime I have is related to the search_date_time, cheking_datetime and checkout_datetime. So these features in my idea can affect the ranking of the hotels. By doing what I explained in my question and then applying OHE I did not get very good result. Do you have any idea of based on this problem what way makes more sense? $\endgroup$ – sariii Nov 17 '20 at 15:07
  • $\begingroup$ You will have to experiment a bit to see what works, but I would start with something like this: -Take the time part separately and represent it just as hour of the day (0-23 for example) -Extract the day of the week and one hot encode it (e.g. Sunday, Monday, etc), or even just weekday vs. weekend -You can leave day of the month as a numerical feature or one hot encode it as beginning, middle and end of the month -Leave the month number and year number as is. This way you have a manageable amount of features. After this you would of course still do normalization/scaling/etc $\endgroup$ – honeybees Nov 18 '20 at 17:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.