0
$\begingroup$

How can we extract information from time-stamp variable for modelling? I have a variable with format mm-dd-yyyy hh:mm:ss I want to predict an outcome variable using time-stamp as input variable. I do not think i can directly use this column for modelling and will need to do some transformation before i can use this in model. Not sure what type of transformation i need to do.

For example, if you have date field, you can create dummy variables of day o week and use that as input variable in model. Not sure how to proceed in the case of time-stamp.

Any help?

$\endgroup$
1
$\begingroup$

Welcome to Datascience.SE!

Like you said, you can extract the day of the week. Also extract the hour of the day, then encode these two variables using sines and cosines with their respective periodicities (7 and 24). Also create a column for the UNIX/epoch time. If there are "special days", such as holidays or sales, create boolean columns for them too.

$\endgroup$
7
  • $\begingroup$ Thanks! From date i have day-of-week, month-of-year. From time i guess i might need hour and minute dummies too. I am sure about the date part. I might have to test for hour and minute. Possibly this will be hour only. But can you imagine 6 dummies from day-of-week, 11 dummies from month-of-year, 23 dummies from hour-of-day. I am just curious if there is any other simple way. Can you elaborate on encode these two variables using sines and cosines with their respective periodicities (7 and 24). Also create a column for the UNIX/epoch time. not sure what this means. $\endgroup$ Jun 5 '16 at 7:11
  • 1
    $\begingroup$ UNIX/epoch time is the number of seconds since 1st january of 1970, this allows for a linear time variable. The hour of the day for example is a cyclic occurence, after 23 you go back to 0 even though they are very close together, using it as a single variable loses that relationship. By mapping it on a circle with the sine and the cosine function you can maintain that relationship. This creates two instead of one variable but requires less dimensions than using a dummy variable which would require 24 variables. $\endgroup$ Jun 5 '16 at 9:46
  • $\begingroup$ @JanvanderVegt would you have some intuitive description or worked-out example for this kind of transformation. I understood from your answer and comment that such a method exists, but i need a bit more detail as in an example of how to do this. Thanks in anticipation!! $\endgroup$ Jun 12 '16 at 17:22
  • $\begingroup$ @JanvanderVegt i got different opinion for using sine and cosine transformation for economical data. Basically i have transactional data stats.stackexchange.com/questions/57652/… $\endgroup$ Jun 18 '16 at 16:01
  • $\begingroup$ If my data only has date and no time. Should i still use a unix/epoch time variable, other than sine and cosine variable for day of week, week of the year? $\endgroup$ Jun 20 '16 at 3:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.