1
$\begingroup$

How can we extract information from time-stamp variable for modelling? I have a variable with format mm-dd-yyyy hh:mm:ss I want to predict an outcome variable using time-stamp as input variable. I do not think i can directly use this column for modelling and will need to do some transformation before i can use this in model. Not sure what type of transformation i need to do.

For example, if you have date field, you can create dummy variables of day o week and use that as input variable in model. Not sure how to proceed in the case of time-stamp.

Any help?

$\endgroup$

1 Answer 1

2
$\begingroup$

Welcome to Datascience.SE!

Like you said, you can extract the day of the week. Also extract the hour of the day, then encode these two variables using sines and cosines with their respective periodicities (7 and 24). Also create a column for the UNIX/epoch time. If there are "special days", such as holidays or sales, create boolean columns for them too.

$\endgroup$
7
  • $\begingroup$ Thanks! From date i have day-of-week, month-of-year. From time i guess i might need hour and minute dummies too. I am sure about the date part. I might have to test for hour and minute. Possibly this will be hour only. But can you imagine 6 dummies from day-of-week, 11 dummies from month-of-year, 23 dummies from hour-of-day. I am just curious if there is any other simple way. Can you elaborate on encode these two variables using sines and cosines with their respective periodicities (7 and 24). Also create a column for the UNIX/epoch time. not sure what this means. $\endgroup$ Commented Jun 5, 2016 at 7:11
  • 1
    $\begingroup$ UNIX/epoch time is the number of seconds since 1st january of 1970, this allows for a linear time variable. The hour of the day for example is a cyclic occurence, after 23 you go back to 0 even though they are very close together, using it as a single variable loses that relationship. By mapping it on a circle with the sine and the cosine function you can maintain that relationship. This creates two instead of one variable but requires less dimensions than using a dummy variable which would require 24 variables. $\endgroup$ Commented Jun 5, 2016 at 9:46
  • $\begingroup$ @JanvanderVegt would you have some intuitive description or worked-out example for this kind of transformation. I understood from your answer and comment that such a method exists, but i need a bit more detail as in an example of how to do this. Thanks in anticipation!! $\endgroup$ Commented Jun 12, 2016 at 17:22
  • $\begingroup$ @JanvanderVegt i got different opinion for using sine and cosine transformation for economical data. Basically i have transactional data stats.stackexchange.com/questions/57652/… $\endgroup$ Commented Jun 18, 2016 at 16:01
  • $\begingroup$ If my data only has date and no time. Should i still use a unix/epoch time variable, other than sine and cosine variable for day of week, week of the year? $\endgroup$ Commented Jun 20, 2016 at 3:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.