I am studying time series analysis to apply on a new project. Well, I am confronting a dilemma that I need some help.

When I read an old version of ggplot2 book (https://ggplot2-book.org/), I guess were the 2nd edition, Wickham applied the following algorithm:

  1. Created some columns based on the date column (month and day of week);

  2. Parsed these columns as factors;

  3. Trained a linear model; and,

  4. Evaluated residuals.

It is important to say that the objective was create a model to analyze the seasonality. In other words, it was not interested to generate a forecasting model. Other important information, as you may guess, this book was written using R as the programing language.

Well, I am transitioning to Python, and I need to accomplish a similar task. In true, I am using the IterativeImputer from scikit-learn to fill the missing data. On the data preparation, I took the first step as before, however I am worried about the second step. Considering this factor column has a cardinality, I did not apply any other transformation, as dummy variables, for example, but I am not sure if I am correct in my decision. I also maintain the column as a float.

More than this, I read some articles about times series forecasting to understand more the tools that are available. Well, one thing that I see was to manually input the lag values and then use a supervised technique to forecast the date. I believe I can improve the results using this picking the autocorrelation to select the lag values.

Summarizing my questions:

  1. When should I apply a dummy transformation for factor features in Python?

  2. When data present cardinality, should I maintain the column as float?

  3. In general, what of the two techniques show better results?

  4. Are there other forms to lead with this situation?



1 Answer 1


It seems that I need to improve my feature engineering. When I read this https://mlcourse.ai/book/topic09/topic9_part1_time_series_python.html tutorial, I understand this, specially the section https://mlcourse.ai/book/topic09/topic9_part1_time_series_python.html#linear-and-not-only-models-for-time-series.

PS.: the cardinality to factorial features stills a thing that I do not figure out.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.