Time Series Data Imputation

Question

I am studying time series analysis to apply on a new project. Well, I am confronting a dilemma that I need some help.

When I read an old version of ggplot2 book (https://ggplot2-book.org/), I guess were the 2nd edition, Wickham applied the following algorithm:

1. Created some columns based on the date column (month and day of week);

2. Parsed these columns as factors;

3. Trained a linear model; and,

4. Evaluated residuals.

It is important to say that the objective was create a model to analyze the seasonality. In other words, it was not interested to generate a forecasting model. Other important information, as you may guess, this book was written using R as the programing language.

Well, I am transitioning to Python, and I need to accomplish a similar task. In true, I am using the IterativeImputer from scikit-learn to fill the missing data. On the data preparation, I took the first step as before, however I am worried about the second step. Considering this factor column has a cardinality, I did not apply any other transformation, as dummy variables, for example, but I am not sure if I am correct in my decision. I also maintain the column as a float.

More than this, I read some articles about times series forecasting to understand more the tools that are available. Well, one thing that I see was to manually input the lag values and then use a supervised technique to forecast the date. I believe I can improve the results using this picking the autocorrelation to select the lag values.

Summarizing my questions:

1. When should I apply a dummy transformation for factor features in Python?

2. When data present cardinality, should I maintain the column as float?

3. In general, what of the two techniques show better results?

4. Are there other forms to lead with this situation?

Thanks

Answer 1

It seems that I need to improve my feature engineering. When I read this https://mlcourse.ai/book/topic09/topic9_part1_time_series_python.html tutorial, I understand this, specially the section https://mlcourse.ai/book/topic09/topic9_part1_time_series_python.html#linear-and-not-only-models-for-time-series.

PS.: the cardinality to factorial features stills a thing that I do not figure out.