How can I weigh observations differently that were provided for a time horizon?

Question

I have 623 observations with one continuous dependent variable and 13 independent variables (continuous, categorical, and ordinal), defined based on researcher experience and literature review. I considered planning to do several regression analysis to estimate the dependent variable and study the predictive factors (if they are positive, negative, and their magnitude) on it. Data are provided for 10 years. Since the latest observations are more important, I’m interested in using weighted observations. How can I approach this problem and validate my approach?

Answer 1

Therefore, the year of the observation was considered as a predictor variable which had a positive impact on the dependent variable. However, the year itself doesn’t have any effect on the dependent variable per se; rather, it is the other factors occurring in the same time period which result in improvements.

This sounds like a challenge for trees with sufficient interaction depth, as you've found that the year interacts with other factors that results in improvements. Ordinary least squares regression here do not capture that type of interaction well.

I would suggest setting up the following regression model:

1. Transform the date column into (CurrentYear - YearOfDateStamp) AS NumOfYearsAway. I'd recommend leaving this as a numerical feature rather than a categorical one. This would allow the tree based model to select cuts like NumOfYearsAway >= 5.5 instead of NumOfYearsAway in (6,7,10). This could also be more helpful when NumOfYearsAway = 0 occurs in your scoring dataset, where you might not have training data for current year data set.
2. Fit a tree based model, I'd pick XGBoost, with the usual CV to tune the hyperparameters such as interaction depth.

The drawback of picking XGBoost in your application however, is that the interpretation of the impact of a particular variable on the target variable is not obvious. You'd need partial dependence plot to observe how the target variable vary with the bespoke input variable. If interpretability is very important, one could pick a single tree regression model like rpart.

Answer 2

Converting the date into an integer timestamp will put a higher weight on the more recent observation.

When you write :

Since the latest observations are more important, I’m interested in using weighted observations.

Do you mean that you know already that the date will be a predictive factor in your analysis, or that you want to artificially make this variable a predictive factor for the regression?

If it is the former, then the integer conversion above could do the trick. If it is the latter, you need to combine the date with your target variable (by multiplication for example).

Answer 3

One thing you can do is calculate the time interval i.e Current time - Observation Time. Let's say this is t. Now create a weight vector as c/t or c*exp(-t). You can use weights in caret using caret::train > weights

Answer 4

I would read up on decay functions. With one of these you can choose exactly to what extent things in the past contribute:

https://en.wikipedia.org/wiki/Exponential_decay