How to deal with mismatch of timesteps in target variable and features in forecasting problem?

Question

Background Info:

I am working with some climate data where I want to predict crop yields with my dataset containing climate- and satellite-derived features.

This is a time series regression forecasting problem and I want to put it through XGBoost and Lasso to generate my predictions. However, there is a mismatch in the sample frequency between my features and target variable; the features are recorded weekly, and the target variable is recorded yearly.

At the moment, I am using a wide-format table as my input dataset to the ML models, but I feel like my models are not generating accurate predictions when the input table is formatted with the wide-format, since there are less samples compared to if I used a long-format table.

Data Table Reference:

For reference, the wide format table looks something like this, where the suffix represents a week number on a feature:

ID	Year	crop_yield	precip1	precip2	precip3	max_temp1	max_temp2	max_temp3
1100	2000	32.1	5.3	3.0	3.1	13.3	15.3	3.1
1100	2001	31.6	6.6	3.2	1.1	11.3	12.3	6.1
5903	2000	41.2	3.4	0.5	2.1	10.3	18.3	8.1
5903	2001	27.7	1.7	3.8	8.1	12.3	16.3	5.1

And the long format table would look something like this:

ID	Year	crop_yield	Week	precip	max_temp
1100	2000	32.1	1	5.3	13.3
1100	2000	32.1	2	3.0	15.3
1100	2000	32.1	3	3.1	3.1
1100	2001	31.6	1	6.6	11.3
1100	2001	31.6	2	3.2	12.3
1100	2001	31.6	3	1.1	6.1
5903	2000	41.2	1	3.4	10.3
5903	2000	41.2	2	0.5	18.3
5903	2000	41.2	3	2.1	8.1
5903	2001	27.7	1	1.7	12.3
5903	2001	27.7	2	3.8	16.3
5903	2001	27.7	3	8.1	5.1

Question:

Would it be advisable to use the long format table as the input to my ML models? I feel like the identical crop yields for each associated ID and year would throw my models off.

In addition, is there a better way to frame my data that I haven't explored yet?

Answer 1

To deal with mismatch of timesteps, you have to find a balance between all those mismatched timesteps according to your objective. For example, if your objective is to have a prediction for the next quarter, your learning data should be adapted to quarters.

learning data < 1 quarter: get the mean values of all weeks that last quarters.

learning data > 1 quarter: repeat the yearly value to every corresponding quarter, or make a (logistic?) regression to simulate the quarterly evolution over years.

This is a simple example to explain the idea, but real cases might require further investigation to define the right timestep according to objectives.