Weather impact on plant growth

Question

I have a data set that includes the following and am using it to learn more about data science. I have googled a bunch - but can't seem to find any examples on what I am trying to do. I am trying to predict days to harvest. I have quite a few years of historical data.

Plant Date	Variety	Days To Harvest	Transplant	Plant Tape
5/16/2022	Wildcat	59	1	0
5/16/2022	Wildcat	81	0	0
5/18/2022	Bearcat	77	0	0

I realize this is a super simple data set, but I am learning! :) What I would like to try to do do is some how incorporate weather data into predicting the days to harvest. What I am unsure of is how to best put the weather data into the data set. For example, do I create a column for each weather feature and day within the X number of days? Example:

Plant Date	Variety	Days To Harvest	Transplant	Plant Tape	Day 0 Min Temp	Day 0 Max Temp	Day 1 Min Temp	Day 1 Max Temp
5/16/2022	Wildcat	59	1	0	40	65	55	72
5/16/2022	Wildcat	81	0	0	40	65	55	72
5/17/2022	Bearcat	77	0	0	55	72	54	80

Would I be on the right track doing it this way? Or would 45 days of climate data be too much and should I be summarizing by the week instead? My goal would be to predict the days until harvest based upon the plant date and first X number of days of weather data.

Answer 1

It's difficult to guess which option is going to give the best results, since it depends on many factors in the data. This is feature engineering, and while there are some general principles it's not a precise science.

In this case:

• Representing every day until harvest is impossible, because the number of features must be fixed for every instance. Additionally this level of detail would likely make it difficult for the algorithm to correctly represent the global trend, and as a result there would be a serious risk of overfitting.
• Ideally the features should "help" the algorithm as much as possible. For example if there is expert knowledge that the number of rainy days is an important factor, then it's worth directly representing the number of days with rain.
• Various relevant statistics could be: min/max/average/std.dev. for temperature and rain, possibly quantiles (e.g. avg temperature for the N% hottest days). This kind of statistics:
  • can be represented as a fixed number of features
  • are more representative of the trend and thus more usable by the algorithm