I have a data set that includes the following and am using it to learn more about data science. I have googled a bunch - but can't seem to find any examples on what I am trying to do. I am trying to predict days to harvest. I have quite a few years of historical data.

Plant Date Variety Days To Harvest Transplant Plant Tape
5/16/2022 Wildcat 59 1 0
5/16/2022 Wildcat 81 0 0
5/18/2022 Bearcat 77 0 0

I realize this is a super simple data set, but I am learning! :) What I would like to try to do do is some how incorporate weather data into predicting the days to harvest. What I am unsure of is how to best put the weather data into the data set. For example, do I create a column for each weather feature and day within the X number of days? Example:

Plant Date Variety Days To Harvest Transplant Plant Tape Day 0 Min Temp Day 0 Max Temp Day 1 Min Temp Day 1 Max Temp
5/16/2022 Wildcat 59 1 0 40 65 55 72
5/16/2022 Wildcat 81 0 0 40 65 55 72
5/17/2022 Bearcat 77 0 0 55 72 54 80

Would I be on the right track doing it this way? Or would 45 days of climate data be too much and should I be summarizing by the week instead? My goal would be to predict the days until harvest based upon the plant date and first X number of days of weather data.


1 Answer 1


It's difficult to guess which option is going to give the best results, since it depends on many factors in the data. This is feature engineering, and while there are some general principles it's not a precise science.

In this case:

  • Representing every day until harvest is impossible, because the number of features must be fixed for every instance. Additionally this level of detail would likely make it difficult for the algorithm to correctly represent the global trend, and as a result there would be a serious risk of overfitting.
  • Ideally the features should "help" the algorithm as much as possible. For example if there is expert knowledge that the number of rainy days is an important factor, then it's worth directly representing the number of days with rain.
  • Various relevant statistics could be: min/max/average/std.dev. for temperature and rain, possibly quantiles (e.g. avg temperature for the N% hottest days). This kind of statistics:
    • can be represented as a fixed number of features
    • are more representative of the trend and thus more usable by the algorithm

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.