0
$\begingroup$

I am trying to train a Model that predicts the solar power generation of my roof. This is my current dataset: https://pastebin.com/gtZcGi2m. It is built using some weather api and the actual power that was produced.

I used this in ML.NET and let it automatically find the best model. The issue is, that e.g. if the solarradiation is 0, but the time is like 2pm, it will still predict generation of 2000-3000. Basically, it performs very poorly. I thought it might make sense to split the datetime column. The time component is the most important one, but also month / day has an influence. How could I prepare my values in order to receive a better model?

$\endgroup$

1 Answer 1

2
$\begingroup$

If using Python, you could extract date and time components from the timestamp automatically utilizing the library Feature-engine.

If the timestamp is in the index, you could execute something as follows:

Create toy dataset:

import pandas as pd

X = {"ambient_temp": [31.31, 31.51, 32.15, 32.39, 32.62, 32.5, 32.52, 32.68],
     "module_temp": [49.18, 49.84, 52.35, 50.63, 49.61, 47.01, 46.67, 47.52],
     "irradiation": [0.51, 0.79, 0.65, 0.76, 0.42, 0.49, 0.57, 0.56],
     "color": ["green"] * 4 + ["blue"] * 4,
     }

X = pd.DataFrame(X)
X.index = pd.date_range("2020-05-15 12:00:00", periods=8, freq="15min")

Extract features automatically:

from feature_engine.datetime import DatetimeFeatures

dtf = DatetimeFeatures(variables="index")

Xtr = dtf.fit_transform(X)

Xtr

The resulting dataframe:

                     ambient_temp  module_temp  irradiation  color  month  \
2020-05-15 12:00:00         31.31        49.18         0.51  green      5
2020-05-15 12:15:00         31.51        49.84         0.79  green      5
2020-05-15 12:30:00         32.15        52.35         0.65  green      5
2020-05-15 12:45:00         32.39        50.63         0.76  green      5
2020-05-15 13:00:00         32.62        49.61         0.42   blue      5
2020-05-15 13:15:00         32.50        47.01         0.49   blue      5
2020-05-15 13:30:00         32.52        46.67         0.57   blue      5
2020-05-15 13:45:00         32.68        47.52         0.56   blue      5

                     year  day_of_week  day_of_month  hour  minute  second
2020-05-15 12:00:00  2020            4            15    12       0       0
2020-05-15 12:15:00  2020            4            15    12      15       0
2020-05-15 12:30:00  2020            4            15    12      30       0
2020-05-15 12:45:00  2020            4            15    12      45       0
2020-05-15 13:00:00  2020            4            15    13       0       0
2020-05-15 13:15:00  2020            4            15    13      15       0
2020-05-15 13:30:00  2020            4            15    13      30       0
2020-05-15 13:45:00  2020            4            15    13      45       0

If the datetime features is in one or more columns instead of the index, you can indicate which columns to use in the variables parameter:

dtf = DatetimeFeatures(variables=["var_1", "var_2"])

The class DatetimeFeatures creates a number of features by default, you can create all possible features by setting the parameter features_to_extract="all" when initializing the transformer. Or alternatively, pass a list with the features you want to create.

More details in the DatetimeFeatures documentation.

DatetimeFeatures runs on top of the pandas dt module.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.