CV Pipeline for Time Series with Differing shape for X and y

Question

I am playing around with ML models to forecast a time series. I'd like to generate a sklearn pipeline like

from sklearn.preprocessing import FunctionTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler

pipe = Pipeline(
    steps=[
        ("gen", FunctionTransformer(generate_features)),
        ("imp_nan", SimpleImputer(strategy="median")),
        ("scale", StandardScaler()),
        ("reg", keras.wrappers.scikit_learn.KerasRegressor(build_model, epochs=35))
    ]
)

Where generate_features takes in my data X and produces a collection of features for a given timestamp. For example a 5 day rolling standard deviation. However, my target y could have a different shape.

For example, I may want to forecast 5 hours in the future (and so my target will have a date time index with period 5 hours), but my input data could have a frequency of 1 min (and so initially an index with period 1 min that gets transformed into a period of 5 hours). This leads to issues when trying to cross validate. For example, if I use

splitter = TimeSeriesSplit(n_splits=3)
grid = GridSearchCV(pipe, {"gen__kw_args": [{"windows": (2,4,6)},]}, cv=splitter)
grid.fit(X_train, y_train)

I immediately get an error telling me the shape of X_train and y_train are not the same. This makes sense because the splitter needs to be used on both X and y.

I want to do this because I'd like to perform a grid cv on the parameters of my generate_features function (as well as the parameters of my model...)

My Question

Is there a nice way to create a pipeline and perform cross validation on time series feature generation when our input shape of X could be different to our input shape of y?

Edit: Here is the traceback

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-11-f0c6e65ecbb8> in <module>
      1 splitter = TimeSeriesSplit(n_splits=3)
      2 grid = GridSearchCV(pipe, {"gen__kw_args": [{"windows": (2,4,6)},]}, cv=splitter)
----> 3 grid.fit(X_train, y_train)

~/.local/lib/python3.8/site-packages/sklearn/model_selection/_search.py in fit(self, X, y, groups, **fit_params)
    648             refit_metric = 'score'
    649 
--> 650         X, y, groups = indexable(X, y, groups)
    651         fit_params = _check_fit_params(X, fit_params)
    652 

~/.local/lib/python3.8/site-packages/sklearn/utils/validation.py in indexable(*iterables)
    246     """
    247     result = [_make_indexable(X) for X in iterables]
--> 248     check_consistent_length(*result)
    249     return result
    250 

~/.local/lib/python3.8/site-packages/sklearn/utils/validation.py in check_consistent_length(*arrays)
    209     uniques = np.unique(lengths)
    210     if len(uniques) > 1:
--> 211         raise ValueError("Found input variables with inconsistent numbers of"
    212                          " samples: %r" % [int(l) for l in lengths])
    213 

ValueError: Found input variables with inconsistent numbers of samples: [522323, 52417]

Answer 1

I've done a bit of searching and have actually found a solution to this using tsfresh.

You can find the sklearn transformers here: https://tsfresh.readthedocs.io/en/latest/text/sklearn_transformers.html

Here is the code snippet from the example.

pipeline = Pipeline(
    [
        ('augmenter', RelevantFeatureAugmenter(column_id='id', column_sort='time')),
        ('classifier', RandomForestClassifier())
    ]
)

df_ts, y = load_robot_execution_failures()
X = pd.DataFrame(index=y.index)

pipeline.set_params(augmenter__timeseries_container=df_ts)
pipeline.fit(X, y)

We pass an empty dataframe to pipeline.fit and specify the input data as a parameter to the tsfresh transformer. df_ts and y have a totally different shape!

:-)