I am playing around with ML models to forecast a time series. I'd like to generate a sklearn pipeline like

from sklearn.preprocessing import FunctionTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler

pipe = Pipeline(
        ("gen", FunctionTransformer(generate_features)),
        ("imp_nan", SimpleImputer(strategy="median")),
        ("scale", StandardScaler()),
        ("reg", keras.wrappers.scikit_learn.KerasRegressor(build_model, epochs=35))

Where generate_features takes in my data X and produces a collection of features for a given timestamp. For example a 5 day rolling standard deviation. However, my target y could have a different shape.

For example, I may want to forecast 5 hours in the future (and so my target will have a date time index with period 5 hours), but my input data could have a frequency of 1 min (and so initially an index with period 1 min that gets transformed into a period of 5 hours). This leads to issues when trying to cross validate. For example, if I use

splitter = TimeSeriesSplit(n_splits=3)
grid = GridSearchCV(pipe, {"gen__kw_args": [{"windows": (2,4,6)},]}, cv=splitter)
grid.fit(X_train, y_train)

I immediately get an error telling me the shape of X_train and y_train are not the same. This makes sense because the splitter needs to be used on both X and y.

I want to do this because I'd like to perform a grid cv on the parameters of my generate_features function (as well as the parameters of my model...)

My Question

Is there a nice way to create a pipeline and perform cross validation on time series feature generation when our input shape of X could be different to our input shape of y?

Edit: Here is the traceback

ValueError                                Traceback (most recent call last)
<ipython-input-11-f0c6e65ecbb8> in <module>
      1 splitter = TimeSeriesSplit(n_splits=3)
      2 grid = GridSearchCV(pipe, {"gen__kw_args": [{"windows": (2,4,6)},]}, cv=splitter)
----> 3 grid.fit(X_train, y_train)

~/.local/lib/python3.8/site-packages/sklearn/model_selection/_search.py in fit(self, X, y, groups, **fit_params)
    648             refit_metric = 'score'
--> 650         X, y, groups = indexable(X, y, groups)
    651         fit_params = _check_fit_params(X, fit_params)

~/.local/lib/python3.8/site-packages/sklearn/utils/validation.py in indexable(*iterables)
    246     """
    247     result = [_make_indexable(X) for X in iterables]
--> 248     check_consistent_length(*result)
    249     return result

~/.local/lib/python3.8/site-packages/sklearn/utils/validation.py in check_consistent_length(*arrays)
    209     uniques = np.unique(lengths)
    210     if len(uniques) > 1:
--> 211         raise ValueError("Found input variables with inconsistent numbers of"
    212                          " samples: %r" % [int(l) for l in lengths])

ValueError: Found input variables with inconsistent numbers of samples: [522323, 52417]
  • $\begingroup$ Could you provide the error traceback?: it doesn't look like the search's fit checks input data; is it the pipeline, or one of the transformers, or keras? $\endgroup$
    – Ben Reiniger
    Commented May 8, 2020 at 16:24
  • $\begingroup$ @BenReiniger I've added the traceback. It seems to be coming from the fit... $\endgroup$ Commented May 8, 2020 at 17:44
  • $\begingroup$ Without the generate_features, it works, right? $\endgroup$
    – Jon Nordby
    Commented May 8, 2020 at 20:41
  • $\begingroup$ You cannot have Y in different length than X. If you want to expand X with more features, that has to be along the features axis, not along the samples axis. $\endgroup$
    – Jon Nordby
    Commented May 8, 2020 at 20:41
  • $\begingroup$ @jonnor so is there a standard way to grid search parameters of feature generation for a time series? $\endgroup$ Commented May 10, 2020 at 11:12

1 Answer 1


I've done a bit of searching and have actually found a solution to this using tsfresh.

You can find the sklearn transformers here: https://tsfresh.readthedocs.io/en/latest/text/sklearn_transformers.html

Here is the code snippet from the example.

pipeline = Pipeline(
        ('augmenter', RelevantFeatureAugmenter(column_id='id', column_sort='time')),
        ('classifier', RandomForestClassifier())

df_ts, y = load_robot_execution_failures()
X = pd.DataFrame(index=y.index)

pipeline.fit(X, y)

We pass an empty dataframe to pipeline.fit and specify the input data as a parameter to the tsfresh transformer. df_ts and y have a totally different shape!



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.