2
$\begingroup$

I am using MinMaxScaler on my training set and applying the transformations to my test set and inverse_transform to my model’s outputs. If this were, say, a stock prediction problem, my training set may have values between 1-200, but in the most recent time steps, the values are toward the high end of that range.

How can I handle future time-steps that should be predicted to land above 200 (since activation function outputs values between 0-1)? I guess my question is, is there a way to manually (or through some “rule of thumb”) specify a maximum value for the training set? So in this case, would it be possible to use MinMaxScaler to use 0-500 as the min-max values to transform to 0-1, that way values in my test set or predictions can be inverse_transformed accordingly? Is this an out-of-the-box scikit-learn feature? What’s the most efficient way to implement this, especially if this has to be done independently for multiple features?

$\endgroup$
1
$\begingroup$

Inverse-transforming with MinMaxScaler should be capable of producing something outside of the training data's range. It seems that, in your use case, using a final activation that lands in $[0,1]$ might not be appropriate. Even if you transform the training data to land in, say, $[0,0.7]$, applying a sigmoid or some-such on the final layer seems to lack motivation.

As to the question of whether it can be done: yes, just with a little roundabout thinking. You can't specify what you want the scaler to think your data's max and min are, but you can specify the output range you want (parameter feature_range), which amounts to the same thing.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.