0
$\begingroup$

I have input data, the magnitudes vary quite a lot between features. I have scaled them using sklearn's StandardScaler(), then used keras to train a NN on this data to predict my target. I have trained/tested this neural network and it performs well.

Training data is a timeseries of input to a function as such of a bigger model, and my target being a timeseries of output. I have trained the model on scaled input data (off-line), but would like it to replace that function in the model. Then it would take unscaled input data (same features) at each time step and give a good prediction for the target (output).

Being trained on the timeseries data offline, it does not seem best suited to then take unscaled data, one tilmestep a time, which it will have to when used in the model. Should I scale that online-input also (and how to make it consistent with scaled offline input?) or should I e.g. train on unscaled data and use "this optimisation method which performs well with unscaled data"?

Does anyone know how to fix this type of issue?

$\endgroup$
1
  • $\begingroup$ My model is in fortran, so I had to use python offline to generate it, fortran to explicitly code it. One option is to extract the "transform" from sklearn, but how do I know how I may extract sklearns transform (for example if for x_1, ... , x_n, x_1 is transformed to x_1/2.5 (say), and x_2=x_2/3.17 and ...) Could I maybe call something like transform.weights (say), or similar, to extract that transform explicitly so I know how much each is scaled by - so that I can then explicitly code what sklearn scales it by into fortran? $\endgroup$
    – Socorro
    Commented Mar 2, 2022 at 9:10

1 Answer 1

0
$\begingroup$

As you mention, you should indeed apply the same transformations that you applied when training your model to any new data. For scikit-learn you can either use the pickle or joblib file format to save your transformation steps/pipelines (as described in the documentation) and then load it again in the script where you are predicting using new data and use the transform method to apply the transformations to the new data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.