0
$\begingroup$

I recently started saving (pickling) my fitted encoders. The thinking was that I would need them to encode previously unseen samples during inference.

  • Encode training features and labels.
  • Train model on encoded data.
  • Export fitted encoders alongside model.
  • New/ unseen data arrives.
  • Encode features of new data.
  • Run model prediction.
  • Decode prediction output.

Is this mandatory?

$\endgroup$
0
$\begingroup$

Yes the encoders need to be rerun during inference time. You basically need to run the exact same data preprocessing steps prior to training on the unseen observations. Without them your model will not be to recognize unseen data and hence fair poorly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.