0
$\begingroup$

Suppose I have input values that represent the change of a stock share from each time step to the next. Now I want to feed these values into an LSTM Neural Net. My problem is that most values are between -0.000001 and 0.000001, but some values go up to 0.1 or -0.1. How can I effectively scale such a dataset. I've tried applying a log function (to the value +1, so e.g. log(1.000001)) and scaling between 0 and 1 and I've tried capping values at around 1.5. Do you have any suggestions how to deal with such extreme outliers better?

$\endgroup$
2
$\begingroup$

Play a bit with your data:

  • try doing the scaling without these outliers and see what happens
  • have you tried training the model regardless and see what happens? then train the model without the "outliers"
  • you could also binarise the feature (and thus transform it into a category) - then in this case you don't have to deal with the large range while the information is still in there.
| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.