Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I am working a price prediction LTSM model for the stock market. I am using multiple features: Open, Close, High and I would like to add the Volume.
The 3 first features are of the same nature but the volume presents much higher values.
What would be the safest way to keep consistency during the normalization process?
You would usually just scale all of them to be within the same range.
You can do this by using something like the Scikit-Learn MinMaxScaler, or just a simple function like this:
def scale(data, new_min=-1, new_max=1)
"""Scale values of data to be within the range [new_min, new_max]
data must be a numpy array or a Pandas Dataframe/Series"""
return (data - data.min()) / (data.max() - data.min()) * (new_max - new_min) + new_min
Between -1 and +1 is just nice, as the data is centered around zero. You could play with those values.
You can think about and perhaps experiment to see if you should scale all variables together (meaning one global min and max in the dataset), or whether to scale the individual columns/features of your dataset, so each one lies in the given range.
A tip for financial data is to use the log returns - that means to take your raw prices, compute the logarithm of those values, then take the difference between the closing prices of each day.
The reason for this is to because the resulting values are normally distributed, which is an underlying assumption of many models you will subsequently use (Boosting, ARIMA, GARCH for volatilities etc.). There are also other reasons of convenience - check out this article
