0
$\begingroup$

I am using crypto currency chart data and was wondering what would be the best process for scaling the time series data. The issue is that some of these currencies are highly volatile and can see extreme swings. The standard process that I see is something like:

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
x_train, y_train, x_test, y_test = train_test_split(x,y, test_size=0.2)
scaler = MinMaxScaler()
scaler.fit(x_train)
x_train = scaler.transform(x_train)
x_test = scaler.transform(x_test)

By default MinMaxScaler scales between 0 and 1 but there are cases where x_test may contain a max that could be double or more what the max was when the scaler was fit. Will this cause model issues or is this acceptable? How do you handle scaling data that you don't know the min/max for? Is min/max scaling best for this?

Also, some currencies have drastically different prices. For example, one could be trading +10x or -10x or more per unit than another. Will having several time series on drastically different scales effect the models? Its seems to me that scaling each time series and each feature to 0-1 would be the best but I am not completely sure. Lets say I have n different time series that have drastically different values. Would something like the following be a decent approach?

#assume data is a list of padas DataFrames
#e.g. data = [pd.DataFrame(asset1), pd.DataFrame(asset2)]
scaled = []
for asset in data:
    fit_data = data.iloc[:int(len(data*0.2))]
    scaler = MinMaxScaler()
    scaler.fit(fit_data)
    scaled.append(scaler.transform(asset))
$\endgroup$
1
$\begingroup$

Rather than using MinMaxScaler, I think StandardScaler would be a better option if you have outliers that are quite different from most of the values, but are still valid data.

StandardScaler is used the same way as MinMaxScaler, but it will scale your values so they have mean equal to 0 and standard deviation equal to 1. Since those values will be found based on all the values in the series, it is much more robust against outliers.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.