I am using crypto currency chart data and was wondering what would be the best process for scaling the time series data. The issue is that some of these currencies are highly volatile and can see extreme swings. The standard process that I see is something like:
from sklearn.model_selection import train_test_split from sklearn.preprocessing import MinMaxScaler x_train, y_train, x_test, y_test = train_test_split(x,y, test_size=0.2) scaler = MinMaxScaler() scaler.fit(x_train) x_train = scaler.transform(x_train) x_test = scaler.transform(x_test)
By default MinMaxScaler scales between 0 and 1 but there are cases where x_test may contain a max that could be double or more what the max was when the scaler was fit. Will this cause model issues or is this acceptable? How do you handle scaling data that you don't know the min/max for? Is min/max scaling best for this?
Also, some currencies have drastically different prices. For example, one could be trading +10x or -10x or more per unit than another. Will having several time series on drastically different scales effect the models? Its seems to me that scaling each time series and each feature to 0-1 would be the best but I am not completely sure. Lets say I have n different time series that have drastically different values. Would something like the following be a decent approach?
#assume data is a list of padas DataFrames #e.g. data = [pd.DataFrame(asset1), pd.DataFrame(asset2)] scaled =  for asset in data: fit_data = data.iloc[:int(len(data*0.2))] scaler = MinMaxScaler() scaler.fit(fit_data) scaled.append(scaler.transform(asset))