I'm encountering a scaling issue in a machine learning project. I'm predicting a target variable from an input sequence (and doing this for many). However, I've encountered a challenge where the predicted target variable sometimes falls outside the range of the input sequence.
Say the input sequence consists of values ranging between -5 and 5. Most of the time, my target variable is in that range but occasionally, it can say go up to 10 or down to -10. Then, having scaled my x values together using a MinMax scaler that I fit only to them, they're between 0 and 1. If i then use that scaler to transform a target variable of 10, I end up with a value above 1. Which is higher than the tanh activation function's range.
So of course, training the model, I get a very high error rate because it doesn't often want to predict above 1 or below 0. I also think I need to scale the target variables because for two otherwise similar sequences, the target variables can be different because each sequence isn't necessarily using the same product.
Any ideas regarding scaling or a way to bypass it even?