# normalization/denormalization for linear regression problem

My question is simple actually, I have two features that have big difference in scale. So I used a simple normalization by dividing the scale=np.max(array) for both data and lables. Then after prediction, I mulitiplied this scale value back.

But since I used a DNN, wouldn't the nonlinear change the scale so make the multiply not valid? e.g.

given input data: X, label: y;
y' = y/scale
X' = X/scale
predicted = f(X')
predicted_update = predicted * scale


Anyone could provide some advice on whether I could do this or it's actually not correct? How do we handle this kind of problem?

I think it is ok, as long as your training and test data have the same maximum values for every feature, approximately. The idea is that the scaling has to be done with the training set (remember that using the test set for anything that is not testing is illegal, not even for scaling).

So, you actually fit $y'$ as a function of $X'$, and you have a model that maps properly $y' = f(X')$. When you get your test data, you just obtain the predictions by doing $f(X_{test}')$. As the paragraph before states, if you have that $scale \approx scale_{test}$, then you can just recover $y_{test}$ by doing $y_{test} = scale \cdot f(X_{test}')$.

Even if the function $f$ is highly nonlinear, it is a function capable of mapping $X'$ to $y'$. If you trust this function and trust the fact that $y = y' \cdot scale$, then there is no need to worry about the way $f$ acts, as function composition makes sense for all kind of functions, both linear and nonlinear.
• Thanks for the comment. For training/test set, I'm splitting the whole data set. And both are scaled before training/testing. And the np.max() is a simplied notation. I use separate scale for different features though I could use one. The question is actually about the bold line you mentioned. Why should i not worry about nonlinearities? I'm doubting the validity of y=y' * scale May 16 '18 at 7:11
• If $scale \approx scale_{test}$, then $y_{test} = scale \cdot y_{test}'$. It is completely valid, scaling and getting back to usual scale are inverse transformations. May 16 '18 at 7:29
• Suppose I use a universal scale for all X, and y, regardless of training/test data. Then y=f(X) is my original problem and what I wanted to learn is f. After scaling, I actually is training with y'_train = g(X'_train) and then I get my $y'_{test} = g(X'_{test})$. To make sure y'_test == y_test, we have to make sure g=~f. Am I right? May 18 '18 at 3:09
• Correction to last sentence: since $y_{test} = f(X_{test})$ and $y'_{test} = g(X'_{test})$, to make sure $y_{test} == scale * y'_{test}$, I have to make sure $f(X_{test}) = scale * g(X'_{test})$ . Which equals to: $f(X_{test}) = scale * g(X_{test}/scale)$ and this is not gauranteed? May 18 '18 at 3:21