3
$\begingroup$

My question is simple actually, I have two features that have big difference in scale. So I used a simple normalization by dividing the scale=np.max(array) for both data and lables. Then after prediction, I mulitiplied this scale value back.

But since I used a DNN, wouldn't the nonlinear change the scale so make the multiply not valid? e.g.

given input data: X, label: y;
y' = y/scale
X' = X/scale
predicted = f(X')
predicted_update = predicted * scale

Anyone could provide some advice on whether I could do this or it's actually not correct? How do we handle this kind of problem?

$\endgroup$
1
$\begingroup$

I think it is ok, as long as your training and test data have the same maximum values for every feature, approximately. The idea is that the scaling has to be done with the training set (remember that using the test set for anything that is not testing is illegal, not even for scaling).

So, you actually fit $y'$ as a function of $X'$, and you have a model that maps properly $y' = f(X')$. When you get your test data, you just obtain the predictions by doing $f(X_{test}')$. As the paragraph before states, if you have that $scale \approx scale_{test}$, then you can just recover $y_{test}$ by doing $y_{test} = scale \cdot f(X_{test}')$.

Edit: Don't worry about nonlinearities

Even if the function $f$ is highly nonlinear, it is a function capable of mapping $X'$ to $y'$. If you trust this function and trust the fact that $y = y' \cdot scale$, then there is no need to worry about the way $f$ acts, as function composition makes sense for all kind of functions, both linear and nonlinear.

$\endgroup$
4
  • $\begingroup$ Thanks for the comment. For training/test set, I'm splitting the whole data set. And both are scaled before training/testing. And the np.max() is a simplied notation. I use separate scale for different features though I could use one. The question is actually about the bold line you mentioned. Why should i not worry about nonlinearities? I'm doubting the validity of y=y' * scale $\endgroup$ May 16 '18 at 7:11
  • $\begingroup$ If $scale \approx scale_{test}$, then $y_{test} = scale \cdot y_{test}'$. It is completely valid, scaling and getting back to usual scale are inverse transformations. $\endgroup$ May 16 '18 at 7:29
  • $\begingroup$ Suppose I use a universal scale for all X, and y, regardless of training/test data. Then y=f(X) is my original problem and what I wanted to learn is f. After scaling, I actually is training with y'_train = g(X'_train) and then I get my $y'_{test} = g(X'_{test})$. To make sure y'_test == y_test, we have to make sure g=~f. Am I right? $\endgroup$ May 18 '18 at 3:09
  • $\begingroup$ Correction to last sentence: since $ y_{test} = f(X_{test}) $ and $ y'_{test} = g(X'_{test}) $, to make sure $y_{test} == scale * y'_{test}$, I have to make sure $f(X_{test}) = scale * g(X'_{test}) $ . Which equals to: $f(X_{test}) = scale * g(X_{test}/scale) $ and this is not gauranteed? $\endgroup$ May 18 '18 at 3:21
0
$\begingroup$

What I saw from another post: How to obtain original coefficients after performing linear regression on normalized data?

and https://stats.stackexchange.com/questions/201909/when-to-normalize-data-in-regression

Looks like if the data is invariant to regression or if it's linear then it's ok to multiply the coefficient back. If not, generally it's not accurate. So it seems if we were to predict the value, better not scale. My various tests shows that it get worse to scale back. Hopefully someone else could have a better answer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.