0
$\begingroup$

Let's say I have 3 dense sequences of uniform length. Should I fit a scaler on them separately or together?

import numpy as np
from sklearn.preprocessing import StandardScaler

arr = np.array([
    [
        [1.1],[2.2],[3.3]
    ],
    [
        [1.2],[2.3],[3.4]
    ],
    [
        [4.0],[5.0],[6.0]
    ]
])

SS = StandardScaler()

Separately:

SS.fit_transform(arr[0])
SS.fit_transform(arr[1])
SS.fit_transform(arr[2])

Or together?

tall_2d = np.concatenate((arr[0],arr[1],arr[2]))
SS.fit(tall_2d)

SS.transform(arr[0])
SS.transform(arr[1])
SS.transform(arr[2])

I suppose I would be performing interpolation on each sequence separately, so should I encode and detect outliers separately too?

$\endgroup$
0
$\begingroup$

After getting hands on with the data, it feels ridiculous not to fit on all samples in the split/fold.

  • In 2D data 'sample==row'. You don't fit on a single sample.
  • In 3D data 'sample==sequence' so you encode on all of the sequences.

This also means less encoders to keep track of for the sake of inverse transform and inference encoding.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.