3
$\begingroup$

I have 3 datasets which I each split into 3 separate classes [Buy/hold/sell]. I randomly up-sample each class's frequency in each dataset to 10,000 data points each.

My question is, should I scale the training set before I do random up-sampling, or afterwards? Does it somehow skew the final training set in some way?

I have provided a function which balances each dataset for me, note that at this point, the data has already been scaled.

def balance_dataset(df): 
    training_set = df[:round(len(df.values) * TRAINING_LENGTH)]

    label_frequencies = training_set['Label'].value_counts(sort = True, ascending = True)

    highest_occurence = resample(training_set[training_set['Label'] == label_frequencies.index[2]], n_samples = 10000, random_state = 0, replace = True)
    middle_occurence = resample(training_set[training_set['Label'] == label_frequencies.index[1]], n_samples = 10000, random_state = 0, replace = True)
    lowest_occurrence = resample(training_set[training_set['Label'] == label_frequencies.index[0]], n_samples = 10000, random_state = 0, replace = True)

    balanced_training_set = pd.concat([highest_occurence, middle_occurence, lowest_occurrence])


    return balanced_training_set[['MACD', 'MFI', 'ROC', 'RSI', 'Ultimate Oscillator', 'Williams %R', 'Awesome Oscillator', 'KAMA', 
            'Stochastic Oscillator', 'TSI', 'Volume Accumulator', 'ADI', 'CMF', 'EoM', 'FI', 'VPT','ADX', 'ADX Negative', 'ADX Positive', 
            'EMA', 'CRA']],  balanced_training_set['Label']

$\endgroup$
3
  • $\begingroup$ Welcome to the community. Before answering, have you seen these answers: stats.stackexchange.com/questions/363312/…, stats.stackexchange.com/questions/60180/…? Let me know if it is not yet clear. Short answer is, rule-of.thumb is do scale sampling, although in random sampling it may become indifference, but you also have a imbalance, so its original class dist. should be respected as well, in short, all operations better to be done before. $\endgroup$ Mar 31, 2020 at 21:35
  • $\begingroup$ Thank your for your comment. The post made sense, the answer said that all scaling etc should be done BEFORE any balancing. However, this was from the point of view of using SMOTE which requires a model to be trained first. However it then went on to say that it shouldn't necessarily affect whether scaling is down before random up/sampling balancing. $\endgroup$ Mar 31, 2020 at 21:40
  • $\begingroup$ That is right. It seems to be a rather open area and one needs to look into case by case. I would be cautious doing either way, and if there is a way to examine both. There are rare cases that scaling (standardization) alters the Euclidean space of the feature space if data is not let's normally distributed, otherwise it is safe. $\endgroup$ Mar 31, 2020 at 22:00

3 Answers 3

3
$\begingroup$

My two cents: a general way to think about this process is in terms of learning and transformations. Scaling (standardization) is a transformation that you apply to every sample both in your training and test/validation/production set. These transformations are done using parameters that are learned using the training set. The aim of up/down sampling is to get a training set where you can better learn the parameters of your transformations - up/down sampling is not by itself a transformation. So, you usually first up/down sample your training set and then apply any scaling. In practice, as noted in the comments and in the response by @Angadishop up/down sampling may not alter the parameters that you learn for your scaling so that doing it before or after may give you the same results (but this depends on the actual scaling function).

$\endgroup$
1
$\begingroup$

Scaling, in general, depends on the min and max values in your dataset and up sampling, down sampling or even smote cannot change those values. So if you are including all the records in your final dataset then you can do it at anytime but, if you are not including all of your original records then you should do it before upsampling.

$\endgroup$
0
$\begingroup$

I would like to contribute to this post because there is little discussion on the examples of the implications of scaling on balancing. SMOTE and Tomek links are based on nearest neighbors algorithms and thus on distance measures. A combined oversampling using SMOTE and undersampling using Tomek links from the imblearn package is a perfect display for how different scales of data may impact the outcome of balancing. First have a look at a simulated bivariate data on the same scale with majority and minority classes. In this case SMOTE oversampling prevails, whereas undersampling with Tomek links has rather modest implications on the data structure.

Example for two variables with a similar scale

Now let us just scale one of the variables by 1000 and see how oversampling and undersampling perform on data with different scales. Obviously the results would be different, but in what way? Below you can see that undersampling may produce suboptimal results while removing (more of) majority class instances which are not even close to the minority class instances. Oversampling behaves in a different way as well - the locations of the synthetic minority class instances are different.

This may be an extreme example with factor 1000 but in large datasets smaller differences in scales may have substantial implications for the data structure. Therefore bringing the data to the same/similar scale would be on the safer side.

Example for two variables with a different scale (factor 1000)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.