Data preprocessing, relative scale problems in features of same type

Question

I am using Keras NN with theanos backend in Python. In my data i have multiple features of the same type but in different columns (on purpose). Here is an example.

   F1   F2
0. 8673 7490
1. 5602 5602
2. 4352 2365

When i go to process this, say MinMaxScaler 0-1, even though '1.' in both features should be the same, because both columns have different min-max values won't the normalized value be different and therefor cause problems in pattern finding? If so, what method can i use to make sure that they are of the same relative scale? Thanks for any help.

Answer 1

If want you want is to scale both columns using the maximum of both and the minimum of both. You can do it like this:

df = pd.DataFrame(data=data, columns=['F1', 'F2'])
columns = ['F1', 'F2']

ma = float(np.amax(df[columns].values))
mi = float(np.amin(df[columns].values))

for column in columns:
    df[column] = df[column].apply(lambda e: (e - mi) / (ma - mi))

According to the documentation of sklearn that is what MaxMinScaler does, if you want to use predefined values for max and min, you can just change ma and mi. If you have more columns you just have to change the list of columns. For rescaling to an arbitrary range (max, min) read the sklearn documentation on MaxMinScaler.