I have a regression problem for which two observations are compared by a siamese-like Multilayer Perceptron.
Each observation 'O' is described by a feature vector 'X' of a certain number 'N' of features 'F', such that the vector pair [X_n, X_m] from observations [O_n, O_m] is fed into the network. After a first passage through the twin channel, resulting embeddings 'E' are pairwise-subtracted:
X_n -> twin_ch -> E_n
X_m -> twin_ch -> E_m
dE = E_n - E_m
The delta embedding 'dE' is generated and then passed through the common channel:
dE -> common_ch -> prediction
A bidirectional similarity score 'S' is provided back. S can be either positive or negative and it's a computed from O pairs.
I would like to know any method or rationale behind the choice of an optimal subset of features in order to reduce the dimensionality of X. I guess that the problem here is to find the best feature which's pair [F_i_n, F_i_m] is optimally contributing in predicting the relative score.
I've tried to use correlation coefficients directly between each feature and the observations array:
R_i = pearson_correlation(F_i, O)
but a part from the poor results, I think it really misses the 'pair' concept. Hence I also tried to operate several empiric 'relation functions' like:
[F_i_n, F_i_m] > F_i_n - F_i_m = d_F_i_m
or
[F_i_n, F_i_m] > (F_i_n - F_i_m) / (F_i_n + F_I_m) = s_F_i_m
compressing each pair in a variation metric and then using the resulting derived feature for the correlation test with S, but despite it's a closer rationale to what I'm looking for, it does not work as expected.
Any idea or paper?
