I have a regression problem for which two observations are compared by a siamese-like Multilayer Perceptron.

Each observation 'O' is described by a feature vector 'X' of a certain number 'N' of features 'F', such that the vector pair [X_n, X_m] from observations [O_n, O_m] is fed into the network. After a first passage through the twin channel, resulting embeddings 'E' are pairwise-subtracted:

X_n -> twin_ch -> E_n

X_m -> twin_ch -> E_m

dE = E_n - E_m

The delta embedding 'dE' is generated and then passed through the common channel:

dE -> common_ch -> prediction

A bidirectional similarity score 'S' is provided back. S can be either positive or negative and it's a computed from O pairs.

I would like to know any method or rationale behind the choice of an optimal subset of features in order to reduce the dimensionality of X. I guess that the problem here is to find the best feature which's pair [F_i_n, F_i_m] is optimally contributing in predicting the relative score.

I've tried to use correlation coefficients directly between each feature and the observations array:

R_i = pearson_correlation(F_i, O)

but a part from the poor results, I think it really misses the 'pair' concept. Hence I also tried to operate several empiric 'relation functions' like:

[F_i_n, F_i_m] > F_i_n - F_i_m = d_F_i_m


[F_i_n, F_i_m] > (F_i_n - F_i_m) / (F_i_n + F_I_m) = s_F_i_m

compressing each pair in a variation metric and then using the resulting derived feature for the correlation test with S, but despite it's a closer rationale to what I'm looking for, it does not work as expected.

Any idea or paper?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.