1
$\begingroup$

I have some vectors {$\mathbf{X_1 ... X_n}$} and they are all of dimension 1 x N. Vectors {$\mathbf{X_1' ... X_n'}$} are also 1 x N and are related to {$\mathbf{X_1 ... X_n}$}, but the relation cannot be modeled by a function.

I want to train a neural network such that for each $\mathbf{X_i}$ I input, it gives me a $\mathbf{Y_i}$ where the loss function to optimize is $\mathbf{X_i' Y_i}$. The reason I do not use $\mathbf{X_i'}$ as the input is that I do not have access to them during testing. The constraint on $\mathbf{Y_i}$ is that the norm is 1.

I tried an implementation similar to this post here (https://stackoverflow.com/questions/46464549/keras-custom-loss-function-accessing-current-input-pattern) which is:

def custom_loss_wrapper(input_tensor):
    def custom_loss(y_true, y_pred):
        return keras.losses.mean_squared_error(y_true, y_pred) + f(input_tensor)
    return custom_loss

However, I found that adding f(input_tensor) only changes the calculated loss, and not the back-propagation itself. The neural network produces same output $\mathbf{Y'}$ with or without the f(input_tensor).

I also tried direct optimization without a neural network, using TensorFlow optimizer.minimize() or similar strategies, and the results are not very good.

Any ideas how I may build this network?

|improve this question
$\endgroup$
  • 1
    $\begingroup$ I don't really get what you are asking. Any good example/reference that you can show? Try rereading and for now as far as your explanation goes I could just throw-in $Y_i$ to be zero-vector and your loss is optimal. $\endgroup$ – Yohanes Alfredo Nov 21 at 16:56
  • $\begingroup$ @Yohanes Alfredo I forgot to mention there is a constraint on Y. The norm of Y must be 1 $\endgroup$ – Y.Z. Nov 21 at 18:56
  • $\begingroup$ Why do you need f(input_tensor)? Isn't y_true = X' and y_pred = Y all you have to pass to custom loss? $\endgroup$ – Valentas Nov 22 at 7:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.