1
$\begingroup$

I'm creating a neural network in tensorflow and need to minimize the following loss function: $\frac{max(y,p)}{min(y,p)}$ Where $y$ represents the true value and $p$ the predicted value. Since the loss function is not differentiable it becomes a problem when using gradient descent.

Update, now I'm trying to implement:
$p > y \rightarrow loss = log(p)-log(y)$
$p < y \rightarrow loss = log(y)-log(p)$
$p = y \rightarrow loss = log(y)-log(y) = log(p)-log(p)$

This is my code:

def custom_loss(y_true, pred):
    if pred > y_true:
        custom_loss = K.log(pred) - K.log(y_true)
    elif pred < y_true:
        custom_loss = K.log(y_true) - K.log(pred)
    else:
        custom_loss = K.log(pred)-K.log(pred)

    return custom_loss


if __name__ == '__main__':
    run = 1

    X = np.load("vectors_normal_2way.npy")

    with open("target2.pickle", "rb") as file:
        target_dict = pickle.load(file)

    target_strings = [*target_dict]

    Y = np.array([])

    for target_value in target_strings:
        Y = np.append(Y, target_dict.get(target_value))

    for i in range(run):
        X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.2)
        model = get_model()
        start = time.time()
        model.fit(X_train, Y_train, verbose = 1, epochs = 1, batch_size = 32)
        end_time = time.time()-start
        pred = model.predict(X_test)[:, 0]

But I get the following error:

tensorflow.python.framework.errors_impl.InvalidArgumentError:  The second input must be a scalar, but it has shape [32]
     [[{{node gradient_tape/custom_loss/cond/StatelessIf/gradient_tape/custom_loss/weighted_loss/Mul/_17}}]] [Op:__inference_train_function_1040]
 

The code can run when I have batch_size = 1

$\endgroup$
4
  • $\begingroup$ Exactly what are you trying to achieve with your loss function? An example might help. Minimizing the ratio of predictions and true values might lead the model to predict too low or too high values anyway, so why not use a simple difference function as loss? $\endgroup$
    – 404error
    Oct 20 '20 at 19:29
  • $\begingroup$ Well, it is important that the relative error is as low as possible. For example, a prediction of p = 1 and y = 2 would generate an error of: max(2,1)/min(2,1) = 2 Another example: p = 100, y = 101 gives max(101,100)/min(101,100) = 101/100 So, in MAE and MSE terms the both examples are equally accurate, but for my specific problem the first example is a worse prediction. A perfect prediction would generate max/min = 1. Hopefully this clears things up. $\endgroup$
    – qer
    Oct 20 '20 at 19:36
  • $\begingroup$ Have you explored mean relative error instead? $\endgroup$
    – 404error
    Oct 20 '20 at 19:45
  • $\begingroup$ Yes, not super satisfied with the results from that. $\endgroup$
    – qer
    Oct 20 '20 at 19:50
1
$\begingroup$

A loss function calculates the error over all the data presented to it. For neural networks, that is an average over the mini-batch. Your code might only handle scalars (hence working when batch=1), it should handle vectors.

The K looks like you are using tf.Keras. The documentation gives examples of custom loss functions. It helps to subclasses tf.keras.losses.Loss which expects a call() method that contains the logic for loss calculation using y_true, y_pred.

class CustomLoss(Loss):

  def call(self, y_true, y_pred):
    pass
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.