# Custom loss function with both min(y, p) and max(y,p)

I'm creating a neural network in tensorflow and need to minimize the following loss function: $$\frac{max(y,p)}{min(y,p)}$$ Where $$y$$ represents the true value and $$p$$ the predicted value. Since the loss function is not differentiable it becomes a problem when using gradient descent.

Update, now I'm trying to implement:
$$p > y \rightarrow loss = log(p)-log(y)$$
$$p < y \rightarrow loss = log(y)-log(p)$$
$$p = y \rightarrow loss = log(y)-log(y) = log(p)-log(p)$$

This is my code:

def custom_loss(y_true, pred):
if pred > y_true:
custom_loss = K.log(pred) - K.log(y_true)
elif pred < y_true:
custom_loss = K.log(y_true) - K.log(pred)
else:
custom_loss = K.log(pred)-K.log(pred)

return custom_loss

if __name__ == '__main__':
run = 1

with open("target2.pickle", "rb") as file:

target_strings = [*target_dict]

Y = np.array([])

for target_value in target_strings:
Y = np.append(Y, target_dict.get(target_value))

for i in range(run):
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.2)
model = get_model()
start = time.time()
model.fit(X_train, Y_train, verbose = 1, epochs = 1, batch_size = 32)
end_time = time.time()-start
pred = model.predict(X_test)[:, 0]


But I get the following error:

tensorflow.python.framework.errors_impl.InvalidArgumentError:  The second input must be a scalar, but it has shape [32]



The code can run when I have batch_size = 1

• Exactly what are you trying to achieve with your loss function? An example might help. Minimizing the ratio of predictions and true values might lead the model to predict too low or too high values anyway, so why not use a simple difference function as loss? Oct 20 '20 at 19:29
• Well, it is important that the relative error is as low as possible. For example, a prediction of p = 1 and y = 2 would generate an error of: max(2,1)/min(2,1) = 2 Another example: p = 100, y = 101 gives max(101,100)/min(101,100) = 101/100 So, in MAE and MSE terms the both examples are equally accurate, but for my specific problem the first example is a worse prediction. A perfect prediction would generate max/min = 1. Hopefully this clears things up.
– qer
Oct 20 '20 at 19:36
• Have you explored mean relative error instead? Oct 20 '20 at 19:45
• Yes, not super satisfied with the results from that.
– qer
Oct 20 '20 at 19:50

A loss function calculates the error over all the data presented to it. For neural networks, that is an average over the mini-batch. Your code might only handle scalars (hence working when batch=1), it should handle vectors.

The K looks like you are using tf.Keras. The documentation gives examples of custom loss functions. It helps to subclasses tf.keras.losses.Loss which expects a call() method that contains the logic for loss calculation using y_true, y_pred.

class CustomLoss(Loss):

def call(self, y_true, y_pred):
pass