My LSTM neural network predicts nominal values between -1 and 1. I would like to set up a custom loss function in Keras that assigns a weight function depending on the predicted sign.

If the predicted sign is positive, a sigmoid weight function should scale prediction errors between 1 (for the most negative prediction error) and 2 (most positive prediction error). If the predicted sign is negative, I would like to use the reverse weight function.

You can reproduce my error with the following code snippet:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, LSTM
from keras import backend as K

# loss function
def lfunc(true,pred):
    diff = pred - true
    def weight(y_hat, error):
        if y_hat>=0: weight = K.sigmoid(error*100)+1
        else: weight = (-K.sigmoid(error*100)) +2    
        return weight

    scale = weight(pred, diff)
    return K.mean(scale*K.square(diff))

# data
trainset = np.random.uniform(low=-1, high=1, size=(100000,13))
no_variables = 3
lookback = 4
train_x, train_y = trainset[:80000][:, :-1], trainset[:80000][:, -1:]
val_x, val_y = trainset[80000:][:, :-1], trainset[80000:][:, -1:]
train_x = train_x.reshape((train_x.shape[0], lookback, no_variables))
val_x = val_x.reshape((val_x.shape[0], lookback, no_variables))

# model set up
model = Sequential()
model.add(LSTM(2, return_sequences=True,
                input_shape=(lookback, no_variables), dropout=0.3))
model.add(LSTM(1, input_shape=(lookback, no_variables), dropout=0.3))
model.compile(loss=lfunc, optimizer='SGD')

#train model
result = model.fit(train_x, train_y, epochs=1, batch_size=2**4, 
                   validation_data=(val_x, val_y), verbose=1, shuffle=False)

The error arises in the model.fit command. The error message looks like:

Traceback (most recent call last):

  File "C:\Users\chris\OneDrive\Dokumente\GitHub\LSTM\data\untitled0.py", line 38, in <module>

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\engine\training.py", line 1158, in fit
    tmp_logs = self.train_function(iterator)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\def_function.py", line 889, in __call__
    result = self._call(*args, **kwds)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\def_function.py", line 933, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\def_function.py", line 763, in _initialize
    self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\function.py", line 3050, in _get_concrete_function_internal_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\function.py", line 3444, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\function.py", line 3279, in _create_graph_function

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\framework\func_graph.py", line 999, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\def_function.py", line 672, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\framework\func_graph.py", line 986, in wrapper
    raise e.ag_error_metadata.to_exception(e)

OperatorNotAllowedInGraphError: in user code:

    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\engine\training.py:830 train_function  *
        return step_function(self, iterator)
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\engine\training.py:813 run_step  *
        outputs = model.train_step(data)
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\engine\training.py:771 train_step  *
        loss = self.compiled_loss(
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\engine\compile_utils.py:201 __call__  *
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\losses.py:142 __call__  *
        losses = call_fn(y_true, y_pred)
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\losses.py:246 call  *
        return ag_fn(y_true, y_pred, **self._fn_kwargs)
    C:\Users\chris\OneDrive\Dokumente\GitHub\LSTM\data\untitled0.py:16 lfunc  **
    C:\Users\chris\OneDrive\Dokumente\GitHub\LSTM\data\untitled0.py:12 weight
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\framework\ops.py:900 __bool__
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\framework\ops.py:503 _disallow_bool_casting
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\framework\ops.py:489 _disallow_when_autograph_enabled
        raise errors.OperatorNotAllowedInGraphError(

    OperatorNotAllowedInGraphError: using a `tf.Tensor` as a Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

Does someone has any suggestions how to implement this problem with a Keras backend?

  • $\begingroup$ I ran this code snippet, and worked fine. What's the error? Please add the whole error message. It also shows the function and line where raises the error. Also add the line where you pass this loss function to your model, which probably is model.compile() $\endgroup$
    – Kaveh
    Jul 9 at 19:19
  • $\begingroup$ Hi Kaveh, I added all further information on the problem $\endgroup$ Jul 11 at 10:50

if y_hat>=0: weight = K.sigmoid(error100)+1
cond = _verify_tf_condition(cond, 'if statement')
verify_tf_condition '; {}'.format(tag, cond, extra_hint))
ValueError: condition of if statement expected to be tf.bool scalar, got Tensor*

If we observe the above logs (trimmed),
We are assuming y_hat to be a scaler. Hence the code will work for batch_size=1 and throws the above error for batch_size>=2

You should handle if y_hat>=0 code for Tensor.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.