0
$\begingroup$

My LSTM neural network predicts nominal values between -1 and 1. I would like to set up a custom loss function in Keras that assigns a weight function depending on the predicted sign.

If the predicted sign is positive, a sigmoid weight function should scale prediction errors between 1 (for the most negative prediction error) and 2 (most positive prediction error). If the predicted sign is negative, I would like to use the reverse weight function.

You can reproduce my error with the following code snippet:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, LSTM
from keras import backend as K

# loss function
def lfunc(true,pred):
    
    diff = pred - true
    
    def weight(y_hat, error):
        if y_hat>=0: weight = K.sigmoid(error*100)+1
        else: weight = (-K.sigmoid(error*100)) +2    
        return weight

    scale = weight(pred, diff)
    
    return K.mean(scale*K.square(diff))

# data
trainset = np.random.uniform(low=-1, high=1, size=(100000,13))
no_variables = 3
lookback = 4
train_x, train_y = trainset[:80000][:, :-1], trainset[:80000][:, -1:]
val_x, val_y = trainset[80000:][:, :-1], trainset[80000:][:, -1:]
train_x = train_x.reshape((train_x.shape[0], lookback, no_variables))
val_x = val_x.reshape((val_x.shape[0], lookback, no_variables))

# model set up
model = Sequential()
model.add(LSTM(2, return_sequences=True,
                input_shape=(lookback, no_variables), dropout=0.3))
model.add(LSTM(1, input_shape=(lookback, no_variables), dropout=0.3))
model.add(Dense(1))
model.compile(loss=lfunc, optimizer='SGD')

#train model
result = model.fit(train_x, train_y, epochs=1, batch_size=2**4, 
                   validation_data=(val_x, val_y), verbose=1, shuffle=False)

The error arises in the model.fit command. The error message looks like:

Traceback (most recent call last):

  File "C:\Users\chris\OneDrive\Dokumente\GitHub\LSTM\data\untitled0.py", line 38, in <module>

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\engine\training.py", line 1158, in fit
    tmp_logs = self.train_function(iterator)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\def_function.py", line 889, in __call__
    result = self._call(*args, **kwds)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\def_function.py", line 933, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\def_function.py", line 763, in _initialize
    self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\function.py", line 3050, in _get_concrete_function_internal_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\function.py", line 3444, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\function.py", line 3279, in _create_graph_function
    func_graph_module.func_graph_from_py_func(

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\framework\func_graph.py", line 999, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\eager\def_function.py", line 672, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)

  File "C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\framework\func_graph.py", line 986, in wrapper
    raise e.ag_error_metadata.to_exception(e)

OperatorNotAllowedInGraphError: in user code:

    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\engine\training.py:830 train_function  *
        return step_function(self, iterator)
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\engine\training.py:813 run_step  *
        outputs = model.train_step(data)
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\engine\training.py:771 train_step  *
        loss = self.compiled_loss(
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\engine\compile_utils.py:201 __call__  *
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\losses.py:142 __call__  *
        losses = call_fn(y_true, y_pred)
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\keras\losses.py:246 call  *
        return ag_fn(y_true, y_pred, **self._fn_kwargs)
    C:\Users\chris\OneDrive\Dokumente\GitHub\LSTM\data\untitled0.py:16 lfunc  **
        
    C:\Users\chris\OneDrive\Dokumente\GitHub\LSTM\data\untitled0.py:12 weight
        
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\framework\ops.py:900 __bool__
        self._disallow_bool_casting()
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\framework\ops.py:503 _disallow_bool_casting
        self._disallow_when_autograph_enabled(
    C:\Users\chris\.conda\envs\snowflake\lib\site-packages\tensorflow\python\framework\ops.py:489 _disallow_when_autograph_enabled
        raise errors.OperatorNotAllowedInGraphError(

    OperatorNotAllowedInGraphError: using a `tf.Tensor` as a Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

Does someone has any suggestions how to implement this problem with a Keras backend?

Improve this question
$\endgroup$
2
  • $\begingroup$ I ran this code snippet, and worked fine. What's the error? Please add the whole error message. It also shows the function and line where raises the error. Also add the line where you pass this loss function to your model, which probably is model.compile() $\endgroup$
    – Kaveh
    Jul 9, 2021 at 19:19
  • $\begingroup$ Hi Kaveh, I added all further information on the problem $\endgroup$
    – Christopher
    Jul 11, 2021 at 10:50

1 Answer 1

Reset to default
0
$\begingroup$

if y_hat>=0: weight = K.sigmoid(error100)+1
cond = _verify_tf_condition(cond, 'if statement')
verify_tf_condition '; {}'.format(tag, cond, extra_hint))
ValueError: condition of if statement expected to be tf.bool scalar, got Tensor*

If we observe the above logs (trimmed),
We are assuming y_hat to be a scaler. Hence the code will work for batch_size=1 and throws the above error for batch_size>=2

You should handle if y_hat>=0 code for Tensor.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.