0
$\begingroup$

Code versions:

python = 3.8.0

tensorflow = 2.2.0

scikitlearn = 0.23.2

I'm training a neural network on an all categorical dataset. Because of the sparsity caused by this dataset, I've attempted to improve results by adding in Entity Embedding as describe in this paper. To test out the embedding, I wrote a simplified version of the network modeled off of this code


X = data.drop(columns='Label')
Y = data['Label']
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, stratify=Y, test_size=0.10, random_state=1234)
X_train, X_val, Y_train, Y_val = train_test_split(X_train, Y_train,stratify=Y_train, test_size=0.10, random_state=5678)

# define the keras model
        inputs = []
        outputs = []
        categorical_columns = self.X_train.columns

        for c in categorical_columns:
            print(c)
            num_unique_vals = 2
            embed_dim = int(max(min(np.ceil(num_unique_vals / 2), 50), 2))
            inp = tf.keras.layers.Input(shape=(1,))
            out = tf.keras.layers.Embedding(num_unique_vals + 1, embed_dim)(inp)

            out = tf.keras.layers.Dropout(0.30)(out)

            out = tf.keras.layers.Reshape(target_shape=(embed_dim,))(out)
            inputs.append(inp)
            outputs.append(out)

        # Concatenate into one single feature vector
        x = tf.keras.layers.Concatenate()(outputs)
        
        # Hidden Layer
        x = Dense(units=30, activation='relu')(x)
        x = tf.keras.layers.Dropout(0.30)(x)

        y = Dense(1, activation='sigmoid')(x)


        model = tf.keras.Model(inputs=inputs, outputs=y)
        model.summary()
        # Compilation
        model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
                      loss='binary_crossentropy',
                      metrics=['accuracy',
                               tf.keras.metrics.Precision(),
                               tf.keras.metrics.Recall(),
                               tf.keras.metrics.AUC()])

        self.history = model.fit(self.X_train,self.Y_train,batch_size=32,
                                           epochs=30,validation_data=(X_val, Y_val), verbose=2)

However, each time I try to run the code I run into the following error at the fit section of the code:

Traceback (most recent call last):
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 2805, in variable_creator_scope
    yield
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 848, in fit
    tmp_logs = train_function(iterator)
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 580, in __call__
    result = self._call(*args, **kwds)
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 627, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 505, in _initialize
    self._stateful_fn._get_concrete_function_internal_garbage_collected(  # pylint: disable=protected-access
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2446, in _get_concrete_function_internal_garbage_collected
    graph_function, _, _ = self._maybe_define_function(args, kwargs)
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2777, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2657, in _create_graph_function
    func_graph_module.func_graph_from_py_func(
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 981, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 441, in wrapped_fn
    return weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 968, in wrapper
    raise e.ag_error_metadata.to_exception(e)
AssertionError: in user code:

    /home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
        outputs = self.distribute_strategy.run(
    /home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:951 run  **
        return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
    /home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
        return self._call_for_each_replica(fn, args, kwargs)
    /home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
        return fn(*args, **kwargs)
    /home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:531 train_step  **
        y_pred = self(x, training=True)
    /home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:927 __call__
        outputs = call_fn(cast_inputs, *args, **kwargs)
    /home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/network.py:717 call
        return self._run_internal_graph(
    /home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/network.py:899 _run_internal_graph
        assert str(id(x)) in tensor_dict, 'Could not compute output ' + str(x)

    AssertionError: Could not compute output Tensor("dense_1/Identity:0", shape=(None, 1), dtype=float32)


Process finished with exit code 1

I'm unsure of what this means. I'm a little worried about the fact that the data I'm using is already one-hot-encoded, so each category is a binary variable, though I'm not sure if that is what is causing the issue.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.