Code versions:
python = 3.8.0
tensorflow = 2.2.0
scikitlearn = 0.23.2
I'm training a neural network on an all categorical dataset. Because of the sparsity caused by this dataset, I've attempted to improve results by adding in Entity Embedding as describe in this paper. To test out the embedding, I wrote a simplified version of the network modeled off of this code
X = data.drop(columns='Label')
Y = data['Label']
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, stratify=Y, test_size=0.10, random_state=1234)
X_train, X_val, Y_train, Y_val = train_test_split(X_train, Y_train,stratify=Y_train, test_size=0.10, random_state=5678)
# define the keras model
inputs = []
outputs = []
categorical_columns = self.X_train.columns
for c in categorical_columns:
print(c)
num_unique_vals = 2
embed_dim = int(max(min(np.ceil(num_unique_vals / 2), 50), 2))
inp = tf.keras.layers.Input(shape=(1,))
out = tf.keras.layers.Embedding(num_unique_vals + 1, embed_dim)(inp)
out = tf.keras.layers.Dropout(0.30)(out)
out = tf.keras.layers.Reshape(target_shape=(embed_dim,))(out)
inputs.append(inp)
outputs.append(out)
# Concatenate into one single feature vector
x = tf.keras.layers.Concatenate()(outputs)
# Hidden Layer
x = Dense(units=30, activation='relu')(x)
x = tf.keras.layers.Dropout(0.30)(x)
y = Dense(1, activation='sigmoid')(x)
model = tf.keras.Model(inputs=inputs, outputs=y)
model.summary()
# Compilation
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
loss='binary_crossentropy',
metrics=['accuracy',
tf.keras.metrics.Precision(),
tf.keras.metrics.Recall(),
tf.keras.metrics.AUC()])
self.history = model.fit(self.X_train,self.Y_train,batch_size=32,
epochs=30,validation_data=(X_val, Y_val), verbose=2)
However, each time I try to run the code I run into the following error at the fit section of the code:
Traceback (most recent call last):
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/contextlib.py", line 131, in __exit__
self.gen.throw(type, value, traceback)
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/ops/variable_scope.py", line 2805, in variable_creator_scope
yield
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 848, in fit
tmp_logs = train_function(iterator)
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 580, in __call__
result = self._call(*args, **kwds)
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 627, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 505, in _initialize
self._stateful_fn._get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2446, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2777, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2657, in _create_graph_function
func_graph_module.func_graph_from_py_func(
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 981, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 441, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 968, in wrapper
raise e.ag_error_metadata.to_exception(e)
AssertionError: in user code:
/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:571 train_function *
outputs = self.distribute_strategy.run(
/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:951 run **
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:2290 call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/distribute/distribute_lib.py:2649 _call_for_each_replica
return fn(*args, **kwargs)
/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py:531 train_step **
y_pred = self(x, training=True)
/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py:927 __call__
outputs = call_fn(cast_inputs, *args, **kwargs)
/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/network.py:717 call
return self._run_internal_graph(
/home/user/anaconda3/envs/deep_learning/lib/python3.8/site-packages/tensorflow/python/keras/engine/network.py:899 _run_internal_graph
assert str(id(x)) in tensor_dict, 'Could not compute output ' + str(x)
AssertionError: Could not compute output Tensor("dense_1/Identity:0", shape=(None, 1), dtype=float32)
Process finished with exit code 1
I'm unsure of what this means. I'm a little worried about the fact that the data I'm using is already one-hot-encoded, so each category is a binary variable, though I'm not sure if that is what is causing the issue.
