I am learning Tensorflow and came across different feature columns used in Tensorflow . Out of these types, two are categorical_identity_column and indicator_column. Both have been defined in the same way. As far as I understand, both convert categorical column to one-hot encoded column.

So my question is what is the difference between the two? When to use one and when to use the other?


indicator_column encodes the input to a multi-hot representation, not one-hot encoding.

The example from https://www.tensorflow.org/api_docs/python/tf/feature_column/indicator_column clarifies more:

name = indicator_column(categorical_column_with_vocabulary_list(
    'name', ['bob', 'george', 'wanda'])
columns = [name, ...]
features = tf.parse_example(..., features=make_parse_example_spec(columns))
dense_tensor = input_layer(features, columns)

dense_tensor == [[1, 0, 0]]  # If "name" bytes_list is ["bob"]
dense_tensor == [[1, 0, 1]]  # If "name" bytes_list is ["bob", "wanda"]
dense_tensor == [[2, 0, 0]]  # If "name" bytes_list is ["bob", "bob"] 

The last two examples describe what is meant by multi-hot encoding. for example if the input be ["bob", "wanda"] the encoding will be [[1, 0, 1]].


You would use categorical_column_with_* to get a _CategoricalColumn to feed into a linear model; this column returns identity values, often using a vocabulary.

On the other hand, indicator_column is a multi-hot representation of a given categorical column and would be used if you want to feed the feature into a DNN, for example; it produces an _IndicatorColumn. embedding_column is analogous but you'd use it if your input is sparse.

  • $\begingroup$ So you are saying Categorical column can't be directly fed to DNN, unless wrapped by indicator or embedded column. $\endgroup$ – Ankit Seth Apr 4 '18 at 12:26
  • $\begingroup$ According to the docs the feature columns "should be instances of classes derived from FeatureColumn" -- embeddings will reduce the complexity of the data representation but I'm not sure that it's a requirement. tensorflow.org/api_docs/python/tf/contrib/learn/DNNClassifier $\endgroup$ – Ethereal Apr 5 '18 at 18:39

Regarding the question in the comments above (by Ankit Seth), the docs here say the following about deep models (as opposed to "wide", i.e. linear):

tf.estimator.DNNClassifier and tf.estimator.DNNRegressor: Only accept dense columns. Other column types must be wrapped in either an indicator_column or embedding_column.

And if you try to pass a categorical column directly to a deep model, TF will throw the following error:

ValueError: Items of feature_columns must be a _DenseColumn. You can wrap a categorical column with an embedding_column or indicator_column.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.