How is this tensorflow command interpreted?

Question

I am reading an introductory tutorial on tensorflow here, and I'm confused about the code that defines an input layer for a word2vec embedding:

# Headline input: meant to receive sequences of 100 integers, between 1 and 10000.
# Note that we can name any layer by passing it a "name" argument.
main_input = Input(shape=(100,), dtype='int32', name='main_input')

Afaik, there are 10.000 words in the library, and the input is a string of 100 words (a news headline). One word is represented as a 32 bit int.

But it seems to me that too little information is passed to this Input layer:

• What does "shape=(100,)" mean? why does it omit the second argument?

• how does it know how to deal with "dtype='int32' "? If I saw that without context, I would guess that it interprets each of the 100 32bit ints as numbers rather than one-shot word vectors, so I would guess it would create a 100 neuron layer, where each neuron receives 1 input of 32 bits which would be the input to the neuron's activation function. But what we really want is a neural net with 10000 neurons (one for each word) times 100 (for each word in the sequence), right?

What am I missing?

Answer 1

This question is very much about code syntax of the Keras Input layer (Tensorflow backend)...

• shape=(100,) meaning - The shape variable takes as input a tuple of integers, not including the batch size. For instance, shape=(100,) indicates that the expected input will be batches of 100-dimensional vectors.

• It knows how to deal with "dtype='int32'" because that is how the Input class was written (I have no better answer). The data type variable takes as input a string (float32, float64, int32...).

The official documentation of the Input layer: tf.keras.layers.Input

This link has all the answers you are looking for.