0
$\begingroup$

I am reading an introductory tutorial on tensorflow here, and I'm confused about the code that defines an input layer for a word2vec embedding:

# Headline input: meant to receive sequences of 100 integers, between 1 and 10000.
# Note that we can name any layer by passing it a "name" argument.
main_input = Input(shape=(100,), dtype='int32', name='main_input')

Afaik, there are 10.000 words in the library, and the input is a string of 100 words (a news headline). One word is represented as a 32 bit int.

But it seems to me that too little information is passed to this Input layer:

  • What does "shape=(100,)" mean? why does it omit the second argument?

  • how does it know how to deal with "dtype='int32' "? If I saw that without context, I would guess that it interprets each of the 100 32bit ints as numbers rather than one-shot word vectors, so I would guess it would create a 100 neuron layer, where each neuron receives 1 input of 32 bits which would be the input to the neuron's activation function. But what we really want is a neural net with 10000 neurons (one for each word) times 100 (for each word in the sequence), right?

What am I missing?

$\endgroup$
1
$\begingroup$

This question is very much about code syntax of the Keras Input layer (Tensorflow backend)...

  • shape=(100,) meaning - The shape variable takes as input a tuple of integers, not including the batch size. For instance, shape=(100,) indicates that the expected input will be batches of 100-dimensional vectors.

  • It knows how to deal with "dtype='int32'" because that is how the Input class was written (I have no better answer). The data type variable takes as input a string (float32, float64, int32...).

The official documentation of the Input layer: tf.keras.layers.Input

This link has all the answers you are looking for.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.