Why does English ELMo model give embeddings for non-English words?

Here's the code from my notebook:

%tensorflow_version 1.x
import tensorflow as tf
import tensorflow_hub as hub

elmo = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)
tf.logging.set_verbosity(tf.logging.ERROR)

def elmo_vectors(x):
embeddings = elmo(x, signature="default", as_dict=True)["elmo"]
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(tf.tables_initializer())
return sess.run(embeddings)


Output for non-English language: (Hindi in this example)

words = ['गोकुल']
v = elmo_vectors(words)
print(v.shape) # (1,1,1024)
print(v[0][0])
# Output: [ 0.3731584   0.5700774  -0.48072845 ... -0.1241736   0.5961436 -0.6986947 ]


The documentation of the pre-trained ELMo on Tensorflow Hub shows that it was trained only on the English language.
That is, the dataset from 1 billion word benchmark is based on monolingual English data. (Source)

So, how/why am I getting embeddings for non-English vocabulary words from ELMo using the TF Hub model?

The input of ELMo is received at character-level. It may happen that the 1B Word data had hindi characters intermixed, case in which your characters would be encoded as they are or, most probably, your characters are encoded as unknown characters (just like the unknown token <unk> for word-level NLP but for characters).
• The input of ELMo is received at character-level. It seems that the tokenizer for ELMo is Moses Tokenizer (example), which is not character-level tokenization. – Gokul NC Jun 25 '20 at 13:26
• The input to ELMo is tokenized first for words and then, for each word, encoded as characters. The input tensor is of dimensions [batch_size, max_seq_length, max_char_length], where max_char_length is by default 50, that is words are assumed to be at most 50 characters long. You can verify this in the original ELMo article or in the very TF Hub documentation, where is says "Computes contextualized word representations using character-based word representations and..." – noe Jun 25 '20 at 16:19