3
$\begingroup$

I got the following warning:

94: UserWarning: Converting sparse IndexedSlices to a dense Tensor with 1200012120 elements. This may consume a large amount of memory.

For the following code:

from wordbatch.extractors import WordSeq
import wordbatch
from keras.layers import Input,Embedding
...
wb = wordbatch.WordBatch(normalize_text, extractor=(WordSeq, {"seq_maxlen": MAX_NAME_SEQ}), procs=NUM_PROCESSOR)
wb.dictionary_freeze = True
full_df["ws_name"] = wb.fit_transform(full_df["name"])
...
name = Input(shape=[full_df["name"].shape[1]], name="name")
emb_name = Embedding(MAX_TEXT, 50)(name)
...

That is I make use of WordSeq (from WordBatch) output from the Embedding layer of a GRU network. How should I modify the code to make it work without converting to dense tensor?

$\endgroup$
4
  • $\begingroup$ I don't think that is an error, that is a warning explicitly put up by the guy written the code to make sure that you know what you are doing. $\endgroup$ Commented Jan 31, 2018 at 5:30
  • 1
    $\begingroup$ @KiriteeGak changed - changing from a sparse matrix to a dense tensor is something not right, hope to find a better method here $\endgroup$
    – william007
    Commented Jan 31, 2018 at 5:34
  • $\begingroup$ hello. Have you found the solution? $\endgroup$
    – Yu Gu
    Commented Oct 6, 2018 at 22:51
  • $\begingroup$ @YuGu Were you working in SUTD? $\endgroup$
    – william007
    Commented Oct 7, 2018 at 6:04

1 Answer 1

3
$\begingroup$

I've had the same issue with the Embedding layer in Keras. The solution is to explicitly use a TensorFlow optimizer, like here:

model.compile(loss='mse', optimizer=TFOptimizer(tf.train.GradientDescentOptimizer(0.1)))

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.