2
$\begingroup$

It is suggested by the author of Keras [1] to use Trainable=False when using the embedding layer in Keras to prevent the weights from being updated during training. But in my experience, I always got better performance (lower error in regression) when setting Trainable=True in text processing.

My question, can I claim my result or I must use Trainable=False? Do I make a cheat when using Trainable=True?

According to my understanding, it just a choice to use (or not to use) previous information to update the weight, so it is allowed to use Trainable=True.

[1] https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html

$\endgroup$
1
$\begingroup$

It depends upon where do you want to submit your results that you claim, and what is the submission criteria.

First, it is unclear if "lower error in regression" is training or validation/test error. I am sure that letting Embedding layer to be trainable will adapt it to fit the training set better, but it might cause overfitting and cause higher error on validation/test set.

If for your set of tasks were you use Embedding layer you consistently get lower test error, and the requirement does not specifically say so to use trainable=False for Embedding layer - why bother? It is not cheating.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.