Using Trainable=True in Keras Embedding obtained better performance

Question

It is suggested by the author of Keras [1] to use Trainable=False when using the embedding layer in Keras to prevent the weights from being updated during training. But in my experience, I always got better performance (lower error in regression) when setting Trainable=True in text processing.

My question, can I claim my result or I must use Trainable=False? Do I make a cheat when using Trainable=True?

According to my understanding, it just a choice to use (or not to use) previous information to update the weight, so it is allowed to use Trainable=True.

[1] https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html

Answer 1

It depends upon where do you want to submit your results that you claim, and what is the submission criteria.

First, it is unclear if "lower error in regression" is training or validation/test error. I am sure that letting Embedding layer to be trainable will adapt it to fit the training set better, but it might cause overfitting and cause higher error on validation/test set.

If for your set of tasks where you use Embedding layer you consistently get lower test error, and the requirement does not specifically say so to use trainable=False for Embedding layer - why bother? It is not cheating.