I'm now learning about deep learning with Keras, and to implement a deep learning model at Keras, you set the initializer to set its initial weights on.
from keras.models import Sequential from keras.layers import Dense model = Sequential() model.add(Dense(12, input_dim=8, kernel_initializer="random_uniform"))
kernel_initializer can take something others, such as
random_normal, which uses Gaussian, not uniform distribution, and
zero, which literally sets all weights to 0.
However, I don't understand why you like to set different weights at the initializer. Specifically, what advantages does it have over setting all initial weights to 0, which sounds more natural for novices like me?
Also, should the initial weights, if needed, be always set a tiny value (e.g.