The simplest example is to have faster/slower learning rates in the upper/lower layers of a network. I found this post on tensorflow.

  • Is there a similar trick in Keras?
  • Going one step further, can we set different learning rates for specific range/set of neurons/weights in a particular layer?
  • 1
    $\begingroup$ Couldn't write this as a comment, sorry. You can follow this thread here: github.com/fchollet/keras/pull/3004 Seems at least that the layer-wise tuning should come. $\endgroup$ – Christian Safka Jan 2 '17 at 14:21
  • $\begingroup$ @ChristianSafka Nice! though still some issues but I will see if I could make it work $\endgroup$ – horaceT Jan 2 '17 at 18:17
  • $\begingroup$ @ChristianSafka You could make your comment above an answer. $\endgroup$ – horaceT Jan 3 '17 at 23:49

You can follow this thread here: https://github.com/fchollet/keras/pull/3004 Seems at least that the layer-wise tuning should come.

| improve this answer | |
  • 2
    $\begingroup$ The pull request is closed by now. You can find the new one here: github.com/fchollet/keras/issues/5920 Until then, depending on your problem, it might be helpful to use Adadelta or Adam as your optimizer, which should adapt the learning rate for each parameter automatically. $\endgroup$ – McLawrence Jun 11 '17 at 21:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.