0
$\begingroup$

Consider simple feed-forward neural network with few layers. I would like to evaluate only gradients of one particular layer, denoted by X. This should be performed repetitively, while parameters of this layers are being always updated. In the same time, the parameters of other layer are remaining intact.

Is there any algorithm to evaluate these gradients of layer X in efficient manner?

$\endgroup$

1 Answer 1

0
$\begingroup$

If you are using Tensorflow, you can use the tf.GradientTape function:

tf.GradientTape(
    persistent=False, watch_accessed_variables=True
)

For example:

import tensorflow as tf
import random


x = tf.constant(random.uniform(0, 1))
with tf.GradientTape() as g:
    g.watch(x)
    y = x * x
dy_dx = g.gradient(y, x)
print(dy_dx)

Documentation: https://www.tensorflow.org/api_docs/python/tf/GradientTape

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.