2
$\begingroup$

I have a loss function without a closed analytic form that I want to implement in tensorflow that has a nice clean gradient (i.e. i have some nice gradient function $f(x)$ that i want to use for backprop but $\int_x f(x) \ dx$ is with no closed form)

Ofcourse i wouldnt be able to evaluate the loss, but i want to use $f(x)$ in the optimization process. How could this be accomplished?

$\endgroup$
1
  • $\begingroup$ @Brian why? I know its gradient and i don't care about actually evaluating the loss, i just want to use that gradient during optimization $\endgroup$
    – mshlis
    Aug 23 '19 at 13:48
1
+50
$\begingroup$

TensorFlow allows for custom gradient functions with tf.custom_gradient.

You could write a decorator that would return specific gradient values for specific values of x. The loss function would not need to be evaluated.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.