0
$\begingroup$

I need to put the prediction into a complex function to calculate loss. This means that i can't build the loss function by tensorflow's operator and can't get gradient from Automatic Differentiation.

I want to get gradient from finite difference in my training step.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

You may consider using a score function estimator (also known as REINFORCE), which defines an estimator of the gradient of a scoring function that does not need to be differentiable. This can be achieved thanks to the log derivative trick.

You may find its mathematical foundations in the original REINFORCE article by Williams or in Ian Goodfellow's seminal book on Deep Learning (section 20.9.1 Back-Propagating through Discrete Stochastic Operations).

Improve this answer
$\endgroup$
2
  • $\begingroup$ +1 Is there any technical condition on the obj for this trick to work? e.g. Lipschitz continuous. $\endgroup$
    – horaceT
    Commented Apr 27, 2018 at 16:41
  • $\begingroup$ Not that I am aware of. However, be aware that the REINFORCE estimator is known for its high variance. $\endgroup$
    – noe
    Commented Aug 7, 2018 at 13:15
0
$\begingroup$

Finite difference is a way to calculate the derivative of a function according to its value. It is based on Taylor's theorem.

You can have a good idea here and may be have look in this tensorflow API.

Let's suppose J is your cost function. The first question is to defined the definition domain of your J in which apply finite difference discretization, I should have taken your different layers (but I have never tried this).

The second question is linked to the fact that your error is "backpropagated" so when constructing your definition domain you should first think of the direction of your domain (from First layer to last or from last to first).

The you have to customize your gradient to use it in your TensorFlow implementation. So I suggest you to see here and to see in StackOverflow if other people has already ask for what you want and if you can find more elaborate answers.

I hope it will help you

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.