I need to put the prediction into a complex function to calculate loss. This means that i can't build the loss function by tensorflow's operator and can't get gradient from Automatic Differentiation.
I want to get gradient from finite difference in my training step.
You may consider using a score function estimator (also known as REINFORCE), which defines an estimator of the gradient of a scoring function that does not need to be differentiable. This can be achieved thanks to the log derivative trick.
You may find its mathematical foundations in the original REINFORCE article by Williams or in Ian Goodfellow's seminal book on Deep Learning (section 20.9.1 Back-Propagating through Discrete Stochastic Operations).
Finite difference is a way to calculate the derivative of a function according to its value. It is based on Taylor's theorem.
You can have a good idea here and may be have look in this tensorflow API.
Let's suppose J is your cost function. The first question is to defined the definition domain of your J in which apply finite difference discretization, I should have taken your different layers (but I have never tried this).
The second question is linked to the fact that your error is "backpropagated" so when constructing your definition domain you should first think of the direction of your domain (from First layer to last or from last to first).
The you have to customize your gradient to use it in your TensorFlow implementation. So I suggest you to see here and to see in StackOverflow if other people has already ask for what you want and if you can find more elaborate answers.
I hope it will help you
