# Non differentiable loss function

I have a loss function that minimizes the error according to what I want the neural network to do. The problem is, that it is a nondifferentiable function. How can I handle this?

the loss function: $$(1-y) \cdot log(1-p) + min((1-y)-(y \cdot log(p)))$$

• $$y$$: target
• $$p$$: prediction
• len((1-y)-(y*log(p))) = len(y) = len(p)

I have tried to smooth the minimum, but I am not sure this is good enough. As you can see, the min operator is nondifferentiable

How to handle a nondifferentiable loss function with Neural Networks?

• min needs at least two variables but you only have one, i.e min(x,y) or min(x,y,z)? Jan 13, 2020 at 17:39
• Can you describe what y and p are in this context? Jan 13, 2020 at 17:56
• y is the target, and p is the prediction @zachdj Jan 13, 2020 at 19:19

## 1 Answer

You can optimize with non-gradient based methods. The field is called derivative-free optimization.

Local Search is one common approach for derivative-free optimization of neural networks.