Neural networks are often used to solve problems where we can't rigorously define what properties the desired solution should have, e.g. you can't define what a "picture of a cat" is and so you have to train a network to learn from examples. However sometimes we can rigorously define some of the properties the solution should have. For example:
The classic example property is robustness. We (nearly) always want our trained network to have the property that when you tweak the input only a tiny bit, you would like the output of the network only to change a tiny bit in response. For example, change a single pixel in your image of a cat, and you would like the network's confidence to remain more or less unchanged.
Other properties are domain-specific. For example, you might be using a neural network to predict the probability of bank-loan application being accepted, and one of the inputs is the applicant's income. In this case a desirable property is that the network's output is monotonic with respect to the applicant's income, i.e. if applicant
Ahas a higher income than applicant
Band all other inputs are equal, then the network should predict that
A's probability of having their loan accepted is at least as high as
Does anyone know of any more examples of neural networks where we can rigorously define properties that it should have?
Note the word "rigorously" is important. One non-example would be the property that when recognising images of cats, we shouldn't care about the colour of the sky. That's definitely a desirable property, but we are no more able to rigorously define what the "sky" is than we are what a "cat" is.