Neural networks are often used to solve problems where we can't rigorously define what properties the desired solution should have, e.g. you can't define what a "picture of a cat" is and so you have to train a network to learn from examples. However sometimes we can rigorously define some of the properties the solution should have. For example:

  1. The classic example property is robustness. We (nearly) always want our trained network to have the property that when you tweak the input only a tiny bit, you would like the output of the network only to change a tiny bit in response. For example, change a single pixel in your image of a cat, and you would like the network's confidence to remain more or less unchanged.

  2. Other properties are domain-specific. For example, you might be using a neural network to predict the probability of bank-loan application being accepted, and one of the inputs is the applicant's income. In this case a desirable property is that the network's output is monotonic with respect to the applicant's income, i.e. if applicant A has a higher income than applicant B and all other inputs are equal, then the network should predict that A's probability of having their loan accepted is at least as high as B's.

Does anyone know of any more examples of neural networks where we can rigorously define properties that it should have?

Note the word "rigorously" is important. One non-example would be the property that when recognising images of cats, we shouldn't care about the colour of the sky. That's definitely a desirable property, but we are no more able to rigorously define what the "sky" is than we are what a "cat" is.


What is the objective in having such properties?

It highly depends on the field you are studying or the kind of neural network you are using. If you want to compare several properties between several models, you may want to read field specific studies where several models are compared like this one. enter image description here

Source: https://medium.com/@MITIBMLab/cnn-cert-a-certified-measure-of-robustness-for-convolutional-neural-networks-fd2ff44c6807

In this example, the Lp norm could be considered as a property, but not all NN have it.

  • $\begingroup$ Thanks for your reply! I'm interested in finding novel use cases for neural network verification technology. Forgive me if I've got this wrong as you haven't provided a link to the study in question, but from the look of the tools used and the use of the phrase Lp-norm, then I think(?) it is comparing tools for verifying whether the networks are robust with respect to the Lp-norm around training data points. I think this is therefore the robustness property that I mentioned as an example in my question. I was hoping someone might know of different properties apart from robustness? $\endgroup$ Jul 23 at 8:49
  • $\begingroup$ Do you mean general properties like accuracy, generalisation or confidence level? $\endgroup$ Jul 23 at 9:06
  • $\begingroup$ No, by properties I mean mathematical relationships between the input and output that you know definitely hold in the true underlying process you're trying to model, and that you would hope that the neural network would learn. Apologies for not being clearer in the question. $\endgroup$ Jul 23 at 9:58
  • $\begingroup$ I don't know specific properties to know relationships between the input and the output. However, you can use explainability tools like Alibi, DeepLift or InterpretML to explain different NN's behaviors that could be useful. Some of them have mathematical functions between input and output. github.com/SeldonIO/alibi github.com/kundajelab/deeplift github.com/interpretml/interpret $\endgroup$ Jul 23 at 15:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.