Most advanced supervised learning techniques are non-deterministic by construction. The final output of the model usually depends on some random parts of the learning process. (Random weight initialization for Neural Networks or variable selection / splits for Gradient Boosted Trees). This phenomenon can be observed by plotting the predictions for a given random seed against the predictions for another seed : the prediction are usually correlated but don't coincide exactly.

Generally speaking it is often not a problem. When trying to separate green tomatoes from red ones, only overall performance of the classifier matters. Individual predictions don't really matter as no tomato will be upset / will sue you. However, for more adavanced problems, more specifically those relating to people (education, work, loan application...) variance in individual scores due to the non-deterministic learning process might become a problem. Basically some people might get a life impacting result based on a given seed and the opposite result had you used another seed. This doesn't seems very fair or ethical to me. Choosing a seed sort of feel like a trolley problem...

Outside of techniques that might be used to reduce this 'seed-dependence' (regularization / ensemble and so on), I would be interested on the ethical aspects of this variance in outputs relating to the random seed. But I can't find any resource on the ethical matter. (I can barely find some resources on the topic of seed dependence - I suspect this phenomenon is not widely disclosed as it might deter people from wanting to use 'Artificial Intelligence').

In the context of models impacting people lives, have the ethical consequences of non-deterministic learning processes been formalized / evaluated?

  • 1
    $\begingroup$ Interesting question and a +1 from me, though I do wonder if this would be a better fit on the AI Stack. $\endgroup$
    – Dave
    Jun 23 at 11:55
  • $\begingroup$ I wondered about it too, but couldn't find a strong argument for the AI SE. $\endgroup$
    – lcrmorin
    Jun 23 at 11:58
  • 1
    $\begingroup$ This is a great question! Summarily there is no (current) way to employ "ethical" requirements into machine learning. So the next best thing is to take the results only as indications and have humans make the final evaluation. IMO $\endgroup$
    – Nikos M.
    Jun 23 at 14:42
  • 2
    $\begingroup$ Interpretability of deep models also plays a role, in that if the model is NOT interpretable, then both the error cannot be accounted through to causes plus there is no way to extract meaning from the model (thus cannot justify ethically the result) $\endgroup$
    – Nikos M.
    Jun 23 at 14:45
  • 1
    $\begingroup$ The random seed phenomenon is not discussed much (I presume) because on average it doesnt matter so much. Training many models the random effects usualy are of no consenquence on average. That does not mean that any concrete model will not suffer and if people's lives are considered have serious ethical consequences as well. Plus I agree that some shortcomings of AI are not discussed, this being one of them. $\endgroup$
    – Nikos M.
    Jun 23 at 16:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.