2
$\begingroup$

In discussions about ML algorithms, in for instance crime prediction, it is often claimed by non-experts that there are problems with feedback loops causing the model to become biased and give the wrong results. Basically saying that the model's predictions give more attention to that type of data, and when retraining with the results, the predictions become skewed so even more attention is given to the same data type, and so on.

Is this true?

I would think that retraining the model with new data would make it more precise, regardless of how that data originated.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

Yes, this is a real problem that manifests once system is used by real users.

Most prominent example is News Echo Chamber (accentuated by ML based recommendation systems)

ML algo sees that you like news / videos related to certain point of view, you watch more of such videos and model becomes more convinced of your choice. So it suggests even more content with similar views.

https://en.wikipedia.org/wiki/Echo_chamber_(media)

http://theconversation.com/explainer-how-facebook-has-become-the-worlds-largest-echo-chamber-91024

https://www.theguardian.com/science/blog/2017/dec/04/echo-chambers-are-dangerous-we-must-try-to-break-free-of-our-online-bubbles

https://www.quora.com/Would-you-say-that-Quoras-generated-news-feed-suffers-from-an-echo-chamber-dilemma

Improve this answer
$\endgroup$
2
  • $\begingroup$ Isn't this only a problem because the user is shown the news items and views them without giving feedback on whether the prediction was true or false. The model then infers wrongly that its predictions were true. $\endgroup$
    – Rugbrød
    Mar 20, 2019 at 12:22
  • $\begingroup$ User gives implicit feedback by viewing the content (and ignoring others), users also provide explicit feedback by like/share/dislike. For example, Youtube allows you to remove a suggestion and also provide feedback on why the suggestion was wrong. $\endgroup$
    – Shamit Verma
    Mar 20, 2019 at 12:26
2
$\begingroup$

Yes feedback loops can happen in much the same way in machine learning. It can happen when the predictions of a model affects the future labels.

Let's say we are predicting crime rate in different neighborhoods. One neighborhood has biased data causing it to be predicted as higher in crime than it actually is. This causes more police presence in this neighborhood which in turn will lead to more real crime being discovered than in the areas that didn't receive extra attention caused by a biased model. This extra discovered crime will then be present for any new models to be trained even if the initial data error/bias is removed. The biased model as enforced its' own bias and produced new data to back it up.

Improve this answer
$\endgroup$
2
  • $\begingroup$ But if you include police activity in the neighbourhood as a variable, won't that compensate for more crime being discovered. $\endgroup$
    – Rugbrød
    Mar 20, 2019 at 12:45
  • $\begingroup$ Probably not. The model is predicting crime rate in a neighborhood and high police activity will probably be correlated with higher crime rates. So adding it will probably only give additional feedback telling future models that this is a high crime neighborhood when it actually was all caused by the initial biased model. $\endgroup$
    – Simon Larsson
    Mar 20, 2019 at 12:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.