0
$\begingroup$

I recently learned about Goodhart's Law. Simply put,

When a measure becomes a target, it ceases to be a good measure.

However, in Data Science, we really do aim at improving our performance by increasing or decreasing a metric, and improve our models based on that. For instance, in Kaggle competitions. Is Goodhart's Law applicable to Data Science? Why or why not?

Improve this question
$\endgroup$
5
  • $\begingroup$ Do the people being measured have a vested interest in the outcome of the metric? If so, beware, because Goodhart was right. $\endgroup$
    – Stephen Rauch ♦
    Nov 6, 2017 at 23:06
  • $\begingroup$ @StephenRauch Rhetorical question? I can certainly see several cases where people do have a vested interest in it. Again, Kaggle competitions are an example. Maybe even some shady "research". But there are cases where people do focus on a single metric, but not with "vested interest" beyond getting good results, or because they used a loss function that the metric can quantify well. Would Goodhart's Law apply there? $\endgroup$
    – learning
    Nov 6, 2017 at 23:14
  • 1
    $\begingroup$ Key element is over time. If you have a fixed data set, Goodhart is not applicable. But if you calculating a metric and collecting data over time, and the people's whose behavior affect the metric can see the metric, they will change behavior to change outcome of the metric in a way that benefits them. $\endgroup$
    – Stephen Rauch ♦
    Nov 6, 2017 at 23:17
  • $\begingroup$ @StephenRauch Ah, so Gerrymandering might be a good example! Kaggle competitions are not. $\endgroup$
    – learning
    Nov 6, 2017 at 23:19
  • $\begingroup$ @StephenRauch But Gerrymandering is also "application of external stress", so not completely sure. Thanks for clarifying, though. $\endgroup$
    – learning
    Nov 6, 2017 at 23:21

1 Answer 1

Reset to default
1
$\begingroup$

The function problem which Goodhart's Law described is the changes of the underlying model from which the data is generated.

One way to look at this is that the data generating process in the past is bias. Say we our try model is a product of two variables $z = xy$. In the past, $y$ has always been constant at $a$, so you come to the conclusion $z$ must equal to $ax$. But this is only true when $y=a$.

In Goodhart's setting (economic policy and social science in general), it is usually the prediction value of $z$ that causing the changes in $y$. It is a dynamic system of which we only have a bias static slice of data.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.