# Goodhart's law applied to data science

I recently learned about Goodhart's Law. Simply put,

When a measure becomes a target, it ceases to be a good measure.

However, in Data Science, we really do aim at improving our performance by increasing or decreasing a metric, and improve our models based on that. For instance, in Kaggle competitions. Is Goodhart's Law applicable to Data Science? Why or why not?

• Do the people being measured have a vested interest in the outcome of the metric? If so, beware, because Goodhart was right. Nov 6, 2017 at 23:06
• @StephenRauch Rhetorical question? I can certainly see several cases where people do have a vested interest in it. Again, Kaggle competitions are an example. Maybe even some shady "research". But there are cases where people do focus on a single metric, but not with "vested interest" beyond getting good results, or because they used a loss function that the metric can quantify well. Would Goodhart's Law apply there? Nov 6, 2017 at 23:14
• Key element is over time. If you have a fixed data set, Goodhart is not applicable. But if you calculating a metric and collecting data over time, and the people's whose behavior affect the metric can see the metric, they will change behavior to change outcome of the metric in a way that benefits them. Nov 6, 2017 at 23:17
• @StephenRauch Ah, so Gerrymandering might be a good example! Kaggle competitions are not. Nov 6, 2017 at 23:19
• @StephenRauch But Gerrymandering is also "application of external stress", so not completely sure. Thanks for clarifying, though. Nov 6, 2017 at 23:21

One way to look at this is that the data generating process in the past is bias. Say we our try model is a product of two variables $z = xy$. In the past, $y$ has always been constant at $a$, so you come to the conclusion $z$ must equal to $ax$. But this is only true when $y=a$.
In Goodhart's setting (economic policy and social science in general), it is usually the prediction value of $z$ that causing the changes in $y$. It is a dynamic system of which we only have a bias static slice of data.