Return to Answer

I am afraid that such situations are fundamentally inherent in predicting/forecasting contexts; quoting from the very recent paper by Taleb et al., On single point forecasts for fat-tailed variables (open access, para 3.7):

3.7. Forecasts can result in adjustments that make forecasts less accurate

It is obvious that if forecasts lead to adjustments, and responses that affect the studied phenomenon, then one can no longer judge these forecasts on their subsequent accuracy.

So, other than communicating this clearly beforehand and reaching an agreement on how the predictions will be assessed in the presence of such adjustments, there is not much else you can do from a modelling or methodology perspective. The advice suggested further in the same paragraph quoted above:

In that sense a forecast can be a warning of the style “if you do not act, these are the costs”.

can form the basis of such a communication and agreement.

The authors offer an example in a footnote regarding the forecasts on the ongoing COVID-19 pandemic:

For instance Dr. Fauci’s warning that the number of (verified) infections could reach 100 K per day (New York Times, June 30, 2020) should not be interpreted as a forecast to be judged according to its accuracy; rather a signal about what could happen should one avoid taking action.

I am afraid that such situations are fundamentally inherent in predicting/forecasting contexts; quoting from the very recent paper by Taleb et al., On single point forecasts for fat-tailed variables (open access, para 3.7):

3.7. Forecasts can result in adjustments that make forecasts less accurate

It is obvious that if forecasts lead to adjustments, and responses that affect the studied phenomenon, then one can no longer judge these forecasts on their subsequent accuracy.

So, other than communicating this clearly beforehand and reaching an agreement on how the predictions will be assessed in the presence of such adjustments, there is not much else you can do from a modelling or methodology perspective. The advice suggested further in the same paragraph quoted above:

In that sense a forecast can be a warning of the style “if you do not act, these are the costs”.

can form the basis of such a communication and agreement.

I am afraid that such situations are fundamentally inherent in predicting/forecasting contexts; quoting from the very recent paper by Taleb et al., On single point forecasts for fat-tailed variables (open access, para 3.7):

3.7. Forecasts can result in adjustments that make forecasts less accurate

It is obvious that if forecasts lead to adjustments, and responses that affect the studied phenomenon, then one can no longer judge these forecasts on their subsequent accuracy.

So, other than communicating this clearly beforehand and reaching an agreement on how the predictions will be assessed in the presence of such adjustments, there is not much else you can do from a modelling or methodology perspective. The advice suggested further in the same paragraph quoted above:

In that sense a forecast can be a warning of the style “if you do not act, these are the costs”.

can form the basis of such a communication and agreement.

I am afraid that such situations are fundamentally inherent in predicting/forecasting contexts; quoting from the very recent paper by Taleb et al., On single point forecasts for fat-tailed variables (open access, para 3.7):

3.7. Forecasts can result in adjustments that make forecasts less accurate

It is obvious that if forecasts lead to adjustments, and responses that affect the studied phenomenon, then one can no longer judge these forecasts on their subsequent accuracy.

So, other than communicating this clearly beforehand and reaching an agreement on how the predictions will be assessed in the presence of such adjustments, there is not much else you can do from a modelling or methodology perspective. The advice suggested further in the same paragraph quoted above:

In that sense a forecast can be a warning of the style “if you do not act, these are the costs”.

can form the basis of such a communication and agreement.

The authors offer an example in a footnote regarding the forecasts on the ongoing COVID-19 pandemic:

For instance Dr. Fauci’s warning that the number of (verified) infections could reach 100 K per day (New York Times, June 30, 2020) should not be interpreted as a forecast to be judged according to its accuracy; rather a signal about what could happen should one avoid taking action.

I am afraid that such situations are fundamentally inherent in predicting/forecasting contexts; quoting from the very recent paper by Taleb et al., On single point forecasts for fat-tailed variables (open access, para 3.7):

3.7. Forecasts can result in adjustments that make forecasts less accurate

It is obvious that if forecasts lead to adjustments, and responses that affect the studied phenomenon, then one can no longer judge these forecasts on their subsequent accuracy.

So, other than communicating this clearly beforehand and reaching an agreement on how the predictions will be assessed in the presence of such adjustments, there is not much else you can do from a modelling or methodology perspective, except perhaps re-framing the whole problem according to the advice suggested further in the same paragraph quoted above:

In that sense a forecast can be a warning of the style “if you do not act, these are the costs”.

I am afraid that such situations are fundamentally inherent in predicting/forecasting contexts; quoting from the very recent paper by Taleb et al., On single point forecasts for fat-tailed variables (open access, para 3.7):

3.7. Forecasts can result in adjustments that make forecasts less accurate

It is obvious that if forecasts lead to adjustments, and responses that affect the studied phenomenon, then one can no longer judge these forecasts on their subsequent accuracy.

So, other than communicating this clearly beforehand and reaching an agreement on how the predictions will be assessed in the presence of such adjustments, there is not much else you can do from a modelling or methodology perspective. The advice suggested further in the same paragraph quoted above:

In that sense a forecast can be a warning of the style “if you do not act, these are the costs”.

can form the basis of such a communication and agreement.