0
$\begingroup$

The problem of predicting the daily number of COVID-19 cases is indeed challenging and many (external) factors should be taken into account to come up with a reasonable predictor. However, we have studied Twitter for a specific country (not English) for the period Mar-Nov 2020, and found out that the volume of daily tweets related to symptom X is highly correlated with the number of confirmed cases in that country (pearson correlation 0.84 with p-value 0.00031).

In the field of data science, would this suffice, at least partially, to say daily tweets of X is a good predictor for the number of COVID-19 cases?

$\endgroup$

2 Answers 2

1
$\begingroup$

Isn't this a variant of the "Google Flu Trends" story from a few years back? (Substitute "Google" with "Twitter" and "Flu" with "Covid-19", "Searches" with "Tweets".

Long story. In a nutshell, frequency of Google Searches for terms like "Flu", "Headache" "Nausea" were an excellent predictor for flu season forecasting, until they weren't. (when people started to search for "flu trends"? I don't remember why). There was a negative feedback loop, and forecasts became less reliable.)

Google finally removed that feature, to avoid criticism and to stay out of trouble.

There are many papers on this, and why it was taken offline.

$\endgroup$
1
  • $\begingroup$ Interesting. Thank you. They should have updated the model periodically. Will check the papers. $\endgroup$
    – seteropere
    Nov 29, 2020 at 10:24
1
$\begingroup$

In this case, I would fit a linear model (lm(tweets ~ cases)) and then use bootstrap or cross-validation to test the model.

$\endgroup$
3
  • $\begingroup$ Thank you. I don't think linear model will fit as the curve is going down. It is more like a bell curve. $\endgroup$
    – seteropere
    Nov 29, 2020 at 10:23
  • 2
    $\begingroup$ Then the correlation coefficient does not indicate tweets are a good predictor. Correlation coefficient assumes linearity. $\endgroup$ Nov 29, 2020 at 11:10
  • 1
    $\begingroup$ You will also find correlation coefficient and the linear regression coefficient are related. See what happens when you center tweets and cases before fitting the linear regression model. $\endgroup$ Nov 29, 2020 at 11:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.