0
$\begingroup$

The problem of predicting the daily number of COVID-19 cases is indeed challenging and many (external) factors should be taken into account to come up with a reasonable predictor. However, we have studied Twitter for a specific country (not English) for the period Mar-Nov 2020, and found out that the volume of daily tweets related to symptom X is highly correlated with the number of confirmed cases in that country (pearson correlation 0.84 with p-value 0.00031).

In the field of data science, would this suffice, at least partially, to say daily tweets of X is a good predictor for the number of COVID-19 cases?

$\endgroup$
1
$\begingroup$

Isn't this a variant of the "Google Flu Trends" story from a few years back? (Substitute "Google" with "Twitter" and "Flu" with "Covid-19", "Searches" with "Tweets".

Long story. In a nutshell, frequency of Google Searches for terms like "Flu", "Headache" "Nausea" were an excellent predictor for flu season forecasting, until they weren't. (when people started to search for "flu trends"? I don't remember why). There was a negative feedback loop, and forecasts became less reliable.)

Google finally removed that feature, to avoid criticism and to stay out of trouble.

There are many papers on this, and why it was taken offline.

$\endgroup$
1
  • $\begingroup$ Interesting. Thank you. They should have updated the model periodically. Will check the papers. $\endgroup$ – seteropere Nov 29 '20 at 10:24
1
$\begingroup$

In this case, I would fit a linear model (lm(tweets ~ cases)) and then use bootstrap or cross-validation to test the model.

$\endgroup$
3
  • $\begingroup$ Thank you. I don't think linear model will fit as the curve is going down. It is more like a bell curve. $\endgroup$ – seteropere Nov 29 '20 at 10:23
  • 2
    $\begingroup$ Then the correlation coefficient does not indicate tweets are a good predictor. Correlation coefficient assumes linearity. $\endgroup$ – Suren Nov 29 '20 at 11:10
  • 1
    $\begingroup$ You will also find correlation coefficient and the linear regression coefficient are related. See what happens when you center tweets and cases before fitting the linear regression model. $\endgroup$ – Suren Nov 29 '20 at 11:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.