0
$\begingroup$

I want to do a sentiment analysis using twitter data. Was thinking about hardcoding a cURL script to download data, from a Google Cloud service (I'll run the data on a neural network on the server, to label each tweet), but I have this question:

  • Am I allowed to do this? I know twitter sells the data, so I am not sure if I can get in trouble for downloading it directly (I have to disclose the data gathering methodology on the paper).
$\endgroup$
1
$\begingroup$

Last time I checked it was not allowed to store the contents of the tweets, instead one is supposed to store the tweet id and retrieve the content of the tweet dynamically.

Afaik this is because users are allowed to delete their tweets at any time, and keeping a tweet that they chose to delete would be against Twitter terms of use (and possibly illegal in some jurisdictions). Using the tweet id solves the problem since the content will simply not be available anymore if the tweet was deleted.

Since you plan to write a paper I assume that you're in academia? If yes in case of doubt it's always safer to ask the data protection and/or legal office in your institution. In this case you're using a secondary source (i.e. you're collecting data which already exists) so it should be straightforward (I think).

| improve this answer | |
$\endgroup$
  • $\begingroup$ Interesting. I would probably have to store tweets at least temporarily to have some people do manual classification on a sample, train the neural network, and then run the rest of the data on the network. But then I would delete all of it and keep just statistical data. Don't know how that fits. I am in grad school but it is a business school, not sure if they had projects like this before but will defo check. $\endgroup$ – Delta Nov 7 '19 at 13:30
  • $\begingroup$ Afaik it's fine to store content temporarily, it's practically unavoidable anyway (but I'm no expert) $\endgroup$ – Erwan Nov 7 '19 at 14:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.