How to retrieve tweets from all cities using Twint package in python?

Question

Using Twint, I'm able to get tweets from each city in the list, but while creating a dataframe I only get tweet from the last city in the list, is it possible to append the tweets based on cities?

india = ["chennai","pune","mumbai"]
for city in india:
    print(city)
    c = twint.Config()
    c.Search = "phone"
    c.Lang= "en"
    c.Pandas = True
    c.Since = "2021-01-01"
    c.Near = city
    c.Limit = 10
    twint.run.Search(c)
data = twint.output.panda.Tweets_df(["date","tweet","near"])
data.head()

        date                              tweet                              near
0   2021-03-23 18:47:09 Cameron FaulknerThe Black Shark 4 is a high-en...   mumbai
1   2021-03-23 18:42:21 Asad just break Zoya's phone so she won't expo...   mumbai
2   2021-03-23 18:40:13 @Babi3Kat Send you phone number an                  mumbai
3   2021-03-23 18:34:12 @sahilbaani @SamsungIndia ????? 5 Saal purane ...   mumbai
4   2021-03-23 18:33:52 @91mobiles They are just started to make more ...   mumbai

Answer 1

You are only seeing the last one since you overwrite your object df at each iteration and when the loop is finished it will store only the last value store which happens to be mumbai

If you want to create a DataFrame with all the results you should do something like this:

ldf = list()
india = ["chennai","pune","mumbai"]
for city in india:
    print(city)
    c = twint.Config()
    c.Search = "phone"
    c.Lang= "en"
    c.Pandas = True
    c.Since = "2021-01-01"
    c.Near = city
    c.Limit = 10
    twint.run.Search(c)
    tmp = twint.output.panda.Tweets_df(["date","tweet","near"])
    ldf.append(tmp)
data = pd.concat(ldf, ignore_index = True)
data.head()