I have a data frame with following structure:


Index(['first_post_date', 'followers_count', 'friends_count', 'last_post_date','min_retweet', 'retweet_count', 'screen_name', 'tweet_count',  'tweet_with_max_retweet', 'tweets', 'uid'],

Inside the tweets series, each cell is another data frame containing all the tweets of an user.


Index(['created_at', 'id', 'retweet_count', 'text'], dtype='object')

I want to perform calculation on the tweets of each users, for example, finding average number of retweets of each user, the tweet with maximum retweets etc.

How can I do these?

Edit link to sample data

  • $\begingroup$ Can you upload a few lines of data so we can present a working solution? $\endgroup$ – Emre May 26 '17 at 18:35
  • $\begingroup$ @Emre, link added $\endgroup$ – Rakib May 26 '17 at 19:04
  • $\begingroup$ @Emre, I loaded the data in data frame using pandas.read_pickle(filename) $\endgroup$ – Rakib May 26 '17 at 19:50

In the absence of a MultiIndex (the Right Way$^\mathrm{TM}$), the apply method can do what you want; e.g.

    max_retweet=df.tweets.apply(lambda x: x.retweet_count.argmax('retweet_count')),
    avg_retweet=df.tweets.apply(lambda x: x.retweet_count.mean())

       avg_retweet  max_retweet  
26662     0.045476          187  
32316     0.821538          427  
25879     0.633681          583  
43411     0.112465          421  
28840     0.472222            8

Also, don't share data with pickle; it's dangerous.

  • $\begingroup$ I have a similar data structure and apply method seems not work for my situation. Can you help me to solve my problem? hackmd.io/s/ryrCfAjZG $\endgroup$ – Po-Ying Chen Dec 11 '17 at 9:40
  • $\begingroup$ Please post a new question at StackOverflow. $\endgroup$ – Emre Dec 11 '17 at 18:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.