1
$\begingroup$

I was thinking of training a neural network that would be able to classify twitter users according to their followers. For example, I would like to know if a user is "gamer" or not by the people he follows. Have a dataset of people who are gamers or not and train them based on that.

The problem that I find is that the number of followers of each person will vary ... One person could follow thousands of users while another could only follow hundreds, so how can I determine my feature there?

Any suggestion? How could I do this?

This is in the way I am representing the data, so you can understand the problem of the features I was referring to

enter image description here

I did not know very well if asking this on StackOverflow, I have finally come here, it is the first time I post here, I hope it is the right place. Thank you.

$\endgroup$
2
  • $\begingroup$ Sounds like a fun project! What are the characteristics of the people being followed that you'll use to classify them? In other words, looking at the people someone follows, what about them would make you classify them as a gamer? $\endgroup$ Jun 4, 2020 at 0:55
  • $\begingroup$ I don't really know, as a human I can identify if they are a gamer or not, So I thought that I could create a csv classifying a user as gamer or not, so that, somehow, my neural network can see what is in common with the followers of each type of class (gamer or not) The problem I see is that the features depend on how many followers a person will have, one person can have more parameters than another (more followers) and I do not know very well how to treat that. @NickKoprowicz $\endgroup$
    – Sharki
    Jun 4, 2020 at 1:03

3 Answers 3

1
$\begingroup$

Your features could be the other twitter accounts that the users follow with a value of 1 if your user follows that account or a value of 0 if the user does not. Here is a very simplified version, using something like your starting point.

# If you start like this:
user_id  feature_1  feature_2  feature_3  label
user_1   account_1  account_5  NA         1
user_2   account_2  account_3  account_4  0
user_3   account_1  account_2  account_5  1
user_4   account_3  NA         NA         0

# Transform your data to look like this:
user_id  account_1  account_2  account_3  account_ 4  account_5  label
user_1   1          0          0          0           1          1
user_2   0          1          1          1           0          0
user_3   1          1          0          0           1          1
user_4   0          0          1          0           0          0

This is a trivial example with only five accounts to follow. You may be dealing with millions of accounts to follow, which would be at best unwieldy. Since you already have a sense that some followed accounts are correlated with your label, you could simplify the features to only check those that are strongly correlated.

In my simple example above, we may only want to keep the features account_1, account_3, and account_5 since they correlate strongly to the label.

$\endgroup$
1
  • $\begingroup$ I agree that this is the correct way to go. The features would be the accounts they follow, and I am sure there are some gamer "influencers" out there which are most predictive. After the df is set up how you suggested, the OP would just need to reduce dimensionality. I think it would be interesting to do feature reduction as opposed to something like PCA so the OP could see which accounts are the best predictors. $\endgroup$ Jul 10, 2020 at 4:50
0
$\begingroup$

This is a really interesting question. So, you are asking for features that you could look at to classify whether Twitter users are gamers, based on the number of followers they have.

That is awesome that you identified variation in number of followers. I would try and plot the variation in number of followers between Twitter users who are known to be gamers and those who are not. If the distribution of number of followers vary significantly (in a broad sense, if their means and standard deviations differ greatly from each other) between gamers and non-gamers, then this would be a great feature to add as part of your data for classification (using a neural network).

a busy cat1

Another avenue you could explore are the Twitter users' posts themselves. One approach, for example, could be looking at frequency of "gaming-related" words and whether these occur more often among gamers than non-gamers.

$\endgroup$
3
  • $\begingroup$ Sorry, my english isn't really good, so I think you misunderstood (or rather, I misspelled) what I was trying to say. I don't see any correlation between the number of followers, but their followers themselve. For example, if you're a gamer, is more likely you will follow accounts of people like EA3, or youtubers whose content is about video game. The number itself don't tell me anything (.... I'm right?) but the list of people they follow, Each person that follows could be a feature, but each person follows a different total number of people. Sorry again for my english... @shepan6 $\endgroup$
    – Sharki
    Jun 5, 2020 at 20:09
  • $\begingroup$ Ah, not a problem. So the features here would be essentially the users which, for example, user A follows (user A friends). So the first pass to convert the list of user A friends into a one-encoded vector. machinelearningmastery.com/… $\endgroup$
    – shepan6
    Jun 6, 2020 at 14:30
  • $\begingroup$ Can I put the list of people that a user follows as a single one feature? For example, Matrix data where data has data[:, 0]and data[:, 1], data0 would contain all of a person's followers, and data1 the label. If I did one-hot encoded or embedding words, would I get the same result that doing it user by user in a different feature? I'll add an image to my previous post so you can see how I'm currently interpreting the data. You will realize that the number of features will be different depending on the number of users that each person follows, thanks again! :) $\endgroup$
    – Sharki
    Jun 6, 2020 at 14:50
0
$\begingroup$

You should limit the number based on judgment or any other info.

General case - When you want to know the interest of user -
In this case, You should use the user's interaction level with the Handle instead of following or not. \begin{array} {|r|r|r|} \hline Most-Interacted &2nd-Most &....... &N^t-Most&Label \\ \hline handle-1 &handle-2 &....... &handle-N&Gamer \\ \hline ...... &..... &..... &.....&Footballer \\ \hline handle-M1 &handle-M2 &....... &handle-MN&Actor \\ \hline\end{array} I am not sure, how will you do the encoding. Maybe you can embed the list of the handles using some approach which catches similarity e.g. Cosine.


Special case - When you want to know If he is a Gamer or not -
In this you should pick 100 (or any good number based on judgement/basic data analysis) Handle which generally people follow. You may get a list from related website

\begin{array} {|r|r|r|} \hline Handle-1 &Handle-2 &....... &Handle-N &Label \\ \hline follow(1) &Not(0) &....... &Follow(1)&Gamer(1) \\ \hline ...... &..... &..... &.....&Gamer(1) \\ \hline Not(0) &Not(0) &....... &Follow(1)&Not-Gamer(0) \\ \hline\end{array} In this case encoding is already there. You may start with a big number too and later reduce the dimension based on variance towards Handles(PCA)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.