0
$\begingroup$

I got a misunderstanding regarding basic statistics (I think) but I can't get my head around that: I did an online survey regarding the usage of some application. The user could answer 1 (I don't know that application), 2 (I know that) or 3 (I use that). Now I want to know, how many applications the average user uses:

df['iuse'].mean()

where df['iuse'] was calculated as the count of answer 3 in one returned answer. The result is something like 2.2.

Now, I want to know, how many applications a user of application X uses:

f = df['q1_1'] == 3 # For application 1, filter all answers where the user uses that
df.loc[f,'iuse'].mean()

That returns a number above 2.2 - for every single app: In short (and ugly):

[(4.235294117647059, 85),
 (4.966666666666667, 60),
 (2.7495274102079397, 1058),
 (4.609195402298851, 87),
 (6.391304347826087, 23),
 (4.122950819672131, 122),
 (4.850746268656716, 67),
 (3.1860068259385668, 586),
 (2.72192513368984, 1122),
 (3.520231213872832, 346),
 (4.276595744680851, 94)]

(left is the mean, and right is the count of answers for that application)

Now: Why is the usage overall less than the usage when seen from a specific application? I would expect to have at least some numbers below the total mean, but they are all above that. I'm confused :(

Thanks for any pointers and help Klaus

$\endgroup$
6
  • $\begingroup$ First: The answer in iuse is encoded (by you) as a number, but it is a so called categorical value. That means that you cannot apply just any operator on it: Not equal works, but a mean definitely does not. For the connoisseur: It seems ordinal, so you could rank it. Second: Could you include a sample of your dataframe? $\endgroup$ Mar 3 '20 at 13:00
  • $\begingroup$ OP: it would be clearer if you provided a sample frame. @SvanBalen, it sounds like the 'iuse' column has been generated. Each row is a user/response, with columns for each answer, like 'q1_1', and those are encoded 1/2/3; but 'iuse' then is just a row-wise count of qi_j==3. $\endgroup$ Mar 3 '20 at 14:42
  • $\begingroup$ Thanks guys, I'll make sure to include a better example next time, but for now @BenReiniger has given a good explanation $\endgroup$ Mar 3 '20 at 16:50
  • $\begingroup$ @BenReiniger Yes it could be a derived feature. Nevertheless it remains a categorial value, and taking the mean is... wait for it ... meaningless $\endgroup$ Mar 4 '20 at 13:18
  • 1
    $\begingroup$ @SvanBalen 'iuse' is a count variable; discrete, but numerical, not categorical. The mean of that is perfectly sensible. I'd agree about the mean of any of the qi_j's... $\endgroup$ Mar 4 '20 at 13:26
1
$\begingroup$

This is not necessarily unexpected (or broken). Imagine that users tend to use none or all of the applications. For a specific example, suppose 90 users use no apps at all, and 10 use all (say) 11 of them.

Then the average apps used by a user is $(90\cdot0+10\cdot11)/100=1.1$, but for each app, the average app-usage of a user who uses that app is $(10\cdot11)/10=11$. (In your case, for a sanity check maybe compute the number of users who use no apps.)

$\endgroup$
1
  • $\begingroup$ Thanks, that seems to explain it (at last for now :)) $\endgroup$ Mar 3 '20 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.