1
$\begingroup$

Let's say I've a dataset with 800 rows(40 entries for each of 20 users). The entries are user session log ( columns are - browser, os, time, date etc for a specific session).

Now each user has unique id (1-20). Let's say user_id=1 is special one and I need to detect it whenever new data comes.

So for new data I need to predict whether that session is of user_id=1 or not.

My question is How to do that ?

One way I thought is make a feature representation for each of the 20 users, and whenever new data comes, take the distance of the data from each of feature and see the minimum distance.

But the problem is when I make a unique feature representation for each user, how to deal with columns like browser, os - because a user can have used multiple browsers in all those 40 sessions ?

$\endgroup$
  • $\begingroup$ Your dataset has 200 rows corresponding to users? And what's on the columns? What variables to you have? $\endgroup$ – Leevo Jun 10 '19 at 7:11
  • $\begingroup$ can you add some sample data? $\endgroup$ – MattR Jun 10 '19 at 12:37
  • $\begingroup$ I might not be clear with the question, so I updated it, have a look again $\endgroup$ – ashukid Jun 10 '19 at 16:37
1
$\begingroup$

If I understand correctly, you want to create features? there are a few ways to do this. I will first talk about Label Encoding and Hot Label Encoding(link is the first I found on Google). This depends on the model, I will explain below.

Since we want to use categorical data in a model, most models behave better with numerical data. So let's convert categorical -> numeric:

import pandas as pd
from sklearn.preprocessing import LabelEncoder

df = pd.DataFrame({'user':['adam','adam','susie','jane'], 
                   'browser':['chrome','firefox','chrome','opera']})

   browser   user
0   chrome   adam
1  firefox   adam
2   chrome  susie
3    opera   jane

label_encoder = LabelEncoder()
# convert browser to numeric
df['browser_encoded'] = label_encoder.fit_transform(df['browser'])

   browser   user  browser_encoded
0   chrome   adam                0
1  firefox   adam                1
2   chrome  susie                0
3    opera   jane                2

In the "real world" you would drop browser column as its now encoded. I left it to show you what's happening.

Now notice how our data is now numeric. But now there is a problem. If you're using something like multiple linear regression, you've just assigned a higher weight to the Opera browser since it's a higher number. (2>1 and 2>0). But since this is categorical, and Opera isn't "weightier" than other browsers in our model, we need to solve this. This is where One Hot Encoding comes in to play.

pd.concat([df,pd.get_dummies(df['browser'])], axis=1)

   browser   user  browser_encoded  chrome  firefox  opera
0   chrome   adam                0       1        0      0
1  firefox   adam                1       0        1      0
2   chrome  susie                0       1        0      0
3    opera   jane                2       0        0      1

Now we have our data in columns. the 1 value appears when someone has used that browser for that record. See the first record? where chrome column is 1, because adam used 'chrome' in the browser column?.

There is one last thing when it comes to label encoding! You don't want to fall in the dummy variable trap with certain models. The basic logic is: If you know the browser is not Chrome or Firefox, it must be Opera. If you know the browser is Chrome, you know that it is not Firefox or Opera. So you only need N-1. If you use all the columns, your model may not perform well. You need to drop only one of the columns. you can do that with drop_first=True

pd.concat([df,pd.get_dummies(df['browser'], drop_first=True)], axis=1)
   browser   user  browser_encoded  firefox  opera
0   chrome   adam                0        0      0
1  firefox   adam                1        1      0
2   chrome  susie                0        0      0
3    opera   jane                2        0      1

Another way to create features is to make them yourself. You mentioned that a user can use more than one browser. I'm making something up for the sake of a simple example - but maybe the number of browsers they use can be a feature? You can create one like so:

df['num_of_browsers_used'] = df.groupby('user', as_index=False).transform('nunique')['browser']
    browser user    browser_encoded num_of_browsers_used
0   chrome  adam    0   2
1   firefox adam    1   2
2   chrome  susie   0   1
3   opera   jane    2   1
|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.