5
$\begingroup$

I am having a problem during feature engineering. Looking for some suggestions. Problem statement: I have usage data of multiple customers for 3 days. Some have just 1 day usage some 2 and some 3. Data is related to number of emails sent / contacts added on each day etc.

I am converting this time series data to column-wise ie., number of emails sent by a customer on day1 as one feature, number of emails sent by a customer on day2 as one feature and so on. But problem is that, the usage can be of either increasing order or decreasing order for different customers.

ie., example 1: customer 'A' --> 'number of emails sent on 1st . day' = 100 . ' number of emails sent on 2nd day'=0

example 2: customer 'B' --> 'number of emails sent on 1st . day' = 0 . ' number of emails sent on 2nd day'=100

example 3: customer 'C' --> 'number of emails sent on 1st . day' = 0 . ' number of emails sent on 2nd day'=0

example 4: customer 'D' --> 'number of emails sent on 1st . day' = 100 . ' number of emails sent on 2nd day'=100

In the first two cases => My new feature will have "-100" and "100" as values. Which I guess is good for differentiating. But the problem arises for 3rd and 4th columns when the new feature value will be "0" in both scenarios Can anyone suggest a way to handle this.

One way to handle this:

I can add "No change" in those scenarios, but I am confused about one thing. If I do that, I will have to make the new feature as categorical, which is not ideal as the other values will be continuous.

Instead, I can have absolute values in the new feature and indicate the trend as "+1" or increasing "-1" for decreasing "no change" for no change and "0" if both the values have been "0". Would that be a good approach though?

The end goal is to predict if a user would continue using the application or not. So it basically would be a two-class model. And I would want to capture even the scale of usage i.e., "A user sending 100 emails every day" should be different from "B user sending 10000 emails every day"

$\endgroup$
  • 1
    $\begingroup$ could you explain a bit better what are you trying to predict? Your question is pretty well explained but the kind of model you plan do train might give some of us better ideas. $\endgroup$ – Pedro Henrique Monforte Apr 11 '19 at 1:40
  • $\begingroup$ I would want to predict if a user would continue using the application or not. So it basically would be a two-class model. Does that answer? $\endgroup$ – SSuram Apr 11 '19 at 2:32
  • $\begingroup$ Yes, just add it to your question and it will be perfect $\endgroup$ – Pedro Henrique Monforte Apr 11 '19 at 2:35
3
$\begingroup$

Well, you want to identify change in usage you could try something like:

$$ f(day_1,day2) = \frac{day_2-day_1 + \delta}{||day_2-day_1+\delta||} \times \Biggr|\Biggr|\frac{day_2+day_1}{(day_2+day_1+1)(day_2-day_1+1)}\Biggl|\Biggl| $$

where $\delta$ is the eps of your machine (minimum value needed to be summed to differ it from other floats)

that will give you $$f(100,0) \approx -98.02$$ $$f(0,100) = 100$$ $$f(100,100) \approx 0.995$$ $$f(0,0) = 0$$

You can look at my experiment here

This will map all non-changes from $[0,1]$ where $f(0,0)$ maps to $0$ and $f(\infty,\infty)$ maps to $1$

Where is it from? Just tuned the function manually. But I think this might suffice for your application

Explaining the idea

You want to have a feature that packs a lot of information: - Is the usage greater than zero? - Is it increasing or decreasing? - If it is stalled, how much is the usage?

Well, your usage vary in integer values so you can map the entire non-changing but above 0 case to a previously non-used interval.

The function above will map in $[0,1]$ all non-changing possibilities, in a exponential kind of way ($a^{(-\frac{1}{usage})}$) also you can extract the actual value from positive changes and the approximate value for negative change (been a better approximation when the drop is high)

This is not the perfect scenario but it is the maximum information I could compress into 1 variable with little loss.

$\endgroup$
  • $\begingroup$ I am not sure if it would answer --- "'''And I would want to capture even the scale of usage i.e., "A user sending 100 emails every day" should be different from "B user sending 10000 emails every day ""''---- part of the question. Could you please explain? $\endgroup$ – SSuram Apr 11 '19 at 2:38
  • $\begingroup$ What would you say about adding the below info to it f = (((d2-d1+eps)/abs(d2-d1+eps))*abs((d2+d1)/(d1+d2+1)*(d2-d1+1)))*(d2/1000)*(d1/1000) where "1000"-- would be max(usage). $\endgroup$ – SSuram Apr 11 '19 at 3:02
  • $\begingroup$ that will actually return zero for near every case $\endgroup$ – Pedro Henrique Monforte Apr 11 '19 at 3:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.