1
$\begingroup$

At my office, I am stuck in a weird situation. I am asked to perform a regression algorithm on the data, in which the target variable is continuous having values range between 0.6 to 0.9 with 8 digits of precision after the decimal. Although I know and have applied many linear and non-linear regression algorithms in the past the case here is something different. There is one variable, which, according to my BU, should have a positive and linear correlation with the target variable. But when I ran Pearson's correlation, the variable is negatively correlated and by plotting a scatter plot I can see that the relationship is not linear at all. What transformations can I perform on the variable so that it can show a positive correlation? I am fairly new to this problem so hoping to get it solved here. Thanks much everyone in advance.

$\endgroup$
  • 2
    $\begingroup$ In order to help can you post a sample of the correlation as well as the major parts of your code? It's rather hard to help without seeing these things. Strictly speaking, if a variable and target is not linear in a relationship then it should not be forced into one. But happy to have a look once you post as requested $\endgroup$ – IsakBosman Jan 15 at 21:23
  • $\begingroup$ Is it possible that there is a positive correlation only when controlling for some other variables? Like, eh, I bet that Walmart's global umbrella sales and rainfall aren't that correlated, but, controlling for whether it's raining or not per store per day, shows a correlation. (Poor example off the top of my head) $\endgroup$ – Sean Owen Jan 16 at 4:17
  • $\begingroup$ @IntuAi Actually I am using client's system and there are many restrictions so couldn't able to post the code and scatter plot. The task at hand is to predict service level in order to optimize the inventories. Now service levels are the probability that tells us that the organization will not hit a stock out. The variable I am talking about is the safety stock hours(converted from safety stock). According to the BU, safety stock hours should have a positive relation with service level. But in reality that is not the case and thus I am facing difficulty to model the data $\endgroup$ – shivanshu dhawan Jan 16 at 7:38
  • $\begingroup$ @SeanOwen I have checked with multiple combinations but couldn't able to find the required results $\endgroup$ – shivanshu dhawan Jan 16 at 7:39
0
$\begingroup$

What's a BU? Business Unit?

Can you tell anything about the variable and target variable? Can you include the scatter plot?

It is still not clear if the problem is not in the domain knowledge/assumptions.

A business example: Profit depends on sales, which is generally assumed to have a positive correlation. But after a threshold marginal sales cost exceeds profits (very expensive to acquire new customers), we can still have a (local) negative correlation. If you only have data in the 'marginal' segment, you can draw incorrect conclusions.

$\endgroup$
  • $\begingroup$ Yes BU is Business unit. The task at hand is to predict service level in order to optimize the inventories. Now service levels are the probability that tells us that the organization will not hit a stock out. The variable I am talking about is the safety stock hours(converted from safety stock). According to the BU, safety stock hours should have a positive relation with service level. But in reality that is not the case and thus I am facing difficulty to model the data $\endgroup$ – shivanshu dhawan Jan 16 at 7:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.