I am not sure if "minimize correlation" is the right title for this issue but I could not find a better sentence to describe what I would like to achieve.

Let's say that I have a black box with multiple inputs and a single output. I know one of the inputs and the output and I have multiple example recordings of both. This known input modifies the output in a way that it is not desired, therefore, I would like to get rid of this "noise" caused by the known input. The transfer function for this input can be safely assumed as linear.

What I am doing right now, it is to loop through the example recordings, creating a linear regression model to predict the unwanted outcome and subtracting it from the real measured output signal, for each example. Afterwards, I compute the average of all the fixed output signals to reveal meaningful data beyond noise.

This strategy seems to work according to the following plot: enter image description here

X axis is the known input signal, Y axis is the output signal, blue and green dots represent the averaged data before and after applying the linear regression algorithm, respectively. Lines are the best fit for each data set.

You can see that the green line ("cleaned" dataset) has the smallest slope, meaning that the output variable is considerably less linearly correlated with the input than it was previously. Therefore, I assume that the regression technique explained before is working as expected.

My question, looking at the plot, is there any mathematical procedure to directly "project" the original dataset in a way that the correlation between the input and output variables is minimized? Is there any math trick to avoid the use of the regression technique on all the example datasets to obtain a similar result?

My written expression is not the best so please feel free to comment the question if you need further explanations.

Any code is welcomed but python (pandas, numpy, etc.) and Matlab are preferred. Theoretical explanations are also very welcomed.

  • 1
    $\begingroup$ I think there are no alternatives to your method! $\endgroup$ Commented May 26, 2015 at 13:58
  • $\begingroup$ Why don't you just see which input has the least pearson correlation(since the relationship is linear) to the output, and then ignore it ? $\endgroup$ Commented May 29, 2015 at 8:49

1 Answer 1


Please allow me to paraphrase your question to make sure I get your question right.

Suppose you have 1 million data entries. Each entry consists of three inputs $X_1$, $X_2$, $X_3$ and one output $Y$. One of the three inputs, $X_1$, is a noise. You want to find the relation between $X_1$ and $Y$. So you can remove the impact of $X_1$ on $Y$.

Let's use a simplified example, $Y = 2X_1 + 3X_2 + 4X_3$ If you update each of the 1 million entries with: $Y_{new} = Y - 2X_1$

Now you can examine the updated 1 million entries to find the relation between input $X_2$, $X_3$ and $Y_{new}$: $Y_{new} = 3X_2 + 4X_3$

The relation between $X_1$ and $Y$ can be computed with different algorithms, such as regression, decision tree. How to measure which algorithm generates the best result, i.e. truly reflects the relation between $X_1$ and $Y$ hence removes the impact of $X_1$ on $Y$ as much as possible?

The problem you're solving is an interesting one. I would use a free tool such as Knime to find out which algorithm generates the best result in modeling the relation between $X_1$ and $Y$. It's faster than coding in Python or Matlab.

Details: to put the problem intuitively, suppose the price of a house is determined by location, size of the house, condition of the house. Assume these three factors are independent from each other. You want to remove the impact of "condition of the house" on "price" as much as possible. In essence, you want to find the relation between "condition of the house" and "price" which may not be linear.

You can let Knime read the 1 million entries of "condition of the house" and "price", try Decision Tree, SVM, or even Ensemble Learning, and see which algorithm generates the best model of $X_1$ and $Y$. In other words, try and error :)

  • 1
    $\begingroup$ You are right! I would like to know what techniques could be used to tackle the problem. Specifically, I was asking whether there is a way to minimize the correlation between X1 and Y without using linear regression. Thank you for the clarification :) $\endgroup$
    – David
    Commented May 28, 2015 at 8:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.