# Minimize correlation between input and output of black box system

I am not sure if "minimize correlation" is the right title for this issue but I could not find a better sentence to describe what I would like to achieve.

Let's say that I have a black box with multiple inputs and a single output. I know one of the inputs and the output and I have multiple example recordings of both. This known input modifies the output in a way that it is not desired, therefore, I would like to get rid of this "noise" caused by the known input. The transfer function for this input can be safely assumed as linear.

What I am doing right now, it is to loop through the example recordings, creating a linear regression model to predict the unwanted outcome and subtracting it from the real measured output signal, for each example. Afterwards, I compute the average of all the fixed output signals to reveal meaningful data beyond noise.

This strategy seems to work according to the following plot: X axis is the known input signal, Y axis is the output signal, blue and green dots represent the averaged data before and after applying the linear regression algorithm, respectively. Lines are the best fit for each data set.

You can see that the green line ("cleaned" dataset) has the smallest slope, meaning that the output variable is considerably less linearly correlated with the input than it was previously. Therefore, I assume that the regression technique explained before is working as expected.

My question, looking at the plot, is there any mathematical procedure to directly "project" the original dataset in a way that the correlation between the input and output variables is minimized? Is there any math trick to avoid the use of the regression technique on all the example datasets to obtain a similar result?

My written expression is not the best so please feel free to comment the question if you need further explanations.

Any code is welcomed but python (pandas, numpy, etc.) and Matlab are preferred. Theoretical explanations are also very welcomed.

• I think there are no alternatives to your method! – stochazesthai May 26 '15 at 13:58
• Why don't you just see which input has the least pearson correlation(since the relationship is linear) to the output, and then ignore it ? – Hristo Buyukliev May 29 '15 at 8:49

Suppose you have 1 million data entries. Each entry consists of three inputs X1, X2, X3 and one output Y. One of the three inputs, X1, is a noise. You want to find the relation between X1 and Y. So you can remove the impact of X1 on Y.

Let's use a simplified example, Y = 2X1 + 3X2 + 4X3 If you update each of the 1 million entries with: Y_new = Y - 2X1

Now you can examine the updated 1 million entries to find the relation between input X2, X3 and Y_new: Y_new = 3X2 + 4X3

The relation between X1 and Y can be computed with different algorithms, such as regression, decision tree. How to measure which algorithm generates the best result, i.e. truly reflects the relation between X1 and Y hence removes the impact of X1 on Y as much as possible?

The problem you're solving is an interesting one. I would use a free tool such as Knime to find out which algorithm generates the best result in modeling the relation between X1 and Y. It's faster than coding in Python or Matlab.

Details: to put the problem intuitively, suppose the price of a house is determined by location, size of the house, condition of the house. Assume these three factors are independent from each other. You want to remove the impact of "condition of the house" on "price" as much as possible. In essence, you want to find the relation between "condition of the house" and "price" which may not be linear.

You can let Knime read the 1 million entries of "condition of the house" and "price", try Decision Tree, SVM, or even Ensemble Learning, and see which algorithm generates the best model of X1 and Y. In other words, try and error :)

• You are right! I would like to know what techniques could be used to tackle the problem. Specifically, I was asking whether there is a way to minimize the correlation between X1 and Y without using linear regression. Thank you for the clarification :) – David May 28 '15 at 8:25