# Finding relationships from group of variables

Given time-series data of a set of input metrics $M_1$, $M_2$, $M_3$ ... $M_n$ and a set of observed metrics $O_1$, $O_2$, $O_3$ ... $O_m$.

For each $M_i$, what is the best way to find equations that best describe the relationship between $M_i$ and the set of observed metrics?

It is given that all $M_i$ are independent of each other.

• The question is a bit fuzzy. You mean each $M_i$ is a time series itself? then set of observations should be 2-Dimensional i.e. $O_{i,j}$. Am I right? – Kasra Manshaei Oct 17 '17 at 7:03
• They are all time-series. The Ms and Os. They are all measured at the same time interval. I just want to equations between each M with the Os. For example, M1 = f1(O1, O4, O5), M2 = f2(O3, O4, O9, O10)... – Sid Prasad Oct 17 '17 at 7:41

It is a pretty difficult task to obtain a function $f_1$ satisfying $M_1(t) = f_1(O_1(t), O_2(t),...)$. It is mostly like a regression problem where you are trying to fit $M_1(t)$.

I could suggest three approaches:

1. Simple multiple regression. http://scikit-learn.org/stable/modules/linear_model.html. If that doesn't work, you could increase the order of your observed metric space $O$ using PolynomialFeatures http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html and then try multiple regression.

I don't really fancy these other two approaches but will list them anyway, in case you want to have a go.

1. Multivariate Adaptive Regression Splines http://contrib.scikit-learn.org/py-earth/content.html You will get a good fit but the function will be discrete in nature.
2. OpenMLC https://github.com/MachineLearningControl/OpenMLC-Python/releases which uses Genetic Algorithms. This will be able to provide you with functions like $sin(O_i), log(O_i), exp(O_i)$ as well. But the problem is that since it uses genetic algorithms, you won't obtain a deterministic function. The function will change each time you run it with varying accuracy.
• Thanks for that. But there is also the problem of selecting which of the Os to use in my equation for Mi. What sort of feature selection technique would you suggest? Also, there is nothing given about the type of functions. The time-series data could be linear, quadratic, exponential, anything. – Sid Prasad Oct 17 '17 at 8:48
• For any of the methods, you don't need to manually select any feature. If the feature is relevant, it will pop up in the function. If you do want to see how relevant each feature $O_i$ is for a given $M$, you could plot a matrix scatterplot. This will give you an idea of how relevant each feature $O_i$ is for every $M$ INDIVIDUALLY. – tomar__ Oct 17 '17 at 9:25
• Isn't there a way to automate the process? – Sid Prasad Oct 17 '17 at 10:36
• You could use Principal Component Analysis for each $M$ and obtain that. – tomar__ Oct 17 '17 at 10:41