# How can I use Machine learning for inter-relationship between Features?

Machine learning is used mostly for prediction and there are numerous algorithms and packages for this.

How can I use machine learning for studying inter-relationships between features? What are major packages and functions for this? Are there any packages for graphics in this area? Can artificial neural networks also be used for this purpose? If so, any particular type is specifically suited for this?

I do not want to limit to any particular language like Python or R.

I use pair plots to study the inter-relationship between features. Pair plot gives the first level information of the features. Seaborn library has pairplot function and even matplotlib has the same.

Another thing you can use heatmap which gives the co-relation between features, by using heat map we can see features that are high co-related and may eliminate one of them. As a word of caution you must good reason or domain knowledge to drop a feature

There is so much detail that you have not specified in your question that it is hard to recommend a specific approach. Certainly, the graphical approach outlined by @Sunil is a great way to start particularly if the data is continuous.

Additionally, you could investigate some of the statistical tests for multicollinearity. For example, the Farrar – Glauber test is a statistical test that may be useful to identify highly correlated attributes that may negatively impact some machine learning techniques.

It is also equally important to realize that not all ML techniques are sensitive to multicollinearity.

• Thanks for your answer. Which ML techniques are not sensitive to multicollinearity?
– rnso
Nov 7 '18 at 2:45
• I believe multicollinearity is more of an issue with statistical approaches like linear and logistic regression. Again, since you have little detail as to what you are trying to accomplish, it’s hard to give suitable recommendations. I might start with a decision tree or random forest if the data is suitable. Nov 7 '18 at 14:42

To study the inter relationship between features, if the features are continuous variables you can use covariance to find their inter relationship. Covariance is a measure of how much a feature is dependent on other. If there are n features, construct a n * n covariance matrix. Cov(X,Y) = 1 implies X is highly related to Y. Negative value of correlation implies they are not correlated.

• What is difference between covariance and correlation?
– rnso
Nov 7 '18 at 10:45
• covariance is a measure of how much two variables are correlated. Nov 7 '18 at 10:48
– rnso
Nov 7 '18 at 10:49

I guess Ridge regression technique is used when the data suffers from multicollinearity.It solves the multicollinearity problem using the shrinkage parameter (lambda).

Also, the presence of multicollinearity does not affect the predictive capability of the model

• You should not put multiple answers on one page unless your answer is becoming very long and new answer is clearly separate from previous one. Also you will get much more upvotes if the answers are well described with explanations, especially with code. Above answer is more like a comment.
– rnso
Nov 7 '18 at 8:25
• Sure @mso . I shall take care of that Nov 7 '18 at 8:36