# Correlation feature selection followed by regression

I have quarterly results data for a company with around 100 variables. Total 60 quarters results are available (total records 60).

sample data: (only few columns & 10 rows)

I would like know following,

1. which ML algorithms / methods to be used to get the information about most important variable(s) affecting the price movement

2. how much each variable is contributing towards price movement (+ve or -ve)

3. predict, if a variable value is changed then how it'll affect the price movement

Thanks!

I think this is a case for linear regression with a lasso/ridge penalty. The lasso/ridge does „shrink“ features/variables, so that it is easy to see which features are important. Since you have 100 variables, you could opt for lasso, since lasso can also „automatically“ exclude features. Here is a lasso example in Python: https://datascience.stackexchange.com/a/53639/71442.

• Thanks Peter. I have tried lasso but most of the time it gives me less than 4 features, some time no variable at all. I have tried RFE method as well (with liner regression) & it gives me more variables. Is there any way I can specify the number of important variables to be given in the output by lasso method? Commented Jun 24, 2019 at 6:59
• No, you cannot specify the number ex ante. However, if you use Ridge or Elastic Net instead of Lasso, no feature will become zero, but they are shrunken anyway. Could be worth a try. Also make sure you tune the choice of regulation parameter for Lasso/Ridge in order to get optimal results. Commented Jun 24, 2019 at 9:00
• Tried Ridge, works for me. One more question, predict function gives the next predicted value for the target feature in both Lasso & Ridge, right? eg. if I have 500 observations in my data then the prediction value is for the 501st observation? Also, how can I get more than one prediction value? Commented Jun 26, 2019 at 12:21
• So your model looks like y = b*x + u. Where b is an estimated coefficient, x are the features you used in regression, and u is the error term. Now you can take any values of x, plug it in the model, and predict y. For each x you get one predicted y (called y_hat). So it is not the "501th" value. When you have "many" x rows in your x_test, you get as many y_hat as you have rows in your x_test. Commented Jun 26, 2019 at 13:25

Try below code, here I have taken other data. pca = PCA(n_components = None) first you give here none to check how much each feature is contributing.

# Importing the libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd

# Importing the dataset
X = dataset.iloc[:, 0:13].values
y = dataset.iloc[:, 13].values

# Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)

# Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)

# Applying PCA
from sklearn.decomposition import PCA
pca = PCA(n_components = None)
X_train = pca.fit_transform(X_train)
X_test = pca.transform(X_test)
explained_variance = pca.explained_variance_ratio_
print explained_variance


Output:

array([0.36884109, 0.19318394, 0.10752862, 0.07421996, 0.06245904,
0.04909   , 0.04117287, 0.02495984, 0.02308855, 0.01864124,
0.01731766, 0.01252785, 0.00696933])


In the above output you can see the contribution in decreasing order. If you choose n =2 , then 57% of variance you are covering, etc. After reducing dimension, you can choose algorithm for this.