# How do I use lagged independent variable in statsmodel OLS regression?

If there is good reason to believe that an independent variable (x) has a lagged effect on dependent variable (y) of a OLS regression model.

import statsmodel
import pandas

# Create DataFrame

sDataFrame = pd.DataFrame({
'Time': ['2012-Q1','2012-Q2','2012-Q3','2012-Q4','2013-Q1','2013-Q2'],
'GDP': ['6.1','6.4','6.8','7.1','6.2','5.8'],
'FDI': ['3.2','2.9','3.1','2.5','1.8','2.3'],
'Unemployment': ['12.1','10.3','11.5','12.4','9.8','11.2']
})

My current formula looks something like this:

model = sm.ols(formula = 'GDP ~ FDI + FDI_Lag + Unemployment', data=sDataFrame).fit()
model.summary()


My question is how do I include FDI_Lag variable in my model, which is FDI - 1 i.e the previous value in DataFrame.

I would do something like that:

import statsmodels.api as sm
import pandas as pd

sDataFrame = pd.DataFrame({
'Time': ['2012-Q1','2012-Q2','2012-Q3','2012-Q4','2013-Q1','2013-Q2'],
'GDP': [6.1,6.4,6.8,7.1,6.2,5.8],
'FDI': [3.2,2.9,3.1,2.5,1.8,2.3],
'Unemployment': [12.1,10.3,11.5,12.4,9.8,11.2]
})

X=sDataFrame.loc[:,['FDI','Unemployment']]
X['FDI_Lag'] = X['FDI'].shift()

y = sDataFrame.loc[:,'GDP']

model = sm.OLS(y,X, missing='drop')
result = model.fit()
result.summary()


@user575406's solution is also fine and acceptable but in case the OP would still like to express the Distributed Lag Regression Model as a formula, then here are two ways to do it - In Method 1, I'm simply expressing the lagged variable using a pandas transformation function and in Method 2, I'm invoking a custom python function to achieve the same thing.

import numpy as np
import pandas as pd
import statsmodels.formula.api as sm

sDataFrame = pd.DataFrame({
'Time':         ['2012-Q1','2012-Q2','2012-Q3','2012-Q4','2013-Q1','2013-Q2'],
'GDP':          np.array(['6.1','6.4','6.8','7.1','6.2','5.8'], dtype='float'),
'FDI':          np.array(['3.2','2.9','3.1','2.5','1.8','2.3'], dtype='float'),
'Unemployment': np.array(['12.1','10.3','11.5','12.4','9.8','11.2'], dtype='float')
})

def lag(x, n):
if n == 0:
return x
if isinstance(x, pd.Series):
return x.shift(n)
else:
x = pd.Series(x)
return x.shift(n)

x = x.copy()
x[n:] = x[0:-n]
x[:n] = np.nan
return x

# Method 1
model1 = sm.ols(formula = 'GDP ~ 1 + FDI + FDI.shift(1) + Unemployment', data=sDataFrame).fit()
model1.summary()

# Method 2
model2 = sm.ols(formula = 'GDP ~ 1 + FDI + lag(FDI, 1) + Unemployment', data=sDataFrame).fit()
model2.summary()


Statsmodel's Formula expression is parsed using patsy python package - More details on that can be found here: https://patsy.readthedocs.io/en/v0.1.0/overview.html

If an independent variable (x) has a lagged effect on dependent variable (y) of a OLS regression model, you must insert its lagged value and not current value in time series data. Your proposed stats model includes both current value and lagged value . This is not justifiable. Therefore, correct your model and proceed.

• I am not sure that is true. There are def models that incorporate current val, and multiple levels of lagged value. – kms Sep 26 '20 at 19:16
• Incorporating the same variable twice as independent variables that are auto-correlated can be dangerous in a multiple regression analysis. The variance analysis presumes I.I.D. – Subhash C. Davar Sep 26 '20 at 23:23
• It is illogical to presume that GDP is influenced by both current and its lagged levels. – Subhash C. Davar Sep 26 '20 at 23:28
• If you are not sure why would you follow a def model ! gross domestic product and foreign direct investment relationship can be better understood from discipline of economics. – Subhash C. Davar Sep 26 '20 at 23:35