0
$\begingroup$

I successfully trained my model using the sklearn's multiple linear regression. This is the code I used:

import pandas as pd

dataset = pd.read_csv('C:\\mylocation\\myfile.csv')
dataset2 = pd.get_dummies(dataset)
y = dataset.iloc[:, 31:32].values
dataset2.pop('Target')
X = dataset2.iloc[:, :180].values

#Split the dataset
from sklearn.cross_validation import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.2, random_state = 0)

#Feature Scaling
#from sklearn.preprocessing import StandardScaler

#sc_X = StandardScaler()
#X_train = sc_X.fit_transform(X_train)
#X_test = sc_X.transform(X_test)

from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
regressor.fit(X_train, y_train)

#Predicting the Test set results
y_pred = regressor.predict(X_test)

According to validation results my y_pred is a reasonable predictor. Now, I would like to take into production this model and I am wondering what are reasonable steps to apply this model to the whole dataset I have stored and future datasets if needed.

$\endgroup$

1 Answer 1

0
$\begingroup$

You should first cross validate your pipeline, making sure that you get an homogeneous y_pred result.

Then you can retrain a model with the same parameters on your full dataset. Pickle the model as well as any preprocessor tools, and reuse them to predict on new data.

$\endgroup$
10
  • $\begingroup$ Thanks. Could you possibly give a few pointers on how to do this specifically? $\endgroup$
    – Taylrl
    Dec 31, 2018 at 11:27
  • $\begingroup$ Check pickle, that's the only thing you need. $\endgroup$ Dec 31, 2018 at 11:32
  • $\begingroup$ Ok thanks. Could you provide a bit more clarity about what I have to do to retrain the model with the same parameters, based on the code I have above? $\endgroup$
    – Taylrl
    Dec 31, 2018 at 14:23
  • 1
    $\begingroup$ regressor.fit(X, y) $\endgroup$ Dec 31, 2018 at 14:28
  • 1
    $\begingroup$ No worries. If X is your new data, then yes, this will work. $\endgroup$ Dec 31, 2018 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.