0
$\begingroup$

I successfully trained my model using the sklearn's multiple linear regression. This is the code I used:

import pandas as pd

dataset = pd.read_csv('C:\\mylocation\\myfile.csv')
dataset2 = pd.get_dummies(dataset)
y = dataset.iloc[:, 31:32].values
dataset2.pop('Target')
X = dataset2.iloc[:, :180].values

#Split the dataset
from sklearn.cross_validation import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.2, random_state = 0)

#Feature Scaling
#from sklearn.preprocessing import StandardScaler

#sc_X = StandardScaler()
#X_train = sc_X.fit_transform(X_train)
#X_test = sc_X.transform(X_test)

from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
regressor.fit(X_train, y_train)

#Predicting the Test set results
y_pred = regressor.predict(X_test)

According to validation results my y_pred is a reasonable predictor. Now, I would like to take into production this model and I am wondering what are reasonable steps to apply this model to the whole dataset I have stored and future datasets if needed.

Improve this question
$\endgroup$
0
$\begingroup$

You should first cross validate your pipeline, making sure that you get an homogeneous y_pred result.

Then you can retrain a model with the same parameters on your full dataset. Pickle the model as well as any preprocessor tools, and reuse them to predict on new data.

Improve this answer
$\endgroup$
  • $\begingroup$ Thanks. Could you possibly give a few pointers on how to do this specifically? $\endgroup$ – Taylrl Dec 31 '18 at 11:27
  • $\begingroup$ Check pickle, that's the only thing you need. $\endgroup$ – Matthieu Brucher Dec 31 '18 at 11:32
  • $\begingroup$ Ok thanks. Could you provide a bit more clarity about what I have to do to retrain the model with the same parameters, based on the code I have above? $\endgroup$ – Taylrl Dec 31 '18 at 14:23
  • 1
    $\begingroup$ regressor.fit(X, y) $\endgroup$ – Matthieu Brucher Dec 31 '18 at 14:28
  • 1
    $\begingroup$ No worries. If X is your new data, then yes, this will work. $\endgroup$ – Matthieu Brucher Dec 31 '18 at 14:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.