# Combine results from multiple models

I am using chunks of 100000 rows at a time from the CSV file to train the a simple LASSO model.

How do i combine all of these models trained from these different chunks? I would like to use all these trained models for prediction

I am familiar with DASK and other alternatives but i would like to use Pandas.

pipelines = {
'lasso' : make_pipeline(StandardScaler(), Lasso(random_state=123))
}

for key, value in pipelines.items():
print( key, type(value) )

# Lasso hyperparameters
lasso_hyperparameters = {
'lasso__alpha' : [0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10]
}

hyperparameters = {
'lasso' : lasso_hyperparameters
}

# Create empty dictionary called fitted_models
fitted_models = {}

# Create cross-validation object from pipeline and hyperparameters
model = GridSearchCV(pipeline, hyperparameters[name], cv=10, n_jobs=-1)

def train(X_train, y_train):
# Fit model on X_train, y_train
model.fit(X_train, y_train)

# Store model in fitted_models[name]
fitted_models[name] = model

# Print '{name} has been fitted'
print(name, 'has been fitted.')
print ("__________________________________")
print (model.cv_results_)

for df in pd.read_csv('train_V2.csv', chunksize=100000):
df = df.dropna()
df = pd.get_dummies(df, columns=['matchType'])
df_train = df.drop(['Id', 'groupId', 'matchId'], axis = 1)
y = df_train.winPlacePerc
X = df_train.drop('winPlacePerc', axis=1)
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2,
random_state=1234)
X_train = np.asarray(X_train)
X_test = np.asarray(X_test)
y_train = np.asarray(y_train)
y_test = np.asarray(y_test)

train(X_train, y_train)

• name should also be defined in your code. – rnso Nov 20 '18 at 1:21

## 1 Answer

What you are looking for is called "stochastic optimization". You don't need to fit separate models and then combine them.

• Thanks. The reason I am doing this is because I have some 40 million rows and total data size is 650 mb. I started getting memory errors and hence decided to go with chunking – dinesh patro Oct 21 '18 at 2:44
• Same thing - do one update per batch. – anymous.asker Oct 21 '18 at 5:21
• Alright. Any reading material or source to read up? Thanks. – dinesh patro Oct 22 '18 at 16:48
• en.wikipedia.org/wiki/Stochastic_gradient_descent For lasso you'll need something different, such as stochastic ADMM or friends. You don;t need to know the details, there are many open-source implementations. – anymous.asker Oct 22 '18 at 16:54