Background:
I have created a basic modeling workflow in sklearn that utilizes sklearn's pipeline object. There are some preprocessing steps within the pipeline, and the last step of the pipeline is to fit an estimator that performs automatic feature-selection, for example, lasso regression. Thus, I begin my workflow with n features, but ultimately only m (where m < n) of them will be used to make predictions for the final model. A simple example is provided below:
from sklearn.datasets import load_breast_cancer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
# Use Wisconsin breast cancer dataset as an example
X, y = load_breast_cancer(return_X_y = True)
# Produce train test split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2)
# Pipeline preprocesses then applies lasso regression
pipe = Pipeline(steps = [
('preprocessing', StandardScaler()),
('lasso', LogisticRegression(penalty = 'l1', solver = 'saga', C = 0.4))
]
)
# Fit the pipeline
pipe.fit(X_train, y_train)
# View the coefficients
pipe.named_steps['lasso'].coef_
print('Count of non-zero coefficients: ',
sum(pipe.named_steps['lasso'].coef_[0] != 0))
print('Count of zero coefficients: ',
sum(pipe.named_steps['lasso'].coef_[0] == 0))
Count of non-zero coefficients: 20
Count of zero coefficients: 10
Objective:
I am now looking to apply my model to a new set of observations to produce predictions for them, but it seems computationally wasteful to run all of the n features through the pipeline, as only m of them will ultimately contribute to the final prediction. The preprocessing steps are not theoretically required on the other n-m features once the model has been fit. Of course this example uses a toy dataset, but on larger datasets the excessive processing could be material.
# Predictions require all features to exist in the X dataset
pipe.predict_proba(X_test)
# Passing a subset of the features causes an error - and understandably so since the pipeline
# lacks the context needed to determine which features are included
pipe.predict_proba(X_test[:,np.where(pipe.named_steps['lasso'].coef_[0] != 0)[0]])
ValueError: X has 20 features, but StandardScaler is expecting 30 features as input.
Is there a solution that would allow me to 'pare down' an already-fitted pipeline object so that it can operate on only the m features that are theoretically needed to make my predictions going forward?
