I am modeling a quality parameter of a chemical process product. I have a list of circa 400 process parameters sampled throughout the process. Most of them should have no meaningful impact on the modeled property. I am looking for an intuitive explanation of the impact these non-relevant parameters may have on the precision of the prediction. Should I try to map what is relevant technologically and exclude the rest from the model?

Thanks in advance


1 Answer 1


Take a look at PCA. PCA is used to reduce the dimension of your feature space. You could use PCA to transform your large set of variables into a smaller one that still contains most of the information in the large set. This helps reduce overfitting and simplifies computation.

Once you have the features selected, you can use your domain knowledge to explain why certain features may have an impact on your model performance. You may also try different subsets of features to determine which produce the best model.

Here's some simple starter code for PCA from sci-kit learn:

import numpy as np
from sklearn.decomposition import PCA
X = np.array([[-1, -1], [-2, -1], [-3, -2], [1, 1], [2, 1], [3, 2]])
pca = PCA(n_components=2)
>>> [0.9924... 0.0075...]
>>> [6.30061... 0.54980...]

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.