I have built several models on the training dataset and i am not happy with the results and I wish to club them all together and generate a new model, so here is my idea as i already have the results for the existing models i would like to create a new dataset with the existing model results as separate features on top of the original feature dataset applying a clustering to filter some data in the original dataset and would like to train the model across all the same models and get the result, Would that be called as stacked modelling?
Stacking takes predictions from diverse and shallow or weak models on a dataset.
Stacks meta-features ( meta-features = predictions ) like columns. And usually a linear meta-model ( like Linear Regression ) is fitted on that dataset of metafeatures. Think of it as if you let multiple models , each one with his own prediction , decide what's the best value for each datapoint. mean across all models? maybe mean across just two? The meta-model decides.
Your approach of using meta-features with the original features ressembles to Boosting which takes each datapoint's residual ( difference between truth value and prediction ) and uses it as a feature to correct the gap iteration by iteration.
Would that be called as stacked modelling?
Yes, this is exactly what stacking models means.
I didn't understand what is the role of clustering in your design?
The standard approach consists in using only the predictions of the $N$ individual learners as feature for the meta-model. The training/testing data split is a bit more complex: the training of the meta-model requires predictions from the individual models, so the data could be split like this:
- training data for individual models
- training data for the meta-model (on which individual models are applied)
- testing data for the full system
Generally it's safer to choose a simple learning algorithm for the meta-model, because there is a higher risk of overfitting (for instance linear regression or majority voting).
[Detailed answer to comments below]
The standard setting would be like this:
- Train $N$ individual models $m_1,...,m_N$ using first-level training set $T_1$
- Apply these models in order to obtain $N$ features for the meta-classifier. At this stage it's a bad idea to use instances from $T_1$, because the individual classifiers have been trained on $T_1$. so one needs a second set of instances $T_2$: for every instance $x\in T_2$, let $m_1(x),...,m_N(x)$ be the predictions resulting from applying classifiers $m_1,...,m_N$.
- Train the meta-classifier model $M$ using $T_2$ as training set, with $m_1(x),...,m_N(x)$ the features for any $x\in T_2$.
- Testing: as usual one needs a fresh set of instances, say $T_3$. For any instance $x\in T_3$ calculate the prediction by applying $M$ to the $N$ predictions obtained by the individual models, i.e. $M(m_1(x),...,m_N(x))$