1
$\begingroup$

Suppose that I have $n$ trained weak base models: $m_1, m_2, ..., m_n$ As I understand after training that models we get their predictions on validation dataset, let's consider single element of sample. Predictions of models $y_1, y_2, ..., y_n$ (meta-features). Then we add true label to our data $y_1, ..., y_n, y_{true}$. And on that meta-features we train our meta-model. Here I have several questions.

  1. If it's supervised learning task what would be label to train meta model? $y_{true}$? If the answer is yes, I have no more questions. Otherwise I have no clue how we will make final predictions on holdout dataset.

Thanks for your help!

$\endgroup$

1 Answer 1

1
$\begingroup$

You are partially correct:

  • Yes, the meta-model is trained to predict the label $y_{true}$.
  • But $y_{true}$ is not used as input feature for the meta model. This would not work in any application / real-world scenario, where $y_{true}$ is not known and the (meta-)model is actually used to predict it.
  • Optionally, the meta model can also get the base features (i.e. the input to the models $m_1, \ldots, m_n$) as additional input. This can be helpful, if the base features help to decide which model is more reliable in a particular case.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.