0
$\begingroup$

My dependent variable is a probability that is sourced from someone else's classification model. I am using this probability as a dependent variable as I don't have the actual data. On building an xgboost algorithm, the accuracy is 100%. There is no data leakage - but I wanted to ask - is a 100% accuracy possible when modeling a previously developed algorithm?

$\endgroup$
1
  • $\begingroup$ How do you know there's no data leakage in the other classification model? If I give you a feature vector that perfectly predicts the target class, it's quite possible that I cheated in the first place by simply copying the target class and giving it to you as a feature vector. $\endgroup$ Jul 15, 2021 at 17:06

1 Answer 1

1
$\begingroup$

Welcome to the community. If I understood correctly:

  1. you have a dataset whose target labels (aka ground truth) you don't know, so you figured these out by assigning the output of another model from someone else --> I guess this pre-trained model was built with the same dataset right?
  2. you built a new model on this dataset, using as target values the other model's output as your ground truth, giving you a 100% accuracy --> you convert the predicted probability to the corresponding label is it?

This might mean:

  • your model is providing the same output labels as the other model trained on the same dataset; this is not strange, mainly if the models are of the same type (e.g. tree-based models as your xgboost), mainly assuming the pretrained model outputs fit the real target values
  • the data is quite easy to "learn", so both models provide similar results over the same target labeling
  • to take into account: accuracy might not be an optimal metric, taking a look at how much unbalanced the dataset is (it is, what is the ratio between the different classes).

Nevertheless, I would make sure to find the real and precise corresponding targets to go on with confidence on your problem.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.